Gate new ScalarSubqueryExec node behind session property

[LiaCastaneda]

Which issue does this PR close?

Related to discussion on #21240 and #21080 (comment).

PR #21240 introduced ScalarSubqueryExec / ScalarSubqueryExpr to execute uncorrelated scalar subqueries during physical execution. The two communicate via shared in process state (a slot in ExecutionProps), which breaks distributed execution that may split execution across a network boundary between the producer (ScalarSubqueryExec) and the consumer expression (ScalarSubqueryExpr). See more details on this explanation in datafusion-contrib/datafusion-distributed#460

What changes are included in this PR?

Adds a new optimizer config option datafusion.optimizer.physical_uncorrelated_scalar_subquery (default true, preserving the current behavior). When true (default), behavior is unchanged from current main; when false, all scalar subqueries are rewritten to left joins by ScalarSubqueryToJoin and ScalarSubqueryExec is never constructed (which was the previous behavior).

Are these changes tested?

Yes all tests pass and added uncorrelated_scalar_subquery_rewritten_when_flag_off to test the negative case.

Are there any user-facing changes?

Yes, a new config option datafusion.optimizer.physical_uncorrelated_scalar_subquery (this just changes the way the query is executed but not the results)

[gabotechs]

Nice, thanks @LiaCastaneda. Is there a chance you could take a look at this one @neilconway?

[LiaCastaneda]

I also ran the tpch queries in my local with the flag turned off (old path), all results match. Maybe it's worth adding it as part of the regular checks

edit: added here d1b9dad

[neilconway]

I think this makes sense as an interim measure if it will be too difficult to adapt df-distributed and/or ballista in the short-term, but long-term I'd prefer not to have a config option that silently produces incorrect query results. Can we add a note that disabling this is not recommended, and that we plan to remove the config option in the future -- say in a few DF releases from now?

[neilconway]

The other similar config options use the phrasing enable_...; we should probably adopt that for consistency.

physical_uncorrelated_scalar_subquery is also a mouthful, although I can't immediately think of a more concise name that is also accurate.

[milenkovicm]

thank you @LiaCastaneda , @gabotechs & @neilconway for driving this

[LiaCastaneda]

Can we add a note that disabling this is not recommended, and that we plan to remove the config option in the future -- say in a few DF releases from now?

Makes sense, I will also create an issue to keep track on this and not forget