RYOW for DuckDB within an explicit txn

[drrtuy]

Add optional Read-Your-Own-Writes for cross-engine DuckDB scans

What

In cross-engine joins, DuckDB previously read non-DuckDB tables through a fiber running a synthetic SELECT in a separate background transaction, so uncommitted writes from the parent transaction were invisible. This PR adds an opt-in session variable that lets DuckDB read those uncommitted writes (RYOW) via a direct handler scan in the parent transaction.

Key changes

• ha_duckdb.cc: new session var duckdb_cross_engine_ryow (default OFF) plus accessor get_thd_cross_engine_ryow().
• cross_engine_scan.cc/.h: per-query RYOW toggle; new _mdb_scan_direct table function with filter_pushdown=false; replacement scan routes to _mdb_scan_direct when RYOW is on.
• ha_duckdb_pushdown.cc: registers the RYOW flag during init_scan().
• architecture.md: documents the RYOW path and its tradeoffs.

How to test

Run cross_engine_ryow.test: with the var OFF uncommitted parent-transaction writes to InnoDB are hidden from the join; with it ON they are visible; after ROLLBACK committed state is intact.

[gemini-code-assist]

Code Review

This pull request introduces the duckdb_cross_engine_ryow session variable, allowing cross-engine joins to read uncommitted writes from non-DuckDB tables via a direct handler scan in the parent transaction. While this enables Read-Your-Own-Writes (RYOW) capabilities, it disables predicate pushdown and index access for those tables. Feedback on the changes highlights a critical thread-safety issue: because DuckDB may execute the direct table function on its worker threads rather than the main connection thread, calling MariaDB handler operations without ensuring current_thd is correctly set could lead to server crashes or undefined behavior.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

When state.direct is true, the scan bypasses the fiber-based execution and runs directly. However, DuckDB may execute the table function on one of its worker threads rather than the main connection thread.

Because MariaDB storage engines (like InnoDB) rely heavily on the thread-local current_thd being set to the correct THD during handler operations (such as ha_rnd_next), calling these methods on a DuckDB worker thread where current_thd is NULL or incorrect will lead to undefined behavior, assertion failures, or server crashes.

To prevent this, we should temporarily set/swap current_thd to tbl->in_use (and restore it afterwards) during the direct handler scan execution if it is running on a different thread, or ensure that the execution of _mdb_scan_direct is strictly confined to the connection thread.