Add support for parquetOptions in GCSToBigQueryOperator

[mlauter]

Description

Add support for parquetOptions in GCSToBigQueryOperator src_fmt_configs.

My team has a lot of workflows that involve loading parquet in gcs to bigquery using airflow. By default (without parquetOptions.enable_list_inference) parquet lists get loaded to bigquery as STRUCT<list ARRAY<STRUCT<element $TYPE>>>. This nested struct is cumbersome to use for querying and analysis.

With the enable_list_inference flag, the same parquet list is loaded simply as ARRAY<$TYPE> which is much easier to work with.

This PR adds support for passing enableListInference as one of the options in src_fmt_configs when the source format is PARQUET. This works both for the external table code path as well as the bq load code path.

Testing

• Added unit tests covering new behavior
• Ran all unit tests for the operator
• Run providers/google/tests/system/google/cloud/gcs/example_gcs_to_bigquery.py (haven't managed to get this running yet)
  • These currently don't work without commenting out all the openlineage stuff slack. I can get it working if this is a requirement, but wondering if my own test (see below) suffices.
• Created a custom dag to exercise the behavior and ran it in the breeze environment (see results below)

Without enableListInference:

    gcs_to_bq_task = GCSToBigQueryOperator(
        task_id="gcs_to_bigquery_parquet_no_options",
        bucket="<my_bucket>",
        source_objects=["<my_parquet_file_with_list"],
        destination_project_dataset_table="<my_table>",
        source_format="PARQUET",
        write_disposition="WRITE_TRUNCATE",
    )

Produces:

With enableListInference:

    gcs_to_bq_task = GCSToBigQueryOperator(
        task_id="gcs_to_bigquery_parquet_no_options",
        bucket="<my_bucket>",
        source_objects=["<my_parquet_file_with_list"],
        destination_project_dataset_table="<my_table>",
        source_format="PARQUET",
        write_disposition="WRITE_TRUNCATE",
        src_fmt_configs={"enableListInference": True},
    )

produces:

And likewise for the external table case which i also tested.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.5 following the guidelines

GenAI tooling was used only for code review and discussion, no lines of code in the PR were written or directly copied from Claude.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[shahar1]

CC: @VladaZakharova @MaksYermak

[shahar1]

Overall LGTM - small comment regarding the newsfragment file, and once you fix it I'll wait a couple of more days to let Google team to review it before merging.
As you tested the changes on a real BigQuery instance and changes are non-breaking, I feel comfortable merging it.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.