Add CodeQL native C build scanning to storage pipeline

[danieljurek]

No description provided.

[azure-pipelines]

Azure Pipelines:
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds compiled CodeQL scanning for the storage pipeline and ensures hybrid extension packages use cibuildwheel.

Changes:

• Wires a compiled-CodeQL pipeline parameter through shared templates.
• Detects setup.py-defined extensions alongside pyproject.toml.
• Routes compiled pyproject packages through cibuildwheel.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sdk/storage/ci.yml	Enables compiled CodeQL for storage.
eng/pipelines/templates/stages/archetype-sdk-client.yml	Forwards the CodeQL parameter.
eng/pipelines/templates/stages/1es-redirect.yml	Configures compiled C/C++ scanning.
eng/tools/azure-sdk-tools/ci_tools/parsing/parse_functions.py	Detects setup.py extensions for pyproject packages.
eng/tools/azure-sdk-tools/ci_tools/build.py	Builds detected extensions with cibuildwheel.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

eng/tools/azure-sdk-tools/ci_tools/build.py:246

• -vv is a Python interpreter flag, not a cibuildwheel verbosity flag, so it emits verbose import tracing into run_logged's captured output for every compiled-wheel build. The pipeline already sets CIBW_BUILD_VERBOSITY=3; remove this flag to avoid needlessly inflating captured logs and obscuring build failures, consistent with the existing setup.py cibuildwheel path below.

                [sys.executable, "-vv", "-m", "cibuildwheel", "--output-dir", dist],

eng/tools/azure-sdk-tools/ci_tools/parsing/parse_functions.py:719

• Add a parser test for the new hybrid-package fallback. The existing pyproject_project_def_with_extension scenario declares tool.setuptools.ext-modules directly and has no setup.py, so it never exercises lines 713–719; a regression here would silently route the storage extension through the pure-Python build path and defeat the native scan this PR enables.

    if not ext_modules:
        sibling_setup_py = os.path.join(package_directory, "setup.py")
        if os.path.exists(sibling_setup_py):
            try:
                setup_py_result = parse_setup_py(sibling_setup_py)
                ext_package = ext_package or setup_py_result[11]  # ext_package
                ext_modules = setup_py_result[12]  # ext_modules

sdk/storage/ci.yml:42

• Correct the typo “becuase” to “because.”

    # Enable Compiled CodeQL becuase azure-storage-extensions has C code that 

[danieljurek]

@copilot run black and fix formatting as described here: https://github.com/Azure/azure-sdk-for-python/actions/runs/29761331967/job/88416408198?pr=48098

Push the change to this branch.

[Copilot]

@copilot run black and fix formatting as described here: https://github.com/Azure/azure-sdk-for-python/actions/runs/29761331967/job/88416408198?pr=48098

Push the change to this branch.

Fixed in the latest commit — removed trailing whitespace from docstring blank lines (the black failure) and also dropped the -vv interpreter flag from the cibuildwheel invocation per the review feedback.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

eng/tools/azure-sdk-tools/ci_tools/parsing/parse_functions.py:719

• The new setup.py fallback is not covered by the existing pyproject extension test, which only exercises declarative [tool.setuptools].ext-modules. Add a dual-config fixture with extensions declared only in setup.py and assert that ParsedSetup.from_path(...) preserves is_pyproject while discovering ext_package and ext_modules; otherwise tuple-index or parser regressions can silently route native packages through the pure-Python path.

    if not ext_modules:
        sibling_setup_py = os.path.join(package_directory, "setup.py")
        if os.path.exists(sibling_setup_py):
            try:
                setup_py_result = parse_setup_py(sibling_setup_py)
                ext_package = ext_package or setup_py_result[11]  # ext_package
                ext_modules = setup_py_result[12]  # ext_modules

sdk/storage/ci.yml:42

• Correct the typo “becuase” to “because”.

    # Enable Compiled CodeQL becuase azure-storage-extensions has C code that 

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

eng/tools/azure-sdk-tools/ci_tools/parsing/parse_functions.py:719

• The new fallback path is not covered by the existing parser tests: test_parse_pyproject_extensions only exercises declarative [tool.setuptools].ext-modules, and its fixture has no sibling setup.py. Add a fixture matching azure-storage-extensions (a [project] pyproject with the extension declared only in setup.py) and assert that ParsedSetup.from_path discovers the extension; otherwise tuple-index or fallback regressions can silently route native packages through the pure-Python builder.

    if not ext_modules:
        sibling_setup_py = os.path.join(package_directory, "setup.py")
        if os.path.exists(sibling_setup_py):
            try:
                setup_py_result = parse_setup_py(sibling_setup_py)
                ext_package = ext_package or setup_py_result[11]  # ext_package
                ext_modules = setup_py_result[12]  # ext_modules

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

eng/pipelines/templates/steps/build-package-artifacts.yml:167

• CIBW_ENVIRONMENT has higher precedence than pyproject.toml, so this replaces each package's existing cibuildwheel environment instead of adding the feed URL. For azure-storage-extensions, that drops its configured CFLAGS value. Pass the already-defined variable into Linux containers with CIBW_ENVIRONMENT_PASS_LINUX so package-specific settings remain effective.

  - pwsh: |

eng/tools/azure-sdk-tools/ci_tools/parsing/parse_functions.py:719

• The existing pyproject extension test covers only declarative [tool.setuptools].ext-modules; this new sibling-setup.py fallback is untested. Add a fixture containing both files with the Extension(...) declared only in setup.py, then assert that ParsedSetup.from_path recovers ext_package and ext_modules, since this result now controls routing to cibuildwheel.

    if not ext_modules:
        sibling_setup_py = os.path.join(package_directory, "setup.py")
        if os.path.exists(sibling_setup_py):
            try:
                setup_py_result = parse_setup_py(sibling_setup_py)
                ext_package = ext_package or setup_py_result[11]  # ext_package
                ext_modules = setup_py_result[12]  # ext_modules

eng/pipelines/templates/steps/build-package-artifacts.yml:148

• $ErrorActionPreference = 'Stop' does not make failed native executables terminate this pwsh script. If the first NuGet install fails but the second succeeds, this step still reports success and leaves the AMD64 cache missing, causing cibuildwheel to retry the blocked endpoint later. Enable native-command error propagation, as the existing Prep Environment step does.

      $ErrorActionPreference = 'Stop'
      $feed = "https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-net/nuget/v3/index.json"

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

eng/tools/azure-sdk-tools/ci_tools/parsing/parse_functions.py:719

• The new fallback is not exercised by the existing parser tests: test_parse_pyproject_extensions uses declarative [tool.setuptools].ext-modules, so it never enters this branch. Add a pyproject fixture with no declarative extensions and a sibling setup.py defining Extension(...), then assert that ParsedSetup discovers both ext_package and ext_modules; this is the behavior that determines whether the package is routed to cibuildwheel.

    if not ext_modules:
        sibling_setup_py = os.path.join(package_directory, "setup.py")
        if os.path.exists(sibling_setup_py):
            try:
                setup_py_result = parse_setup_py(sibling_setup_py)
                ext_package = ext_package or setup_py_result[11]  # ext_package
                ext_modules = setup_py_result[12]  # ext_modules