FEAT: conda recipe for mssql-python (self-contained repackage of the signed wheel) - #734
Merged
Merged
Conversation
Copilot started reviewing on behalf of
Jahnvi Thakkar (jahnvi480)
August 31, 2026 08:07
View session
Jahnvi Thakkar (jahnvi480)
requested review from
Gaurav Sharma (bewithgaurav) and
Sumit Sarabhai (sumitmsft)
August 31, 2026 08:10
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a dormant conda-build recipe (conda/mssql-python/) that repackages the prebuilt, ESRP-signed mssql-python wheel into a self-contained conda package by co-locating the mssql_python wheel and the mssql-python-odbc payload wheel in the same site-packages, with a Linux-only patchelf step to ensure the vendored driver resolves conda-provided runtime libraries.
Changes:
- Introduces
meta.yamlwith repackaging-safe conda-build settings (no relocation/prefix scanning) plus runtime dependency declarations for the bundled ODBC Driver payload. - Adds
build.sh/bld.batto install or extract wheels fully offline fromWHEELS_DIR, including Linux RUNPATH stamping and a win-arm64 bulk-copy limitation workaround. - Adds
conda/.gitattributesto enforce LF line endings for*.shscripts.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| conda/mssql-python/meta.yaml | Conda recipe metadata, build safeguards for signed binaries, and runtime dependency declarations. |
| conda/mssql-python/build.sh | Linux/macOS repackaging script; installs/extracts wheels and applies Linux RUNPATH “$ORIGIN climb” fix. |
| conda/mssql-python/bld.bat | Windows repackaging script; installs/extracts wheels and removes mssql_py_core on win-arm64. |
| conda/.gitattributes | Forces LF EOL for shell scripts to keep conda-build working across platforms. |
Suppressed comments (1)
conda/mssql-python/bld.bat:53
- Same wildcard issue as above for
ODBC_WHL: if the wheel isn't present, theforloop will still assign the literal pattern, so the intended explicit error message won't fire. Also switch the failure exits toexit /b 1for consistent batch behavior.
set "ODBC_WHL="
for %%W in ("%WHEELS_DIR%\mssql_python_odbc-%MSSQL_ODBC_VERSION%-py3-none-win_*.whl") do set "ODBC_WHL=%%~fW"
if not defined ODBC_WHL (
echo ERROR: no mssql_python_odbc==%MSSQL_ODBC_VERSION% py3-none-win wheel in "%WHEELS_DIR%"
exit 1
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…signed wheel) Repackages the prebuilt, ESRP-signed mssql-python wheel into a conda package offline (no compile), vendoring the ODBC Driver 18 payload inside it (the v1.11.0 self-contained model -- no separate mssql-python-odbc conda package). - meta.yaml: python >=3.10 floor (host + run); repackage-safe conda-build flags (error_overlinking / error_overdepending off, binary_relocation off, skip pyc on the osx/win cross legs); Linux run deps openssl >=3,<4 / krb5 / libtool for driver reachability (#563); win vc14_runtime. - build.sh: Linux/macOS repackage; patchelf $ORIGIN climb (#563); the cross osx-arm64 leg extracts the universal2 wheel without Python, filtered on CONDA_PY so it can never grab another interpreter's cpXY wheel. - bld.bat: Windows repackage; the cross win-arm64 leg extracts without Python and strips the x64 mssql_py_core (no arm64 build yet -> bulk copy raises a clean "not available" error); guarded wheel globs; exit /b on failure. - .gitattributes: *.sh eol=lf, *.bat eol=crlf.
Jahnvi Thakkar (jahnvi480)
force-pushed
the
jahnvi/conda-recipe
branch
from
August 31, 2026 08:22
3e6d637 to
6cce9af
Compare
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. 📋 Files Needing Attention📉 Files with overall lowest coverage (click to expand)mssql_python.pybind.logger_bridge.cpp: 58.9%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 75.5%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.6%
mssql_python.pybind.connection.connection.cpp: 84.4%
mssql_python.logging.py: 85.5%
mssql_python.connection.py: 85.9%🔗 Quick Links
|
…g tweaks - meta.yaml: pin azure-identity >=1.12.0 in run: -- conda does NOT inherit the wheel's install_requires, so a solve against conda-forge could otherwise pick <1.12.0. - build.sh: count libmsodbcsql (driver) and libodbcinst (driver manager) separately so a Linux payload missing EITHER fails loudly (a lone libodbcinst would ship no SQL driver). - Condense the verbose block comments in meta.yaml / build.sh / bld.bat to concise, meaningful lines while keeping the essential "why".
…cross legs
The osx-arm64 (built on Intel) and win-arm64 (built on x64) legs extract the wheel
without running the target Python, and the pipeline can't import-test them. A real
Apple-Silicon test found every osx-arm64 build (py3.11-3.14) shipped the cp310
ddbc_bindings, so only 3.10 could import. The cp${CONDA_PY} wheel filter already picks
the right wheel; this adds a static post-extract assert that the packaged
mssql_python/ddbc_bindings.cp<CONDA_PY>-* actually exists -- the python-tag twin of the
existing win-arm64 PE-arch assert -- so a wrong-Python binding fails the build instead
of the user's import.
Jahnvi Thakkar (jahnvi480)
added a commit
that referenced
this pull request
Aug 31, 2026
… hardening Cross-built osx-arm64 (on Intel) extracts the code wheel without running the arm64 Python, via a first-match glob that grabbed the alphabetically-first cp310 universal2 wheel for EVERY python -- so py3.11-3.14 shipped the cp310 ddbc_bindings and only 3.10 could import (found on a real Apple-Silicon Mac; the pipeline can't catch it because it never runs the arm64 import). Filter the code wheel by cp<CONDA_PY> (mirrors bld.bat's win_arm64 filter) and add a static post-extract assert that the packaged mssql_python/ddbc_bindings.cp<CONDA_PY>-* exists on the un-runnable cross legs. Also brings the recipe byte-in-line with PR #734: comment trim, azure-identity >=1.12.0 floor (conda does not inherit the wheel's install_requires), per-type Linux driver counters, guarded wheel globs, exit /b, and *.bat CRLF.
…a-build) The `python >=3.10` floor added in review broke conda-build's per-Python variant: a version range in `host: python` overrides the `--python` matrix, so every `--python 3.10..3.14` build collapsed into ONE package pinned to python_abi cp<newest=3.14> (build string `py>=310`). Every leg (win-64/arm64, osx-64/arm64, linux-64/aarch64) then failed validation with: package mssql-python-1.14.0-py314_0 requires python_abi 3.14.* *_cp314 ... python =3.10 is not installable `host: python` must be bare so conda-build binds the per-Python variant. The >=3.10 floor is enforced implicitly -- only py310-py314 are built and each package's python_abi pins its exact minor. Verified with `conda-build --output`: --python 3.10 -> py310_0, --python 3.13 -> py313_0. azure-identity >=1.12.0 (a real run floor) is kept.
Gaurav Sharma (bewithgaurav)
requested changes
Sep 1, 2026
… driver dlopens OpenSSL from fixed system paths, so the conda-env copy is unreachable; user note in description + maintainer comment on why in-env fixes dont work
…py3-none tag hides arch-specific driver DLLs, so py3-none-win_* could vendor an x64 driver into a win-arm64 package; mirror the code-wheel arch handling (Sumit review PR #734)
Jahnvi Thakkar (jahnvi480)
requested review from
Gaurav Sharma (bewithgaurav) and
Sumit Sarabhai (sumitmsft)
September 2, 2026 06:15
Sumit Sarabhai (sumitmsft)
left a comment
Sumit Sarabhai (sumitmsft)
left a comment
Contributor
There was a problem hiding this comment.
Automated review (proposal-only workflow), posting one high-confidence packaging reliability finding. Non-blocking comment.
Native Windows ARM64 selects the x64 ODBC wheel. On a native Windows ARM64 conda-build host the ARM64 Python is executable, so the else branch runs and installs the ARM64 code wheel — but ODBC_ARCH is hardcoded to win_amd64, so the later ODBC extraction targets the x64 payload. The build then either fails looking for mssql_python_odbc-...-py3-none-win_amd64.whl or vendors the wrong driver DLLs, leaving the package without the libs\windows\arm64 payload the loader expects. Consider deriving ODBC_ARCH from the conda target platform (target_platform/CONDA_SUBDIR) rather than from whether Python can execute.
…n executability bld.bat keyed ODBC_ARCH off whether %PYTHON% could run: the cross (arm64-on-x64) leg set win_arm64, the native else leg hardcoded win_amd64. On a NATIVE Windows arm64 conda-build host the arm64 Python IS executable, so it fell into the else leg and selected the win_amd64 ODBC wheel -- the package would fail to find mssql_python_odbc-...-py3-none-win_amd64.whl or vendor the x64 driver DLLs, leaving no libs\windows\arm64 payload for the loader. Derive the arch once from the conda target platform (target_platform, with CONDA_SUBDIR fallback): win-arm64 -> win_arm64, else win_amd64. Use it for the code-wheel glob and the ODBC wheel, and guard the x64 mssql_py_core strip on win_arm64. Python executability now only chooses pip-install (native) vs tar-extract (cross). (Sumit review, PR #734.)
…ql_py_core in win-arm64 wheels)
Jahnvi Thakkar (jahnvi480)
added a commit
that referenced
this pull request
Sep 2, 2026
…n executability bld.bat keyed ODBC_ARCH off whether %PYTHON% could run: the cross (arm64-on-x64) leg set win_arm64, the native else leg hardcoded win_amd64. On a NATIVE Windows arm64 conda-build host the arm64 Python IS executable, so it fell into the else leg and selected the win_amd64 ODBC wheel -- the package would fail to find mssql_python_odbc-...-py3-none-win_amd64.whl or vendor the x64 driver DLLs, leaving no libs\windows\arm64 payload for the loader. Derive the arch once from the conda target platform (target_platform, with CONDA_SUBDIR fallback): win-arm64 -> win_arm64, else win_amd64. Use it for the code-wheel glob and the ODBC wheel, and guard the x64 mssql_py_core strip on win_arm64. Python executability now only chooses pip-install (native) vs tar-extract (cross). (Sumit review, PR #734.)
…e bulk copy) Restore path for PR #737: the win-arm64 wheel now vendors the matching arm64 mssql_py_core, so stop unconditionally stripping it. bld.bat keeps the core when the extracted wheel has a matching-arch native ext (mssql_py_core.cp<ver>-<arch>.pyd) -- shipping bulk copy on win-arm64 -- and strips only when a legacy x64 core is present (a pre-#737 wheel), so the package never carries a core that can't load on the target. Self-healing: bulk copy turns on automatically once the arm64-core wheel is the conda-build input, with no build failure during the rollout gap.
Jahnvi Thakkar (jahnvi480)
requested a review
from Sumit Sarabhai (sumitmsft)
September 2, 2026 09:21
Sumit Sarabhai (sumitmsft)
approved these changes
Sep 2, 2026
Sumit Sarabhai (sumitmsft)
left a comment
Sumit Sarabhai (sumitmsft)
left a comment
Contributor
There was a problem hiding this comment.
Re-reviewed at 95c62f64. The earlier finding is resolved: conda/mssql-python/bld.bat now derives ODBC_ARCH from the conda target platform (target_platform/CONDA_SUBDIR) and uses the exact arch-specific ODBC wheel tag, so the native Windows ARM64 x64-wheel mis-selection no longer applies.
No new high-confidence correctness, security, reliability, or breaking-change findings. The outstanding macOS OpenSSL concern was already raised by another reviewer and is not duplicated here.
Gaurav Sharma (bewithgaurav)
approved these changes
Sep 2, 2026
Jahnvi Thakkar (jahnvi480)
added a commit
that referenced
this pull request
Sep 3, 2026
Second slice of the conda onboarding split (#720), after the recipe (#734, merged). Adds the OneBranch conda-build pipeline that runs conda-build against the recipe already on main and produces a consolidated conda/ artifact tree, plus the pure, no-DB validation tooling the pipeline gates on. Pipeline (OneBranchPipelines/): - conda-build-pipeline.yml + the per-platform build/validate step templates (conda-build-validate-step{,-posix}.yml) and the consolidate-artifacts job (win-arm64 py3.12-3.14 included in the accounting). - build-conda-packages.{ps1,sh}: drive conda-build per leg; idempotent builder env (set -e safe). The TLS-probe connection string is sourced from a SECRET variable (variable group / Key Vault) mapped into the step env, never a plaintext queue-time parameter that ADO would leave unmasked in logs. Validation tooling (conda/, eng/scripts/) + pure no-DB tests (tests/test_028-033): - tls_connect_probe.py / driver_load_probe.py: import-time TLS-handshake and driver-load probes. The connection-string splitter honors MS-ODBCSTR }} brace escaping so a braced password is never mis-split at an internal ';'; neutral failure labels for triage. - assert_pe_machine.py: Windows PE machine assert -- verifies the native binding (ddbc_bindings*.pyd) AND the vendored ODBC driver DLLs are both present and match the package arch (win-arm64 skips the runtime import, so this is its presence gate). - audit_bundled_binaries.py: Linux RUNPATH self-containment audit + ELF e_machine arch gate (linux-64==x86_64, linux-aarch64==aarch64). Scope: BUILD pipeline only. The release/publish steps, release-metadata validator, and re-solve drift gate move to a follow-up PR. The osx-arm64 slice is trusted from the universal2 wheel tag (no Mach-O audit is claimed); the enforced arch guards are the PE and ELF checks. The GitHub conda-audit workflow and the product-code changes are separate slices. All probe/audit unit tests pass; black clean. Validated on a NonOfficial ADO build run.
Jahnvi Thakkar (jahnvi480)
added a commit
that referenced
this pull request
Sep 3, 2026
Second slice of the conda onboarding split (#720), after the recipe (#734, merged). Adds the OneBranch conda-build pipeline that runs conda-build against the recipe already on main and produces a consolidated conda/ artifact tree, plus the pure, no-DB validation tooling the pipeline gates on. Pipeline (OneBranchPipelines/): - conda-build-pipeline.yml + the per-platform build/validate step templates (conda-build-validate-step{,-posix}.yml) and the consolidate-artifacts job (win-arm64 py3.12-3.14 included in the accounting). - build-conda-packages.{ps1,sh}: drive conda-build per leg; idempotent builder env (set -e safe). The TLS-probe connection string is sourced from a SECRET variable (variable group / Key Vault) mapped into the step env, never a plaintext queue-time parameter that ADO would leave unmasked in logs. Validation tooling (conda/, eng/scripts/) + pure no-DB tests (tests/test_028-033): - tls_connect_probe.py / driver_load_probe.py: import-time TLS-handshake and driver-load probes. The connection-string splitter honors MS-ODBCSTR }} brace escaping so a braced password is never mis-split at an internal ';'; neutral failure labels for triage. - assert_pe_machine.py: Windows PE machine assert -- verifies the native binding (ddbc_bindings*.pyd) AND the vendored ODBC driver DLLs are both present and match the package arch (win-arm64 skips the runtime import, so this is its presence gate). - audit_bundled_binaries.py: Linux RUNPATH self-containment audit + ELF e_machine arch gate (linux-64==x86_64, linux-aarch64==aarch64). Scope: BUILD pipeline only. The release/publish steps, release-metadata validator, and re-solve drift gate move to a follow-up PR. The osx-arm64 slice is trusted from the universal2 wheel tag (no Mach-O audit is claimed); the enforced arch guards are the PE and ELF checks. The GitHub conda-audit workflow and the product-code changes are separate slices. All probe/audit unit tests pass; black clean. Validated on a NonOfficial ADO build run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work Item / Issue Reference
Summary
First of a series splitting the large conda onboarding branch (#720) into smaller, reviewable PRs.
Adds the conda-build recipe that repackages the prebuilt, ESRP-signed
mssql-pythonwheel into a self-contained conda package (it vendors the ODBC Driver 18 payload — there is no separate companion package). The recipe compiles nothing; it repackages the already-signed wheel, fully offline.Files
conda/mssql-python/meta.yaml— recipe manifest (version fromMSSQL_PYTHON_VERSION; repackage-safe build flags so conda-build never rewrites/scans the signed binaries).conda/mssql-python/build.sh— Linux/macOS: install/extract the code + odbc wheels into site-packages, then stamp the relative$ORIGINRUNPATH climb (Publish mssql-python as a conda-forge package #563) onto the vendored Linux driver so it resolves the env's own krb5/openssl.conda/mssql-python/bld.bat— Windows: extract the code + odbc wheels; on win-arm64 it strips the x64mssql_py_core(no arm64 build exists yet, so bulk copy is a documented limitation on that platform).conda/.gitattributesSafety: this recipe is dormant on
main— nothing runs it until the standalone conda build pipeline (a follow-up PR) invokes it, so merging is safe and changes no existing behavior.Follow-up PRs in this split: (2) conda build pipeline + the gates/probes it runs, (3) conda release pipeline, (4) the GitHub conda-audit workflow.