Rust Driver EngSys Prototype - #48867
Draft
Daniel Jurek (danieljurek) wants to merge 10 commits into
Draft
Conversation
Take eng/ wholesale from main to pick up the compiled-package build work, then reapply this branch's msrustup pipeline changes on top. Route builds to cibuildwheel when a package declares [tool.cibuildwheel], in addition to the existing ext_modules check. ext_modules only covers setuptools Extension objects, so packages built by other backends (maturin/PyO3) compile native code but declare none and were misrouted to `python -m build`, yielding a wheel tagged for whatever toolchain the agent happened to have. Also widen CIBW_ENVIRONMENT_PASS_LINUX to a superset. That variable replaces a package's own environment-pass list rather than merging with it, which silently dropped the msrustup credentials the manylinux container needs. cibuildwheel skips names unset on the host, so the extra entries are inert for other packages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
System.AccessToken is scoped to the azure-sdk organization, so RustInstaller could not authenticate against the DevDiv Rust.Sdk feed and failed with HTTP 401 on both Linux and Windows. Default the feed to an azure-sdk feed whose upstream points at azure-feed://devdiv/DevDiv/Rust.Sdk@Release so the upstream performs the cross-organization fetch. The feed is organization scoped, so the URL carries no project segment. Callers now forward MsRustToolchainFeed only when they set it, leaving the template default as the single source of truth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
Build agents run under 1ES network isolation, which does not allow index.crates.io, so cargo failed to resolve dependencies on Windows with "Could not connect to server ... index.crates.io port 443". Configure cargo to pull from the same Azure Artifacts feed azure-sdk-for-rust uses, following the approach in that repository: write a cargo config that replaces the crates-io source with the feed, then authenticate with NuGetAuthenticate and CargoAuthenticate. This is part of installing the internal toolchain rather than a separate opt-in, since an agent that needs that toolchain cannot reach crates.io either. Also set LOGLEVEL for extension builds so sdk_build streams cibuildwheel output live instead of withholding it until the command finishes, which left the step silent for minutes and then collapsed the whole build onto one timestamp. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
Linux wheels are built inside a container that cannot see the cargo config written on the agent, so cargo there still tried to reach index.crates.io, which build agents cannot resolve. Cargo reads registries.<name>.index and its credentials from the environment but ignores source replacement set that way, so the credentials are forwarded into the container and a small config naming the same registry is written by a before-all hook. The index URL and token stay in the environment and are never written to disk. The hook is a no-op when no index was forwarded, so local builds with direct crates.io access are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
The ARM64 Windows wheel is cross-compiled on an x64 agent, where only an x64 interpreter is available, so maturin had nothing to take the Python import library from and failed with "Need a Python interpreter to compile for Windows without PyO3's generate-import-lib feature". Enabling that feature makes pyo3 synthesize the import library for the target architecture instead. It only does anything for Windows targets, so the other platforms are unaffected. This pulls in python3-dll-a, the only lock change. Also corrects a stale comment: the driver has been a pinned git dependency for a while, not a path dependency on a sibling clone. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
An unpinned maturin let every target resolve its own version. The emulated aarch64 container found no aarch64 wheel for the version the other targets picked, fell back to a much older release, and produced a wheel holding the Python sources with no compiled extension. auditwheel was the only thing that noticed. Pin the version so all targets share one known-good maturin and a missing wheel fails loudly. Emulated aarch64 also needs far longer than a native build, so raise the build timeouts for packages that compile Rust. The increase is keyed off the existing toolchain parameter and leaves every other package unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
cibuildwheel builds the Windows ARM64 wheel with the x64 interpreter, since an ARM64 python cannot run on an x64 agent. maturin rejects that interpreter once it knows the target, and with no PYO3_CROSS_LIB_DIR set it searches the host for an ARM64 interpreter that cannot be there. Pointing the variable at the ARM64 CPython the pipeline already downloads moves maturin onto its abi3 Windows cross-compilation path, where a placeholder interpreter stands in for a real one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226b86d8-4eb1-4ad1-8507-207950b34cc7
|
Azure Pipelines: 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.