Enable one-click crates.io release for mxc-sdk

[dhoehna]

Summary

Adds a one-click Azure DevOps pipeline that publishes the mxc-sdk crate and its first-party dependency closure to crates.io, plus the crate-metadata changes required to make that closure publishable.

This branch contains two logical commits:

1. Make the mxc-sdk crate closure publishable (11 Cargo.toml files) — registry-agnostic metadata: workspace repository, per-crate description/license/repository, dual { path, version } on internal path deps, and making sandbox_spec publishable. Required to publish from anywhere.
2. Add the release pipeline (2 files) — .azure-pipelines/1ES.Crate.Release.yml and .azure-pipelines/scripts/publish_crates_to_cratesio.py.

How it works

• Run pipeline with dryRun = true (the default) previews every crate via cargo package --list and publishes nothing.
• With dryRun = false, a Publish stage waits for manual approval on the MXC-CratesIo-Production environment, then publishes leaf-first, waiting for index propagation between crates. Re-runs are idempotent (a version already on crates.io is skipped).

Secrets stay out of the repo

CARGO_REGISTRY_TOKEN comes from the ADO variable group MXC-CratesIo-Publish and is mapped to the publish step's env only. Nothing secret is committed.

One-time setup (not codeable in the repo)

Variable group, environment + approval, crates.io egress allowlist, crate-name reservation / OSPO sign-off, and registering the pipeline. See the checklist at the bottom of 1ES.Crate.Release.yml.

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull request overview

This PR enables releasing the Rust mxc-sdk crate to crates.io by (1) making the crate’s first-party dependency closure publishable via Cargo metadata/manifest adjustments, and (2) adding an Azure DevOps “one-click” pipeline plus a helper script to publish the closure leaf-first.

Changes:

• Add crates.io publish metadata to mxc-sdk and its first-party dependency closure (license/repository/description) and make sandbox_spec publishable.
• Convert internal path dependencies used by the publishable closure to dual { path, version } so cargo publish can strip path and publish clean manifests.
• Add an ADO pipeline (1ES.Crate.Release.yml) and a Python script to dry-run/package or publish the crate closure with index-propagation waits.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Cargo.toml	Adds workspace repository and sets { path, version } for publishable internal workspace deps.
src/core/wxc_common/Cargo.toml	Adds publish metadata and ensures optional nanvix_common path dep includes a version for publishing.
src/core/mxc-sdk/Cargo.toml	Adds workspace repository metadata needed for crates.io publishing.
src/core/mxc_pty/Cargo.toml	Adds publish metadata (repository/description) for the closure.
src/mxc_telemetry/Cargo.toml	Adds workspace license/repository metadata for publishing.
src/core/generated/base_container_specification/Cargo.toml	Removes publish = false by making sandbox_spec publishable and adds repository metadata.
src/backends/appcontainer/common/Cargo.toml	Adds publish metadata for publishing appcontainer_common.
src/backends/bubblewrap/common/Cargo.toml	Adds publish metadata for publishing bwrap_common.
src/backends/lxc/common/Cargo.toml	Adds publish metadata for publishing lxc_common.
src/backends/seatbelt/common/Cargo.toml	Adds publish metadata for publishing seatbelt_common.
src/backends/nanvix/common/Cargo.toml	Adds workspace repository metadata so nanvix_common is publishable.
.azure-pipelines/1ES.Crate.Release.yml	New ADO pipeline that runs dry-run preview and gated publish to crates.io.
.azure-pipelines/scripts/publish_crates_to_cratesio.py	New helper script to dry-run package or publish crates leaf-first and wait for sparse-index propagation.

[bbonaby]

issue: To publish to crates.io using the official Microsoft open source account we will need to publish it through the ESRP release system. See: https://eng.ms/docs/microsoft-security/identity/trust-and-security-services/tss-release-distribute/tss-release-esrp-parent/oss-publishing/releasing-open-source/cratesio . You will need to follow that and do something similar to what we do in the 1ES.Release.yml. To be honest, we should be able to reuse that same yaml file to release to either Npm, Crates.io or both.

At the very least that one should be the entry point to all releases, but we can still have template yml files that we point to to keep it clean. Everything in terms of set up on the back end for ESRP should already be set up for us. So, let me know if you have any issues with that.

In the ESRP docs they say the azure rust sdk folks have this working, where they publish all their crate dependencies first then publish their main crate. I believe the yml file they do this in is this one https://github.com/Azure/azure-sdk-for-rust/blob/95eb8c3878cb2c7d3239b5f584b8179ee0483c31/eng/pipelines/templates/stages/archetype-rust-release.yml . We can probably do something similar.

[dhoehna]

Thanks Brandon -- followed this. Reworked the crate release to go through ESRP and folded it into 1ES.Release.yml so one pipeline releases npm, crates.io, or both:

• 1ES.Release.yml now takes publishNpm / publishCrates boolean params. The existing NPM stage is gated on publishNpm, and a new Publish_to_CratesIo stage runs the ESRP path.
• Following the azure-sdk-for-rust approach, the official build packages the closure leaf-first in a single cargo package invocation (all --package flags together, so intra-closure deps resolve locally) into the mxc-crates-package artifact. The release job then runs one EsrpRelease@12 task per crate (contenttype: Rust, account microsoft-oss-releases), leaf-first, verifying each on the crates.io sparse index before the next.
• Deleted the old token-based 1ES.Crate.Release.yml + publish_crates_to_cratesio.py -- no more CARGO_REGISTRY_TOKEN. New templates: templates/Package.Crates.Job.yml (packaging) and templates/Publish.CratesIo.Job.yml (ESRP release loop).

This also obsoletes the two Copilot comments above (they were on the now-deleted script/pipeline).

One heads-up before we flip publishing on -- publishCrates defaults to false for now. Packaging the true --all-features closure of mxc-sdk surfaced that it now routes through the mxc_engine hub crate and expands to 17 first-party crates. Before a first real publish we still need to: (1) add a version to the 7 path deps in src/Cargo.toml that currently only carry path; (2) resolve isolation_session_bindings being publish = false; and (3) do the OSPO OSS-release registration + reserve the crate names. Happy to take that as a follow-up -- let me know if you'd rather have it in this PR or a separate one.

[bbonaby]

issue: we should probably add a conditional here to only do this packaging when the isOfficialBuild parameter is true.

[bbonaby]

note: the ESRP task above will actually keep polling until the package is published so we probably don't need this. It's done when the waitforreleasecompletion input to the task is set to true. That said, I'm not against the extra wait task.

[bbonaby]

issue: I like the parameters, lets make both false so it's clear in the UI that we expect the person initiating the pipeline run to select one or both.

[bbonaby]

issue: So this won't worked in the 1ES pipeline due to network isolation. You'd need to use the private feed's sparse index. Should be this one

mxc/.azure-pipelines/.cargo/config.toml

Line 8 in 3fd07f5

Mxc-Azure-Feed = { index = "sparse+https://microsoft.pkgs.visualstudio.com/Dart/_packaging/Mxc-Azure-Feed/Cargo/index/" }

So it would look something like this:

AZURE_FEED_REGISTRY = "Mxc-Azure-Feed"
AZURE_FEED_SPARSE_INDEX = (
    "https://microsoft.pkgs.visualstudio.com/"
    "Dart/_packaging/Mxc-Azure-Feed/Cargo/index"
)
SYSTEM_ACCESS_TOKEN_ENV = "SYSTEM_ACCESSTOKEN"

then you can have a helper function in here somewhere:

def _feed_request(url: str) -> urllib.request.Request:
    """Create an Azure Artifacts request authenticated by the pipeline token."""
    token = os.environ.get(SYSTEM_ACCESS_TOKEN_ENV)
    if not token:
        raise RuntimeError(
            f"{SYSTEM_ACCESS_TOKEN_ENV} is not set. Map $(System.AccessToken) "
            f"to that environment variable in the pipeline task."
        )

    return urllib.request.Request(
        url,
        headers={"Authorization": f"Bearer {token}"},
    )

Then your _published_versions function below can look similar to:

def _published_versions(crate: str) -> set[str]:
    """Return the versions of `crate` already in the Azure Artifacts feed."""
    url = f"{AZURE_FEED_SPARSE_INDEX}/{_sparse_index_path(crate)}"
    with urllib.request.urlopen(_feed_request(url), timeout=30) as response:
        body = response.read().decode("utf-8")
    versions: set[str] = set()
    for line in body.splitlines():
        line = line.strip()
        if line:
            versions.add(json.loads(line)["vers"])
    return versions

You also have have references to crates.io in the comments of this file as well. You'd want to update those to just say Azure Artifacts Feed or something like that.

[bbonaby]

issue: We should update this to include all the crates that will be published as part of the mxc-sdk. That said, The crate names themselves should actually be prefixed with mxc- so they are easily discoverable in crates.io.

I think we also need a follow up PR to add a GitHub PR workflow/action to check that all crates are publishable.

[bbonaby]

issue: most of this comment can be removed or reduced now I believe.

[bbonaby]

Note: for your testing, you can add a dry-run boolean in here too, that when set won't go into the EsrpRelease@10 tasks. You can actually test this out if you create an in repo branch (noticed this PR is coming from your fork). But yea if you create an in repo branch e.g user/dhoehna/test-rust-publishing with these same changes, you should be able to use the branch to run the official pipeline via https://microsoft.visualstudio.com/Dart/_build?definitionId=190018 and then see any errors that pop up so you can fix them in this PR. Let me know if you don't have access.