Skip to content

Use shared localization submission and pull-only release imports - #1971

Open
Bill Schnurr (bschnurr) wants to merge 7 commits into
mainfrom
users/bschnurr/shared-localization
Open

Bill Schnurr (bschnurr) wants to merge 7 commits into
mainfrom
users/bschnurr/shared-localization

Conversation

@bschnurr

Copy link
Copy Markdown
Member

Summary

  • Use the shared Python localization core and composite action rather than copying helper scripts into this extension.
  • Keep GitHub English submission separate from pull-only DevDiv stable/pre-release imports.
  • Use drop branch dev and canonical extension ID ms-python.vscode-python-envs; remove tracked generated locale snapshots where present and keep generated files in VSIX selection.
  • Preserve existing runtime metadata, extraction sources, feeds, platform matrices, and version/validation behavior.
  • Validate readiness independently in every required real-language manifest/runtime table; allow individual untranslated entries and reject missing/wholly untranslated tables before packaging.

Shared dependencies

The GitHub action and Azure tooling checkout are pinned to d1877b84bb10ff7caa6bd12ab4e75498ba760790. Azure templates temporarily reference the shared feature branch for verification; switch to Pylance-Eng main after the dependency merges, before merging this caller.

Verification

Executable guards read the actual YAML and preserve existing triggers, build steps, publication defaults, validation contacts, version flags, and matrices. Both release YAMLs expanded with publishing explicitly disabled (environments used the enabled validation definition because its native definitions are disabled). Actual English export used the pinned CLI through the internal feed. Changed YAML/JSON formatting and whitespace checks passed.

Required setup and release blockers

Canonical English/output directories are absent from drop dev. Both native DevDiv release definitions are disabled; the exact YAML was expanded through the enabled isort validation definition without enabling them.

No repository-level PYTHON_LOCALIZATION_APP_ID variable or PYTHON_LOCALIZATION_PRIVATE_KEY secret is currently visible here. Organization-level access could not be verified with current permissions. Configure or expose them through the owners; do not copy credentials between repositories. Authorize Github-Pylance for both release resources/pipelines.

Keep this PR draft while dependencies/setup/handbacks are pending. No merge or publication is requested.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bschnurr Bill Schnurr (bschnurr) added the debt Code quality issues label Oct 30, 2026
@bschnurr

Copy link
Copy Markdown
Member Author

Current CI blocker: Smoke Tests (windows-latest, 3.15) fails in the before-all hook because environment discovery does not complete within 45000ms. This PR does not change the runtime discovery implementation or smoke-test workflow. The failure has not been reproduced on the base branch, so it is not being classified as confirmed pre-existing. Left unresolved and outside the localization change; keep this PR draft pending CI investigation as well as its documented localization onboarding and disabled release definitions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the selecting manager separate from the fallback environment owner when clearing explicit global selections. Capture non-Linux E2E and integration host logs to diagnose remaining failures.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6166fde-f261-4e66-abe4-3ef5c1313713
@bschnurr

Bill Schnurr (bschnurr) commented Nov 3, 2026 •

Copy link
Copy Markdown
Member Author

Fixed and pushed in 5e192fc and 385c9c6. Global interpreter routing now retains the selecting manager instead of inferring it from a fallback environment owner; regression coverage verifies that clearing an explicit selection follows the configured manager on subsequent refreshes. Added non-Linux E2E/integration failure diagnostics using the existing test-host hooks, without weakening assertions or adding automatic test retries. Validation: full local unit suite, 72 targeted manager tests including the new regression, TypeScript compilation, webpack, ESLint, actionlint, actual-YAML diagnostic checks and the existing Python-matrix guard. Real Windows discovery and controlled stdlib/UV venv removal probes passed. Final revision 385c9c6: all 53 jobs in the pull-request workflow passed (https://github.com/microsoft/vscode-python-environments/actions/runs/41703916165); the duplicate push workflow also passed (https://github.com/microsoft/vscode-python-environments/actions/runs/41703906838). Reran only jobs that failed before executing tests because of Marketplace HTTP 503 or VS Code download timeouts. The original Windows discovery/removal failures did not recur. The test-file policy check now passes. No failed checks remain; the separate external VS Code PR Check is still pending.

Verify that a later configured-manager refresh publishes and reads its new environment rather than routing through the stale fallback owner.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6166fde-f261-4e66-abe4-3ef5c1313713
@bschnurr
Bill Schnurr (bschnurr) marked this pull request as ready for review November 4, 2026 17:48
@rchiodo

Rich Chiodo (rchiodo) commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🔒 Automated review in progress — Rich Chiodo (@rchiodo) is auto-reviewing this PR.

@rchiodo

Copy link
Copy Markdown
Contributor

Result: ⚠️ partially-verified

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: Test compilation and all 30 targeted environment-manager tests passed, including the PR’s new regression test. Offline dependency preparation succeeded; local YAML contract checks and generated-localization file selection through VSCE also passed. Initial discovery encountered missing dependencies and absent container Git metadata, not a demonstrated PR defect. Shared localization execution, real handback readiness, Azure template expansion, and non-Linux diagnostic collection remain unverified. **Reviewer dependency follow-up (nonblocking):** Suggest a separate PR adding `mocha ^10.8.2`, `typescript ^5.1.3` to the official reviewer dependency declarations, using `package.json` as the dependency/version evidence. Do not automatically create that PR. This is reviewer environment maintenance, not a required change to the reviewed PR.

Test runs: 5 passed, 1 failed, 2 not run

  • ⚠️ Not run | Shared localization submission, release import and readiness validation
  • ⚠️ Not run | Non-Linux E2E and integration failure diagnostics
  • ❌ Failed | unrelated to this PR | Dependency and test discovery | git status --short; git log -3 --oneline; printf '\nEnvironment\n'; printenv AUTOMATION_SANDBOX_PROFILE; uname -s; node --version; npm --version; printf '\nDependency and test discovery\n'; node -e "const fs=require('fs'); const p=require('./package.json'); console.log(JSON.stringify({scripts:p.scripts,devDependencies:p.devDependencies,dependencies:p.dependencies},null,2)); for(const f of ['node_modules','node_modules/mocha/bin/mocha.js','node_modules/typescript/bin/tsc','out/test/features/envManagers.unit.test.js']) console.log(f,fs.existsSync(f));"; git diff --stat HEAD~1 HEAD; git ls-files 'localization' 'unit.test' 'test.py'
  • ✅ Passed | Compile TypeScript tests | npm run compile-tests
  • ✅ Passed | Environment manager unit tests | node ./node_modules/mocha/bin/mocha.js --no-config --require source-map-support/register --require ./out/test/unittests.js --ui tdd --timeout 180000 out/test/features/envManagers.unit.test.js
  • ✅ Passed | Offline dependency preparation | npm ci --offline
  • ✅ Passed | Local localization and diagnostics YAML contracts | node <<'JS'
    const assert = require('node:assert/strict');
    const fs = require('node:fs');
    const yaml = require('js-yaml');
    const read = p => yaml.load(fs.readFileSync(p, 'utf8'));
    const workflow = read('.github/workflows/localization.yml');
    assert.deepEqual(workflow.permissions, {contents:'read'});
    assert.deepEqual(workflow.on.push.branches, ['main']);
    assert.equal(workflow.concurrency['cancel-in-progress'], false);
    assert.equal(workflow.jobs.localization.if, "github.repository == 'microsoft/vscode-python-environments'");
    const action = workflow.jobs.localization.steps[0];
    const pin = 'd1877b84bb10ff7caa6bd12ab4e75498ba760790';
    assert.equal(action.uses, microsoft/vscode-common-python-lsp/.github/actions/localization@${pin});
    assert.equal(action.with['app-id'], '${{ vars.PYTHON_LOCALIZATION_APP_ID }}');
    assert.equal(action.with['private-key'], '${{ secrets.PYTHON_LOCALIZATION_PRIVATE_KEY }}');
    for (const quality of ['stable', 'pre-release']) {
    const pipeline = read(build/azure-devdiv-pipeline.${quality}.yml);
    const repos = Object.fromEntries(pipeline.resources.repositories.map(r => [r.repository, r]));
    assert.equal(repos.translations.name, 'microsoft/vscode-python-extensions-loc');
    assert.equal(repos.translations.ref, 'refs/heads/dev');
    assert.equal(repos.translations.endpoint, 'Github-Pylance');
    assert.equal(repos.localizationTools.name, 'microsoft/vscode-common-python-lsp');
    assert.equal(repos.localizationTools.ref, pin);
    assert.equal(repos.localizationTools.endpoint, 'Github-Pylance');
    assert.equal(pipeline.extends.template, azure-pipelines/extension/${quality}.yml@templates);
    assert.equal(pipeline.extends.parameters.pullLocalization, true);
    assert.equal(pipeline.extends.parameters.standardizedVersioning, true);
    assert.equal(pipeline.parameters.find(p => p.name === 'publishExtension').default, true);
    assert.deepEqual(pipeline.parameters.find(p => p.name === 'buildPlatforms').default.map(p => p.vsceTarget), ['web','linux-arm64','linux-armhf','linux-x64','alpine-arm64','alpine-x64','darwin-arm64','darwin-x64','win32-arm64','win32-x64']);
    assert.deepEqual(pipeline.parameters.find(p => p.name === 'buildSteps').default.map(s => s.script), ['npm ci','python ./build/update_package_json.py', quality === 'stable' ? 'python ./build/update_ext_version.py --release --for-publishing' : 'python ./build/update_ext_version.py --for-publishing','npm run package']);
    if (quality === 'stable') {
    assert.equal(pipeline.extends.parameters.validationNotifyUsers, 'plseng@microsoft.com');
    assert.deepEqual(pipeline.trigger.branches.include, ['release/*']);
    } else {
    assert.equal(pipeline.trigger, 'none');
    assert.equal(pipeline.schedules[0].cron, '0 10 * * 1-5');
    }
    console.log(PASS ${quality}: pinned pull-only caller wiring, version flags, build steps, publication default and 10 targets);
    }
    const pr = read('.github/workflows/pr-check.yml');
    for (const [job, id] of [['e2e-tests','e2e-non-linux'], ['integration-tests','integration-non-linux']]) {
    const steps = pr.jobs[job].steps;
    const run = steps.find(s => s.id === id);
    assert.equal(run.if, "runner.os != 'Linux'");
    assert.equal(run.env.VSCODE_TEST_DIAGNOSTICS_DIR, '${{ runner.temp }}/vscode-test-diagnostics');
    const upload = steps.find(s => s.if === '${{ failure() && steps.' + id + ".outcome == 'failure' }}");
    assert.ok(upload);
    assert.equal(upload.with.path, run.env.VSCODE_TEST_DIAGNOSTICS_DIR);
    assert.equal(upload.with['retention-days'], 7);
    assert.equal(upload.with['if-no-files-found'], 'error');
    console.log(PASS ${job}: non-Linux diagnostic capture and failure-only upload wiring);
    }
    console.log('PASS English submission workflow pin and read-only caller token');
    console.log('LIMITATION: parsed local YAML only; external actions/templates and live services were not executed.');
    JS
  • ✅ Passed | VSCE generated localization file selection | node <<'JS'
    const fs = require('node:fs');
    const assert = require('node:assert/strict');
    const {execFileSync} = require('node:child_process');
    const manifest = 'package.nls.de.json';
    const runtime = 'l10n/bundle.l10n.de.json';
    assert.ok(!fs.existsSync(manifest) && !fs.existsSync(runtime), 'fixture paths must not overwrite existing translations');
    const hadDir = fs.existsSync('l10n');
    try {
    fs.mkdirSync('l10n', {recursive:true});
    fs.writeFileSync(manifest, JSON.stringify({description:'Deutsche Testübersetzung'}));
    fs.writeFileSync(runtime, JSON.stringify({'Test string':'Testzeichenfolge'}));
    const files = execFileSync(process.execPath, ['./node_modules/@vscode/vsce/vsce', 'ls', '--no-dependencies'], {encoding:'utf8'}).split(/\r?\n/);
    assert.ok(files.includes(manifest), 'generated manifest must be selected by VSCE');
    assert.ok(files.includes(runtime), 'generated runtime table must be selected by VSCE');
    assert.ok(files.includes('package.nls.json'), 'English manifest must remain selected');
    console.log('PASS VSCE selects package.nls.json, generated package.nls.de.json and l10n/bundle.l10n.de.json despite localization ignore patterns');
    console.log('LIMITATION: fixture file selection only; no translated drop import or VSIX publication performed.');
    } finally {
    fs.rmSync(manifest, {force:true});
    fs.rmSync(runtime, {force:true});
    if (!hadDir) fs.rmdirSync('l10n');
    }
    JS
⚠️ Shared localization submission, release import and readiness validation diagnostic output
Only caller YAML was checked locally. The referenced shared action and Azure templates were not executed or expanded, and no real 14-locale handback was imported. External resources and services are unavailable in the network-isolated verification container.
⚠️ Non-Linux E2E and integration failure diagnostics diagnostic output
Discovery reported Linux. Both changed diagnostic-enabled test steps require runner.os != 'Linux'; Windows/macOS collection and artifact upload were not exercised. Their YAML wiring passed a separate static check.
❌ Dependency and test discovery diagnostic output
AUTOMATION_SANDBOX_PROFILE=typescript
Linux
Node v22.21.1; npm 10.9.4
package.json declares mocha ^10.8.2 and typescript ^5.1.3.
node_modules false
node_modules/mocha/bin/mocha.js false
node_modules/typescript/bin/tsc false
out/test/features/envManagers.unit.test.js false
fatal: not a git repository (or any parent up to mount point /)
[container exit=128]
Dependency gap subsequently resolved with npm ci --offline; compilation and targeted tests then passed. Git metadata remained unavailable.

@rchiodo Rich Chiodo (rchiodo) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@rchiodo Rich Chiodo (rchiodo) added the review-auto:approved Automated review: no blocking findings (approval posted). label Nov 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member Author

Investigated the failing Windows/Python 3.15 E2E job:

  • Can trigger environment refresh fails with RpcTimeoutError: Request 'refresh' timed out after 30000ms.
  • Discovers at least one environment then exceeds its 90000ms test timeout.
  • Uploaded Python Environments diagnostics show PET server refreshes repeatedly hitting their 30-second limit and restarting. The JSON CLI fallback eventually succeeds after approximately 48 seconds; logs show slow Conda discovery under C:\Miniconda, including missing package metadata and fallback probing.
  • The remaining 11 E2E matrix jobs passed, including Windows/Python 3.10, 3.12, and 3.14.
  • Both src/managers/common/nativePythonFinder.ts and src/test/e2e/environmentDiscovery.e2e.test.ts match current main. This failure is in runtime discovery, not localization extraction/import. A baseline run on the same runner image has not been performed, so this is not being labeled conclusively pre-existing or a harmless flake.

No tests were skipped, timeouts increased, or production discovery behavior changed. The underlying reason for slow Conda discovery still needs investigation; this is separate from the PR's current merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debt Code quality issues review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants