Complete optional Python project modernization

[tylerlong]

Motivation

After the compatibility, reproducibility, CI, and documentation bugs are fixed, several optional modernization opportunities will remain. They can improve maintainability and release security, but they are not required to restore currently promised behavior and should not be mixed into the bug fixes.

Candidate scope

• Evaluate retaining, upgrading, replacing, or removing observable; version 1.x changes behavior used by the SDK and is a compatibility migration rather than a routine version bump.
• Upgrade websockets, documentation, development, coverage, and build dependencies with focused compatibility checks.
• Upgrade GitHub Actions and choose an explicit third-party action pinning policy.
• Upgrade or replace the GitHub Pages deployment action.
• Remove superseded setup.py, setup.cfg, MANIFEST.in, and requirements files only after artifact and workflow parity is established.
• Reconsider whether tests should ship in distributions and whether demos should be included, without silently changing artifact contents.
• Add useful optional project metadata.
• Evaluate PyPI Trusted Publishing and uv publish while preserving or explicitly redesigning tag-driven version and release behavior.
• Consider optional credentialed live compatibility smoke tests outside deterministic unit CI.

Acceptance criteria

• Each proposed dependency or tooling upgrade has a documented compatibility reason and focused verification.
• Observable compatibility is resolved deliberately rather than treated as an automatic version bump.
• Any packaging-file or artifact-content cleanup occurs only after parity is demonstrated and the intended change is explicit.
• Any release-authentication or publishing change has a non-uploading dry run and documented rollback path.
• Deterministic unit CI remains independent of live credentials.
• The enhancement is split into independently deliverable child tickets before implementation begins.