Extract shared docs.rs build library and add a local build CLI

[syphar]

Motivation

• Give crate authors a way to run the docs.rs build pipeline in their own CI, catching documentation failures before publishing a release.
• To support this without duplicating the production builder, extract the build logic into a shared library. Separating execution from storage, database updates, and queue handling makes individual build steps and error policies easier to test.
• Use that separation to expand coverage of build failures, retries, artifact retention, and publication behavior that previously had little or no testing.

Summary

Extract the shared docs.rs build pipeline into docs_rs_rustwide, use it in the production builder, and introduce docs_rs_build for local and CI builds.

The main goal is the shared library: separate building documentation from database updates, storage, and queue handling. There are plenty of further refactoring opportunities, but this PR focuses on that separation and the
changes needed to use it.

• docs_rs_build is an initial prerelease. The intention is to distribute it, try it in libraries, and invite crate authors to test it before settling the interface. GitHub Actions–generated binary releases are likely the initial distribution method; crates.io publication is not decided.
• Remove local-crate support from the production builder. Local builds move to the standalone CLI, simplifying docs_rs_builder.
• Make build-limit types usable without a database. Database-specific functionality in docs_rs_build_limits is feature-gated.
• Reuse the existing fixtures. Fixture crates are primarily moved into the library’s tests, with a few small adjustments rather than a new fixture suite.
• Expand test coverage. Library tests cover build steps, artifacts, lifecycle, and retry policies. Production-builder tests cover previously untested publication failures, persisted results, and queue-reattempt
  decisions.
• Expose richer results. The library captures per-step status, duration, and logs, plus sandbox statistics. The production builder does not yet persist all this information; it continues publishing HTML and JSON build logs.

Design

BuildEnvironment manages the workspace, toolchain, sandbox configuration, and maintenance. Callers can build a complete release or invoke individual steps with their own ordering and error handling.

Step results preserve diagnostics on success and failure. HTML and JSON outputs are retained separately so subsequent steps cannot overwrite them. Fetching sources is a distinct lifecycle phase, allowing the production builder to archive them before build preparation can fail.

Publication remains in the production builder, separate from execution.

Behavior changes

• Toolchain update checks run once per hour, rather than before every build. Workspace refresh remains daily. Essential-file publication still checks the last successfully published compiler version.
• Default-target HTML preparation or command failures can trigger one retry with a regenerated lockfile. Regeneration failures retain the original build failure and regeneration diagnostics without requesting a queue reattempt.
• JSON publication happens after the release lifecycle completes. Late cleanup failures prevent documentation publication and request a reattempt.
• JSON build-log upload failures now request a reattempt, matching HTML log failures. JSON artifact parsing and upload failures remain nonfatal.
• Compiler metrics collection failures are nonfatal.
• Complete release builds collect coverage only for the default target.
• Build-std dependencies are fetched when command arguments request them, replacing the unconditional release-wide fetch.
• An exclusive workspace lock prevents concurrent users from interfering with builds or retained artifacts.

The CLI automatically packages local crates, reports results and timings in a table, and prints retained artifact paths. Auxiliary failures are nonfatal by default; --strict makes them affect the exit status.

Not yet

• Parallel builds within one workspace: Rustwide supports this, but the new library does not expose it yet.
• Binary or crates.io release automation and an installer. Git installation is documented temporarily.
• A reusable GitHub Action that handles setup and Docker-image caching.
• A README example for using a locally built image to test new system dependencies.
• Persisting the additional statuses, logs, and timings now available from the library.

[syphar]

@GuillaumeGomez I think this is ready for first review.

Sorry about the size :)

Happy to explain / extend anything needed.