diff --git a/.agents/skills/polylith/README.md b/.agents/skills/polylith/README.md index 438db3da..98de999d 100644 --- a/.agents/skills/polylith/README.md +++ b/.agents/skills/polylith/README.md @@ -3,7 +3,14 @@ > **Note for contributors:** this README is a **human reference**. The agent loads each `*/SKILL.md` independently via the skill loader; this file is **not** auto-loaded with any skill. Anything an agent must know to act has to live in the relevant `SKILL.md` itself, not here. -## Available Skills +## Skill loading model + +Two kinds of skill live under this directory; the distinction matters when picking an entry point: + +- **Atomic skills (`polylith-*`).** Each maps to one `poly` CLI command (or one focused concept). Safe to load in isolation; individually composable. These cover everyday Polylith workflows — creating bricks, syncing, checking, inspecting, and so on. +- **Orchestrated skill set (`migrate-project/migrate-*`).** A multi-phase workflow with shared state (`migration//state.md`) and a git safety net. **Never load an individual `migrate-*` skill directly** — always load `migrate-orchestrator` and let it drive the phases in order. See [`migrate-project/README.md`](./migrate-project/README.md). This is an advanced, explicit-opt-in workflow used for migrating a **non-Polylith** project into a Polylith workspace, **not** part of daily Polylith use. + +## Available Skills (daily Polylith workflows) | Skill | Command | Purpose | |---------------------------|--------------------|----------------------------------------------------------------------------------------------------------| @@ -15,8 +22,13 @@ | [Sync](./polylith-sync/SKILL.md) | `poly sync` | Update each project's brick list to match actual imports. | | [Workspace Inspection](./polylith-workspace-inspection/SKILL.md) | `poly info` | Show brick × project usage (which projects use which bricks). | | [Dependency Visualization](./polylith-dependency-visualization/SKILL.md) | `poly deps` | Show brick × brick dependencies and interface compliance. | +| [Dependency Management](./polylith-dependency-management/SKILL.md) | — | Add or manage third-party libraries for a brick or project. | | [Testing](./polylith-testing/SKILL.md) | `poly test diff` | List bricks/projects affected by **test-code** changes since a tag. | | [Diff](./polylith-diff/SKILL.md) | `poly diff` | List bricks whose **implementation** changed since a tag. | | [Check](./polylith-check/SKILL.md) | `poly check` | Validate the workspace (CI gate; exits 1 on failure). | | [Libs](./polylith-libs/SKILL.md) | `poly libs` | Inspect third-party libraries per project. | -| [Concepts](./polylith-concepts/SKILL.md) | — | Provides foundational knowledge about Polylith architecture and terminology. | +| [Concepts](./polylith-concepts/SKILL.md) | — | Foundational knowledge about Polylith architecture and terminology. | + +## Advanced workflow + +For migrating an existing **non-Polylith** Python project into a Polylith workspace, see [`migrate-project/README.md`](./migrate-project/README.md). This is a destructive, multi-phase, explicit-opt-in workflow — start with the `migrate-orchestrator` skill, not any individual `migrate-*` sub-skill. diff --git a/.agents/skills/polylith/migrate-project/README.md b/.agents/skills/polylith/migrate-project/README.md new file mode 100644 index 00000000..ac771520 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/README.md @@ -0,0 +1,111 @@ +# Project Migration Skills + +> **Note for contributors:** this README is a **human reference** and a skill **index**. Agents load each `migrate-*/SKILL.md` independently via the skill loader; this README is **not** auto-loaded with any skill. Anything an agent must know to act has to live in the relevant `SKILL.md` itself, not here. + +This directory contains skills for migrating **non-Polylith Python projects** into a Polylith workspace. They cooperate via two artifacts under `migration//`: + +- `state.md` — a flat `KEY=value` file. Canonical schema lives in [`migrate-discover/SKILL.md`](./migrate-discover/SKILL.md). +- `manifest.md` — a human-readable structural inventory of the source project. + +--- + +## ⚠️ When to use these skills + +**Only** when: +1. A human has **explicitly instructed** to migrate a specific project (e.g., "migrate `projects/my-app` to Polylith"). +2. The target project lives under `projects//` of this Polylith workspace. +3. The goal is to refactor the project into Polylith bricks (bases and components). + +**Do not use** for: +- Automated or unattended migrations. +- Projects that are already structured as Polylith bricks. +- Daily Polylith development tasks — for those, see the sibling `polylith-*` skills. + +--- + +## How to invoke + +Load the orchestrator and let it drive the rest: + +``` +Load the `migrate-orchestrator` skill and migrate `projects/`. +``` + +The orchestrator will: +1. Ask the user for explicit confirmation. +2. Establish a git safety net (dedicated branch + per-phase commits). +3. Load and execute each phase skill in order, verifying after each. + +> **Why every sub-skill says "do not load directly".** Each `migrate-` skill's `description:` starts with a redirect to `migrate-orchestrator`. This is intentional: it makes the orchestrator the only valid entry point regardless of which sub-skill the agent's fuzzy-match initially favours. The sub-skills depend on state (`migration//state.md`) and a git safety net that only the orchestrator sets up. + +--- + +## Downstream installation + +When this skill set is installed into another Polylith workspace (e.g. via a skills package), the in-skill `[ENTRY POINT]` / `[Internal sub-skill of migrate-orchestrator …]` markers in each `description:` are the primary routing signal — they ship with the package. + +For an extra-strong signal, downstream consumers should add the following snippet to their own repo-level `AGENTS.md` (or equivalent agent-routing file). It is **not** required — the in-skill markers are usually sufficient — but it removes any ambiguity for agents that read `AGENTS.md` before scanning skill descriptions. + +```markdown +## Polylith migration instructions + +When the user asks to migrate a non-Polylith Python project to Polylith +(e.g. "migrate `projects/` to Polylith"), load the +`migrate-orchestrator` skill first and let it drive the workflow. + +Never load `migrate-discover`, `migrate-extract-to-base`, or any other +`migrate-*` sub-skill directly — they are phases the orchestrator +invokes in order, with per-phase verification and git checkpoints +between them. +``` + +--- + +## Workflow at a glance + +The **authoritative** phase order, numbering, and dependencies live in +[`migrate-orchestrator/SKILL.md`](./migrate-orchestrator/SKILL.md) — it is the +**single source of truth**. This README intentionally does **not** repeat the +table, to avoid drift (sub-skills must not hardcode phase numbers either; they +reference `` from the orchestrator table). + +At a glance, the flow is: + +> **discover → analyze-imports** (choose `shim`/`shimless`) **→ extract-to-base → +> update-imports →** *[optional shim sub-track, only when `SHIM_STRATEGY=shim`]* **→ +> prepare-project → verify-stability → isolate-base-and-big-component → +> split-big-component → extract-standalone-modules → +> isolate-shared-and-project-logic → distribute-wiring → split-component-internals → +> refactor-tests → definition-of-done** + +### Optional skills (triggered during `migrate-discover`) + +| Skill | Purpose | Trigger / dependency | +|-------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------| +| [`migrate-convert-linter`](./migrate-convert-linter/SKILL.md) | Align the project's linter/formatter with the **workspace's** configured tool. | `CONVERT_LINTER=yes` in `state.md`. Runs between `migrate-discover` and `migrate-analyze-imports`. | +| [`migrate-convert-type-checker`](./migrate-convert-type-checker/SKILL.md) | Align the project's type checker with the **workspace's** configured tool. | `CONVERT_TYPE_CHECKER=yes` in `state.md`. Runs between `migrate-discover` and `migrate-analyze-imports`. | +| [`migrate-convert-package-manager`](./migrate-convert-package-manager/SKILL.md) | Convert the project's `pyproject.toml` to uv workspaces. **Opinionated about uv** — only run when the workspace itself uses uv. | `CONVERT_PACKAGE_MANAGER=yes` in `state.md`. Runs between `migrate-discover` and `migrate-analyze-imports`. | +| [`migrate-dedupe`](./migrate-dedupe/SKILL.md) | Identify and apply controlled deduplication discovered during refactoring. | User approval. Runs after `migrate-split-big-component` or `migrate-extract-standalone-modules`. | + +--- + +## Scope of the four "splitting" skills + +These skills overlap in vocabulary but address different scopes. Use this matrix to decide which one applies: + +| Skill | Scope | Trigger | +|---------------------------------------------|------------------------------------------------|---------------------------------------------------------------------------| +| `migrate-split-big-component` | Within one project; component → multiple components. | The temporary big component (from `migrate-isolate-base-and-big-component`) is too large. | +| `migrate-extract-standalone-modules` | Within one project; pulls foundational modules out of the residual. | Residual still contains `consts.py`/`exceptions.py`/`models.py`. | +| `migrate-split-component-internals` | Within one already-extracted shared component; `core.py` → multiple files. | A component's `core.py` mixes multiple domains internally. | +| `migrate-isolate-shared-and-project-logic` | Cross-project; separate shared vs project-specific in components used by ≥ 2 projects. | Migrating a 2nd+ project that overlaps with an already-extracted one. | + +> 💡 In a fresh migration of a single project, you usually run `migrate-split-big-component` → `migrate-extract-standalone-modules` → `migrate-split-component-internals`, and skip `migrate-isolate-shared-and-project-logic` until a second project is migrated. + +--- + +## Files in this directory + +- `README.md` — this file (human reference + index). +- `migrate-orchestrator/SKILL.md` — entry point; defines the phase order and the git safety net. +- `migrate-/SKILL.md` — one per phase referenced in the orchestrator table. diff --git a/.agents/skills/polylith/migrate-project/migrate-analyze-imports/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-analyze-imports/SKILL.md new file mode 100644 index 00000000..c001a3e2 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-analyze-imports/SKILL.md @@ -0,0 +1,99 @@ +--- +name: migrate-analyze-imports +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Analyze the project's import graph to find how the original namespace is referenced, choose the namespace-rewrite strategy (shim vs shimless), detect circular imports, and list symbols exported by the original namespace." +--- + +# Skill: migrate-analyze-imports + +## Goal +Analyze the project's import graph to: +1. Find **every** reference to the original namespace (in all three forms — see below). +2. Choose the namespace-rewrite strategy: `SHIM_STRATEGY=shim|shimless`. +3. List symbols exported by the original namespace's `__init__.py`. +4. Detect potential circular imports. + +This drives the namespace rewrite (phase 4) and, when `SHIM_STRATEGY=shim`, the +shim sub-track (phase 4b). See the `migrate-orchestrator` table for phase numbers. + +## Inputs +- Project name (from `migration//state.md`) +- Original namespace `ORIG_TOP_NS` (from `migration//state.md`) + +## The three reference forms (cover all of them) +A namespace rewrite is **incomplete** unless it covers every form below. A naive +"replace `from .` " misses forms 2 and 3: + +1. **Dotted import** — `from ${ORIG_TOP_NS}. import …`, `import ${ORIG_TOP_NS}.`. +2. **Bare submodule import** — `from ${ORIG_TOP_NS} import ` — single, multi-name + (`a, b, c`), and **mixed** lines where only some names move. Easy to miss: there is + no dot after the namespace. +3. **Quoted string module paths** — `mock.patch("${ORIG_TOP_NS}.x.Y")`, logging + dict-config `"${ORIG_TOP_NS}.logging.HealthFilter"`, `importlib` / `getattr` + targets. Not import statements, but they must be rewritten too. + +> ⚠ Do **not** rewrite unquoted local variables that merely share a name with the +> namespace (e.g. a FastAPI `app = FastAPI()` instance's `app.include_router(...)`). +> Target import statements and quoted module paths only. + +## Steps + +### 1. Identify references to the original namespace +1. Search for all three forms above across **all** `.py` files in the project (and + `pyproject.toml` / config files for string paths). +2. Record the paths and statements in `migration/${PROJECT}/import_analysis.md`, + grouped as: **internal** (inside `${ORIG_TOP_NS}/`), **external consumers** + (entrypoints, `alembic`, scripts), and **tests**. The external + test groups are + what the strategy decision below hinges on. + +### 2. List symbols exported by the original namespace +1. Inspect `${ORIG_TOP_NS}/__init__.py` (typically `projects/${PROJECT}/src/${ORIG_TOP_NS}/__init__.py` + or `projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py`) and list public symbols + (not starting with `_`). +2. Record them, and **note whether `__init__.py` is effectively empty** (docstring only). + +### 3. Decide the rewrite strategy (`SHIM_STRATEGY`) +Choose based on the findings and record it in `state.md`: + +- **`shimless`** — when imports are predominantly **submodule-qualified** + (`from ${ORIG_TOP_NS}. import …` / `from ${ORIG_TOP_NS} import `) and + `${ORIG_TOP_NS}/__init__.py` exports little or nothing. A single top-level + re-export shim would resolve **none** of those submodule paths, and a full package + shim mirroring every module is high-effort/high-risk. Phase 4 rewrites **all** + references (internal + external + tests) directly to the new namespace, and the + **phase 4b sub-track is skipped**. +- **`shim`** — when consumers import top-level symbols (`from ${ORIG_TOP_NS} import X`) + that a single `${ORIG_TOP_NS}/__init__.py` re-export can satisfy, and you want to + defer rewriting external consumers. Run the phase 4b sub-track after phase 4. +- **When in doubt, prefer `shimless`** — it leaves no transitional shim to remove + later (the definition-of-done forbids undocumented shims), at the cost of a wider + but mechanical rewrite. Confirm with the user if the consumer surface is large. + +Record: +``` +SHIM_STRATEGY= +``` + +### 4. Detect potential circular imports +1. Inspect the import graph for cycles — especially base ↔ shim once a shim is in + place (`shim` strategy only). +2. Record any chains in `import_analysis.md`. A pure namespace rename (shimless) + preserves the original graph, so cycles there usually mean the project already had + them. + +## Output +- `migration//import_analysis.md` with: references by form & group, + exported symbols, the chosen strategy + rationale, and circular-import chains (if any). +- `SHIM_STRATEGY` set in `migration//state.md`. + +## Verify +1. `migration//import_analysis.md` exists and is not empty. +2. It records all three reference forms, the exported symbols, and any cycles. +3. `SHIM_STRATEGY` is set in `state.md` (`shim` or `shimless`) with a recorded rationale. + +## Commit +```bash +git add migration/${PROJECT}/import_analysis.md migration/${PROJECT}/state.md +git commit -m "migrate(${PROJECT}): phase — analyze-imports" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single +> source of truth) — do not hardcode it. diff --git a/.agents/skills/polylith/migrate-project/migrate-automate-import-updates/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-automate-import-updates/SKILL.md new file mode 100644 index 00000000..a244e4cb --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-automate-import-updates/SKILL.md @@ -0,0 +1,76 @@ +--- +name: migrate-automate-import-updates +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Update imports in the new base location to reference the new namespace instead of the original namespace." +--- + +# Skill: migrate-automate-import-updates + +## Goal +Update imports to reference the new namespace instead of the original namespace, ensuring that the codebase remains functional after the namespace migration. + +> **Scope depends on `SHIM_STRATEGY`** (set by `migrate-analyze-imports`): +> - `shim`: rewrite imports **in the new base location only** — external consumers keep +> using the original namespace via the shim (phase 4b). +> - `shimless`: rewrite **every** reference to the original namespace — base internals, +> entrypoints / run-scripts, infra (e.g. `alembic/env.py`), and tests — so nothing +> imports the old namespace and no shim is needed. +> +> Cover **all three reference forms** (see `migrate-analyze-imports`): dotted +> `from . import …`, bare-submodule `from import ` (incl. multi-name +> and **mixed** lines), and quoted string paths (`mock.patch(".x.Y")`, logging +> dict-config). A naive "`from .`" replace silently misses the bare and quoted forms. +> +> 💡 **Script the rewrite for anything non-trivial.** A small text-in → text-out helper +> (a pure function: file text + an `{old → new}` mapping → rewritten text) handles all +> three forms — including splitting a mixed bare-import line into moved vs. not-moved +> names — far more reliably than ad-hoc edits across dozens of files. Keep it as plain +> functions (no classes), run it over `bases/`, `components/`, `test/`, and the project +> dir, then verify by grepping for any residual `` reference. The same helper is +> reusable for the component-extraction rewrites in `migrate-split-big-component`. + +## Inputs +- Project name (from `migration//state.md`) +- Original namespace (from `migration//state.md`) +- New namespace (from `migration//state.md`) +- Import analysis report (from `migration//import_analysis.md`) + +## Steps + +### 1. Identify files in the new base location importing from the original namespace +1. Review the import analysis report to identify files in the new base location (at `bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/`) that import from the original namespace. + +### 2. Update imports to reference the new namespace +1. For each file in the new base location that imports from the original namespace: + - Replace `from ${ORIG_TOP_NS} import ...` with `from ${TARGET_TOP_NS}.${INITIAL_BASE_NAME}. import ...` + - Replace `import ${ORIG_TOP_NS}` with `import ${TARGET_TOP_NS}.${INITIAL_BASE_NAME}` + +Example: +```python +# Before +from myproject.core import MyClass +import myproject.utils + +# After +from mynamespace.mybase.core import MyClass +import mynamespace.mybase.utils +``` + +### 3. Record updated files +1. Record all updated files in `migration/${PROJECT}/import_updates.md`. + +## Output +- Updated files in the new base location +- A report file: `migration//import_updates.md` listing all updated files + +## Verify +1. Confirm that all files listed in `migration//import_updates.md` have been updated. +2. Check that no files in the new base location import from the original namespace. +3. Verify that the codebase remains functional by running the test command from `migration//state.md`. + +## Commit +```bash +git add bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/ +git add migration/${PROJECT}/import_updates.md +git commit -m "migrate(${PROJECT}): phase — automate-import-updates" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single source of truth) — do not hardcode it. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-convert-linter/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-convert-linter/SKILL.md new file mode 100644 index 00000000..ee806610 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-convert-linter/SKILL.md @@ -0,0 +1,85 @@ +--- +name: migrate-convert-linter +description: "[Internal sub-skill of `migrate-orchestrator` (optional, runs only when opted in during `migrate-discover`). Do not load directly — load `migrate-orchestrator` first.] Align the project's linter and formatter with the **workspace's** configured tool (whatever it is — ruff, black+isort+flake8, pylint, etc.). Removes project-specific config and consolidates rules into the workspace root." +--- + +# Skill: migrate-convert-linter + +## Goal +Align the project with the **workspace's** linter and formatter. This skill is **not** opinionated about ruff — it reads the workspace's configured tool from the root `pyproject.toml` and aligns the project to that. + +## When to Skip +Skip this step if the project's `LINTER` and `FORMATTER` in `migration//state.md` already match the workspace's tools. + +## Inputs +From `migration//state.md`: +- `PROJECT_DIR` +- `LINTER`, `FORMATTER` +- `RUN_TEST_CMD` (optional: `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD`) + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 0. Identify the workspace's tool +Open the **workspace root** `pyproject.toml` and identify the configured linter and formatter using the same detection table as `migrate-discover` (`[tool.ruff]` → ruff, `[tool.black]` → black, `[tool.flake8]` or `.flake8` → flake8, etc.). Record what you found — every step below refers to "the workspace's linter/formatter" and means **this** tool, not necessarily ruff. + +If the workspace has no linter configured at all, stop and ask the user how to proceed (introduce ruff? skip linting alignment? abort the conversion?). + +### 1. Remove Project-Specific Configs and Dependencies +- Remove the following sections from the project's `pyproject.toml`: + - `[tool.black]`, `[tool.isort]`, `[tool.flake8]`, `[tool.pylint.*]`, `[tool.autopep8]`, `[tool.pycodestyle]` +- Remove standalone config files: `.flake8`, `.pylintrc`, `.isort.cfg`, and lint sections in `setup.cfg` or `tox.ini`. +- Remove old linter/formatter dependencies from the project's `pyproject.toml`: + - `flake8`, `flake8-*` plugins, `pylint`, `pylint-*` plugins, `black`, `isort`, `autopep8`, `pyflakes`, `pycodestyle`, `bandit` + +### 2. Assess Workspace Linting Config +- Review the workspace root's linting and formatting configuration. +- Identify any project-specific rules or ignores that differ from the workspace's standards. + +### 3. Merge Project-Specific Rules +- If the project has unique linting rules or ignores, merge them into the workspace root's linting configuration (e.g., `[tool.ruff]`). +- For conflicts (e.g., stricter rules in the project), ask the user to provide guidance on whether to: + - Adopt the project's rules in the workspace. + - Suppress the project's rules in favor of the workspace's. + - Defer the decision and document the conflict in `migration//state.md`. + +### 4. Run Workspace Linting and Formatting Tools +- Run the workspace's linting tool to assess violations: + - **Few new violations**: Fix them now. + - **Many new violations**: Ask the user whether to fix, suppress, or defer. +- Run the workspace's formatting tool to reformat the code. + +### 5. Update `state.md` +- Set `LINTER` and `FORMATTER` to the workspace's tool(s). +- Update `RUN_LINT_CMD` to use the workspace's linting and formatting commands. + +## Verify +- The workspace's linting tool passes (or remaining violations are user-approved). +- The workspace's formatting tool passes. +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_TYPECHECK_CMD` succeeds. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| The workspace's linter surfaces hundreds of violations the project's old linter didn't catch | Stricter ruleset; not a "fix everything now" situation. | Ask the user: fix now, suppress via `[tool.]` ignores in the project subsection, or defer in a follow-up branch. Do **not** auto-fix silently — record the decision in `state.md`. | +| Project-specific lint rules are stricter than the workspace's standard | Project had higher discipline; lowering it would regress quality. | Add the project's rule as a per-path override in the workspace's lint config (most linters support per-directory rule overrides). Do not weaken the rule globally. | +| Old linter config file (`.flake8`, `.pylintrc`, etc.) lingers and confuses developers' local editors | Step 1 missed a config file. | Delete it; mention in the commit message so reviewers update their editor configs. | + +## Done When +- No project-specific linter/formatter config files or dependencies remain. +- Project-specific linting rules are merged into the workspace root's configuration. +- `LINTER` and `FORMATTER` in `migration//state.md` match the workspace's tools. +- Tests pass via the workspace's tooling. + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase optional — convert-linter" +``` + +Substitute `` from `state.md`. This is an **optional** skill off the numbered main line, so the commit uses the literal `phase optional` label (no ``). Do not proceed without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-convert-package-manager/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-convert-package-manager/SKILL.md new file mode 100644 index 00000000..38bc35a1 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-convert-package-manager/SKILL.md @@ -0,0 +1,104 @@ +--- +name: migrate-convert-package-manager +description: "[Internal sub-skill of `migrate-orchestrator` (optional, runs only when opted in during `migrate-discover`). Do not load directly — load `migrate-orchestrator` first.] Convert the project's `pyproject.toml` to PEP 621/uv-workspaces format and register it as a uv-workspace member. **Opinionated about uv** — only applies when the workspace itself uses uv. Skip otherwise." +--- + +# Skill: migrate-convert-package-manager + +> ⚠ **Opinionation gate.** This skill is **explicitly opinionated about uv**. It does not generalize to Poetry, PDM, or Hatch workspaces. If your Polylith workspace uses one of those, **do not run this skill** — align the project to the workspace's manager via a manual step instead, and leave `CONVERT_PACKAGE_MANAGER=no` in `state.md`. + +## Goal +Convert the project's `pyproject.toml` to PEP 621/uv format and register it as a uv-workspace member. This makes the project share the workspace's single lock file and virtual environment, preventing version skew. + +## When to Skip +Skip this skill if **any** of the following holds: +- `PACKAGE_MANAGER=uv` already in `migration//state.md` (nothing to convert). +- The workspace root does **not** use uv (this skill does not apply — see the opinionation gate above). +- `CONVERT_PACKAGE_MANAGER=no` in `state.md` (user opted out during `migrate-discover`). + +### Verify the workspace uses uv before proceeding +Open the **workspace root** `pyproject.toml` and look for `[tool.uv.workspace]` and/or a sibling `uv.lock`. If neither is present, **stop**: this skill does not apply. + +## Inputs +From `migration//state.md`: +- `PROJECT_DIR` +- `PACKAGE_MANAGER` +- `RUN_TEST_CMD` (optional: `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD`) + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Ask for User Approval +- Ask the user if they want to convert the project's `pyproject.toml` to PEP 621/uv format. +- Record their choice in `state.md`: + ```text + CONVERT_PACKAGE_MANAGER= + ``` + +### 2. Rewrite `pyproject.toml` to PEP 621/uv Format +- Move `[tool.poetry.dependencies]` to `[project] dependencies`. Keep only **runtime** (non-dev, non-test) dependencies in the project, listed **without version constraints**. +- Remove Poetry-specific sections: `[tool.poetry]`, `[tool.poetry.group.*]`, `[[tool.poetry.source]]`, and `[build-system]` with `poetry-core`. +- Add a `[build-system]` with `hatchling` (or the workspace's build backend). +- Preserve `[tool.*]` sections for other tools (e.g., pytest, ruff, mypy). +- Add `[tool.uv]` only if project-level uv configuration is needed. + +### 3. Register as a Workspace Member +- Add the project path to the workspace root `pyproject.toml` under `[tool.uv.workspace] members`. + Example: + ```toml + members = ["projects/example-service-b"] + ``` + +### 4. Consolidate Dependencies +- Add all **third-party runtime dependencies with version constraints** to the workspace root `pyproject.toml` `[project] dependencies`. +- Move all dev/test/tooling dependencies to the workspace root `[dependency-groups]` (e.g., `dev = [...]`, `test = [...]`). +- Ensure the project's `pyproject.toml` lists runtime dependencies **without version numbers**. + +### 5. Lock and Sync +- Run `uv lock` from the workspace root to regenerate `uv.lock` with the new member. +- Run `uv sync` to install all dependencies into the shared `.venv`. +- Resolve any version conflicts that arise during `uv lock`. + +### 6. Delete Old Lock Files +- Remove `poetry.lock`, `Pipfile.lock`, and generated `requirements*.txt` from the project directory. + +### 7. Update Verification Commands +- Replace any `poetry run`, `pipenv run`, or bare commands with `uv run` equivalents in `migration//state.md`. + Example: + ```text + RUN_TEST_CMD=uv run pytest + RUN_LINT_CMD=uv run ruff check + ``` +- Update `PACKAGE_MANAGER=uv` in `state.md`. + +## Verify +- Run the updated `RUN_TEST_CMD` and confirm the same pass/fail counts as before the conversion. +- If set, run `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD`. +- Ensure `uv lock` and `uv sync` succeed from the workspace root. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Workspace root does not use uv (no `[tool.uv.workspace]`, no `uv.lock`) | This skill does not apply — the opinionation gate at the top of this file rules it out. | Stop. Set `CONVERT_PACKAGE_MANAGER=no` in `state.md`. Align the project to the workspace's actual manager (Poetry/PDM/Hatch) via a manual step instead. | +| `uv lock` fails with "no version satisfies …" after adding the project as a workspace member | Project's old version constraints conflict with the workspace root's pins. | Relax the workspace root's range, or, if the project legitimately needs a different version, pin it explicitly in the project's `pyproject.toml`. As a last resort, exclude the project from the workspace and use a separate environment. | +| Project depends on a private/internal package that the workspace root doesn't know about | Private index or path-dependency not declared at the root. | Add the dependency (and its source — `[tool.uv.sources]` or `[[tool.uv.index]]`) to the workspace root `pyproject.toml`. | + +## Done When +- The project is listed as a workspace member in the root `pyproject.toml`. +- `uv lock` and `uv sync` succeed from the workspace root. +- Old lock files (`poetry.lock`, `Pipfile.lock`, generated `requirements*.txt`) are deleted from the project directory. +- `PACKAGE_MANAGER=uv` is recorded in `migration//state.md`. +- Verification commands in `state.md` use `uv run`. +- Tests pass via `uv run`. + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase optional — convert-package-manager" +``` + +Substitute `` from `state.md`. This is an **optional** skill off the numbered main line, so the commit uses the literal `phase optional` label (no ``). Do not proceed without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-convert-type-checker/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-convert-type-checker/SKILL.md new file mode 100644 index 00000000..bf1b6123 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-convert-type-checker/SKILL.md @@ -0,0 +1,83 @@ +--- +name: migrate-convert-type-checker +description: "[Internal sub-skill of `migrate-orchestrator` (optional, runs only when opted in during `migrate-discover`). Do not load directly — load `migrate-orchestrator` first.] Align the project's type checker with the **workspace's** configured tool (whatever it is — mypy, pyright, ty, etc.). Removes project-specific config and consolidates settings into the workspace root." +--- + +# Skill: migrate-convert-type-checker + +## Goal +Align the project with the **workspace's** type checker. This skill is **not** opinionated about ty — it reads the workspace's configured tool from the root `pyproject.toml` and aligns the project to that. + +## When to Skip +Skip this step if the project's `TYPE_CHECKER` in `migration//state.md` already matches the workspace's type-checking tool. + +## Inputs +From `migration//state.md`: +- `PROJECT_DIR` +- `TYPE_CHECKER` +- `RUN_TEST_CMD` (optional: `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD`) + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 0. Identify the workspace's tool +Open the **workspace root** `pyproject.toml` and identify the configured type checker using the same detection table as `migrate-discover` (`[tool.mypy]` or `mypy.ini` → mypy, `[tool.pyright]` or `pyrightconfig.json` → pyright, `[tool.ty]` → ty). Record what you found — every step below refers to "the workspace's type checker" and means **this** tool, not necessarily ty. + +If the workspace has no type checker configured at all, stop and ask the user how to proceed (introduce one? skip type-check alignment? abort the conversion?). + +### 1. Remove Old Configs and Dependencies +- Remove the following sections from `pyproject.toml`: + - `[tool.mypy]`, `[[tool.mypy.overrides]]`, `[mypy-*]`, `[tool.pyright]`, `[tool.pytype]` +- Remove standalone config files: `mypy.ini`, `.mypy.ini`, `pyrightconfig.json`. +- Remove old type checker dependencies from `pyproject.toml`: + - `mypy`, `mypy-extensions`, `types-*` stub packages, `pyright`, `pytype`, `sqlalchemy-stubs`, `django-stubs`. + +### 2. Clean Up Type Ignore Comments +- Remove `# type: ignore[]` comments that reference mypy-specific error codes. +- Leave comments that suppress real issues and document them in `state.md`. + +### 3. Adopt Workspace Type-Checking Config +- The workspace root `pyproject.toml` may define the type-checking configuration. The project inherits this config. +- Add project-specific overrides if needed. + +### 4. Run the Workspace's Type-Checking Tool +- Run the workspace's type-checking tool to assess errors: + - **Same or fewer errors**: No action needed. + - **New errors**: Ask the user whether to fix, suppress, or adjust the config. + - **Missing stub errors**: Silence with per-module ignores in the workspace's type-checking config. + +### 5. Update `state.md` +- Set `TYPE_CHECKER` to the workspace's type-checking tool. +- Update `RUN_TYPECHECK_CMD` to use the workspace's type-checking command. + +## Verify +- The workspace's type-checking tool runs cleanly (or remaining errors are user-approved). +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` succeeds. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| The workspace's checker surfaces type errors the old checker didn't | Stricter type rules expose real bugs that were always there. | Fix the bugs now where cheap (small handful). For systemic gaps, suppress per-module in the workspace's checker config and document the debt in `state.md`. Do **not** blanket-ignore at file scope. | +| Stub packages (`types-*`, `django-stubs`, `sqlalchemy-stubs`) were required by the old checker but the new one bundles its own | Dependency carryover. | Remove the stub packages from `pyproject.toml`. Re-run the workspace's checker to confirm it picks up its own stubs. | +| `# type: ignore[]` comments still reference the old checker's error codes | Step 2 missed some. | Strip the `[]` bracket. If the suppression is still needed, leave a bare `# type: ignore` **with a comment explaining why**; otherwise remove the suppression entirely. | + +## Done When +- No old type checker config files remain. +- Old type checker dependencies are removed from `pyproject.toml`. +- Stale `# type: ignore` comments referencing tool-specific codes are removed (or documented if intentionally kept). +- `TYPE_CHECKER` in `migration//state.md` matches the workspace's type-checking tool. +- The workspace's type-checking tool runs cleanly or known issues are documented in `state.md`. +- Tests pass via the workspace's tooling. + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase optional — convert-type-checker" +``` + +Substitute `` from `state.md`. This is an **optional** skill off the numbered main line, so the commit uses the literal `phase optional` label (no ``). Do not proceed without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-dedupe/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-dedupe/SKILL.md new file mode 100644 index 00000000..ee53aeda --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-dedupe/SKILL.md @@ -0,0 +1,118 @@ +--- +name: migrate-dedupe +description: "[Internal sub-skill of `migrate-orchestrator` (optional, runs only when opted in during `migrate-discover`). Do not load directly — load `migrate-orchestrator` first.] Identify and execute controlled deduplication of code during migration (if the user opts in)." +--- + +# Skill: migrate-dedupe + +> 📐 **Scope vs sibling skills.** This skill is **opportunistic deduplication** that may be triggered any time during refactoring when duplication candidates surface. It is **not** the canonical place for the structural decompositions: +> - For "split this big component into smaller ones", use `migrate-split-big-component` (it already includes a dedup-analysis subsection — usually sufficient on a first migration). +> - For "this component's `core.py` mixes domains", use `migrate-split-component-internals`. +> - For "two projects have overlapping code, split shared from project-specific", use `migrate-isolate-shared-and-project-logic`. +> +> Use `migrate-dedupe` when none of the above fits cleanly — e.g., duplication discovered across already-extracted components that don't map to a structural split. + +## Goal +Identify duplication candidates during the migration process and execute controlled deduplication for user-approved candidates. + +## When to Use +- After splitting the big component or extracting standalone modules. +- When potential duplication between components is suspected. + +## Classification + +Use this table when deciding whether a candidate is a real duplicate: + +| Class | Definition | Action | +|-------|------------|--------| +| **Identical** | Same logic, same control flow, only trivial differences (variable names, formatting, ordering of independent statements). | Extract into a shared component. Both call sites import from it. | +| **Similar** | Same purpose, slightly different behaviour (e.g., different default arguments, project-specific fields on an otherwise shared model). | Extract a **parameterized** shared component. Project-specific behaviour passes in as arguments or subclass hooks. Avoid forcing a one-size-fits-all signature. | +| **Coincidental** | Looks similar (same function name, same shape) but serves unrelated purposes. | Leave alone. Sharing here would couple two domains that should evolve independently. | + +## When to parameterize vs. keep separate + +- **Parameterize** when the core logic is identical and only data/config differs. +- **Keep separate** when control flow or structure diverges (different frameworks, different patterns) — forcing a shared abstraction here creates a brittle "shared core" that grows project-specific flags over time. +- **Extract shared base + per-project wrappers** when there's a significant shared core but non-trivial project-specific logic around it. + +### Worked example — logging + +Two projects each had their own `init_logging`. The core (structlog setup, base log levels, JSON formatter) was identical; the differences were: + +- Project A added loggers for `httpx`, `backoff`. +- Project B added a logger for `confluent_kafka_helpers`. + +The shared component exposed an `init(config, *, extra_loggers=None, cache_logger_on_first_use=False)` function. Each project's base calls `init` with its own `extra_loggers` dict. No coincidental coupling, no version skew, and adding a third project requires only its own dict — not a change to the shared component. + +## Shared-component naming + +When creating a shared component to deduplicate code, name it after the **domain or capability** it represents, never after how it's used. Good: `logging`, `kafka_client`, `merchant_serializer`. Bad: `shared_utils`, `common`, `helpers`, `misc`. Generic-named bricks attract more code over time and become the next thing that needs decomposing. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- Verification commands (`RUN_TEST_CMD`, `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD`). + +From `migration//manifest.md`: +- Module map of components. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Identify Duplication Candidates +- Use `directory_tree` and `grep` to scan for overlapping logic between components. +- Classify candidates by type: + - **Identical**: Code that is exactly the same. + - **Similar**: Code that serves the same purpose but with minor differences. + - **Coincidental**: Code that looks similar but serves unrelated purposes. + +### 2. Present Candidates to the User +- Provide a list of duplication candidates, including: + - Component names. + - File paths. + - Type of duplication (identical, similar, coincidental). + - Risk assessment (low, medium, high). +- Ask the user to approve or reject each candidate for deduplication. + +### 3. Execute Deduplication for Approved Candidates +- For each approved candidate: + - **Identical Code**: Extract the shared logic into a new component and update imports. + - **Similar Code**: Refactor to use shared logic or parameterize differences. + - **Coincidental Code**: Leave as-is. +- Update `pyproject.toml` to include any new components. +- Run `POLY_CMD_PREFIX sync` to synchronize the workspace. + +### 4. Verify Changes +- Run `RUN_TEST_CMD` to ensure no regressions. +- Run `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` if set. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. + +## Verify +- All tests pass (`RUN_TEST_CMD`). +- Linting and type-checking pass (if set). +- The workspace structure is valid (`POLY_CMD_PREFIX check`). + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Two pieces of code look identical but operate on different domains (e.g., both are `validate(...)` but one is for users, the other for transactions) | Coincidental similarity, not real duplication. | Classify as "coincidental"; leave both in place. Resist the urge to share. | +| The candidate shared component would pull in framework-specific dependencies (e.g., a "logging" shared brick that needs both `confluent_kafka_helpers` and `httpx`) | Wrong shared abstraction — you're sharing the union of two project surfaces. | Revert and parameterize instead: keep the shared core minimal and pass project-specific values as arguments. See the "Pattern: Parameterize the shared component" guidance in `migrate-split-big-component`. | +| Tests break after deduplication because `mock.patch("")` no longer hits anything | Patch strings reference the pre-dedup module path. | Update patch strings to the new shared module path. Validate by deliberately breaking the patched function and confirming the test fails. | + +## Done When +- Duplication candidates are identified and presented to the user. +- User-approved candidates are deduplicated. +- All tests and checks pass. +- The workspace structure is valid. + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase optional — dedupe" +``` + +Substitute `` from `state.md`. This is an **optional** skill off the numbered main line, so the commit uses the literal `phase optional` label (no ``). Do not proceed without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-definition-of-done/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-definition-of-done/SKILL.md new file mode 100644 index 00000000..dc3196b4 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-definition-of-done/SKILL.md @@ -0,0 +1,95 @@ +--- +name: migrate-definition-of-done +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Define the criteria for completing the migration process." +--- + +# Skill: migrate-definition-of-done + +## Done When + +### Structure +- Temporary migration base is gone or thin. +- Bases contain only entrypoints/wiring. +- All non-entrypoint code lives in components. + +### Source Project +- `projects//` contains only: + - Packaging config (`pyproject.toml`). + - Runner scripts and task runners (`Makefile`, `Justfile`). + - Project-specific config (e.g., `alembic.ini`). +- Project `pyproject.toml` references all required bricks. +- Brick names are meaningful and non-generic. +- Base names are project-prefixed to avoid collisions. + +### Tests +- Tests are moved from `projects//tests/` to workspace level (**required**). +- Unit-test layout — **one** of the following (see `migrate-refactor-tests`): + - **Per-brick (theme-aligned):** `loose` → `test/bases///` and `test/components///`; `tdd` → `bases//test///` and `components//test///`. **Or** + - **Workspace-level service dir** `test/_service/` — valid when shared test helpers can't cheaply become fixtures (the namespace-merge hazard makes a per-brick split unsafe otherwise). Record the chosen layout in `state.md`. +- Integration tests live in a shared location (e.g., `test/integration/` or `test/_service/integration/`). +- Shared fixtures live in a `conftest.py` (e.g. `test//conftest.py`, `test/conftest.py`, or the service dir's `conftest.py`). +- `RUN_TEST_CMD` points to the test root **and collects the same number of tests as the pre-migration baseline**. + +### Infrastructure +- Infrastructure folders are moved to `infra///`, **or** the move is **deferred with a documented rationale** in `state.md` when the deploy cannot be verified in the migration environment (see `migrate-prepare-project`). Either way, `alembic/` stays with `alembic.ini` in the project. + +### Interfaces +- Each component defines its public API via `__init__.py`. +- Bricks import each other via those APIs. + +### Linting and Type-Checking +- Linting/formatting and type-checking use the workspace's configured tool(s). +- `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` pass **if set**. They may be intentionally empty when the project's pre-migration baseline already failed these gates (recorded in `migrate-discover`); **pre-existing** violations are not the migration's responsibility — note them as a follow-up rather than blocking the migration. + +### Dependencies +- Workspace root `pyproject.toml` contains all third-party dependencies with version constraints. +- Project `pyproject.toml` lists runtime dependencies without version numbers. + +### Cleanup +- Migration artifacts (`migration//state.md`, `migration//manifest.md`, and any `migration/shims.md`) are either removed or kept under `migration//` for reference (user's choice). +- The migration branch (`GIT_BRANCH` from `state.md`) is ready to be merged or rebased into the main branch. Per-phase commits remain available for review/bisect. + +## End-to-end checks + +These checks go beyond per-phase verification and exercise the migrated project as a whole. **Both must pass.** + +### 1. Baseline test count restored + +Compare collected test count to the baseline recorded by `migrate-discover`: + +```bash + +``` + +The count must **equal** the baseline. A lower count means `pytest` discovery is misconfigured (see `migrate-refactor-tests` failure modes). A higher count means tests were inadvertently duplicated during the move. + +### 2. No undocumented shims remain + +Inspect `migration//shims.md` (if it exists): + +- **Empty or absent** → ✅ proceed. +- **Lists active shims** → either remove them now (rewrite imports to the new namespace and delete the shim modules) or schedule their removal as a follow-up PR and reference that PR in the migration branch description. Do **not** merge with undocumented shims. + +## Verify +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX info` to inspect the workspace and confirm all projects and bricks are correctly registered. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| `RUN_TEST_CMD` passes but collected test count is lower than the baseline from `migrate-discover` | `pytest` discovery is misconfigured after the test reorganisation. | Update `[tool.pytest.ini_options].testpaths` (or pass paths explicitly in `RUN_TEST_CMD`). Run `pytest --collect-only` and diff against baseline collection. See `migrate-refactor-tests` for details. | +| `poly check` is green and tests pass, but a base imports something that no longer exists at the expected path | Entrypoint wiring regression introduced while moving code into bases/components. | Verify each base's entrypoint module imports cleanly and that its wiring matches the original. Revisit `migrate-distribute-wiring`. | +| `migration/shims.md` still lists active shims | Shims from `migrate-extract-to-base` (namespace change) were never removed. | Either rewrite imports to the new namespace and delete the shims, or schedule shim removal as a follow-up PR and note it in the migration branch description. Do not merge with undocumented shims. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — definition-of-done" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-detect-circular-imports/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-detect-circular-imports/SKILL.md new file mode 100644 index 00000000..be86ef2b --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-detect-circular-imports/SKILL.md @@ -0,0 +1,48 @@ +--- +name: migrate-detect-circular-imports +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Detect and report circular imports introduced by the namespace migration." +--- + +# Skill: migrate-detect-circular-imports + +> ⛓ **Conditional phase (4b).** Runs **only when `SHIM_STRATEGY=shim`** (chosen in `migrate-analyze-imports`). On the **shimless** path this phase is **skipped** — a pure namespace rename introduces no base↔shim cycles. See the `migrate-orchestrator` workflow. + +## Goal +Detect and report circular imports introduced by the namespace migration, particularly between the new base location and the compatibility shim. + +## Inputs +- Project name (from `migration//state.md`) +- Original namespace (from `migration//state.md`) +- New namespace (from `migration//state.md`) +- Import updates report (from `migration//import_updates.md`) + +## Steps + +### 1. Analyze the import graph +1. Manually inspect the import graph to identify circular dependencies: + - Between the new base location (`bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/`) and the compatibility shim (`projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py`) + - Within the new base location itself + +### 2. Report circular imports +1. Record all circular import chains in `migration/${PROJECT}/circular_imports.md`. +2. For each circular import chain, document: + - The files involved (use full paths like `bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/module.py`) + - The import statements causing the circularity + - Suggested resolution steps + +## Output +- A report file: `migration//circular_imports.md` listing all circular import chains + +## Verify +1. Confirm that `migration//circular_imports.md` exists. +2. Verify that the report includes: + - A list of circular import chains + - The files involved in each chain + - Suggested resolution steps + +## Commit +```bash +git add migration/${PROJECT}/circular_imports.md +git commit -m "migrate(${PROJECT}): phase — detect-circular-imports" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single source of truth) — do not hardcode it. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-discover/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-discover/SKILL.md new file mode 100644 index 00000000..aa15ecfb --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-discover/SKILL.md @@ -0,0 +1,274 @@ +--- +name: migrate-discover +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Create `migration//state.md` and `migration//manifest.md` by inspecting the existing project under `projects//`." +--- + +# Skill: migrate-discover + +## Goal +Inspect the project and create two artifacts that drive every subsequent migration phase: +- `migration//state.md` — a flat `KEY=value` file (see schema below). +- `migration//manifest.md` — a human-readable structural inventory. + +> 💡 `` is the project subfolder name (e.g., `api` for `projects/api/`). Use that exact string everywhere — paths, filenames, branch names. + +## Canonical `state.md` schema + +All later skills read `state.md` as a flat `KEY=value` file. Use **exactly** this format — no markdown tables, no fenced TOML, no inline comments. One key per line. + +```ini +# migration//state.md +PROJECT_DIR=projects/ +ORIG_TOP_NS= +TARGET_TOP_NS= +INITIAL_BASE_NAME= +ALIAS= +GROUP= + +PACKAGE_MANAGER= +LINTER= +FORMATTER= +TYPE_CHECKER= +POLY_CMD_PREFIX= +BRICK_IMPORT_MECHANISM=.* bricks for import> + +SHIM_STRATEGY= + +CONVERT_LINTER= +CONVERT_TYPE_CHECKER= +CONVERT_PACKAGE_MANAGER= + +RUN_TEST_CMD= +RUN_LINT_CMD= +RUN_TYPECHECK_CMD= + +GIT_BRANCH= +GIT_BASE_SHA= +``` + +### Field reference + +| Key | Description | Source | +|-----|-------------|--------| +| `PROJECT_DIR` | Project subfolder path. | The orchestrator's ``. | +| `ORIG_TOP_NS` | Current top-level Python package name. | First non-`tests` directory under `projects//src/` or `projects//`. | +| `TARGET_TOP_NS` | Desired Polylith namespace. | `workspace.toml` `[tool.polylith].namespace`, or `ORIG_TOP_NS` if no workspace exists yet. | +| `INITIAL_BASE_NAME` | Name of the **single temporary base** used to hold all code during early phases. Becomes the **default base name** in `migrate-isolate-base-and-big-component`. Final migrations usually contain *several* bases — this is just the starting one. | Derived from `[project.name]`, confirmed by user. | +| `ALIAS` | Short alias shown in `poly info` / `poly deps` tables. Optional. | Derived, confirmed by user. | +| `GROUP` | Polylith project group. Optional. | Asked from user. | +| `PACKAGE_MANAGER` / `LINTER` / `FORMATTER` / `TYPE_CHECKER` | Detected tooling. | Detection table below. | +| `POLY_CMD_PREFIX` | Prefix every `poly …` command in later skills uses. | Derived from `PACKAGE_MANAGER`. | +| `BRICK_IMPORT_MECHANISM` | How the workspace makes `.*` bricks importable for tests/dev. `editable-root` (root project is editable-installed; e.g. Hatch `dev-mode-dirs`), `pytest-pythonpath` (root `[tool.pytest.ini_options].pythonpath = ["bases","components","development"]`), or `other`. | Step 4: confirmed/established below. | +| `SHIM_STRATEGY` | `shim` or `shimless` — chosen in phase 2 (`migrate-analyze-imports`). May be empty at discover. | `migrate-analyze-imports`. | +| `CONVERT_*` | Whether the user opted in to a tooling conversion. | Asked from user. | +| `RUN_TEST_CMD` etc. | The exact shell commands the migration verifies against after each phase. | Derived from project config, confirmed by user if ambiguous. | +| `GIT_BRANCH` / `GIT_BASE_SHA` | Set by the orchestrator's Phase 0; recorded here so later skills know where to roll back to. | Orchestrator. | + +### Deriving `INITIAL_BASE_NAME` and `ALIAS` from `[project.name]` + +| `[project.name]` | `INITIAL_BASE_NAME` (snake) | `ALIAS` (kebab) | +|-------------------------|-----------------------------|-----------------| +| `example-service-a` | `example_a` | `svc-a` | +| `order-management-api` | `order_management` | `order-mgmt` | +| `payment-worker` | `payment` | `payment` | + +### Validation rules + +When any later phase loads `state.md`, validate before proceeding: + +1. **File exists** at `migration//state.md`. +2. **Format**: every line is one of: blank, `# comment`, or `KEY=value`. No markdown tables, no fenced TOML, no inline comments after a value. +3. **Schema coverage**: every key from the schema above is present. A value may be empty (for optional keys), but the key line must exist. +4. **Enumerations**: `PACKAGE_MANAGER`, `LINTER`, `FORMATTER`, `TYPE_CHECKER`, `BRICK_IMPORT_MECHANISM`, `SHIM_STRATEGY` (when set), and the three `CONVERT_*` flags use only the documented values. +5. **Required non-empty**: `PROJECT_DIR`, `ORIG_TOP_NS`, `TARGET_TOP_NS`, `INITIAL_BASE_NAME`, `PACKAGE_MANAGER`, `POLY_CMD_PREFIX`, `BRICK_IMPORT_MECHANISM`, `RUN_TEST_CMD`, `GIT_BRANCH`, `GIT_BASE_SHA` must all be non-empty. (`SHIM_STRATEGY` may be empty until phase 2 sets it.) +6. **Consistency**: `POLY_CMD_PREFIX` matches `PACKAGE_MANAGER` per the mapping table. + +If validation fails, abort the phase, surface the offending line(s) to the user, and ask them to fix `state.md` before retrying. Never silently coerce values. + +## Steps + +> Run these in order. Every step writes to `state.md` or `manifest.md`. Do not skip the confirmation gates (steps 4 and 7). + +### 1. Record project metadata +Read `projects//pyproject.toml` (or `setup.cfg`/`setup.py`) and fill in `PROJECT_DIR`, `ORIG_TOP_NS`, `TARGET_TOP_NS`, and the **derived** `INITIAL_BASE_NAME`, `ALIAS` per the tables above. + +### 2. Detect tooling +Scan project config files and fill in `PACKAGE_MANAGER`, `LINTER`, `FORMATTER`, `TYPE_CHECKER`: + +| Tool | Detection criteria | +|------|--------------------| +| **Package Manager** | | +| Poetry | `poetry.lock` or `[tool.poetry]` in `pyproject.toml` | +| Pipenv | `Pipfile` or `Pipfile.lock` | +| Pip | `requirements.txt` (no lock file) | +| UV | `uv.lock` or `[tool.uv]` in `pyproject.toml` | +| Setuptools | `setup.py` or `setup.cfg` only | +| **Linter** | | +| Flake8 | `setup.cfg`, `tox.ini` `[flake8]` section, or `.flake8` | +| Pylint | `[tool.pylint]` or `.pylintrc` | +| Ruff | `[tool.ruff]` | +| **Formatter** | | +| Black | `[tool.black]` | +| Isort | `[tool.isort]` | +| Ruff | `[tool.ruff.format]` | +| **Type Checker** | | +| Mypy | `mypy.ini`, `.mypy.ini`, or `[tool.mypy]` | +| Pyright | `[tool.pyright]` or `pyrightconfig.json` | +| Ty | `[tool.ty]` | + +### 3. Derive `POLY_CMD_PREFIX` +Map `PACKAGE_MANAGER` to the command prefix: + +| `PACKAGE_MANAGER` | `POLY_CMD_PREFIX` | +|-------------------|-------------------| +| `poetry` | `poetry poly` | +| `pipenv` | `pipenv run poly` | +| `pdm` | `pdm run poly` | +| `hatch` | `hatch run poly` | +| `uv` | `uv run poly` | +| `pip` / `setuptools` / activated venv | `poly` | + +### 4. Discover verification commands +1. **Check for Virtualenv**: Verify if the project has a virtualenv or if dependencies are installed. For example: + - For `uv`: Check for `uv.lock` or `.venv`. If no virtualenv exists, guide the user to run `uv sync`. + - For `pdm`: Run `pdm venv list` to check for a virtualenv. If none exists, guide the user to run `pdm install`. + - For `poetry`: Run `poetry env list` to check for a virtualenv. If none exists, guide the user to run `poetry install`. + - For `pip`: Check if a `venv` or `.venv` directory exists. If not, guide the user to create and activate one. + +2. **Reconcile the Python version**: Compare the project's `requires-python` with the workspace's (`requires-python` in the **root** `pyproject.toml` and the root `.python-version`). If they disagree (e.g. project `>=3.13`, workspace `>=3.12`), the baseline test command can silently resolve the wrong interpreter. Resolve **before** establishing the baseline by either: + - aligning the workspace (bump the root `requires-python` / `.python-version`) — confirm with the user, as it affects every project; or + - recording a per-command interpreter override in the verification commands (e.g. `uv run --python 3.13 …`). + +3. **Establish the brick-import mechanism**: Tests and entrypoints import bricks as `.` from `bases/` and `components/`. Confirm the workspace actually exposes them — this is a prerequisite for `RUN_TEST_CMD` to work **after** code is moved into a base (phase 3+). Inspect the **root** `pyproject.toml`: + - If the root project is editable-installed so the namespace resolves (e.g. Hatch `dev-mode-dirs = ["components","bases","development", …]` **and** the root is actually installed — *not* `[tool.uv] package = false`), set `BRICK_IMPORT_MECHANISM=editable-root`. + - Otherwise add `pythonpath = ["bases","components","development"]` to the root `[tool.pytest.ini_options]` and set `BRICK_IMPORT_MECHANISM=pytest-pythonpath`. + - Sanity-check in the workspace env: ` run python -c "import "` (once at least one brick exists, e.g. after phase 3). It must succeed. + > ⚠ **Common trap:** a root `pyproject.toml` with both `dev-mode-dirs` **and** `[tool.uv] package = false` looks configured but installs nothing — `.*` is then unimportable and **every** post-`extract-to-base` test run fails with `ModuleNotFoundError: No module named ''`. Prefer `pytest-pythonpath` (it avoids changing the install model), or make the root installable. + +4. **Verify `RUN_TEST_CMD`**: Run the test command in the project's directory to ensure it works. + > ⚠ **You are executing untrusted code.** Running the project's tests and the + > install/sync commands below executes arbitrary code from the project (e.g. + > `setup.py`, `conftest.py`, build hooks) **and** from resolved third-party + > packages (post-install scripts). Only run these on a project the user trusts; + > do not proceed on an unknown or untrusted codebase. + + For example: + - For `uv`: Run `uv run pytest tests --collect-only -q | tail -1`. If the command fails, guide the user to install test dependencies (e.g., `uv sync --extra tests`). + - For `pdm`: Run `pdm run pytest tests --collect-only -q | tail -1`. If the command fails, guide the user to install test dependencies (e.g., `pdm install --group tests`). + - For `poetry`: Run `poetry run pytest tests --collect-only -q | tail -1`. If the command fails, guide the user to install test dependencies (e.g., `poetry install --with tests`). + - For `pip`: Run `python -m pytest tests --collect-only -q | tail -1`. If the command fails, guide the user to install test dependencies (e.g., `pip install -e ".[tests]"`). + +5. **Record Baseline**: Record the baseline test count (e.g., number of tests collected) in `state.md`. If the command differs (e.g., `python -m pytest`), update `RUN_TEST_CMD` to match the working command. + +6. **Inspect Config Files**: Inspect `Makefile`, `Justfile`, `tox.ini`, `pyproject.toml` `[tool.pytest.ini_options]`, and CI config (`.github/workflows/*.yml`, `.circleci/config.yml`, etc.) to identify the project's existing commands. Fill `RUN_TEST_CMD`, and `RUN_LINT_CMD` / `RUN_TYPECHECK_CMD` when present. If a command can't be found, leave the value empty. + > 🔒 **Never store secrets in `state.md`.** `state.md` is committed. When a derived + > command embeds a credential (an inline token, a `--token=…` flag, a database URL + > with a password, an API key), do **not** copy the literal value. Reference the + > environment variable name instead (e.g. `RUN_TEST_CMD=DATABASE_URL=$DATABASE_URL uv run pytest …`), + > and have the user supply the secret via their environment at run time. Redact any + > literal credential before writing the file. The same applies to `manifest.md` — + > it captures structure, not secrets. + +7. **Proceed Only After Verification**: Only proceed to the next phase if `RUN_TEST_CMD` succeeds. If it fails, guide the user to resolve the issue before continuing. + +### 5. Determine tooling-conversion eligibility +Read the **workspace root** `pyproject.toml` to determine the workspace's standard linter, formatter, type checker, and package manager. + +- If the project's `LINTER`/`FORMATTER` already matches the workspace's → set `CONVERT_LINTER=no` (skip). +- If the project's `TYPE_CHECKER` already matches the workspace's → set `CONVERT_TYPE_CHECKER=no` (skip). +- If the project's `PACKAGE_MANAGER` is already `uv` **and** the workspace uses uv → set `CONVERT_PACKAGE_MANAGER=no` (skip). +- If the workspace does **not** use uv, `migrate-convert-package-manager` does not apply at all — set `CONVERT_PACKAGE_MANAGER=no` and skip the question below. + +### 6. Create `manifest.md` + +Write `migration//manifest.md` using **exactly** this template — fixed headings, fixed shapes. Later phases parse this file by heading. + +`````markdown +# migration//manifest.md + +## Directory tree + +/`, fenced as a code block> + +## Module map + +| Path | Role | +|------|------| +| `` | | + +## Entrypoints + +- ``: + +## Tests + +- Root: `` (relative to project) +- File count: +- Fixture files: ``, ``, … + +## Infrastructure + +- ``: +````` + +> ⚠ Keep the five `##` headings literal. Downstream phases reference them by name; renaming a heading silently breaks input discovery. + +### 7. Present derived values, then confirm with the user +**Do not proceed past this step without explicit user confirmation.** Present the derived state in one block: + +``` +Derived from projects//: + INITIAL_BASE_NAME = ← initial base name; you will likely add more bases later + ALIAS = ← short alias for poly info/deps tables (optional) + GROUP = ← project group (optional) + +Detected tooling: + PACKAGE_MANAGER = + LINTER = + FORMATTER = + TYPE_CHECKER = + +Verification commands: + RUN_TEST_CMD = + RUN_LINT_CMD = + RUN_TYPECHECK_CMD = + +Optional conversions you can opt into: + - Convert linter/formatter to match workspace standard? (default: no) + - Convert type checker to match workspace standard? (default: no) + - Convert package manager to uv (workspace-uv only)? (default: no) + +Confirm the values above, or correct any of them. +``` + +Wait for the user's response. Update `state.md` with corrections and the `CONVERT_*` answers. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Derived `INITIAL_BASE_NAME` collides with an existing brick under `bases//` or `components//` | Two projects derive the same base name from a generic `[project.name]`. | Append a project-specific suffix (e.g., `payment_api` instead of `payment`) and re-confirm with the user. Check before writing `state.md`. | +| Project has no detectable test command | No `pytest` / `make test` / CI config that reveals a runnable test command. | Ask the user explicitly. If none exists, set `RUN_TEST_CMD=` empty and record that **every later phase loses its primary safety check** — flag the heightened risk and, after each phase, verify that each base's entrypoint module imports cleanly and its wiring matches the original. | +| Multiple linters or formatters are configured simultaneously (e.g., black + ruff format both active) | Project history accumulated tools without a cleanup. | Record both in `state.md` (comma-separated values are acceptable in this one case), and flag for resolution during `migrate-convert-linter`. Do **not** silently pick one. | +| Baseline test command resolves the wrong Python (e.g. "incompatible with the project's Python requirement") | Project `requires-python` disagrees with the workspace root `requires-python` / `.python-version`. | Reconcile per step 4.2 — align the workspace version (confirm with user) or record a per-command `--python ` override in the verification commands. | +| `ModuleNotFoundError: No module named ''` once code is in a base | The workspace doesn't expose bricks (e.g. root `[tool.uv] package = false` with `dev-mode-dirs` that never takes effect). | Establish `BRICK_IMPORT_MECHANISM` per step 4.3 — add a root pytest `pythonpath`, or make the root editable-installable. | + +## Done When +The following artifacts and conditions all hold: + +- [ ] `migration//state.md` exists and contains **every** key in the schema (empty values where N/A, but no missing keys). +- [ ] `migration//manifest.md` exists with all five sections (directory tree, module map, entrypoints, tests, infrastructure). +- [ ] The user has explicitly confirmed `INITIAL_BASE_NAME`, `ALIAS`, `GROUP`, and the three `CONVERT_*` flags. +- [ ] `RUN_TEST_CMD` is set and **runs successfully on the project's current code** (the migration's baseline pass-rate), with the Python version reconciled (step 4.2). +- [ ] `BRICK_IMPORT_MECHANISM` is set and the workspace can import `.*` (step 4.3). +- [ ] `GIT_BRANCH` and `GIT_BASE_SHA` are populated (from orchestrator Phase 0). + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — discover" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-distribute-wiring/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-distribute-wiring/SKILL.md new file mode 100644 index 00000000..5b7dda9b --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-distribute-wiring/SKILL.md @@ -0,0 +1,79 @@ +--- +name: migrate-distribute-wiring +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Distribute app-wiring code from the residual component into the appropriate bases and shared components." +--- + +# Skill: migrate-distribute-wiring + +## Goal +Distribute app-wiring code from the residual component into the appropriate bases and shared components. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- `INITIAL_BASE_NAME` +- Verification commands. +- Base names. + +From `migration//manifest.md`: +- Current module map, including what remains in the residual component. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Read the Residual Component +- List every public function remaining in the residual module. +- Confirm it contains only app-wiring code. + +### 2. Identify the Split +- Trace callers of each function (use `grep`). +- Group callers by base or runner script. +- Map each function to its natural base: + +| Function Pattern | Belongs In | Rationale | +|------------------|------------|-----------| +| `init_consumer`, `close_consumer` | Handler/consumer base | Kafka consumer lifecycle is handler-specific | +| `init_job` | Jobs base | Job bootstrap is jobs-specific | +| `init_api` / app factory | API base | HTTP server setup is API-specific | + +### 3. Extract Shared Helpers +- Identify shared init functions (e.g., `init_logging`, `init_db`). +- Move shared helpers to a `bootstrap` component if needed. + +### 4. Move Composite Functions +- Move composite functions to their respective bases. +- Update runner scripts to import from the base directly. + +### 5. Update Tests +- Update integration tests to monkeypatch the correct module. + +### 6. Clean Up +- Delete the residual component directory. +- Update `pyproject.toml` to remove the residual brick. + +## Verify +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX sync` to synchronize the `[tool.polylith.bricks]` table with actual imports. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Two bases now import the same `init_` function and the residual still exists | Step 6 wasn't reached — the function moved into one base but the residual reference wasn't deleted. | Confirm the residual is gone; if `init_` is genuinely shared by 2+ bases, it belongs in a shared `bootstrap` component (step 3), not in either base. | +| `mock.patch("..init_api")` fails after the move | The test references the old residual path; the function now lives in a base. | Update mock patch strings to the new location, typically `..`. | +| `poly check` says the residual brick is still declared but its files are gone | `pyproject.toml` `[tool.polylith.bricks]` still lists the deleted brick. | Remove the line from `[tool.polylith.bricks]` and re-run `POLY_CMD_PREFIX sync --quiet`. | +| Application starts but a runtime feature is missing (e.g., logging, DB) | A shared init helper was moved into one base only. The other base never calls it. | Promote the helper to a shared `bootstrap` component and call it from every base's startup path. | +| Verification fails and you can't quickly diagnose | Phase commit not yet made. | `git reset --hard HEAD` to roll back to the previous phase's commit and consult the user. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — distribute-wiring" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-extract-standalone-modules/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-extract-standalone-modules/SKILL.md new file mode 100644 index 00000000..6f2482dc --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-extract-standalone-modules/SKILL.md @@ -0,0 +1,65 @@ +--- +name: migrate-extract-standalone-modules +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Extract foundational modules (e.g., `consts.py`, `exceptions.py`, or similar) from the residual component into standalone components." +--- + +# Skill: migrate-extract-standalone-modules + +## Goal +Extract **foundational modules** (e.g., `consts.py`, `exceptions.py`, `models.py`) from the residual component into standalone components. This skill is for zero-dependency or low-dependency modules that serve as building blocks for other components. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- `INITIAL_BASE_NAME` +- Verification commands. + +From `migration//manifest.md`: +- Current module map, including what remains in the residual component. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Analyze the Residual Component +- Use `directory_tree` and `grep` to list modules remaining in the residual component. +- Classify each module: + - **Zero internal deps**: Modules with only stdlib/third-party imports (e.g., `exceptions.py`, `consts.py`). + - **Low internal deps**: Modules that depend on already extracted or zero-dep modules (e.g., `models.py`). + - **App-wiring**: Modules that compose infrastructure setup. These stay in the residual. + +### 2. Extract Modules in Dependency Order +- Extract zero-dep modules first, followed by modules that depend on them. + +### 3. For Each Extraction +1. Check for naming collisions. +2. Create the component directory with `__init__.py` and `core.py`. +3. Update imports in all consumers. +4. Add the new brick to `pyproject.toml`. +5. Run verification. +6. Delete the original module from the residual component. + +## Verify +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX sync` to synchronize the `[tool.polylith.bricks]` table with actual imports. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Module classified as zero-dep actually pulls in a transitive runtime dependency (e.g., reads an env var via the residual's config module) | The classification missed an indirect import. | Reclassify as low-dep, extract the config module first, then retry. | +| Two consumers now import the same constant via different paths (e.g., `from .consts` and `from ..consts`) | The original module wasn't deleted from the residual after extraction. | Delete the original from the residual (step 3.6), pick one canonical import path, and update every caller. Verify with `grep -r '..consts'`. | +| Extracting `exceptions.py` causes `except` clauses elsewhere to stop catching what they used to | Exception classes are identity-based: two definitions are two different classes. | Ensure the new standalone module is the **only** definition. Delete the residual copy and update every `raise`/`except` site. | +| Verification fails and you can't quickly diagnose | Phase commit not yet made. | `git reset --hard HEAD` to roll back to the previous phase's commit and consult the user. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — extract-standalone-modules" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-extract-to-base/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-extract-to-base/SKILL.md new file mode 100644 index 00000000..22827041 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-extract-to-base/SKILL.md @@ -0,0 +1,84 @@ +--- +name: migrate-extract-to-base +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Extract all application code from `projects//` into a temporary migration base." +--- + +# Skill: migrate-extract-to-base + +## Goal +Extract all application code from `projects//` into a temporary migration base (`bases///`). + +## Inputs +From `migration//state.md`: +- `PROJECT_DIR` +- `ORIG_TOP_NS` +- `TARGET_TOP_NS` (default: `ORIG_TOP_NS`) +- `INITIAL_BASE_NAME` +- `RUN_TEST_CMD` (optional: `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD`) + +From `migration//manifest.md`: +- Directory tree and module map. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Create the Base Directory +- Create `bases///`. + +### 2. Move Application Code +- Move application packages/modules from `projects//` to the base: + - For `src/` layout: Move `projects//src//` under the base. + - For flat layout: Move `projects///` under the base. +- Leave non-code files (Dockerfiles, k8s manifests, deploy scripts, `pyproject.toml`) in `projects//`. + +### 3. Update `pyproject.toml` +- Add the base to `[tool.polylith.bricks]`: + ```toml + [tool.polylith.bricks] + "../../bases//" = "/" + ``` + +### 4. Fix Imports +- Update imports minimally to ensure tests and linting pass. + +### 5. Update `manifest.md` +- Reflect the new structure in `migration//manifest.md`. + +### 6. Handle Namespace Changes +If `TARGET_TOP_NS != ORIG_TOP_NS`, the migration orchestrator will handle this in subsequent phases: + +1. `migrate-generate-shim` will create a compatibility shim at `projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py` that re-exports from `${TARGET_TOP_NS}.${INITIAL_BASE_NAME}`. +2. `migrate-automate-import-updates` will update imports in the new base location (at `bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/`) to reference the new namespace. +3. `migrate-update-tests` will update test files to use the compatibility shim. + +### 7. Use Shims if Needed +- If imports break during this phase, add minimal temporary shims to re-export names from the new brick API. +- Track shims in `migration/shims.md`. +- Note that comprehensive shim generation will be handled in the `migrate-generate-shim` phase. + +## Verify +- `RUN_TEST_CMD` succeeds with the same pass/fail counts as the pre-migration baseline recorded in `migrate-discover`. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX sync` to synchronize the `[tool.polylith.bricks]` table with actual imports. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| `ModuleNotFoundError: No module named '.'` after move | `TARGET_TOP_NS != ORIG_TOP_NS` and imports weren't rewritten or shimmed. | This is resolved by later phases per `SHIM_STRATEGY` (chosen in `migrate-analyze-imports`): `shimless` rewrites every reference in `migrate-automate-import-updates` (phase 4); `shim` adds a re-export shim under `ORIG_TOP_NS` in the phase 4b sub-track (`migrate-generate-shim`). To keep this phase green in the meantime, add a minimal temporary shim and record it in `migration/shims.md` (step 7). | +| `RUN_TEST_CMD` collects 0 tests after the move | Test files moved but `pytest` rootdir / `testpaths` still points at the old `projects//tests` location. | Update `pyproject.toml` `[tool.pytest.ini_options].testpaths` or pass explicit dirs in `RUN_TEST_CMD`. (Note: `migrate-prepare-project` moves tests properly — if extract-to-base touched tests at all, consider undoing that part.) | +| `poly check` reports "brick imports another brick that is not in `[tool.polylith.bricks]`" | Step 3 only added the base; imports inside the base may pull in components not yet listed. | Run `POLY_CMD_PREFIX sync --quiet` to populate the rest, then re-run `check`. | +| Editable install / build fails (`error: package directory '' does not exist`) | The base move broke the previous `[tool.setuptools]` or `[tool.hatch.build]` `packages` setting in `projects//pyproject.toml`. | Update `packages` to point at the new `` namespace, or remove the explicit `packages` setting and let Polylith's build hook handle it. | +| Verification fails and you can't quickly diagnose | Phase commit not yet made. | `git reset --hard HEAD` to roll back to the previous phase's commit and consult the user. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — extract-to-base" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-generate-shim/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-generate-shim/SKILL.md new file mode 100644 index 00000000..78d0502e --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-generate-shim/SKILL.md @@ -0,0 +1,63 @@ +--- +name: migrate-generate-shim +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Generate a compatibility shim that re-exports all symbols from the new base location to maintain backward compatibility during namespace migration." +--- + +# Skill: migrate-generate-shim + +> ⛓ **Conditional phase (4b).** Runs **only when `SHIM_STRATEGY=shim`** (chosen in `migrate-analyze-imports`). On the **shimless** path this phase is **skipped** — the namespace rewrite in `migrate-automate-import-updates` (phase 4) covers base internals, external consumers, and tests directly, leaving no shim to maintain or remove. See the `migrate-orchestrator` workflow. + +## Goal +Generate a compatibility shim (`projects///__init__.py`) that re-exports all symbols from the new base location to maintain backward compatibility during namespace migration. + +## Inputs +- Project name (from `migration//state.md`) +- Original namespace (from `migration//state.md`) +- New namespace (from `migration//state.md`) +- Import analysis report (from `migration//import_analysis.md`) + +## Steps + +### 1. List symbols exported by the new base location +1. Inspect the new base location's `__init__.py` (at `bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/__init__.py`) to list all public symbols. +2. Compare with the symbols exported by the original namespace from the import analysis report. + +### 2. Generate the shim +1. Create a shim file at `projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py`. +2. Add re-export statements for all symbols from the new base location using `from ${TARGET_TOP_NS}.${INITIAL_BASE_NAME}. import `. +3. Include an `__all__` list with all re-exported symbols. + +Example shim content: +```python +# Compatibility shim for the migration from `${ORIG_TOP_NS}` to `${TARGET_TOP_NS}.${INITIAL_BASE_NAME}`. +# This module re-exports all public symbols from the new base location. + +from ${TARGET_TOP_NS}.${INITIAL_BASE_NAME}.core import MyClass +from ${TARGET_TOP_NS}.${INITIAL_BASE_NAME}.utils import my_function + +__all__ = [ + "MyClass", + "my_function", +] +``` + +### 3. Verify the shim +1. Ensure the shim's `__all__` includes all symbols exported by the new base location. +2. Record any missing symbols in `migration/${PROJECT}/shim_report.md`. + +## Output +- A shim file: `projects///__init__.py` +- A report file: `migration//shim_report.md` listing all re-exported symbols + +## Verify +1. Confirm that `projects///__init__.py` exists and is not empty. +2. Verify that `migration//shim_report.md` exists and lists all re-exported symbols. +3. Ensure the shim's `__all__` includes all symbols exported by the new base location. + +## Commit +```bash +git add projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py +git add migration/${PROJECT}/shim_report.md +git commit -m "migrate(${PROJECT}): phase — generate-shim" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single source of truth) — do not hardcode it. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-isolate-base-and-big-component/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-isolate-base-and-big-component/SKILL.md new file mode 100644 index 00000000..cc40aa97 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-isolate-base-and-big-component/SKILL.md @@ -0,0 +1,103 @@ +--- +name: migrate-isolate-base-and-big-component +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Shrink the temporary migration base into thin base(s) + one big component. Bases contain only entrypoints/wiring, while the big component contains everything else." +--- + +# Skill: migrate-isolate-base-and-big-component + +## Goal +Shrink the temporary migration base into thin base(s) + one big component: +- **Bases**: Contain only entrypoints/wiring (e.g., FastAPI endpoints, CLI wiring, consumer wiring). +- **Big Component**: Contains all other code. + +> ⚠ **Base vs component naming (read first).** A base and a component **cannot share +> the same brick name** — both would resolve to the Python package `.` +> and collide. So the **big component takes `INITIAL_BASE_NAME`** +> (`.`) and each **base is renamed** to a distinct, +> entrypoint-specific name (the `` used in step 5 — e.g. `_api`, +> `_handler`). This is *why* the temporary base must be renamed when +> the component claims its name. +> +> **Multiple entrypoints → multiple thin bases.** A real service often has several +> deployable entrypoints (an HTTP API + one or more queue consumers + scheduled jobs). +> Create **one thin base per entrypoint** (e.g. `_api`, `_handler`, `_jobs`), +> each exposing only its run/wiring function and importing logic from components. Do +> **not** force them into a single base. (Shared startup wiring — `init_db`, +> `init_logging`, … — goes into a small shared component, not duplicated per base; see +> `migrate-distribute-wiring`.) + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- `INITIAL_BASE_NAME` +- Verification commands. + +From `migration//manifest.md`: +- Entrypoints list. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Create the Big Component +- Create `components///`. +- ⚠ If the temporary base currently occupies `bases///`, **rename the base(s) first** — they must not keep the same name as the big component (see the naming callout above). Create one renamed thin base per entrypoint (`` in step 5), and update the project's run scripts / Dockerfile to point at the new base name(s). + +### 2. Move Non-Entrypoint Code +- Move non-entrypoint code from the base(s) to the big component. + +### 3. Define Public API +- Define a minimal public API in `components///__init__.py`. + +### 4. Update Bases +- Update bases to import only from component APIs: + ```python + from . import ... + ``` + +### 5. Update `pyproject.toml` +- Add the new component to `[tool.polylith.bricks]`: + ```toml + [tool.polylith.bricks] + "../../bases//" = "/" + "../../components//" = "/" + ``` + +### 6. Update `manifest.md` +- Reflect the new structure in `migration//manifest.md`. + +### 7. FastAPI Guidance +| Stays in Base | Moves to Big Component | +|---------------|------------------------| +| `app = FastAPI(...)` | Domain/business logic | +| Middleware, router registration | Persistence/repositories | +| Route handlers (endpoints) | External integrations | +| Startup/shutdown/lifespan wiring | Reusable parsing/validation | + +## Verify +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX sync` to synchronize the `[tool.polylith.bricks]` table with actual imports. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Circular import: base imports from component, component imports back from base | Some "non-entrypoint" code was moved but still references base-only helpers (e.g. the FastAPI `app` instance). | Move the helper down into the component, or invert the dependency by passing the needed value as a function argument. The base's `app` instance must **never** be imported by a component. | +| `poly check` / import error: base and big component both named `` | The base wasn't renamed when the component claimed `INITIAL_BASE_NAME` — two bricks now map to `.`. | Rename the base to an entrypoint-specific name (`_api`, `_handler`, …), update `[tool.polylith.bricks]` and the run scripts, then `POLY_CMD_PREFIX sync`. | +| `ImportError: cannot import name '' from '.'` | The big component's `__init__.py` doesn't re-export ``. Code that previously reached into submodules now needs the public API. | Add `from .. import ` to the component's `__init__.py`, or have the caller import the submodule directly (and accept the brick-interface violation that `poly deps --interface` will flag). | +| Base file becomes near-empty after the split | Good — that's the goal. But check: is there *any* wiring left, or did you accidentally move the entrypoint itself? | If the entrypoint (e.g., `app = FastAPI(...)`) ended up in the component, move it back to the base. The base must own the entrypoint object. | +| `poly check` flags the component as not used by any project | The base's imports go to the wrong namespace (e.g., `from ...`) so the import graph doesn't reach the component. | Update base imports to `from . import …` and re-run `POLY_CMD_PREFIX sync`. | +| Tests for moved code now fail to find fixtures | `conftest.py` was left in the base or moved to the wrong scope. | Move test fixtures alongside the code they cover; usually that's under `test/components///`. | +| Verification fails and you can't quickly diagnose | Phase commit not yet made. | `git reset --hard HEAD` to roll back to the previous phase's commit and consult the user. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — isolate-base-and-big-component" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-isolate-shared-and-project-logic/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-isolate-shared-and-project-logic/SKILL.md new file mode 100644 index 00000000..43a08373 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-isolate-shared-and-project-logic/SKILL.md @@ -0,0 +1,93 @@ +--- +name: migrate-isolate-shared-and-project-logic +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Identify and isolate shared and project-specific logic in monolithic components (e.g., `models`, `schemas`, or similar)." +--- + +# Skill: migrate-isolate-shared-and-project-logic + +> 📐 **Scope vs sibling skills.** This skill is **cross-project**: it separates code used by **two or more projects** (shared) from code used by **a single project** (project-specific), potentially creating new shared *and* project-specific components. Don't confuse with: +> - `migrate-split-big-component` — within one project; splits one component into multiple components. +> - `migrate-split-component-internals` — within one component; splits one `core.py` into multiple files. +> - `migrate-dedupe` — opportunistic deduplication, broader than just shared-vs-project-specific. +> +> 💡 **When to skip this skill.** On the **first** project migration there is usually no second project to compare against. Skip this skill until at least one prior project has been migrated. The orchestrator still calls it after `migrate-extract-standalone-modules` — that's the right place when overlap exists. + +## Goal +Identify and isolate shared and project-specific logic in monolithic components (e.g., `models`, `schemas`). Extract shared logic into reusable components and isolate project-specific logic into project-specific components. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- `INITIAL_BASE_NAME` +- Verification commands (`RUN_TEST_CMD`, `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD`). + +From `migration//manifest.md`: +- Module map of components. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Identify Monolithic Components +- Scan the workspace for monolithic components (e.g., `models`, `schemas`). +- Focus on components with large `core.py` files or mixed domain logic. + +### 2. Analyze Usage +- Use `grep` to trace imports of each definition in the component. +- Classify definitions as: + - **Shared**: Used by multiple projects. + - **Project-Specific**: Used by only one or a few projects. + - **Similar**: Definitions that could reuse shared logic (e.g., similar models). + +### 3. Extract Shared Logic +- Create a new shared component (e.g., `models_shared`, `schemas_shared`). +- Move shared definitions into the new component. +- Update imports in all projects to reference the shared component. + +### 4. Isolate Project-Specific Logic +- Create project-specific components (e.g., `models_project_a`, `schemas_project_b`). +- Move project-specific definitions into the appropriate component. +- Update imports in the relevant projects. + +### 5. Refactor Similar Models +- For similar models, extract shared logic into the shared component. +- Update the project-specific models to reuse the shared logic. + +### 6. Update `pyproject.toml` +- Add the new shared and project-specific components to the workspace's `pyproject.toml`. +- Update the project's `pyproject.toml` to reference the new components. + +### 7. Verify Changes +- Run `RUN_TEST_CMD` to ensure no regressions. +- Run `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` if set. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. + +## Verify +- All tests pass (`RUN_TEST_CMD`). +- Linting and type-checking pass (if set). +- The workspace structure is valid (`POLY_CMD_PREFIX check`). + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| Only one project exists in the workspace, so there's nothing to compare against | Running this skill on the **first** project migration. | Skip the skill entirely. Record `migrate-isolate-shared-and-project-logic: skipped (single-project workspace)` in `migration//state.md` and proceed to `migrate-distribute-wiring`. Revisit when a second project is migrated. | +| A definition looks shared but is actually used by one project via two different bases inside that project | Bases within one project both use it — still single-project usage. | Leave it project-specific. Cross-project sharing requires consumers in **different** projects under `projects/`. | +| Two projects each have a "same" class with subtle field differences (extra fields, different defaults) | Silent merging would change behaviour. | Do **not** merge silently. Either create a shared base class + project-specific subclasses, or keep the implementations separate and accept the duplication. Confirm with the user. | + +## Done When +- Shared logic is extracted into reusable components. +- Project-specific logic is isolated into project-specific components. +- Similar models reuse shared logic where possible. +- All tests and checks pass. +- The workspace structure is valid. + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — isolate-shared-and-project-logic" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-orchestrator/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-orchestrator/SKILL.md new file mode 100644 index 00000000..91d5a586 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-orchestrator/SKILL.md @@ -0,0 +1,188 @@ +--- +name: migrate-orchestrator +description: "[ENTRY POINT] Load this skill first when the user asks to migrate a non-Polylith Python project to Polylith (e.g. \"migrate `projects/` to Polylith\"). Drives the full migration workflow plus optional tooling conversions; do not load any other `migrate-*` skill directly — they are sub-skills this orchestrator invokes." +--- + +# Skill: migrate-orchestrator + +> 🧭 **You are in the right place.** This is the **entry point** for migrating a non-Polylith Python project into a Polylith workspace. If you arrived here from a fuzzy match on a sub-skill name (e.g., `migrate-discover`, `migrate-extract-to-base`), **stay here** — those sub-skills depend on state and a git safety net that only this orchestrator sets up. Loading them in isolation is undefined behaviour. Execute the phases below in order. + +## Goal +Define and execute the workflow for migrating a non-Polylith Python project to a Polylith workspace. +**This skill must be explicitly invoked by a human with the project name/path.** + +## Usage +To migrate a project, load the `migrate-orchestrator` skill and provide the project name (the subdirectory under `projects/`): + +``` +Load the `migrate-orchestrator` skill and migrate `projects/`. +``` + +> 💡 **How sub-skills are loaded.** Each phase points to another skill named `migrate-` (e.g., `migrate-discover`, `migrate-extract-to-base`). Load each via your skill loader before executing the phase. Do not interleave phases — finish and verify one before starting the next. + +## Pre-flight + +### 0. User Confirmation +Ask the user to confirm the project path and migration intent before doing anything else: + +``` +You are about to migrate `projects/` to Polylith. This will refactor +the project into bases and components and move files. Proceed? (yes/no) +``` + +If the user declines, abort: +``` +Migration aborted by user. +``` + +### Phase 0. Safety Net (git checkpoint) +Migration is destructive — files move, directories are deleted, `pyproject.toml`s are rewritten. **Before loading `migrate-discover`, establish rollback points:** + +1. Confirm the working tree is clean: + ```bash + git status + ``` + If there are uncommitted changes, ask the user to commit/stash before proceeding. Do not start a migration on top of a dirty tree. (If the repo has **no commits yet**, create an initial baseline commit so there is a `GIT_BASE_SHA` to roll back to.) +2. **Secret-hygiene precondition (do this before any phase stages files).** Later + phases stage changes broadly, and the migration *creates* untracked files + mid-flow (e.g. `.venv/` from `uv sync`/`poetry install`, regenerated lock files). + A "clean" tracked tree can still leave **untracked** secrets (`.env*`, `*.pem`, + `*.key`, `*_rsa`, `*service-account*.json`, credential files) that a broad stage + would commit. Before proceeding: + - Confirm `.gitignore` covers `.venv/`, `.env*`, and common secret material. + - Review `git status --porcelain` for untracked sensitive files; have the user + remove, relocate, or ignore them. **Do not start** until no untracked secret + material remains stageable. +3. Create a dedicated migration branch: + ```bash + git checkout -b migrate/ + ``` +4. After each completed phase, commit per that phase's `## Commit` section. The commit message follows the pattern `migrate(): phase — ` so phases can be located in `git log` later. + - **Stage narrowly.** Prefer scoped `git add ` over `git add -A`, limiting + the stage to migration-relevant paths (the bricks/components/bases touched, the + project dir, and `migration//`). Where a phase's `## Commit` + section still shows `git add -A`, first run `git status --porcelain` and + **exclude anything matching secret patterns** (`.env*`, `*.pem`, `*.key`, + `*_rsa`, `*service-account*.json`, credential files) or build artifacts + (`.venv/`, caches). Never stage a file you have not accounted for. + ```bash + git add && git commit -m "migrate(): phase — " + ``` + This gives the user (and the agent) a discrete, named rollback point per phase. If a later phase fails verification, the agent can `git reset --hard HEAD~1` to back out exactly one phase without losing earlier progress. +5. Record the branch name and starting commit SHA in `migration//state.md` (the `migrate-discover` skill defines that file). + +> ⚠ Never `git reset --hard` past the start of the migration branch without explicit user approval — the user's pre-migration work lives there. +> ⚠ Because phases may stage broadly, a `git reset --hard HEAD` *before* a commit also discards untracked work. This is a second reason to stage narrowly (step 4). + +## Workflow + +The table below is the **single source of truth** for phase order and numbering. +Execute phases top to bottom. **Verify each phase's `Verify` section before +starting the next, and commit between phases** (see Phase 0 step 3). Where a +sub-skill's own `## Commit` section or `description:` hardcodes a *different* +phase number, **ignore that number** — use the `#` from this table in the +commit message. + +### Main line (always run) + +| # | Phase | Skill | Depends on | +|----|----------------------------------------|------------------------------------------------|------------| +| 1 | Discover | `migrate-discover` | — | +| 2 | Analyze imports + choose rewrite strategy | `migrate-analyze-imports` | 1 | +| 3 | Extract to base | `migrate-extract-to-base` | 2 | +| 4 | Update imports in the new base | `migrate-automate-import-updates` | 3 | +| 5 | Prepare project | `migrate-prepare-project` | 4 (+ 4b if taken) | +| 6 | Verify stability | `migrate-verify-stability` | 5 | +| 7 | Isolate base and big component | `migrate-isolate-base-and-big-component` | 6 | +| 8 | Split big component | `migrate-split-big-component` | 7 | +| 9 | Extract standalone modules | `migrate-extract-standalone-modules` | 8 | +| 10 | Isolate shared and project logic | `migrate-isolate-shared-and-project-logic` | 9 | +| 11 | Distribute wiring | `migrate-distribute-wiring` | 10 | +| 12 | Split component internals | `migrate-split-component-internals` | 11 | +| 13 | Refactor tests | `migrate-refactor-tests` | 12 | +| 14 | Definition of done | `migrate-definition-of-done` | 13 | + +### Conditional namespace-shim sub-track (phase 4b) + +`migrate-analyze-imports` (phase 2) sets a `SHIM_STRATEGY` in `state.md`: + +- **`shimless`** (recommended when imports are submodule-qualified — e.g. + `from . import …` — and `/__init__.py` exports little or nothing; + a top-level re-export shim would resolve nothing there): **phase 4 rewrites all + references** — base internals, entrypoints, infra, and tests — directly to the + new namespace. **Skip the sub-track below.** +- **`shim`** (a top-level re-export shim is viable): after phase 4, run this + sub-track before phase 5, then continue the main line: + + | # | Phase | Skill | Depends on | + |-------|-----------------------------|------------------------------------|------------| + | 4b.i | Generate compatibility shim | `migrate-generate-shim` | 3, 4 | + | 4b.ii | Detect circular imports | `migrate-detect-circular-imports` | 4b.i | + | 4b.iii| Resolve circular imports | `migrate-resolve-circular-imports` | 4b.ii | + | 4b.iv | Update test files | `migrate-update-tests` | 4b.iii | + +> **Dependency note (why this is a *post-extraction* sub-track):** the shim +> re-exports the **original** namespace *from the new base location*, so it can +> only be generated **after** `migrate-extract-to-base` (phase 3) and after the +> base's own imports point at the new namespace (`migrate-automate-import-updates`, +> phase 4). A shim generated before extraction would re-export from a location +> that does not exist yet. + +### Skippable / mergeable phases + +Some phases are no-ops for certain projects. Skip with a one-line rationale +recorded in `state.md`, and still commit the (possibly empty) phase so the +`git log` stays complete: + +- **Phase 10 (isolate shared and project logic)** — skip on the **first** project + migrated into the workspace (there is no second project to compare against). + Revisit when a 2nd overlapping project is migrated. +- **Phase 12 (split component internals)** — skip when components are already + cohesive (no monolithic `core.py` mixing multiple domains). +- **Infra relocation / per-brick test layout** — if a step cannot be verified in + the migration environment (e.g. a deploy cycle is required), it may be + **deferred with a documented rationale** rather than blocking the migration + (see `migrate-prepare-project` and `migrate-definition-of-done`). + +## Optional Skills + +These are not part of the linear flow above. They are triggered when the user opts in during `migrate-discover` (or, for `migrate-dedupe`, when duplication candidates surface). When triggered, **insert them at the indicated point** in the flow. + +| Skill | When to run | Trigger | +|------------------------------------|--------------------------------------------------------------------|-------------------------------------------------| +| `migrate-convert-linter` | After `migrate-discover`, before `migrate-analyze-imports`. | User opts in during `migrate-discover`. | +| `migrate-convert-type-checker` | After `migrate-discover`, before `migrate-analyze-imports`. | User opts in during `migrate-discover`. | +| `migrate-convert-package-manager` | After `migrate-discover`, before `migrate-analyze-imports`. | User opts in during `migrate-discover` **AND** the workspace itself uses uv. The skill is opinionated about uv — see its header for the gating rule. | +| `migrate-dedupe` | After `migrate-split-big-component` or `migrate-extract-standalone-modules` surfaces duplication candidates. | Duplication candidates surfaced and user approves. | + +> ⚠ `migrate-convert-package-manager` only converts **to uv**. If the workspace uses Poetry, PDM, or Hatch as its standard, **skip this skill entirely** — the project should be aligned to the workspace's manager via a manual step instead. + +### Ordering when multiple converters are opted in + +When the user opts into more than one of the optional `migrate-convert-*` skills during `migrate-discover`, run them in **this order** between `migrate-discover` and `migrate-analyze-imports`: + +1. `migrate-convert-package-manager` — runs first because it rewrites `pyproject.toml` wholesale; subsequent skills must operate on the final layout. +2. `migrate-convert-linter` — runs second so workspace-level lint config consolidation happens against the final `pyproject.toml`. +3. `migrate-convert-type-checker` — runs last; type-checker config is the most localized of the three. + +`migrate-dedupe` is triggered later (after the big component is split and duplication candidates surface) and has **no ordering dependency** with the converters. + +Commit between each optional skill the same way the main phases commit (see each skill's `## Commit` section). + +## Execution checklist + +For each phase: +1. **Validate `state.md`** against the rules in `migrate-discover` (`### Validation rules`). Abort the phase if validation fails. +2. Load the skill (`migrate-`). +3. Execute its `Steps` in order. +4. Run its `Verify` section. **If verification fails, do not commit and do not proceed.** Either fix the issue, or `git reset --hard` to back out the phase and consult the user. +5. On success, commit per the phase's `## Commit` section, using the `#` from **this** orchestrator table (not any number baked into the sub-skill). + +## Validation +- **The phase graph is a DAG.** Every `Depends on` entry references an + earlier-numbered phase (or an earlier step of the 4b sub-track), so a strict + top-to-bottom execution always satisfies dependencies. (An earlier version of + this table violated that — it listed `generate-shim` *before* `extract-to-base`; + the shim phases are now a post-extraction sub-track, phase 4b.) +- Every skill referenced above exists as `migrate-/SKILL.md` under `.agents/skills/polylith/migrate-project/`. +- Each phase's `Verify` block uses the commands recorded in `migration//state.md` (`RUN_TEST_CMD`, optionally `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD`, plus `POLY_CMD_PREFIX check`). diff --git a/.agents/skills/polylith/migrate-project/migrate-prepare-project/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-prepare-project/SKILL.md new file mode 100644 index 00000000..a82620db --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-prepare-project/SKILL.md @@ -0,0 +1,107 @@ +--- +name: migrate-prepare-project +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Clean up the project subfolder and consolidate dependencies after extracting application code into bases." +--- + +# Skill: migrate-prepare-project + +## Goal +Clean up the project subfolder (`projects//`) and consolidate dependencies. After this step, the project subfolder should contain only: +- Project `pyproject.toml` (with brick references). +- Task runners (`Makefile`, `Justfile`). +- Project-specific config (e.g., `alembic.ini` **and** its `alembic/` directory). +- Compatibility shim (only when `SHIM_STRATEGY=shim`). + +## Inputs +From `migration//state.md`: +- `PROJECT_DIR` +- `TARGET_TOP_NS` +- `ALIAS` (optional: `GROUP`) +- Verification commands. + +From `migration//manifest.md`: +- Infra files list. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Verify Project Subfolder +- Run `directory_tree` on `projects/${PROJECT}/` to confirm only infra and config files remain. + +### 2. Update `pyproject.toml` +- Add brick references to `[tool.polylith.bricks]`: + ```toml + [tool.polylith.bricks] + "../../bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}" = "${TARGET_TOP_NS}/${INITIAL_BASE_NAME}" + ``` +- Register the project alias and group in `workspace.toml` (if provided): + ```toml + [tool.polylith.projects.alias] + ${PROJECT} = "${ALIAS}" + + [tool.polylith.projects.groups] + ${GROUP} = ["${PROJECT}"] + ``` + +### 3. Move Tests to Workspace Level +- Move `tests/` to `test//`, where `` is a **valid Python package name**. ⚠ `${PROJECT}` frequently contains hyphens (e.g. `order-management-api`), which are **illegal in a package name** and break pytest's package-style import. Derive an underscore name — the workspace's `_service` convention is a good default (e.g. `order_management_service`). Record `` in `state.md`. +- If the moved tests import the old package by name (e.g. `from tests.fixtures import …` / `from tests.helpers import …`), rewrite those to the new name (`from .fixtures import …`). Brick imports (`.*`) are absolute and need no change. +- Update `RUN_TEST_CMD` in `migration/${PROJECT}/state.md` to point to the new location. +- Update mock patch strings in test files if paths changed. +> Per-brick test redistribution (`test////`) is handled later in `migrate-refactor-tests` — this step only lifts tests to the workspace level. See that skill for the shared-helper / namespace-merge caveats before splitting further. + +### 4. Move Infrastructure Folders +- Move **deployment** infra folders (e.g., `helm/`, `k8s/`, `kustomize/`, `skaffold/`) to `infra//${PROJECT}/`. +- 🔒 **Secret & `.gitignore` check before the move.** Infra folders frequently carry + secret material (e.g. `values-secret.yaml`, `*.env`, sealed-secret sources, TLS + keys). Two hazards when relocating them: + - **`.gitignore` scope shifts.** Ignore rules scoped to the **old** path stop + matching at the new `infra//${PROJECT}/` path, so a previously-ignored + secret file can become **tracked** after the move. Verify the destination is + still covered by `.gitignore` for anything that was ignored at the source. + - **Inline secrets.** Scan the moved manifests/values for inline credentials + before committing. Do **not** commit newly-tracked secrets; have the user move + them to an ignored path or a secret manager first. +- **Keep `alembic/` together with `alembic.ini` in the project** — do **not** split them. `alembic.ini`'s `script_location` points at `alembic/` relatively, and `alembic/env.py` imports the project's bricks; separating them breaks migrations. (Alembic is project-specific config, which the project subfolder is allowed to keep.) ⚠ `alembic.ini` commonly holds a `sqlalchemy.url` with DB credentials — if it is not already tracked, do not newly commit it with a literal credential; reference an env var instead. +- ⚠ **Deploy-path breakage:** moving infra breaks deploy scripts / `skaffold` / CI that hardcode the old paths. If you **cannot verify the deploy** in the migration environment (no deploy / manifest-diff cycle available), it is acceptable to **defer the infra move and document it** in `state.md` rather than relocate blindly. Otherwise, update the referencing scripts and record the change. Prefer updating references over committing a **symlink**: a committed symlink can point outside the repo and behaves inconsistently across checkouts/OSes — if you must use one as a transition, keep it within the repo, record it in `state.md`, and schedule its removal. + +### 5. Consolidate Dependencies +- Move third-party dependencies with version constraints to the workspace root `pyproject.toml`. +- Move dev/test/tooling dependencies to the workspace root. +- List runtime dependencies **without version numbers** in the project `pyproject.toml`. +- Sync the virtual environment with the project's package manager (e.g. `uv sync`, + `pdm install`, `poetry install` — match `PACKAGE_MANAGER` in `state.md`). +- Verify dependencies, substituting your package manager's run prefix (`` = + `uv run` / `pdm run` / `poetry run` / bare in an activated venv): + ```bash + pip list # Confirm all dependencies are installed + python -c "import " # Verify key dependencies are available + ``` + +## Verify +- `RUN_TEST_CMD` succeeds against the **new** test location. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX info` to inspect the workspace and confirm the project is correctly registered. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| `RUN_TEST_CMD` collects 0 tests after step 3 | The command in `state.md` still references the old `projects/${PROJECT}/tests` path. | Update `RUN_TEST_CMD` to the new `test/${PROJECT}/` location. Also check `[tool.pytest.ini_options].testpaths` / `rootdir` / `conftest.py` discovery. | +| Tests fail with `ModuleNotFoundError` on internal imports | Tests use `from tests.fixtures import …` and the `tests` package name changed. | Update test imports to the new test root path. If many tests reference the old name, consider keeping `tests` as the leaf directory and only renaming the parent. | +| `mock.patch("")` fails with `AttributeError` | Mock patch strings reference moved modules. | Update patch strings to the new module paths. Use `grep -r 'patch("' test/` to find them all. | +| Infra folder move breaks deploy scripts that hardcoded paths (e.g., `helm//values.yaml`) | Deploy scripts haven't been updated to the new `infra/): phase — prepare-project" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-refactor-tests/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-refactor-tests/SKILL.md new file mode 100644 index 00000000..2795d32e --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-refactor-tests/SKILL.md @@ -0,0 +1,121 @@ +--- +name: migrate-refactor-tests +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Restructure unit tests to align with the workspace's Polylith theme." +--- + +# Skill: migrate-refactor-tests + +## Goal +Restructure unit tests so they live next to the brick they test, following the workspace's Polylith theme (`loose` or `tdd`). + +## Scope +- **Unit tests only.** Integration tests typically stay in a shared location (e.g., `test/integration/` or `test//integration/`). Do not redistribute them across bricks. +- **Structure only.** Reorganize test files and update imports/mock patch strings. Do not rewrite test logic or fixtures unless an import/path change makes it strictly necessary. + +## Shared test helpers & the namespace-merge hazard (read before moving) + +Two real traps when moving unit tests next to their bricks: + +1. **Shared test-support packages stop resolving.** If tests import shared + helpers/fixtures by package (e.g. `from _service.helpers import …`, + `from _service.fixtures import …`), those resolve today only because the + service test dir sits on `sys.path` (pytest inserts the topmost non-package dir). + Once a test moves to `test/components///` (no `__init__.py`), + pytest inserts the **leaf** test dir instead, and the shared import breaks. +2. **Do NOT "fix" it by adding `test/` to `pythonpath`.** That makes + `test/<…>///` directories **merge into the real + `` namespace** and collide with `components//` + / `bases//` — an ambiguous, hard-to-debug namespace package. + +### Two sanctioned layouts — pick one + +- **Per-brick (the target table below).** Use it **after** making shared helpers + reachable *without* putting `test/` on the path: convert them to `conftest.py` + **fixtures** (pytest injects fixtures with no import), or move pure helper functions + to an on-path support location. Unit tests then live under `test////`. +- **Workspace-level service dir.** Keep the project's tests under a single + `test/_service/` directory (the importable, valid-package name that + `migrate-prepare-project` created). Shared helpers stay co-located and importable. + Choose this when converting helpers to fixtures isn't worth it; record the choice in + `state.md`. This still satisfies "tests at workspace level". + +If unsure, the **workspace-level service dir is the lower-risk default**. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- `RUN_TEST_CMD` (will be updated by this skill) +- `RUN_LINT_CMD`, `RUN_TYPECHECK_CMD` (optional) + +From `migration//manifest.md`: +- List of all bricks (bases and components) with their module maps. + +From `workspace.toml`: +- `[tool.polylith.structure].theme` (`loose` or `tdd`). + +From `test/`: +- Current test directory structure. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Target layout + +| Theme | Test path for a base | Test path for a component | +|----------|------------------------|-----------------------------| +| `loose` | `test/bases///test_*.py` | `test/components///test_*.py` | +| `tdd` | `bases//test///test_*.py` | `components//test///test_*.py` | + +## Steps + +### 1. Classify each unit test file +For each `test_*.py` under the current test root: +1. Read its `import` statements and `mock.patch("...")` strings. +2. Identify the *primary* brick under test — usually the one most imported or the one mock-patched. Tie-break by file name (`test_user_handler.py` → ``). +3. Record the classification in a table you keep in scratch (do **not** commit a separate file for this — it's transient): + +| Test file | Primary brick | Target path | +|-----------|---------------|-------------| + +If a test exercises 2+ bricks at integration level, classify it as integration and move it to `test/integration/` instead. + +### 2. Move test files +- Create the target directories per the theme matrix above. +- Move each `test_*.py` to its brick's test directory. +- For each `conftest.py`: + - **Brick-scoped fixtures** (referenced only by tests under one brick) → move to that brick's test directory. + - **Workspace-shared fixtures** (cross-brick) → keep one `test//conftest.py` or `test/conftest.py`. + +### 3. Update imports and mock patch strings +- Update any `from tests.` imports that survived to the new layout. +- Update `mock.patch("")` strings. Brick reshuffling earlier in the migration may have changed where the patched symbol now lives — use `grep` to locate the target symbol's new path and align the patch string. Patching a wrong path silently succeeds and the test will pass for the wrong reason — verify by intentionally breaking the target function and checking that the test fails. + +### 4. Update `RUN_TEST_CMD` +- Set `RUN_TEST_CMD` in `state.md` to a command that collects from the **new** test root (typically ` pytest test/` for `loose`, or per-brick collection for `tdd`). +- Verify the test count after the move equals the baseline recorded in `migrate-discover`. **A drop in collected tests means files were lost or pytest discovery is misconfigured.** + +## Verify +- `RUN_TEST_CMD` succeeds and **collects the same number of tests** as the baseline from `migrate-discover`. +- If set, `RUN_LINT_CMD` succeeds. +- `POLY_CMD_PREFIX check` is still green. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| `pytest` collects fewer tests than before | The old test root is no longer on `pytest`'s path; or duplicate `conftest.py` files silently shadow each other. | Update `[tool.pytest.ini_options].testpaths` (or pass paths explicitly in `RUN_TEST_CMD`). Run `pytest --collect-only` and diff against baseline collection. | +| `fixture '' not found` | The fixture was in a `conftest.py` you moved into a brick's test dir; tests in another brick can no longer see it. | Either move the fixture up to a higher-scope `conftest.py`, or duplicate it (only if cheap). Don't import fixtures across `conftest.py` files. | +| Tests pass but assert nothing useful (`mock.patch` no longer hits anything) | Patch string still points at the pre-migration module path. | Re-derive the patch path: `...`. Validate by deliberately breaking the patched function and confirming the test fails. | +| `ImportError: cannot import name 'fixture_'` from `conftest.py` | A `conftest.py` `import`s a moved test helper module that didn't follow it. | Move the helper next to the new `conftest.py`, or import it from its new brick path. | +| Tests for moved code suddenly find themselves under a brick name that doesn't match their content | Misclassification in step 1. | Re-read the test's imports — the brick most imported is the one that owns the test. Move and update. | +| `ModuleNotFoundError: No module named '_service'` after moving a test to `test////` | Shared test-support package is no longer on `sys.path`. | Use a sanctioned layout (above): convert the helpers to conftest fixtures, or keep tests in the workspace-level `_service/` dir. Do **not** add `test/` to `pythonpath` (namespace merge with the real bricks). | +| Verification fails and you can't quickly diagnose | Phase commit not yet made. | `git reset --hard HEAD` to roll back to the previous phase's commit and consult the user. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — refactor-tests" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-resolve-circular-imports/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-resolve-circular-imports/SKILL.md new file mode 100644 index 00000000..7b8c5113 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-resolve-circular-imports/SKILL.md @@ -0,0 +1,57 @@ +--- +name: migrate-resolve-circular-imports +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Resolve circular imports by updating imports in the new base location to avoid referencing the compatibility shim." +--- + +# Skill: migrate-resolve-circular-imports + +> ⛓ **Conditional phase (4b).** Runs **only when `SHIM_STRATEGY=shim`** (chosen in `migrate-analyze-imports`) and only if `migrate-detect-circular-imports` found cycles. **Skipped** on the shimless path. See the `migrate-orchestrator` workflow. + +## Goal +Resolve circular imports by updating imports in the new base location to avoid referencing the compatibility shim, ensuring a clean dependency graph. + +## Inputs +- Project name (from `migration//state.md`) +- Original namespace (from `migration//state.md`) +- New namespace (from `migration//state.md`) +- Circular imports report (from `migration//circular_imports.md`) + +## Steps + +### 1. Review the circular imports report +1. Review the circular imports report to identify all circular import chains. + +### 2. Resolve circular imports +1. For each circular import chain: + - Update imports in the new base location (`bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/`) to avoid referencing the compatibility shim + - Use direct imports from the new namespace instead + - Consider using forward references or dependency injection where necessary + +Example: +```python +# Before (circular import with shim) +from myproject.core import MyClass + +# After (direct import from new namespace) +from mynamespace.mybase.core import MyClass +``` + +### 3. Record resolved circular imports +1. Record all resolved circular imports in `migration/${PROJECT}/circular_imports_resolved.md`. + +## Output +- Updated files in the new base location +- A report file: `migration//circular_imports_resolved.md` listing resolved circular imports + +## Verify +1. Confirm that all circular imports listed in `migration//circular_imports.md` have been resolved. +2. Verify that no circular imports remain by re-running the circular import detection. +3. Ensure the codebase remains functional by running the test command from `migration//state.md`. + +## Commit +```bash +git add bases/${TARGET_TOP_NS}/${INITIAL_BASE_NAME}/ +git add migration/${PROJECT}/circular_imports_resolved.md +git commit -m "migrate(${PROJECT}): phase — resolve-circular-imports" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single source of truth) — do not hardcode it. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-split-big-component/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-split-big-component/SKILL.md new file mode 100644 index 00000000..ab159c16 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-split-big-component/SKILL.md @@ -0,0 +1,193 @@ +--- +name: migrate-split-big-component +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Split the big component (`components///`) into multiple focused components." +--- + +# Skill: migrate-split-big-component + +> 📐 **Scope vs sibling skills.** This skill operates **within one project** and turns *one component into many components*. Don't confuse with: +> - `migrate-extract-standalone-modules` — same scope (one project) but pulls *foundational modules* (`consts.py`, `exceptions.py`) out of the residual big component into their own standalone components. +> - `migrate-split-component-internals` — operates **inside one already-extracted component**, splitting its `core.py` into multiple files. No new components are created. +> - `migrate-isolate-shared-and-project-logic` — **cross-project** scope; identifies shared-vs-project-specific logic when migrating a 2nd+ project. +> - `migrate-dedupe` — opportunistic deduplication; this skill includes a dedup-analysis subsection so for a single project you usually don't need `migrate-dedupe` separately. + +## Goal +Split the big component (`components///`) into multiple focused components to improve maintainability, clarity, and reusability. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- `INITIAL_BASE_NAME` +- Verification commands. + +From `migration//manifest.md`: +- Module map of the big component. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### Phase 1: Plan the Split +1. **Review the Big Component**: Use `directory_tree` and `grep` to analyze the big component's structure and identify natural slices. +2. **Define Component Names**: Name components after the **domain or functionality** they represent (e.g., `domain_a_serializer`, `data_transformations`). Avoid generic names like `utils` or `helpers`. +3. **Create a Split Plan**: Record the following in `migration//split_plan.md`: + - Brick name for each new component. + - Files/modules to move into each component. + - Public API (key functions/classes to export). + - Bricks that will import from the new component. + +**When NOT to Extract**: +- The module is tightly coupled to other modules in the component. +- The module is very small (< 20 lines) and extraction adds more indirection than value. + +**Extraction Order**: +- Extract modules with **zero internal dependencies** first (e.g., `exceptions.py`, `consts.py`). +- Extract modules that depend on already-extracted modules next (e.g., `models.py` that imports `exceptions` and `consts`). + +### Examples of Component Naming +Name components after the **domain or functionality** they represent: + +| Original module name | Content (after inspection) | Component name | +|----------------------|----------------------------|----------------| +| `serializers.py` | Serializes data for ERP | `domain_a_serializer` | +| `transformations/` | Maps data between formats | `data_transformations` | +| `parsers.py` | Parses event payloads | `event_parser` | +| `validators.py` | Validates records | `record_validator` | + +#### Avoiding circular imports when extracting modules + +Extracting a module into a separate component can create circular imports if the new component imports from the parent component and the parent still imports from the new component. This commonly happens when a component's `__init__.py` eagerly imports from many submodules. + +**Diagnosis:** The cycle typically looks like: + +``` +new_component.core → parent.__init__ → parent.submodule → new_component +``` + +Python triggers `parent.__init__` whenever any submodule of `parent` is imported (e.g. `from parent.consts import X` loads `parent/__init__.py` first). + +**Resolution strategies (in order of preference):** + +1. **Extract the circular part into its own component.** A circular dependency often signals that the code involved is isolated enough to be its own component. Extract the module that causes the cycle into a standalone component — this breaks the cycle structurally. A component doesn't have to be a "feature"; it can be a utility, a data definition, a pure technical module, or a single ORM model. If the code has a clear responsibility and can be imported without pulling in the rest of the parent, it belongs in its own brick. + + ``` + # Before (cycle): new_component → parent.consts → parent.__init__ → parent.handlers → new_component + # After (no cycle): new_component → consts_component (standalone, no __init__ chain) + ``` + +2. **Trim `__init__.py` exports.** Remove the problematic import from the parent's `__init__.py` and have callers import the submodule directly. This makes the dependency graph explicit and often eliminates the cycle without creating a new brick. + + ```python + # Before: parent/__init__.py imports everything eagerly + from parent.command_handler import CommandHandler # triggers handler → new_component cycle + + # After: remove from __init__.py, callers import directly + from parent.command_handler import CommandHandler # in the base that needs it + ``` + +3. **Standalone component instead of submodule.** If the new component would be a submodule of an existing package (e.g. `myns.models.example_transaction`), importing it triggers the parent package's `__init__.py` and all its eager imports. Make it a standalone component at the namespace level instead (e.g. `myns.example_transaction`). + +4. **Deferred import (last resort).** Move the import inside the function that uses it. This works but hides the dependency and makes the code harder to reason about. Prefer strategies 1–3 first. + +**Pre-flight check:** Before extracting a module, trace the import chain: +1. The new component imports `parent.submodule_X` → Python loads `parent.__init__` +2. Does `parent.__init__` (directly or transitively) import from the new component? +3. If yes → apply one of the strategies above (extract, trim, or restructure) before proceeding. + +#### Refactoring shared infrastructure components + +When a second project needs a component that already exists (e.g. `myns.logging`, `myns.kafka`), compare the implementations closely. Common refactoring patterns: + +**Pattern: Parameterize the shared component.** When two implementations are 80%+ identical with project-specific extras, refactor the shared component to accept optional parameters rather than duplicating code. + +Example — logging with project-specific loggers: +```python +# Shared component: myns.logging +def init(config, *, extra_loggers=None, cache_logger_on_first_use=False): + loggers = {**_BASE_LOGGERS} + loggers.update(_verbosity_overrides(config.LOG_VERBOSITY_LEVEL)) + if extra_loggers: + loggers.update(extra_loggers) + ... + +# Project A base: +init(config, extra_loggers={"httpx": {...}, "backoff": {...}}, + cache_logger_on_first_use=config.LOG_CACHE_LOGGER_ON_FIRST_USE) + +# Project B base: +init(config, extra_loggers={"confluent_kafka_helpers": {...}}) +``` + +**When to parameterize vs. keep separate:** +- **Parameterize** when the core logic is identical and only data/config differs. +- **Keep separate** when the control flow or structure diverges (different frameworks, different patterns). +- **Extract shared base + project-specific wrappers** when there's a significant shared core but non-trivial project-specific logic around it. + +#### Splitting Component Internals + +Components with generic names like `models`, `schemas`, `exceptions`, or `consts` may start with a single `core.py` file. As the workspace grows, these components can accumulate code from different domains. Splitting `core.py` into multiple domain-focused modules **inside the component** can improve maintainability and clarity. + +**When to Split:** +- If the `core.py` file contains definitions from multiple distinct domains (e.g., `domain_a` and `domain_b`). +- If the file contains helper/utility functions alongside class definitions. +- If preparing for a second project migration that will contribute to the same component. + +**Approach:** +- Group definitions by the domain concept they serve. +- Name each module after the domain or functionality it represents (e.g., `domain_a.py`, `domain_b.py`). +- Ensure the public API remains unchanged to avoid breaking existing imports. + +#### Cross-component duplication analysis + +After drafting the split plan (but before executing any moves), analyze the planned components — and any _already existing_ components in the workspace — for duplication: + +1. **Identify overlap:** For each planned component, check whether an existing component already contains similar logic. Look for: + - Functions/classes with the same or very similar names. + - Modules that operate on the same domain concept (e.g., two different `domain_a_serializer` implementations). + - Copy-pasted utility functions (string helpers, date formatting, retry wrappers, etc.). + +2. **Classify the overlap:** + - **Identical or near-identical:** the code does the same thing with trivial differences (variable names, formatting). → Extract to a shared component. + - **Same purpose, different behavior:** the code solves the same problem but with project-specific logic (e.g., different serialization schemas). → Keep separate, but extract any genuinely shared helpers. + - **Coincidental similarity:** the code looks similar but serves unrelated purposes. → Leave separate. + +3. **Propose shared extractions:** When genuinely duplicated code is found, add a step to the split plan: + - Create a new shared component (or extend an existing one) containing the common logic. + - Have both the existing and the new component depend on the shared one. + - Record this in `/split_plan.md` with a rationale. + +4. **Always confirm with the user** before creating shared components — "is this code genuinely shareable?" is a judgment call that depends on how the projects will evolve. + +### Phase 2: Execute the Split +For each planned component in `split_plan.md`: +1. **Create the Component**: Create the component directory with `__init__.py`. +2. **Move Files/Modules**: Move the relevant files/modules into the new component. +3. **Define the Public API**: Update `__init__.py` to re-export the public API. +4. **Update Callers**: Update all imports to reference the new component. For anything beyond a handful of call sites, drive this with the small text-in → text-out rewrite helper described in `migrate-automate-import-updates` (it covers dotted, bare-submodule, and quoted-string references and splits mixed import lines), then grep for residual references to the old path. +5. **Update `pyproject.toml`**: Add the new brick to the project's `[tool.polylith.bricks]`. +6. **Run Verification**: Ensure tests, linting, and type-checking pass. + +## Verify +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. +- Run `POLY_CMD_PREFIX check` to validate the workspace structure. +- Run `POLY_CMD_PREFIX sync` to synchronize the `[tool.polylith.bricks]` table with actual imports. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| New component is named `utils`, `helpers`, `common`, or `misc` | Naming taken from old module names instead of the domain the code serves. | Rename to a domain-specific name (see the "Examples of Component Naming" table). Generic-named bricks attract more code and become the next big component. | +| Extracted component imports back into the residual via the residual's `__init__.py` | Circular import — see the "Avoiding circular imports" subsection above. | Apply strategies 1–3 from that subsection (extract the cyclic part, trim `__init__.py` exports, or restructure to standalone). Strategy 4 (deferred import) only as last resort. | +| `poly check` flags the newly extracted component as not used by any project | The project's base still imports from the residual path (`..`) instead of the new component. | Update the base's imports to the new component's public API, then `POLY_CMD_PREFIX sync --quiet` and re-run check. | +| Verification fails and you can't quickly diagnose | Phase commit not yet made. | `git reset --hard HEAD` to roll back to the previous phase's commit and consult the user. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — split-big-component" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. diff --git a/.agents/skills/polylith/migrate-project/migrate-split-component-internals/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-split-component-internals/SKILL.md new file mode 100644 index 00000000..d9b5f333 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-split-component-internals/SKILL.md @@ -0,0 +1,70 @@ +--- +name: migrate-split-component-internals +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Split monolithic `core.py` files in shared components into domain-focused modules." +--- + +# Skill: migrate-split-component-internals + +> 📐 **Scope vs sibling skills.** This skill operates **inside one already-extracted component**, splitting its `core.py` into multiple sibling Python files. **No new components are created** — only the internal file layout changes. Don't confuse with: +> - `migrate-split-big-component` — splits **one component into multiple components**. Use that when the unit being broken up is a component, not a file. +> - `migrate-isolate-shared-and-project-logic` — separates shared vs project-specific code **across components and projects**; may create new components. +> - `migrate-extract-standalone-modules` — pulls foundational modules out of the residual big component into new standalone components. + +## Goal +Split monolithic `core.py` files in **shared components** (e.g., `models_shared`, `schemas_shared`, or similar) into domain-focused modules. This skill ensures that shared components remain well-organized and maintainable. + +## Inputs +From `migration//state.md`: +- `TARGET_TOP_NS` +- Verification commands. + +From `migration//manifest.md`: +- Current component list and structure. + +> All inputs from `state.md` are assumed to satisfy the validation rules in `migrate-discover` (`### Validation rules`). Validate before proceeding. + +## Steps + +### 1. Identify Candidates +- Scan components for large `core.py` files. +- A component is a candidate if: + - The file contains definitions from multiple domains. + - The file contains helper/utility functions alongside class definitions. + - The file exceeds a reasonable size threshold. + +### 2. Group Definitions by Domain +- Group definitions by the domain concept they serve. +- Example domains: ORM models, schema/dataclass clusters, helper functions. + +### 3. Create New Modules +- Create domain-focused modules (e.g., `merchant.py`, `transaction.py`). +- Move relevant definitions from `core.py` to the new modules. + +### 4. Update `__init__.py` +- Re-export public names from the new modules in `__init__.py`. + +### 5. Handle `core.py` +- Delete `core.py` if all definitions have been moved. +- Keep `core.py` if it serves as a composition point. + +## Verify +- `RUN_TEST_CMD` succeeds. +- If set, `RUN_LINT_CMD` and `RUN_TYPECHECK_CMD` succeed. + +## Common failure modes + +| Symptom | Likely cause | Remediation | +|---------|--------------|-------------| +| `core.py` is small (<100 lines) but mixes two domains | The component itself is too small to warrant an internal split. | Leave it. Splitting adds indirection without benefit at this size; revisit when the file grows. | +| After splitting, an import like `from . import ` fails | `__init__.py` was not updated to re-export the symbol from its new module. | Add `from .. import ` to `__init__.py`. The component's public API must remain stable across the split. | +| New files inside the component now circularly import each other (e.g., `models/user.py` ↔ `models/transaction.py`) | Domain split was too aggressive; the two files genuinely share a concept. | Extract the shared concept into a third file (e.g., `models/_base.py`) and have both depend on it. | + +## Commit + +After verification passes, commit this phase to the migration branch: + +```bash +git add -A && git commit -m "migrate(): phase — split-component-internals" +``` + +Substitute ``, ``, and `` from `state.md` and the orchestrator's phase table. Do not proceed to the next phase without a clean commit — the per-phase commit is the rollback point for the next phase's failure-mode tables. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-update-tests/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-update-tests/SKILL.md new file mode 100644 index 00000000..6e4bf593 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-update-tests/SKILL.md @@ -0,0 +1,69 @@ +--- +name: migrate-update-tests +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Update test files to import from the compatibility shim or the new namespace, ensuring test stability after namespace migration." +--- + +# Skill: migrate-update-tests + +> ⛓ **Conditional phase (4b).** Runs **only when `SHIM_STRATEGY=shim`** (chosen in `migrate-analyze-imports`), to point test imports at the compatibility shim. On the **shimless** path this phase is **skipped** — test imports are rewritten to the new namespace as part of `migrate-automate-import-updates` (phase 4). (Physical test relocation to the workspace happens later in `migrate-prepare-project` / `migrate-refactor-tests`, regardless of strategy.) See the `migrate-orchestrator` workflow. + +## Goal +Update test files to import from the compatibility shim or the new namespace, ensuring that tests remain functional after the namespace migration. + +## Inputs +- Project name (from `migration//state.md`) +- Original namespace (from `migration//state.md`) +- Import analysis report (from `migration//import_analysis.md`) + +## Steps + +### 1. Identify test files importing from the original namespace +1. Review the import analysis report to identify test files (typically in `projects/${PROJECT}/tests/`) that import from the original namespace. + +### 2. Update imports in test files +The compatibility shim re-exports the original namespace's public symbols at the +**top level** of `${ORIG_TOP_NS}` (from `projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py`). +So test imports keep the `${ORIG_TOP_NS}` prefix — only **submodule-qualified** imports +need collapsing onto that top-level surface: + +1. For each test file importing from the original namespace: + - Collapse submodule-qualified imports onto the shim: rewrite + `from ${ORIG_TOP_NS}. import ` to `from ${ORIG_TOP_NS} import ` + (the symbol is re-exported by the shim). + - Leave top-level imports (`from ${ORIG_TOP_NS} import `, `import ${ORIG_TOP_NS}`) + unchanged — they already resolve through the shim. + - If a symbol is **not** re-exported by the shim, either add it to the shim's `__all__` + (see `migrate-generate-shim`) or import it from its new namespace path directly. + +Example: +```python +# Before +from myproject.core import MyClass + +# After (using the compatibility shim) +from myproject import MyClass +``` + +### 3. Verify test discovery +1. Run test discovery to ensure the test count matches the baseline from `migration/${PROJECT}/state.md`. +2. Record any discrepancies in `migration/${PROJECT}/test_updates.md`. + +### 4. Record updated test files +1. Record all updated test files in `migration/${PROJECT}/test_updates.md`. + +## Output +- Updated test files +- A report file: `migration//test_updates.md` listing all updated test files + +## Verify +1. Confirm that all test files listed in `migration//test_updates.md` have been updated. +2. Verify that test discovery produces the expected number of tests (as recorded in `migration//state.md`). +3. Ensure that a representative subset of tests passes by running the test command from `migration//state.md`. + +## Commit +```bash +git add projects/${PROJECT}/tests/ +git add migration/${PROJECT}/test_updates.md +git commit -m "migrate(${PROJECT}): phase — update-tests" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single source of truth) — do not hardcode it. \ No newline at end of file diff --git a/.agents/skills/polylith/migrate-project/migrate-verify-stability/SKILL.md b/.agents/skills/polylith/migrate-project/migrate-verify-stability/SKILL.md new file mode 100644 index 00000000..4fb24835 --- /dev/null +++ b/.agents/skills/polylith/migrate-project/migrate-verify-stability/SKILL.md @@ -0,0 +1,58 @@ +--- +name: migrate-verify-stability +description: "[Internal sub-skill of `migrate-orchestrator`. Do not load directly — load `migrate-orchestrator` first, which drives all phases.] Verify that the migration did not break the project by running test discovery, checking for circular imports, and ensuring the compatibility shim covers all necessary symbols." +--- + +# Skill: migrate-verify-stability + +## Goal +Verify that the migration did not break the project by: +1. Running test discovery and comparing the test count with the baseline +2. Checking for circular imports +3. Ensuring the compatibility shim covers all necessary symbols + +## Inputs +- Project name (from `migration//state.md`) +- Test command (from `migration//state.md`) +- Baseline test count (from `migration//state.md`) + +## Steps + +### 1. Run test discovery +1. Run the test discovery command from `migration/${PROJECT}/state.md`. +2. Compare the test count with the baseline from `migration/${PROJECT}/state.md`. +3. Record any discrepancies in `migration/${PROJECT}/stability_report.md`. + +### 2. Run a subset of tests +1. Run a representative subset of tests using the test command from `migration/${PROJECT}/state.md`. +2. Record any test failures in `migration/${PROJECT}/stability_report.md`. + +### 3. Check for circular imports +1. Verify that no circular imports remain by reviewing `migration/${PROJECT}/circular_imports_resolved.md`. +2. Record any remaining circular imports in `migration/${PROJECT}/stability_report.md`. + +### 4. Verify shim coverage +1. Ensure the compatibility shim (`projects/${PROJECT}/${ORIG_TOP_NS}/__init__.py`) covers all necessary symbols by comparing it with the original namespace's exports. +2. Record any missing symbols in `migration/${PROJECT}/stability_report.md`. + +## Output +- A report file: `migration//stability_report.md` summarizing: + - Test count before and after migration + - Test failures (if any) + - Circular import status + - Shim coverage status + +## Verify +1. Confirm that `migration//stability_report.md` exists and is not empty. +2. Verify that: + - The test count matches the baseline + - No test failures are reported + - No circular imports remain + - The shim covers all necessary symbols + +## Commit +```bash +git add migration/${PROJECT}/stability_report.md +git commit -m "migrate(${PROJECT}): phase — verify-stability" +``` +> `` is this phase's number from the `migrate-orchestrator` table (the single source of truth) — do not hardcode it. \ No newline at end of file diff --git a/projects/polylith_cli/pyproject.toml b/projects/polylith_cli/pyproject.toml index 464b1bf7..05af6b8b 100644 --- a/projects/polylith_cli/pyproject.toml +++ b/projects/polylith_cli/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "polylith-cli" -version = "1.47.2" +version = "1.48.0" description = "Python tooling support for the Polylith Architecture" authors = ['David Vujic'] homepage = "https://davidvujic.github.io/python-polylith-docs/"