UN-3479 [FEAT] SDK subpackage for org-to-org data migration

[chandrasekharan-zipstack]

Summary

New unstract.migration sub-package — moves an org's configured resources (adapters, connectors, tags, prompt-studio projects, workflows, tool instances, workflow endpoints, ETL/TASK pipelines, API deployments) into another org via two admin-issued Platform API keys. Works cloud↔cloud, on-prem↔on-prem, or within the same deployment.

Tracked in UN-3479. Backend companion: Zipstack/unstract#1987.

Architecture

• OrgEndpoint — base URL + org slug + Platform API key + configurable api_path_prefix (api/v1 default).
• PlatformClient — entity-scoped HTTP wrapper; Bearer auth; DRF-OPTIONS-driven writable-field set via get_post_schema(entity_path).
• MigrationContext — bundles source/target clients, options, per-run RemapTable (source UUID → target UUID).
• Phase base + per-entity subclasses; each phase: list → per-id GET → POST (or adopt-by-name).
• walker.remap_uuids — JSON walker that rewrites embedded UUIDs using RemapTable.resolve_any before POST.
• MigrationReport — rich table summary + Source → Target UUID Map for audit.
• click CLI: unstract-migrate migrate --source-url ... --source-org ... --target-url ... --target-org ... with env-var fallback for keys, plus --include / --exclude for phase selection and --dry-run.

Phase order is owned by orchestrator.PHASES — phases never call each other:

adapter → connector → tag → custom_tool → workflow → tool_instance
       → workflow_endpoint → pipeline → api_deployment

Phases

Phase	Notes
Adapter	Skips is_friction_less=True. Encrypted adapter_metadata_b survives via per-id GET.
Connector	Same list→GET→POST pattern. Adopt-by-name idempotent.
Tag	Backend now exposes POST/PATCH/DELETE; SDK creates tags on target.
CustomTool	Composite — tool + profile_managers + prompts + republish to registry. Resolves profile adapters by name (UUIDs differ across orgs).
Workflow	tool_metadata UUID-walk during remap.
ToolInstance	≤1 per workflow; FK-rewrites via remap.
WorkflowEndpoint	Rewrites connector + workflow refs.
Pipeline	ETL + TASK only. DEFAULT (legacy v1) and APP (Streamlit) skipped. Warns when source has >1 active key.
APIDeployment	Adopts by api_name. Same key-warning behaviour. Sequenced after workflow_endpoint (serializer requires configured source+destination endpoints).

Constraints (from design ADRs)

• Secrets carried verbatim through the SDK (same posture as the FE; ADR 0006).
• No local state file (ADR 0007); idempotency = name-based GET against target.
• No UUID preservation; embedded refs rewritten via the walker.
• No DB / schema migrations.

Carry-forward gotchas

1. List endpoints often serve stripped payloads (e.g. AdapterListSerializer omits adapter_metadata_b) — every phase does list → per-id GET before POST.
2. PATH_PREFIX is env-controlled on the backend; --api-prefix flag exposed.
3. AuditSerializer auto-adds creator to shared_users M2M — relevant for any cleanup scripting.
4. Service-account is a real User row (is_service_account=True) + OrganizationMember.
5. Backend auto-provisions one active API key per Pipeline/APIDeployment on POST; key UUIDs are editable=False. SDK does not preserve source key values — operator must rotate. Warning is emitted when source has >1 active key.

Test plan

• 56 unit tests across all phases (happy / dry-run / adopt / abort / extra-keys-warning paths) + remap table + walker + HTTP client + CLI.
• Local end-to-end smoke against docker stack (org_Q4qgjLWIbaJlfSts → org_migration_target) — full pipeline creates on first run, adopts idempotently on re-run.
• Cloud↔cloud smoke against staging before un-drafting.
• On-prem↔on-prem smoke before un-drafting.

Out of scope (separate work)

• shared_to_org / shared_users propagation (resource-ownership semantics).
• Prompt-studio uploaded document blobs (file migration).
• API-key value preservation (would need backend design change).
• Pipeline types DEFAULT (legacy) and APP (Streamlit).

Related

• JIRA: UN-3479
• Backend PR: Zipstack/unstract#1987 (feat/org-migration-platform-api-gaps)
• KB (internal): ~/Documents/Obsidian Vault/zipstuff/org-data-migration/

[greptile-apps]

Greptile Summary

This PR introduces the unstract.clone sub-package, a CLI and SDK for migrating configured org resources (adapters, connectors, tags, prompt-studio projects, workflows, tool instances, endpoints, pipelines, API deployments) between Unstract orgs via Platform API keys. It's a well-structured, phased migration tool with a solid test suite (56 unit tests) and clear idempotency semantics.

• Architecture: OrgEndpoint + PlatformClient + CloneContext/RemapTable + per-entity Phase subclasses, orchestrated in strict topological order; remap_uuids walker rewrites embedded UUIDs before each POST.
• Correctness improvements already landed (in prior commits on this branch): build_post_payload preserves False/0, pipeline/api_deployment phases now do per-id GET before POST, workflow_endpoint skips rather than silently detaches on unresolvable connectors, dry-run adopt path no longer fires export_custom_tool, list_custom_tools hoisted above loop in CustomToolPhase.
• One gap remains: WorkflowPhase builds its POST payload directly from the list-endpoint summary — the only create-path phase that skips the get_workflow(src_id) detail fetch — making it susceptible to the same field-stripping problem that was already fixed for pipelines and API deployments.

Confidence Score: 4/5

Safe to merge after confirming whether the workflow list serializer includes source_settings/destination_settings; if it does, the current code is correct; if not, workflows would be created with stripped connector settings.

The package is well-designed with a thorough test suite and most previously identified correctness gaps already addressed. The one remaining concern is that WorkflowPhase builds its POST payload from the list-endpoint summary without calling get_workflow(src_id) first — the only create-path phase in the codebase that follows this pattern. Given the PR's own guidance that list serializers strip fields, and that the two flagged fields (source_settings, destination_settings) embed connector UUIDs that also need UUID-walking, a silent miss here would result in workflows created without their connector configuration.

src/unstract/clone/phases/workflow.py — confirm the backend workflow list serializer includes all fields needed for POST (especially source_settings and destination_settings) or add a per-id GET call.

Important Files Changed

Filename	Overview
src/unstract/clone/phases/workflow.py	Migrate workflows; uses list-response payload directly without a per-id GET, which is inconsistent with the stated pattern and could silently omit fields stripped by the list serializer.
src/unstract/clone/phases/files.py	Prompt-Studio document file migration with retry logic; calls list_custom_tools() once per tool inside the main loop for a cosmetic log-only lookup, causing N redundant API requests.
src/unstract/clone/phases/custom_tool.py	Prompt-Studio project migration; dry-run adopt path fixed (no longer fires export_custom_tool), target tool list hoisted above loop, adapter fallback UUID dropped.
src/unstract/clone/phases/pipeline.py	ETL/TASK pipeline migration; now does per-id GET before POST (fixing earlier list-serializer gap), correctly skips DEFAULT/APP types and warns on extra source API keys.
src/unstract/clone/phases/api_deployment.py	API deployment migration; per-id GET added before POST, adopt-by-name idempotency, extra-key warning present.
src/unstract/clone/phases/workflow_endpoint.py	Workflow endpoint PATCH migration; fixed to skip instead of silently detach when source connector has no remap, null connection_type no longer coerced to empty string.
src/unstract/clone/phases/base.py	Base phase helpers; build_post_payload correctly preserves False and 0 after the prior fix, server-managed fields stripped universally.
src/unstract/clone/client.py	Thin HTTP client scoped per org; OPTIONS-driven schema caching, entity-scoped methods, Bearer auth, clean context-manager lifecycle.
src/unstract/clone/walker.py	JSON UUID walker; cleanly rewrites UUID-shaped strings via RemapTable, unknown UUIDs pass through, dict keys not walked (correct for JSON field names).
src/unstract/clone/phases/tool_instance.py	ToolInstance migration; properly detects broken adapter sentinels, strips source identity fields, two-step POST+PATCH pattern correctly mirrors backend behavior.
src/unstract/clone/orchestrator.py	Top-level clone() entry point; strict topological phase order, client lifecycle managed in finally block, CloneError aborts cleanly.
src/unstract/clone/context.py	Shared state dataclasses; RemapTable correctly implements resolve_any for the UUID walker, OrgEndpoint is frozen, CloneContext bundles clients and remap.

Sequence Diagram

sequenceDiagram
    participant CLI
    participant Orchestrator
    participant Phase
    participant SrcClient as Source PlatformClient
    participant TgtClient as Target PlatformClient
    participant RemapTable

    CLI->>Orchestrator: clone(src_endpoint, tgt_endpoint, options)
    Orchestrator->>SrcClient: create (Bearer key)
    Orchestrator->>TgtClient: create (Bearer key)

    loop For each Phase in topological order
        Orchestrator->>Phase: run(report)
        Phase->>TgtClient: OPTIONS entity/ → get_post_schema()
        Phase->>SrcClient: list_entities()
        loop For each source entity
            Phase->>SrcClient: get_entity(id) [detail fetch]
            Phase->>TgtClient: "list_entities(name=...) [adopt check]"
            alt Entity exists on target
                Phase->>RemapTable: record(src_id, tgt_id)
            else dry_run
                Note over Phase: skip, no write
            else create
                Phase->>RemapTable: remap_uuids(payload)
                Phase->>TgtClient: create_entity(payload)
                Phase->>RemapTable: record(src_id, tgt_id)
            end
        end
    end
    Orchestrator->>CLI: CloneReport (counts + remap snapshot)

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
src/unstract/clone/phases/workflow.py:81-83
**Workflow payload built from list response, not per-id GET**

`WorkflowPhase._clone_one` applies `remap_uuids` and `build_post_payload` directly to the list-endpoint summary (`src`), skipping the `get_workflow(src_id)` call that every other phase in this package performs for the same reason. The PR's own "carry-forward gotcha #1" states: _"List endpoints often serve stripped payloads … every phase does list → per-id GET before POST."_ If the backend's workflow list serializer strips `source_settings` or `destination_settings` (the two fields called out as non-trivial in this module's own docstring), those blobs — and any connector UUIDs inside them that need UUID-walking — would be silently absent from the POST, creating a misconfigured workflow on the target. `PipelinePhase` and `APIDeploymentPhase` had identical gaps that were fixed in 9391ce3; `WorkflowPhase` appears to have been missed.

### Issue 2 of 2
src/unstract/clone/phases/files.py:70-71
`_lookup_tool_name` fires a full `list_custom_tools()` API call on every loop iteration — one call per tool in `tool_remap`. For N tools this means N round-trips that each fetch the entire tool list, just to produce a name for log messages. The same antipattern was identified and fixed in `CustomToolPhase` (hoisted to before the loop). Caching the list once before the loop removes the redundant requests.

```suggestion
        try:
            _all_tgt_tools = self.ctx.target.list_custom_tools()
        except Exception:
            _all_tgt_tools = []
        _tgt_name_by_id = {t["tool_id"]: t["tool_name"] for t in _all_tgt_tools}

        for src_tool_id, tgt_tool_id in tool_remap.items():
            tool_name = _tgt_name_by_id.get(tgt_tool_id) or src_tool_id
```

_{Reviews (11): Last reviewed commit: "fix(clone): address PR #15 review feedba..." | Re-trigger Greptile}

[jaseemjaskp]

Automated multi-agent review of the clone subpackage (Code Reviewer, Silent-Failure Hunter, Type-Design, Test Analyzer, Comment Analyzer, Code Simplifier). Findings below are grouped by file; severity is tagged inline (P1 highest). Findings already covered by existing review comments (e.g. the greptile dry-run write on custom_tool.py:180, the _extract_adapter_name / list_custom_tools / base.build_post_payload / workflow_endpoint items) are intentionally omitted as fixed/duplicate. Tests pass (75 green).

[jaseemjaskp]

P1 — dry-run performs a real PATCH on the adopt path. The dry_run guard is only the elif at line 132, which is reachable just when not existing. When the target workflow already has a tool instance (the adopt path, if existing: at line 125), dry_run=True still falls through to update_tool_instance_metadata at line 175 — a real write during a dry run. The existing dry-run test seeds an empty target so it never exercises this. This is the same bug class as the greptile finding on custom_tool.py:180.

Fix: short-circuit before the metadata PATCH when dry-run, e.g. once existing is determined add if self.ctx.options.dry_run: result.skipped += 1; return (and don't bump adopted), or guard the PATCH block (lines 158-184) with if not self.ctx.options.dry_run:.

[chandrasekharan-zipstack]

Fixed in 9391ce3. Adopt branch now checks dry_run before falling through to the PATCH. Records the remap (so downstream dry-run output stays coherent) and returns. New test_dry_run_on_adopt_path_does_not_repatch_target exercises this.

[jaseemjaskp]

P1 — malformed source DM rows are silently dropped. If a row lacks document_name/document_id (renamed field, partial serializer response), the bare continue skips it with no counter, no errors entry, and no warning — indistinguishable from "nothing to do". The operator sees a green run while a document never migrated.

Fix: increment result.failed (or at least result.skipped) and append to result.errors/report.failed_files, e.g. result.errors.append(f"src DM row for tool {tool_name} missing document_id/name: keys={list(doc)}") before continuing.

[chandrasekharan-zipstack]

Fixed in 9391ce3. Malformed source DM rows now bump result.skipped and append an error entry (and warn-level log). New test_malformed_source_dm_row_bumps_skipped_with_error.

[jaseemjaskp]

P1 — unsupported-mime and oversize files are recorded but never counted, so the run still reports success. Both this unsupported_files.append(...) path (line 189) and the oversize_files.append(...) path (line 200) return without touching result.failed/result.skipped. The report's headline verdict (report.py:108-114) is driven solely by totals["failed"], so a run that left every Excel/oversize test doc behind still prints Completed successfully in green — contradicting the unsupported_files/oversize_files sections it also renders.

Fix: result.skipped += 1 on both paths, and/or have render() factor non-empty unsupported_files/oversize_files/failed_files into the final verdict line.

[chandrasekharan-zipstack]

Fixed in 9391ce3. Both unsupported_files and oversize_files paths now bump result.skipped so the phase counters surface them. Existing oversize/unsupported tests extended with explicit result.skipped == 1 assertions. Note: I chose skipped over failed because these are known limitations the operator chose (cap setting, mime support), not transport failures — the per-file report list still carries the details.

[jaseemjaskp]

P2 — registry-remap lookup failure is swallowed at WARNING with no counter. A failed list_registries on source/target logs a warning and returns, leaving the prompt_studio_registry remap unrecorded. Downstream ToolInstancePhase then hits if not tgt_tool_id (tool_instance.py:108) and silently skips the tool instance — the workflow lands with no tool wired, and both phases report zero failures. A genuine API/network error is thereby indistinguishable from the legitimate "tool unpublished on source" case.

Fix: append to result.errors (and consider result.failed += 1) here so the lookup failure is visible in the structured report, distinct from the unpublished-tool path.

[chandrasekharan-zipstack]

Fixed in 9391ce3. The exception branch now bumps result.failed and appends registry remap lookup {tool_name}: {e} so the operator sees the partial-success in the phase counters.

[jaseemjaskp]

P2 — resolve_any can silently rewrite a UUID to an unrelated target id. remap_uuids resolves any UUID-shaped string against all entity maps and returns the first hit (context.py:78-83). A UUID embedded in a JSON blob (tool_instance metadata, workflow settings, endpoint config) that is not actually a foreign key — or that collides across entity maps — gets rewritten to an unrelated target UUID, producing a structurally valid payload pointing at the wrong row, with no log at all.

Fix: at minimum logger.debug each rewrite (which entity matched, src→tgt). Better: have known-FK fields pass an expected entity type, and have resolve_any warn when a UUID matches in more than one entity map.

[chandrasekharan-zipstack]

Acknowledged but deferring. UUID collisions across entity tables are astronomically unlikely in practice (UUIDv4 birthday bound), and the walker's contract is documented as 'first hit wins'. The right fix is to make the walker entity-aware at each call site rather than scoping resolve_any — that's a broader refactor I'd like to do in a follow-up rather than as part of this review. Tracking as future work.

[jaseemjaskp]

Test gap (P1) — clone() is never exercised end-to-end. Untested: phase ordering, opts.includes() include/exclude → skipped_phases, the CloneError → abort/break path (does an abort actually halt subsequent phases?), and remap_snapshot population on the returned report. This is the integration seam the whole package hangs on.

Suggest a test that monkeypatches PHASES with two trivial fake phases (one raising NameConflictError) and asserts the second never runs, report.aborted is True, abort_reason is set, and --exclude lands a name in skipped_phases.

[chandrasekharan-zipstack]

Addressed in 9391ce3. New tests/clone/test_orchestrator.py (6 tests) covers: phase ordering, include/exclude → skipped_phases, CloneError abort + remaining phases skipped, non-CloneError propagation, client close on both paths, remap snapshot on the report.

[jaseemjaskp]

Type design (P2) — the five file-row buckets are loose list[dict[str, Any]] with divergent, implicit schemas. Each is built ad hoc in files.py with a different key set (uploaded_files has size_bytes+mime_type; oversize_files adds cap_bytes; failed_files has error), and the reader _describe_file_row defensively .get()s keys that may or may not exist. A typo in a producer key surfaces as "?" in the report rather than failing anywhere.

Suggest a small frozen dataclass FileOutcome(tool_id, file_name, tool_name=None, size_bytes=None, mime_type=None, error=None) for these fields; as_dict() becomes asdict(x) and the defensive .get()s in the renderer go away.

[chandrasekharan-zipstack]

Fair point. Dataclasses for the five buckets would tighten the contract, but the report layer is read-only (built once, rendered once) and the loose dict shape isn't currently a foot-gun anywhere. Tracking as a follow-up if/when a consumer outside the renderer starts depending on these structures.

[jaseemjaskp]

Type design (P3) — closed option sets use magic strings. on_name_conflict ("adopt"/"abort") and file_strategy ("platform_api"/"skip") are compared by string literal far away (adapter.py:74 == "abort", files.py:84 == "skip"), so a typo on either side is undetectable until runtime and the CLI must re-validate by hand.

Suggest class NameConflict(str, Enum) / class FileStrategy(str, Enum) — str-based keeps CLI/JSON ergonomics while making illegal values unconstructable and the comparison sites self-documenting.

[chandrasekharan-zipstack]

Click's type=Choice(...) already validates at the CLI boundary and the option-set is small (4 strings across 2 fields). A StrEnum would tighten the in-process checks but the readability win is marginal at this scale. Deferring.

[jaseemjaskp]

Simplification (P2) — six phases duplicate the same run() preamble and adopt/create skeleton. adapter, connector, tag, workflow, pipeline, api_deployment each repeat: try get_post_schema(PATH) → on error append OPTIONS <name> and return; try list_<x>() → on error append list source <x> and return; loop. The adopt/dry-run/create branch (with NameConflictError raise + remap.record) is likewise near-identical, differing only in the create call and entity label.

Suggest hoisting _fetch_writable(path, result, label) and _adopt_or_create(...) helpers onto this base class — removes ~15 duplicated lines per module across six phases while preserving the (self.name-derived) error strings. Lower priority: _warn_if_extra_source_keys is byte-identical in pipeline.py and api_deployment.py and could also move here.

[chandrasekharan-zipstack]

Acknowledged. The per-phase divergence (CustomTool composite, ToolInstance two-step POST+PATCH, Files DM-row enumeration, WorkflowEndpoint PATCH-not-POST) makes a single run() template fragile — three of the ten phases would need template hooks. I'd rather wait until the divergent phases stabilize before extracting. Tracking for a future cleanup.

[jaseemjaskp]

Comment accuracy (P3) — this explanatory comment is self-contradicting and hard to parse. "0 in (None, "") is False, but 0 not in (...) falsely returns True" — those two statements are consistent, not contradictory, so the comment muddles the reasoning behind the (correct) code below it.

Suggest: "Use is not None and != '' (not membership in (None, '')) so meaningful falsy values like False and 0 survive — membership tests against (None, '') are easy to get subtly wrong."

[sergiojrdotnet]

Addressed in 2b80d2a0 — the test now covers 9 behavioral scenarios: confirm-path (resolve via searchPersons → flatten → POST bulk-assign with the correct persona ids), already-tagged-filter (drops personas carrying the target tag), already-assigned-set filter, all-pre-tagged short-circuit (no POST + friendly toast + onComplete not called), partial-failure pass-through to onComplete, bulk-assign 500 error path, and search 500 error path. Uses the same vi.mock approach the SearchBuilderDrawer test uses for PersonSearchPicker — mocks the drawer to expose onConfirm deterministically without coupling to its internal filter/review wiring.

[greptile-apps]

Using or "" coerces a None connection_type to an empty string. If the backend's connection_type is a DRF ChoiceField (typical values are "filesystem", "api", etc.), sending "" will be rejected with a 400 error because allow_blank is False by default. Pass None when the field is absent so the target can apply its own default, or omit the key entirely from the PATCH.

Suggested change

	payload: dict[str, Any] = {
	"connection_type": src_ep.get("connection_type") or "",
	"configuration": remap_uuids(src_ep.get("configuration") or {}, self.ctx.remap),
	"connector_instance_id": tgt_conn_id,
	}
	payload: dict[str, Any] = {
	"configuration": remap_uuids(src_ep.get("configuration") or {}, self.ctx.remap),
	"connector_instance_id": tgt_conn_id,
	}
	if src_ep.get("connection_type"):
	payload["connection_type"] = src_ep["connection_type"]

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/unstract/clone/phases/workflow_endpoint.py
Line: 139-143

Comment:
Using `or ""` coerces a `None` `connection_type` to an empty string. If the backend's `connection_type` is a DRF `ChoiceField` (typical values are `"filesystem"`, `"api"`, etc.), sending `""` will be rejected with a 400 error because `allow_blank` is `False` by default. Pass `None` when the field is absent so the target can apply its own default, or omit the key entirely from the PATCH.

```suggestion
        payload: dict[str, Any] = {
            "configuration": remap_uuids(src_ep.get("configuration") or {}, self.ctx.remap),
            "connector_instance_id": tgt_conn_id,
        }
        if src_ep.get("connection_type"):
            payload["connection_type"] = src_ep["connection_type"]
```

How can I resolve this? If you propose a fix, please make it concise.

[chandrasekharan-zipstack]

Fixed in 9391ce3. Dropped the or "" coercion; connection_type is now only added to the PATCH payload when source has a non-None value. New test_endpoint_with_null_connection_type_omits_key_in_payload locks the regression.

[greptile-apps]

Bare return breaks the PhaseResult contract

Line 83 returns None from a method declared -> PhaseResult. Every other early-exit in this file (e.g. the matching return result on line 70) and in every other run() method throughout the codebase returns result. If the target list call fails, the orchestrator gets None back — and any caller that relies on the return value would receive the wrong type. The fix is return result.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/unstract/clone/phases/custom_tool.py
Line: 83

Comment:
**Bare `return` breaks the `PhaseResult` contract**

Line 83 returns `None` from a method declared `-> PhaseResult`. Every other early-exit in this file (e.g. the matching `return result` on line 70) and in every other `run()` method throughout the codebase returns `result`. If the target list call fails, the orchestrator gets `None` back — and any caller that relies on the return value would receive the wrong type. The fix is `return result`.

How can I resolve this? If you propose a fix, please make it concise.

[chandrasekharan-zipstack]

Same root cause as #3296606829 — fixed in 9391ce3 by returning result from the phase-init error branch.

[greptile-apps]

Pipeline and API-deployment phases skip the per-id GET, risking incomplete payloads

The PR's own "carry-forward gotcha #1" documents that list endpoints serve stripped payloads and that every phase should follow the list → per-id GET → POST pattern. AdapterPhase and ConnectorPhase both call get_adapter/get_connector before building the POST payload; PipelinePhase and APIDeploymentPhase build the payload directly from the list summary using remap_uuids(src, ...) / build_post_payload(remapped, ...). Fields absent from the list serializer (e.g. cron_string for TASK pipelines, or detailed scheduler config) would be silently omitted from the POST, creating a misconfigured entity on the target. The client already exposes get_pipeline and get_api_deployment — they should be called before constructing the payload, mirroring the adapter/connector pattern.

The same issue applies at api_deployment.py line 106–108.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/unstract/clone/phases/pipeline.py
Line: 109-111

Comment:
**Pipeline and API-deployment phases skip the per-id GET, risking incomplete payloads**

The PR's own "carry-forward gotcha #1" documents that list endpoints serve stripped payloads and that every phase should follow the `list → per-id GET → POST` pattern. `AdapterPhase` and `ConnectorPhase` both call `get_adapter`/`get_connector` before building the POST payload; `PipelinePhase` and `APIDeploymentPhase` build the payload directly from the list summary using `remap_uuids(src, ...)` / `build_post_payload(remapped, ...)`. Fields absent from the list serializer (e.g. `cron_string` for TASK pipelines, or detailed scheduler config) would be silently omitted from the POST, creating a misconfigured entity on the target. The client already exposes `get_pipeline` and `get_api_deployment` — they should be called before constructing the payload, mirroring the adapter/connector pattern.

The same issue applies at `api_deployment.py` line 106–108.

How can I resolve this? If you propose a fix, please make it concise.

[chandrasekharan-zipstack]

Fixed in 9391ce3. Both PipelinePhase and APIDeploymentPhase now do get_pipeline/get_api_deployment per source id before remapping + posting, so list-only serializer omissions can't strip required fields. New test_create_uses_per_id_get_not_stripped_list_payload in the pipeline tests proves the path.