Skip to content

Data loss — same table name in two schemas maps to identical Iceberg table; second archive silently destroys first schema's cold data #53

Description

@imtiazqa

Steps to Reproduce:

  1. Create public.events2 and analytics.events2 — identical structure, different payloads (public-N vs analytics-N)
  2. archiver register --schema public --table events2 ...
  3. archiver register --schema analytics --table events2 ...
  4. ./bin/archiver --config cf.yaml

Actual Result:

  • Both schemas map to "ice"."default"."events2" — same Iceberg table
  • Second archive (analytics) runs phase 0 wipe on the shared Iceberg table, destroying public cold data
  • SELECT * FROM public.events2 returns analytics-* rows — wrong schema's data
  • public-* rows are permanently lost from cold storage

Root Cause:

// cmd/archiver/main.go:415
iceTable: pgx.Identifier{"ice", cfg.Iceberg.Namespace, t.SourceTable}.Sanitize()
// schema (t.SourceSchema) is never included
Fix: Include schema in Iceberg table name:

iceTable: pgx.Identifier{"ice", cfg.Iceberg.Namespace, t.SourceSchema + "__" + t.SourceTable}.Sanitize()
// public.events2 → ice.default.public__events2
// analytics.events2 → ice.default.analytics__events2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions