Skip to content

Productionize direct AllThePlaces enrichment after a successful pilot #400

Description

@Medformatik

Provider / source name

Direct AllThePlaces (ATP) business extract; conditional production follow-up to pilot #397.

Provider website

https://www.alltheplaces.xyz/

Domain

POI / places; Knowledge / enrichment

API / data documentation

ATP upstream project, its published run format and data/software license. Pin the exact chosen run/schema/checksum during the pilot; a floating latest URL is not a release contract.

Access requirements

Fully open — no key required

Geographic coverage

Source-dependent business coverage; launch in the pilot region only. This issue does not claim uniform global coverage or guaranteed operating-hours freshness.

Data license & attribution terms

ATP output data is released under CC0; spider software is MIT. Preserve canonical source/run/observation metadata and donor provenance even where attribution is optional. Merged OSM/Overture content retains its own source-specific license/notice requirements. Do not infer permission to make automatic OSM edits from possession of a business fact.

Maps to an existing service?

Extend data-manager/PostGIS ingestion and the existing place/conflation/resolver pipeline. Decide whether the current generic POI-ingest acquisition/staging mechanisms can host this source; a new daemon is not assumed.

Would you like to build it?

Maybe, with some guidance. Conditional backlog; implementation has not started.

Verified need and scope

Rechecked at a8e03bc3efc49207d91f7c1855402c6cf149b8fd; existing issues/discussions contain only pilot #397 for direct ATP. Existing Overture source normalization already credits ATP-derived records, but no direct ATP importer/job is present. Reuse data-manager ingestion contracts, conflation safeguards and Overture fact enrichment, adapting contracts where necessary rather than weakening identity rules.

Productionize only after #397 demonstrates useful correct delta. Add source-aware normalization, staged/atomic generations, repeatable update jobs, field-level donation/conflict policy and rollback. Feed the place identities established by #399. Out of scope: rerunning ATP spiders in the shared API, replacing OSM facts blindly, scraping proprietary providers or expanding to a planet-scale deployment.

Acceptance criteria

  • The pilot's go/no-go decision and selected region/schema are explicit; a no-go leaves production deferred.
  • Acquisition verifies run/checksum/coverage and handles corrupt/changed/missing sources without replacing the previous generation.
  • Updates are idempotent, resumable/cancellable and observable, with bounded resource preflight, retention and rollback.
  • Canonical donor IDs and observation times survive into per-field facts; contradictory/missing/stale facts follow a documented precedence policy.
  • Same-brand branches, tenants/floors, closures and coordinate ambiguity have negative conflation tests and sampled quality thresholds.
  • API/search/details and ambient product resolve the same entity; incremental source changes invalidate affected caches/generations coherently.
  • OSM contribution eligibility remains separate from display enrichment; attribution/provenance and operator docs are updated.
  • Repeat the pilot's useful-place/hour/contact metrics after publication, with no false improvement from raw row counts.

Dependencies and verification

Blocked by positive ATP pilot #397 and the place identity/publication contract in #399. Related #301 (Overture scaling) and #303 (OSM replication) are separate work, not required planet-scale prerequisites. Resolve donor precedence and closure/freshness policy before implementation.

Use staging failure/retry/rollback and real migrated PostGIS tests, provider/conflation negative fixtures, source notices and resolver/cache tests. Run pnpm lint, pnpm check-types, pnpm test, applicable pnpm test:database, pnpm check:policy, and docs build; publish a bounded-region rollout/rollback report.

Activity

  1. added
    enhancementNew feature or request
    integrationNew or changed data provider, source, or backend integration
    needs-triageNeeds initial review and categorization
    on Oct 6, 2026
  2. Medformatik commented on Oct 7, 2026

    @Medformatik
    CollaboratorAuthor

    Overnight dependency check: #397 remains an uncompleted measured pilot, and #399's common place identity/publication work remains open. This issue explicitly requires both a positive ATP pilot and that identity contract before production ingestion.

    Current code already maps ATP-derived Overture donor metadata (packages/core/src/utils/overtureSource.ts) and has staged Overture/conflation mechanisms, but those do not prove a useful incremental direct-ATP delta. I left this conditional production task open and did not start a source job during the at-most-Medium batch. The #397 preflight records the exact-generation and independent-label prerequisites. Resume only after its evidence supports a go and the shared publication identity/rollback contract is established; otherwise retain a no-go/deferred production decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestintegrationNew or changed data provider, source, or backend integrationneeds-triageNeeds initial review and categorization

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions