Skip to content

[Feature]: Adopt the CE-migration ADRs and build the integration-test scaffold (Phase1) #9

Description

@aldbr

User Story

As an interCEde maintainer and contributor,
I want the migration ADRs merged and a shared integration-test scaffold in place,
So that the in-flight backend PRs (#6, #7, #8) harmonize onto one agreed layout instead of
diverging, and every later stack has a template to slot into.

Feature Description

This issue tracks the two foundational PRs and links both:

  1. The ADR PR — docs: ADR for migration #4. Adds docs/adr/ with IC-ADR-001 (Computing Element
    interfaces & the DIRAC → interCEde migration), IC-ADR-002 (Integration testing against
    containerized backends) and IC-ADR-003 (backend credential model — typed credentials,
    declared requirements, provider-supplied), plus the template and index.

  2. The scaffold PR (new, branched off docs: ADR for migration #4 ). Lands the tests/integration/ skeleton
    that IC-ADR-002 describes — the target the three in-flight backend PRs converge onto. Stacked
    on the ADR branch (so its diff is scaffold-only); retargeted to main once docs: ADR for migration #4 merges.

    Scaffold contents (Phase 0 — native-binary friendly, no dependency on the not-yet-existing
    intercede package):

    • tests/integration/stacks.yml — manifest seeded with the three stacks' entries so each author
      sees where their line goes;
    • tests/integration/README.md — the harmonization guide / stack contract + checklist;
    • tests/integration/stacks/_template/compose.yml skeleton (network, credentials volume,
      HEALTHCHECK block) + config/basic/intercede-client.toml template;
    • tests/integration/stacks/_images/ — Dockerfile conventions (ARG + # renovate: annotation);
    • tests/integration/conftest.py — thin --stack/--config options + capability-skip helper;
    • integration-test markers registered in pyproject.toml
      (remote scheduler arc htcondor slurm ssh destructive_fetch);
    • a skeleton .github/workflows/integration.yml that reads stacks.yml, so authors can
      delete their bespoke per-PR workflows against a real target.

Once this lands, harmonizing a stack = move files under stacks/<id>/ + add a manifest entry +
follow the checklist
— not "invent structure" or "rewrite against interCEde".

Definition of Done

  • PR docs: ADR for migration #4 merged — docs/adr/{IC-ADR-001,IC-ADR-002,IC-ADR-003,IC-ADR-XXX_template,index} on main
  • Scaffold PR merged (based on adr_migration, retargeted to main after docs: ADR for migration #4)
  • tests/integration/ skeleton in place: stacks.yml (3 seeded stacks), README.md
    checklist, stacks/_template/, stacks/_images/ conventions, conftest.py, markers in
    pyproject.toml, skeleton integration.yml
  • A stack is runnable locally with the documented two-liner
    (docker compose … up --wait + pixi run pytest … --stack=<id>)
  • Harmonization guide linked from [Feature]: Implement an HTCondor CI Workflow #5, #, # so all three authors
    work from the same target
  • Follow-up issues filed for the deferred consolidation: manifest-driven CI matrix (fleshing
    out integration.yml), prebuilt GHCR images, Renovate two-lane + version window, and the
    arc-condor / ssh-condor stacks

Alternatives Considered

  • Merge the three prototypes as-is, consolidate later. Rejected: without a shared target they
    re-diverge on the exact axes they already differ on (systemd vs foreground, Podman vs Docker,
    Ubuntu vs EL9), making the later consolidation larger than doing the scaffold first.
  • Fold the scaffold into the ADR PR (docs: ADR for migration #4). Rejected: keeps the ADR reviewable as pure docs and
    lets the scaffold evolve without re-opening the ADR; the stacked-PR link keeps them associated.
  • Full protocol-based contract suite now. Deferred — the intercede Protocols
    (IC-ADR-001) don't exist yet, so Phase 0 tests use native client tools.

Related Issues

Implemented by: #4 (ADR docs) and the new scaffold PR
Related to: #5 (htcondor stack), # (PR #7), # (PR #8) — all three
harmonize onto this scaffold
Blocks: #5, #, # (they need the scaffold as their target)

Additional Context

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions