Develop v06#42

Open

stevenhsd wants to merge 24 commits intomainfrom

Commits on Jan 24, 2026

build: add flexible versions to most core dependencies and removed some legacy dependencies
georgeRobertson2
committed

Commits on Jan 26, 2026

feat: Change how error messages are generated (by writing in batches). Duckdb no long relies on pandas- use of pyarrow, multiprocessing and background thread batch writing to avoid memory pressure
stevenhsd
committed

Commits on Jan 27, 2026

style: run black, isort and resolve linting issues
stevenhsd
committed

Commits on Jan 28, 2026

Commits on Feb 3, 2026

refactor: merge in main and resolve conflicts and linting issues
stevenhsd
committed

Commits on Feb 6, 2026

Commits on Feb 9, 2026

feat: added reference data loading of arrow ipc files including enhanced test coverage for reference data loaders
stevenhsd
committed

Commits on Feb 10, 2026

Commits on Feb 11, 2026

refactor: amended process pool to use executor supplied to duckdb pipeline/data contract rather than always instantiating new pool
stevenhsd
committed

Commits on Feb 12, 2026

Merge pull request #37 from NHSDigital/feature/ndit-655_refdata_arrow_integration
stevenhsd
authored

Commits on Feb 13, 2026