Skip to content

repository: single config/config text object (repo version 5) - #10377

Open
ThomasWaldmann wants to merge 1 commit into
borgbackup:masterfrom
ThomasWaldmann:repo-config-file
Open

ThomasWaldmann wants to merge 1 commit into
borgbackup:masterfrom
ThomasWaldmann:repo-config-file

Conversation

@ThomasWaldmann

@ThomasWaldmann ThomasWaldmann commented Sep 16, 2026

Copy link
Copy Markdown
Member

Follow-up to #10371 .. #10374, which emptied the manifest. Opening a repository loaded four store objects: config/readme, config/version, config/id and config/manifest (only read for its key type byte). Now it loads one:

# This is a Borg Backup repository.
# See https://borgbackup.readthedocs.io/

[repository]
version = 5
id = <64 hex digits>
encryption = aes256-ocb
id_hash = sha256

The config object

  • config/config is a plain text INI object (configparser), the readme text lives on as comment lines. encryption / id_hash record the crypto suite by the --encryption / --id-hash names; key_factory() selects the key class from them without reading any repository object. Both names are recorded or none: a config with one but not the other is invalid, an unsupported suite raises InvalidRepositoryConfig. Key storage (keyfile vs repokey) stays a property of each key.
  • Repository version 5, only 5 accepted. No code for the old layout, no migration (betas are for new repos only): an older beta repository fails to open with "repository version 4 is not supported".

No manifest object for borg 2 repositories. Manifest is only the in-memory container for key, repo_objs, repository and archives. Removed: write() and its callers, get_manifest() / put_manifest(), the manifest checks and rebuild in borg check, borg debug dump-manifest, key_from_repository() and the ROBJ_MANIFEST type (no stored object ever had it, borg 1.x objects carry no type at all). borg 1.x repositories are still read via their manifest (legacy_key_factory), since it holds their archives list. NoManifestError stays defined (never raised) so rc 26 remains reserved.

repo-create leaves nothing behind when it fails. Repository.create() creates all namespace directories in advance (store.create_levels()) and writes the config (version, id, no key info) by default, so a repository created via the Python API is an openable key/value store. borg repo-create alone uses create_config=False and writes the config once, last, after the key exists, so nothing that looks like a repository exists until it succeeds: any failure or interruption before that destroys the store and, in keyfile mode, the keyfile just written; a failure inside create() itself destroys the store as well.

A store without config (only possible after a hard kill, or a repository that lost its config):

  • on open it is reported as not a valid repository (InvalidRepository); DoesNotExist is only for a missing store backend, the rest:// case from repo-info on a non-existent remote repo raises BackendDoesNotExist #10365;
  • repo-create on it raises the new IncompleteRepository error (rc 11, "has no repository config"), as the store backend refuses any non-empty directory;
  • borg repo-delete --force destroys it (without --force: "Deleting this requires the --force option"), so that ssh/rest/s3 users can remove it without other access to the storage, but only if it has the packs, archives, index and config namespaces every store borg created has: a data directory, a home directory or a borg 1.x repository is never destroyed by borg;
  • key_factory() on a config without key info raises RepositoryKeyInfoMissing (rc 54).

Cache config: the [cache]/[integrity] cross-check on the manifest id could never detect anything (every borg that can open the repository knows the integrity section) and is dropped, together with its warning and test.

Docs: a new "Repository config" section replaces the manifest section, including how to recreate a lost config by hand (check --repair can not restore it); the layout list, packs and security internals (where the protection against a swapped crypto suite lives: the security directory and the unknown-unencrypted prompt for swaps to a non-encrypting suite, the security directory for swaps between encrypting suites), the repo-info example, the error list (rc 11, 54) and the remaining manifest mentions are updated. Sphinx builds without warnings.

Tests: manifest-specific tests removed; new tests for the config round trip, a store without config (open: invalid, create: incomplete, repo-delete --force: destroyed), a repository that lost its config (invalid, destroyed with --force), a data directory (refused by repo-delete --force), bad configs (wrong version, no section, missing / invalid / half-present entries, not text), key_class_for over all creatable suites, repo-create failing while writing the config or inside create() leaving neither store nor keyfile behind, and a plain data directory refused by repo-create and repo-delete --force.

Full suite passes locally (FUSE mount tests deselected, macFUSE is unavailable on this machine).

🤖 Generated with Claude Code

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.64706% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.10%. Comparing base (21caafd) to head (77db875).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/borg/repository.py 93.81% 5 Missing and 1 partial ⚠️
src/borg/archiver/repo_delete_cmd.py 83.33% 2 Missing and 1 partial ⚠️
src/borg/archiver/_common.py 75.00% 1 Missing and 1 partial ⚠️
src/borg/cache.py 50.00% 1 Missing and 1 partial ⚠️
src/borg/archiver/check_cmd.py 83.33% 1 Missing ⚠️
src/borg/archiver/debug_cmd.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10377      +/-   ##
==========================================
- Coverage   88.15%   88.10%   -0.05%     
==========================================
  Files         103      103              
  Lines       18879    18885       +6     
  Branches     2917     2929      +12     
==========================================
- Hits        16642    16638       -4     
- Misses       1554     1562       +8     
- Partials      683      685       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread docs/internals/packs.rst Outdated
Comment thread src/borg/crypto/key.py Outdated
Comment thread src/borg/legacy/archives.py
Comment thread src/borg/testsuite/archiver/compact_cmd_test.py Outdated
Comment thread src/borg/repository.py
@ThomasWaldmann
ThomasWaldmann marked this pull request as draft September 16, 2026 04:31
@ThomasWaldmann ThomasWaldmann changed the title repository: single config/config text object replaces readme, version, id and manifest (repo version 5) repository: single config/config text object (repo version 5) Sep 16, 2026
@ThomasWaldmann
ThomasWaldmann force-pushed the repo-config-file branch 3 times, most recently from c7a58a1 to 285c9f5 Compare September 16, 2026 17:18
Comment thread docs/internals/data-structures.rst Outdated
Comment thread docs/internals/packs.rst Outdated
Comment thread docs/internals/packs.rst Outdated
Comment thread docs/internals/packs.rst Outdated
Comment thread docs/internals/security.rst Outdated
Comment thread src/borg/archiver/repo_delete_cmd.py Outdated
Comment thread src/borg/archiver/repo_delete_cmd.py Outdated
Comment thread src/borg/archiver/repo_delete_cmd.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
…, id and manifest (repo version 5)

Opening a repository loaded four store objects: config/readme (sanity check),
config/version, config/id and config/manifest (only read for its key type
byte since the manifest lost all other content). Now there is one plain text
INI object, config/config, e.g.:

    # This is a Borg Backup repository.
    # See https://borgbackup.readthedocs.io/

    [repository]
    version = 5
    id = <64 hex digits>
    encryption = aes256-ocb
    id_hash = sha256

The crypto suite of the key is recorded by the --encryption / --id-hash
names, so key_factory() selects the key class from the config without
reading any repository object. Both names are recorded or none; a config
with one but not the other is invalid, an unsupported suite raises
InvalidRepositoryConfig. The repository version is 5; only version 5 is
accepted, there is no code to read the old layout and no migration (betas
are for new repositories only).

The manifest object is gone for borg 2 repositories: Manifest is now only
the in-memory container for key, repo_objs, repository and archives; its
write() and all callers, get_manifest()/put_manifest(), the manifest checks
and rebuild in "borg check", "borg debug dump-manifest", the key type
detection from stored objects (key_from_repository) and the ROBJ_MANIFEST
type are removed. borg 1.x repositories are still read via their manifest
(legacy_key_factory), as it holds their archives list. NoManifestError is
kept (never raised) so that rc 26 stays reserved.

Repository.create() creates all namespace directories in advance
(store.create_levels()) and writes the config (version, id, no key info)
right away, so a repository created via the Python API is an openable
key/value store. "borg repo-create" alone defers the config write
(create_config=False) until after the key was created, so nothing that
looks like a repository exists until repo-create succeeds: any failure or
interruption before that destroys the store (and the keyfile, in keyfile
mode), and a failure inside create() destroys the store as well.

A store without repository config is reported as not a valid repository
(InvalidRepository; DoesNotExist is only for a missing store backend, the
rest:// case from borgbackup#10365). repo-create on it raises IncompleteRepository
(rc 11, "has no repository config"), as the store backend refuses any
non-empty directory. "borg repo-delete --force" destroys such a store, so
that ssh/rest/s3 users can remove it without other access to the storage,
but only if it has the packs, archives, index and config namespaces every
store borg created has: a data directory, a home directory or a borg 1.x
repository is never destroyed by borg. key_factory() on a config without
key info raises RepositoryKeyInfoMissing (rc 54).

The cache config's [cache]/[integrity] cross-check (manifest id) could
never detect anything and is dropped.

Docs: new "Repository config" section replaces the manifest section
(including how to recreate a lost config by hand, as check --repair can
not), the layout list, packs and security internals (where the protection
against a swapped crypto suite lives), repo-info example, error list and the
remaining manifest mentions are updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ThomasWaldmann
ThomasWaldmann marked this pull request as ready for review September 17, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant