remove the repository feature flags mechanism

docs/internals/data-structures.rst

@@ -253,8 +253,7 @@ manifest object if the content changed. It looks like this:
 
 Borg 2 always writes *version* 2. Reading also accepts version 1, which is what
 borg 1.x repositories have (they are supported read-only, e.g. for
-``borg transfer``). The versions differ in the way feature flags are handled,
-described below.
+``borg transfer``).
 
 A *timestamp* entry, as written by borg 1.x and by older borg 2 versions, is
 accepted and ignored when reading.
@@ -264,148 +263,12 @@ manifest, each archive has its own pointer object in the ``archives/``
 namespace, see :ref:`archive`.
 
 *config* is a general-purpose location for additional metadata. All versions
-of Borg preserve its contents. Borg stores these keys in there:
-
-*config['feature_flags']* are the feature flags of the repository, see below.
+of Borg preserve its contents. Currently, borg does not store anything in there.
 
 A *config['item_keys']* list (written by older borg 2 versions) or a top-level
 *item_keys* list (borg 1.x) is accepted and ignored when reading: *borg check*
 does not validate item keys against such a list anymore, see Item_.
 
-Feature flags
-+++++++++++++
-
-Feature flags are used to add features to data structures without causing
-corruption if older versions are used to access or modify them. The main issues
-to consider for a feature flag oriented design are flag granularity,
-flag storage, and cache_ invalidation.
-
-Feature flags are divided in approximately three categories, detailed below.
-Due to the nature of ID-based deduplication, write (i.e. creating archives) and
-read access are not symmetric; it is possible to create archives referencing
-chunks that are not readable with the current feature set. The third
-category are operations that require accurate reference counts, for example
-archive deletion and check.
-
-As the manifest is always read (and rewritten whenever its content changes), it is the ideal place to store
-feature flags, comparable to the super-block of a file system. The only problem
-is to recover from a lost manifest, i.e. how is it possible to detect which feature
-flags are enabled, if there is no manifest to tell. This issue is left open at this time,
-but is not expected to be a major hurdle; it doesn't have to be handled efficiently, it just
-needs to be handled.
-
-Lastly, cache_ invalidation is handled by noting which feature
-flags were and which were not understood while manipulating a cache.
-This allows borg to detect whether the cache needs to be invalidated,
-i.e. rebuilt from scratch. See `Cache feature flags`_ below.
-
-The *config* key stores the feature flags enabled on a repository:
-
-.. code-block:: python
-
-    config = {
-        'feature_flags': {
-            'read': {
-                'mandatory': ['some_feature'],
-            },
-            'check': {
-                'mandatory': ['other_feature'],
-            }
-            'write': ...,
-            'delete': ...
-        },
-    }
-
-The top-level distinction for feature flags is the operation the client intends
-to perform,
-
-| the *read* operation includes extraction and listing of archives,
-| the *write* operation includes creating new archives,
-| the *delete* (archives) operation,
-| the *check* operation requires full understanding of everything in the repository.
-|
-
-These are weakly set-ordered; *check* will include everything required for *delete*,
-*delete* will likely include *write* and *read*. However, *read* may require more
-features than *write* (due to ID-based deduplication, *write* does not necessarily
-require reading/understanding repository contents).
-
-Each operation can contain several sets of feature flags. Only one set,
-the *mandatory* set is currently defined.
-
-Upon reading the manifest, the Borg client has already determined which operation
-should be performed. If feature flags are found in the manifest, the set
-of feature flags supported by the client is compared to the mandatory set
-found in the manifest. If any unsupported flags are found (i.e. the mandatory set is
-not a subset of the features supported by the Borg client used), the operation
-is aborted with a *MandatoryFeatureUnsupported* error:
-
-    Unsupported repository feature(s) {'some_feature'}. A newer version of Borg is required to access this repository.
-
-Older Borg releases do not have this concept and do not perform feature flags checks.
-These are locked out with manifest version 2, which is what Borg 2 always writes:
-the only difference between manifest versions 1 and 2 is that the latter is only
-accepted by Borg releases implementing feature flags.
-
-.. _Cache feature flags:
-.. rubric:: Cache feature flags
-
-:ref:`The local cache <cache>` does not have its separate set of feature flags.
-Instead, Borg stores which flags were used to create or modify a cache (as the
-*mandatory_features* / *ignored_features* keys in the cache ``config`` file).
-
-All mandatory manifest features from all operations are gathered in one set.
-Then, two sets of features are computed;
-
-- those features that are supported by the client and mandated by the manifest
-  are added to the *mandatory_features* set,
-- the *ignored_features* set comprised of those features mandated by the manifest,
-  but not supported by the client.
-
-Because the client previously checked compliance with the mandatory set of features
-required for the particular operation it is executing, the *mandatory_features* set
-will contain all necessary features required for using the cache safely.
-
-Conversely, the *ignored_features* set contains only those features which were not
-relevant to operating the cache. Otherwise, the client would not pass the feature
-set test against the manifest.
-
-When opening a cache and the *mandatory_features* set is not a subset of the features
-supported by the client, the cache is wiped out and rebuilt,
-since a client not supporting a mandatory feature that the cache was built with
-would be unable to update it correctly.
-The assumption behind this behaviour is that any of the unsupported features could have
-been reflected in the cache and there is no way for the client to discern whether
-that is the case.
-Meanwhile, it may not be practical for every feature to have clients using it track
-whether the feature had an impact on the cache.
-Therefore, the cache is wiped.
-
-When opening a cache and the intersection of *ignored_features* and the features
-supported by the client contains any elements, i.e. the client possesses features
-that the previous client did not have and those new features are enabled in the repository,
-the cache is wiped out and rebuilt.
-
-While the former condition likely requires no tweaks, the latter condition is formulated
-in an especially conservative way to play it safe. It seems likely that specific features
-might be exempted from the latter condition.
-
-.. rubric:: Defined feature flags
-
-Currently no feature flags are defined.
-
-From currently planned features, some examples follow,
-these may/may not be implemented and purely serve as examples.
-
-- A mandatory *read* feature could be using a different encryption scheme (e.g. session keys).
-  This may not be mandatory for the *write* operation - reading data is not strictly required for
-  creating an archive.
-- Any additions to the way chunks are referenced (e.g. to support larger archives) would
-  become a mandatory *delete* and *check* feature; *delete* implies knowing correct
-  reference counts, so all object references need to be understood. *check* must
-  discover the entire object graph as well, otherwise the "orphan chunks check"
-  could delete data still in use.
-
 .. _archive:
 
 Archives
@@ -1400,8 +1263,6 @@ the file's name (see :ref:`the files cache <cache>` about that name):
     version = 1
     repository = 3c4...e59
     manifest = 10e...21c
-    ignored_features =
-    mandatory_features =
 
     [integrity]
     manifest = 10e...21c

src/borg/archive.py

@@ -2322,7 +2322,7 @@ def check(
             rebuild_manifest = True
         else:
             try:
-                self.manifest = Manifest.load(repository, (Manifest.Operation.CHECK,), key=self.key)
+                self.manifest = Manifest.load(repository, key=self.key)
             except IntegrityErrorBase as exc:
                 logger.error("Repository manifest is corrupted: %s", exc)
                 self.error_found = True

src/borg/archiver/_common.py

@@ -60,32 +60,7 @@ def get_repository(location, *, create, exclusive, lock_wait, lock, args, v1_leg
     return repository
 
 
-def compat_check(*, create, manifest, key, cache, compatibility, decorator_name):
-    if not create and (manifest or key or cache):
-        if compatibility is None:
-            raise AssertionError(f"{decorator_name} decorator used without compatibility argument")
-        if type(compatibility) is not tuple:
-            raise AssertionError(f"{decorator_name} decorator compatibility argument must be of type tuple")
-    else:
-        if compatibility is not None:
-            raise AssertionError(
-                f"{decorator_name} called with compatibility argument, " f"but would not check {compatibility!r}"
-            )
-        if create:
-            compatibility = Manifest.NO_OPERATION_CHECK
-    return compatibility
-
-
-def with_repository(
-    create=False,
-    lock=True,
-    exclusive=False,
-    manifest=True,
-    cache=False,
-    secure=True,
-    compatibility=None,
-    allow_v1=False,
-):
+def with_repository(create=False, lock=True, exclusive=False, manifest=True, cache=False, secure=True, allow_v1=False):
     """
     Method decorator for subcommand-handling methods: do_XYZ(self, args, repository, …)
 
@@ -96,20 +71,8 @@ def with_repository(
     :param manifest: load manifest and repo_objs (key), pass them as keyword arguments
     :param cache: open cache, pass it as keyword argument (implies manifest)
     :param secure: do assert_secure after loading manifest
-    :param compatibility: mandatory if not create and (manifest or cache), specifies mandatory
-           feature categories to check
     :param allow_v1: (bool) allow legacy Borg 1.x repositories
     """
-    # Note: with_repository decorator does not have a "key" argument (yet?)
-    compatibility = compat_check(
-        create=create,
-        manifest=manifest,
-        key=manifest,
-        cache=cache,
-        compatibility=compatibility,
-        decorator_name="with_repository",
-    )
-
     # We may need to modify `lock` inside `wrapper`. Therefore we cannot use the
     # `nonlocal` statement to access `lock` as modifications would also
     # affect the scope outside of `wrapper`. Subsequent calls would
@@ -154,7 +117,7 @@ def wrapper(self, args, **kwargs):
                         from ..legacy.repoobj import RepoObj1
 
                         ro_cls = RepoObj1
-                    manifest_ = Manifest.load(repository, compatibility, other=False, ro_cls=ro_cls)
+                    manifest_ = Manifest.load(repository, other=False, ro_cls=ro_cls)
                     kwargs["manifest"] = manifest_
                     if "compression" in args:
                         manifest_.repo_objs.compressor = args.compression.compressor
@@ -177,7 +140,7 @@ def wrapper(self, args, **kwargs):
     return decorator
 
 
-def with_other_repository(manifest=False, cache=False, compatibility=None, required=False):
+def with_other_repository(manifest=False, cache=False, required=False):
     """
     this is a simplified version of "with_repository", just for the "other location".
 
@@ -186,15 +149,6 @@ def with_other_repository(manifest=False, cache=False, compatibility=None, requi
     :param required: the command can not work without the other repository, refuse to run if it is not given.
     """
 
-    compatibility = compat_check(
-        create=False,
-        manifest=manifest,
-        key=manifest,
-        cache=cache,
-        compatibility=compatibility,
-        decorator_name="with_other_repository",
-    )
-
     def decorator(method):
         @functools.wraps(method)
         def wrapper(self, args, **kwargs):
@@ -231,7 +185,7 @@ def wrapper(self, args, **kwargs):
                         from ..legacy.repoobj import RepoObj1
 
                         ro_cls = RepoObj1
-                    manifest_ = Manifest.load(repository, compatibility, other=True, ro_cls=ro_cls)
+                    manifest_ = Manifest.load(repository, other=True, ro_cls=ro_cls)
                     assert_secure(repository, manifest_)
                     if manifest:
                         kwargs["other_manifest"] = manifest_

src/borg/archiver/analyze_cmd.py

@@ -9,7 +9,7 @@
 from ..helpers import ProgressIndicatorPercent
 from ..helpers.argparsing import ArgumentParser
 from ..helpers import GroupBySpec
-from ..manifest import AI_GROUP_BY_KEYS, Manifest, archive_group_key, format_group_key
+from ..manifest import AI_GROUP_BY_KEYS, archive_group_key, format_group_key
 from ..repository import Repository
 
 from ..logger import create_logger
@@ -445,7 +445,7 @@ def report_hotspots(self, hotspots):
 
 
 class AnalyzeMixIn:
-    @with_repository(compatibility=(Manifest.Operation.READ,))
+    @with_repository()
     def do_analyze(self, args, repository, manifest):
         """Analyzes archives."""
         ArchiveAnalyzer(args, repository, manifest).analyze()

src/borg/archiver/compact_cmd.py

@@ -14,7 +14,6 @@
 from ..hashindex import ChunkIndex
 from ..helpers import set_ec, EXIT_ERROR, Error, sig_int, format_file_size, bin_to_hex, hex_to_bin, IntegrityError
 from ..helpers import ProgressIndicatorPercent
-from ..manifest import Manifest
 from ..repoobj import object_validator
 from ..repository import Repository
 
@@ -456,7 +455,7 @@ def compact_packs(self):
 
 
 class CompactMixIn:
-    @with_repository(exclusive=True, compatibility=(Manifest.Operation.DELETE,))
+    @with_repository(exclusive=True)
     def do_compact(self, args, repository, manifest):
         """Collects garbage in the repository."""
         if not args.dry_run:

src/borg/archiver/copy_cmd.py

@@ -2,15 +2,14 @@
 from ..constants import *  # NOQA
 from ..helpers import archivename_validator, bin_to_hex
 from ..helpers.argparsing import ArgumentParser
-from ..manifest import Manifest
 
 from ..logger import create_logger
 
 logger = create_logger()
 
 
 class CopyMixIn:
-    @with_repository(cache=True, compatibility=(Manifest.Operation.CHECK,))
+    @with_repository(cache=True)
     @with_archive
     def do_copy(self, args, repository, manifest, cache, archive):
         """Copy an archive to a new archive name."""

src/borg/archiver/create_cmd.py

@@ -36,7 +36,6 @@
 from ..helpers import MakePathSafeAction
 from ..helpers import Error, CommandError, BackupWarning, FileChangedWarning
 from ..helpers.argparsing import ArgumentParser
-from ..manifest import Manifest
 from ..patterns import PatternMatcher
 from ..platform import is_win32, get_flags
 
@@ -64,7 +63,7 @@ def stat_root(path):
 
 
 class CreateMixIn:
-    @with_repository(compatibility=(Manifest.Operation.WRITE,))
+    @with_repository()
     def do_create(self, args, repository, manifest):
         """Creates a new archive."""
         if args.read_special_timeout is not None and not args.read_special: