fix: preserve legacy path bytes and repair tag, tree and ref edge cases

CONTRIBUTING.md

@@ -48,6 +48,12 @@ Attributing AI assistance in commit metadata, for example with a `Co-authored-by
 trailer, is welcome but not required. Code is reviewed the same way regardless of its
 origin.
 
+## Documenting subclass interfaces
+
+Use Sphinx's `:meta public:` field in docstrings of underscore-prefixed methods
+that form a supported interface for subclasses. This includes their documentation
+in the API reference without exposing unrelated private implementation details.
+
 ## Temporary test directories
 
 Tests require Python 3.11+, Git 2.52+, Bash, and the `test` extra, which installs

README.md

@@ -11,11 +11,9 @@ bindings to execute supported operations through Gitoxide while exposing GitPyth
 high-level repository, commit, tree and index interfaces.
 
 Supported object reads, tree traversal, reference/index queries and commit history
-can run without launching Git subprocesses. GitPython also retains and reuses a Gix
-repository handle across operations, reducing repeated initialization work. These
-changes reduce subprocess overhead, particularly for repeated repository queries;
-the [backend benchmarks](test/performance/README.md) measure results and CLI launch
-counts for both backends.
+can run without launching Git subprocesses. Each native operation opens a fresh
+Gix repository handle. The [backend benchmarks](test/performance/README.md) measure
+results and CLI launch counts for both backends.
 
 ### Activate the Gix backend
 
@@ -161,8 +159,12 @@ GitPython is not suited for long-running processes (like daemons) as it tends to
 leak system resources. It was written in a time where destructors (as implemented
 in the `__del__` method) still ran deterministically.
 
-In case you still want to use it in such a context, you will want to search the
-codebase for `__del__` implementations and call these yourself when you see fit.
+Use `Repo` as a context manager (`with Repo(path) as repo:`), or call
+`repo.close()` explicitly when finished, to release its native repository handle
+and persistent Git object readers. Close every repository instance and any
+separately opened readers before removing its directory. On Windows,
+`git.util.rmtree(path)` also handles Git's read-only files, which plain
+`shutil.rmtree()` may be unable to remove even after handles are closed.
 
 Another way assure proper cleanup of resources is to factor out GitPython into a
 separate process which can be dropped periodically.

doc/source/changes.rst

@@ -47,34 +47,43 @@ Git executable and safety
 * Known command outcomes retain established GitPython/configparser exceptions
   where distinguishable. Other failures raise ``GitCommandError`` with Git's
   exit status and diagnostic output; exact error text may differ.
+  Text and streamed command output preserve undecodable bytes with
+  ``surrogateescape``, including fetch/push results, clone errors and timeout
+  diagnostics containing byte names.
 
 API changes
 -----------
 
+* Packed tag names containing non-UTF-8 bytes are represented losslessly as
+  surrogate-escaped strings. ``safe_encode()`` recovers their original bytes,
+  and Gix reference validation and resolution receive those bytes directly.
+  Reference enumeration, fetch results, stale references and remote URLs preserve
+  Unicode line separators within names instead of treating them as record breaks.
+* Raw diffs preserve non-UTF-8 filename bytes in ``a_rawpath``, ``b_rawpath``,
+  and raw rename fields. Staged-file diffs no longer fail while decoding legacy
+  filenames; blob contents remain byte streams regardless of their encoding.
+  Text paths and rename properties use ``surrogateescape`` instead of replacing
+  undecodable bytes. Diff blob paths can be used for tree lookup, and renames
+  between distinct byte names are detected correctly. Patch rename fields
+  contain unquoted filename bytes.
+* Tree serialization now delegates construction to Git or the supported Gix
+  backend. ``Tree.cache`` and ``TreeModifier`` edit one tree's in-memory entries;
+  ``set_done()`` orders entries by filename bytes, including undecodable names.
+  Object IDs and parent trees do not change automatically. Prefer
+  ``IndexFile.write_tree()`` for changes to staged paths.
 * ``Commit.stats`` preserves Unicode line separators in filenames when
   ``core.quotePath=false``. Git's LF-delimited output remains intact for root,
-  subsequent and empty commits, including the Gix backend's existing filename
-  quoting fallback.
-* ``GitConfigParser[section] = values`` again replaces a nonempty section,
-  restoring Kas's Git identity cleanup. Git performs each mutation and locks
-  it independently; replacement is not one atomic transaction. Empty sections
-  and ``add_section()`` remain unsupported, including Kas's SSH key setup.
-* Explicit ``from git import Union`` and ``Optional`` imports remain available
-  for Demisto SDK's validator registration and MkDocs RSS Plugin imports,
-  retaining the historical ``typing`` aliases.
-* Editable/source imports expose the release version in ``git.__version__``,
-  matching built distributions. Salt's GitPython provider checks can parse the
-  version instead of rejecting the unstamped ``"git"`` build placeholder.
+  subsequent and empty commits, including the Gix backend's filename
+  quoting fallback. File keys contain the actual names, unquoting Git's C-style
+  escapes and preserving undecodable bytes with ``surrogateescape``.
+* Imports from a source checkout now read ``git.__version__`` from ``VERSION``.
+  Built wheels and source distributions continue to carry their stamped release
+  version.
 * ``Repo`` construction validates repository configuration through Git/Gix.
   A subclass override of ``config_reader()`` no longer controls discovery or
   suppresses its include graph. Malformed configuration and invalid repository
-  extensions can fail before downstream custom validation runs. sbxloop's
-  metadata-reader subclass exposes this remaining compatibility break; callers
-  must prepare their trusted metadata view before opening it with ``Repo``.
-* Revision root trees such as ``Repo.rev_parse("HEAD:")`` retain their empty
-  path and can be iterated. Missing paths within a valid tree revision retain
-  ``Tree.join()``'s ``KeyError`` and missing-component message, restoring MCP
-  Server Git's directory display and path-error reporting.
+  extensions can fail before downstream custom validation runs. Callers using
+  custom metadata views must prepare them before opening with ``Repo``.
 * ``Head.tracking_branch()`` returns a local ``Head`` when Git configures the
   current repository (``branch.<name>.remote=.``) as the upstream, instead of
   constructing an invalid ``refs/remotes/./...`` reference. Git resolves the
@@ -122,17 +131,23 @@ API changes
   including updates when ``logmsg`` is omitted.
 * ``GitConfigParser`` uses Git's configuration syntax and canonical key spelling,
   and no longer subclasses ``configparser.RawConfigParser``.
+  Subsection names and values preserve undecodable bytes with ``surrogateescape``.
   Common getters, typed values, duplicate values, file/stream inputs and mutation
-  methods remain. Invalid Git syntax previously accepted by Python is rejected.
-  Empty-section creation and valueless writes are removed; valueless reads remain.
-  Relative includes from streams are rejected by Git. Locks cover each native
-  mutation rather than the writer object's lifetime.
+  methods remain, including ``GitConfigParser[section] = values`` for replacing
+  nonempty sections. Invalid Git syntax previously accepted by Python is rejected.
+  Empty-section creation, including ``add_section()``, and valueless writes are
+  removed; valueless reads remain. Relative includes from streams are rejected
+  by Git. Locks cover each native mutation rather than the writer object's
+  lifetime; replacing a section is not one atomic transaction.
 * ``Submodule.rename()`` is removed. Moving a submodule preserves its logical name,
   matching Git. Normal removal retains Git's recoverable submodule metadata.
   Re-adding with ``no_checkout=True`` can reuse that metadata without changing refs;
   incompatible URLs, branches, and clone-only options are rejected before mutation.
   Fetch/pull results are derived from command output rather than ``FETCH_HEAD``
-  file parsing. Revision strings follow Git's native revision grammar.
+  file parsing. Revision strings follow Git's native revision grammar. Root trees
+  such as ``Repo.rev_parse("HEAD:")`` have an empty path and can be iterated.
+  Missing paths within a valid tree revision raise ``Tree.join()``'s ``KeyError``
+  with its missing-component message.
 
 * ``IndexFile.entries`` returns an immutable snapshot keyed by ``(path, stage)``
   for compatibility with existing readers. Each access queries the current
@@ -156,8 +171,8 @@ API changes
 * Commit/tag metadata uses structured Gix decoders when the Gix backend supports
   the repository. The CLI backend retains raw decoding because Git has no faithful
   formatted query for all commit headers or arbitrary tag objects. ``gpgsig``
-  remains readable. The small ``Actor.from_string()`` and
-  ``parse_actor_and_date()`` identity helpers remain supported.
+  remains readable. ``Actor.from_string()`` and ``parse_actor_and_date()`` provide
+  identity parsing.
 * ``parse_date()`` delegates text syntax and timezone interpretation to Git;
   aware datetimes retain direct conversion and invalid inputs raise ``ValueError``.
   ISO/RFC dates now apply their timezone to the UTC timestamp, correcting the
@@ -172,9 +187,10 @@ API changes
   ``Actor.from_string``), ``git.util.Iterable`` (use ``IterableObj``),
   ``git.compat.is_win/is_posix/is_darwin`` (use ``os.name``/``sys.platform``), and
   ``git.types.Lit_commit_ish`` (use ``Literal["commit", "tag"]`` or
-  ``GitObjectTypeString``). Top-level typing exports and private module aliases
-  are removed; import from ``typing`` or the owning module. ``git.util`` now
-  exposes the actual utility module. Abstract ``Traversable.traverse`` and
+  ``GitObjectTypeString``). Top-level typing exports other than explicit
+  ``Union`` and ``Optional`` imports are removed, along with private module
+  aliases; import from ``typing`` or the owning module. ``git.util`` now exposes
+  the actual utility module. Abstract ``Traversable.traverse`` and
   ``list_traverse`` raise ``NotImplementedError``; use concrete implementations.
 * Paths and URLs no longer expand ``$VAR``/``%VAR%`` automatically. ``Repo``
   still accepts ``expand_vars=False`` for callers that explicitly select literal
@@ -1135,7 +1151,6 @@ Please note that due to breaking changes, we have to increase the major version.
 * CRITICAL: fixed incorrect `Commit` object serialization when authored or commit date had timezones which were not
   divisiblej by 3600 seconds. This would happen if the timezone was something like `+0530` for instance.
 * A list of all additional fixes can be found `on GitHub <https://github.com/gitpython-developers/GitPython/issues?q=milestone%3A%22v1.0.2+-+Fixes%22+is%3Aclosed>`__
-* CRITICAL: `Tree.cache` was removed without replacement. It is technically impossible to change individual trees and expect their serialization results to be consistent with what *git* expects. Instead, use the `IndexFile` facilities to adjust the content of the staging area, and write it out to the respective tree objects using `IndexFile.write_tree()` instead.
 
 1.0.1 - Fixes
 =============

doc/source/intro.rst

@@ -6,7 +6,7 @@ Overview / Install
 
 GitPython is a python library used to interact with git repositories, high-level like git-porcelain, or low-level like git-plumbing.
 
-It provides Python objects for repository data and delegates Git operations to the Git executable. The default backend supports both SHA-1 and SHA-256 object IDs and both files and reftable reference storage. The legacy pure-Python GitDB backend is removed.
+It provides Python objects for repository data and delegates Git operations to the Git executable. The default backend supports both SHA-1 and SHA-256 object IDs and both files and reftable reference storage. ``GitDB`` is an alias for ``GitCmdObjectDB``; the pure-Python storage implementation is unavailable.
 
 The object database implementation is optimized for handling large quantities of objects and large datasets, which is achieved by using low-level structures and data streaming.
 

doc/source/reference.rst

@@ -91,7 +91,6 @@ Objects.Util
 .. automodule:: git.objects.util
    :members:
    :undoc-members:
-   :private-members: _get_intermediate_items, _list_traverse, _traverse
    :special-members:
 
 Index.Base

doc/source/tutorial.rst

@@ -63,6 +63,32 @@ Archive the repository contents to a tar file.
     :start-after: # [6-test_init_repo_object]
     :end-before: # ![6-test_init_repo_object]
 
+Closing a repository
+====================
+
+Use a :class:`~git.repo.base.Repo` context manager, or call
+:meth:`~git.repo.base.Repo.close` explicitly when finished. Exiting the context
+releases its native handle and persistent Git object readers, including when an
+exception is raised.
+
+.. literalinclude:: ../../test/test_docs.py
+    :language: python
+    :dedent: 8
+    :start-after: # [close-repo-context]
+    :end-before: # ![close-repo-context]
+
+Close every repository instance and any separately opened readers before deleting
+the directory. Windows prevents deletion while files remain open; Git's read-only
+object files can also make plain ``shutil.rmtree()`` fail. The existing
+:func:`git.util.rmtree` helper handles those read-only files while still reporting
+deletion failures:
+
+.. literalinclude:: ../../test/test_docs.py
+    :language: python
+    :dedent: 8
+    :start-after: # [remove-repo-directory]
+    :end-before: # ![remove-repo-directory]
+
 Advanced Repo Usage
 ===================
 
@@ -522,9 +548,10 @@ resolves abbreviated object IDs through persistent ``git cat-file`` processes::
     # Equivalent explicit selection:
     repo = Repo("path/to/repo", odbt=GitCmdObjectDB)
 
-Only ``GitCmdObjectDB`` and subclasses can be selected through ``odbt``. The
-legacy ``GitDB`` backend is removed. The ``gitdb`` package remains a dependency
-for shared types and utilities.
+Only ``GitCmdObjectDB`` and subclasses can be selected through ``odbt``.
+``GitDB`` is an alias for ``GitCmdObjectDB``; passing ``odbt=GitDB`` selects the
+same backend. The pure-Python storage implementation is unavailable. The
+``gitdb`` package provides shared types and utilities.
 
 Git Command Debugging and Customization
 ***************************************

fuzzing/README.md

@@ -71,8 +71,8 @@ Contains Python files for each fuzz test.
 **Things to Know**:
 
 - Each fuzz test targets a specific part of GitPython's functionality.
-- Repository and object targets exercise Git-backed workflows. Removed Python index/tree binary parsers are no longer
-  fuzz targets; malformed configuration and object IDs can now be rejected by Git or by Python input validation.
+- Repository and object targets exercise Git-backed workflows. Git or Python input validation may reject malformed
+  configuration and object IDs.
 - Test files adhere to the naming convention: `fuzz_<API Under Test>.py`, where `<API Under Test>` indicates the
   functionality targeted by the test.
 - Any functionality that involves performing operations on input data is a possible candidate for fuzz testing, but