Skip to content

git-recursive: resolve relative submodule URLs - #218

Open
marvil6 wants to merge 1 commit into
tuna:masterfrom
marvil6:fix-relative-submodule-urls
Open

marvil6 wants to merge 1 commit into
tuna:masterfrom
marvil6:fix-relative-submodule-urls

Conversation

@marvil6

@marvil6 marvil6 commented Sep 29, 2026 •

Copy link
Copy Markdown

git-recursive.sh passes each submodule's .gitmodules URL straight to git clone --mirror / git remote set-url. When that URL is relative (e.g. url = ../system.git), git treats it as a filesystem path instead of resolving it against the parent repository's own URL, so the clone fails. The return value of the recursive git_sync_recursive call is never checked, so the job still reports success while those submodules are left empty. This is the root cause identified in #161 by @ZenithalHourlyRate.

What changed

  • Add resolve_submodule_url(). A URL starting with ./ or ../ is resolved against the parent repository's URL the way git does (see gitsubmodules(7)). Every other form (https://, ssh://, scp-like, file://, absolute paths, ...) passes through unchanged.
  • checkout_repo takes the current repository's own upstream URL as a fourth argument (the caller already has it as $upstream) and resolves each submodule URL before recursing. Nested submodules resolve against their already-resolved parent.
  • If a relative URL climbs above what the parent URL can supply, the submodule is skipped with a message naming both URLs. In that case git's own resolver keeps eating into the scheme/authority and returns a malformed URL, so this patch refuses instead of reproducing it (see the table below).

Results

offline, file:// repos. Columns are exit status / gitlinks in the parent / submodule mirrors populated:

Fixture Before After
Flat, 5 relative submodules 0 / 5 / 0 0 / 5 / 5
Flat, 50 relative submodules 0 / 50 / 0 0 / 50 / 50
Nested (relative submodule with its own relative submodule) 0 / 1 / 0 of 2 0 / 1 / 2 of 2
./-relative submodule 0 / 1 / 0 0 / 1 / 1
Mixed relative + absolute 0 / 2 / 1 0 / 2 / 2
One relative + one genuinely unreachable 0 / 2 / 0 0 / 2 / 1 (the unreachable one still fails)
Absolute-URL control 0 / 5 / 5 0 / 5 / 5

For the absolute-URL control, the generated client script is identical before and after. A fresh git clone --recurse-submodules from the patched output has every submodule at its pinned commit.

Real world: boostorg/boost (all 173 submodules use relative URLs):

Before After
Exit status 0 0
Submodule mirrors populated 0 of 173 173 of 173
Wall time 20 s (superproject only) 458 s (173 additional clones)
Mirror size 217 MB 2.2 GB

Environment and caveats

  • Tested on a Linux host with bash 5.2.21, git 2.43.0 and GNU sed 4.9, and re-run inside the tunathu/tunasync-scripts image itself (bash 5.2.37, git 2.47.3) with identical results.
  • shellcheck reports no new warnings on the changed lines.
  • The resolver is validated against git's own resolution and against boostorg/boost, not every URL scheme git accepts.

Fixes #161

.gitmodules URLs like "../system.git" (used throughout boostorg/boost)
were passed straight to git clone/remote, which treats them as
filesystem paths rather than resolving them against the parent
repository's own URL. Add resolve_submodule_url() to do that
resolution the way git itself does, threaded through checkout_repo
via a new fourth parameter. Absolute URLs are unaffected.

Fixes tuna#161.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

git-recursive脚本同步git仓库时,工作不正常

1 participant