scalar: add --[no-]prefetch

[Derrick Stolee (derrickstolee)]

When using the GVFS protocol with scalar clone, the first git fetch
issues a /gvfs/prefetch request to download the commits and trees that
back the checked-out branch, so history operations are usable immediately
after cloning. For large repositories this prefetch can dominate the clone
time.

Some users would rather optimize for the initial usability of the working
tree and do not need full history right away; they are content to let a
later fetch (including background maintenance) download the prefetch data
for them.

This PR adds a --[no-]prefetch option to scalar clone. With
--no-prefetch, the initial /gvfs/prefetch request is skipped so the
worktree becomes ready as quickly as possible.

Implementation

The prefetch-during-fetch behavior is gated by the GVFS_PREFETCH_DURING_FETCH
bit (1 << 7) in core.gvfs, which scalar clone sets as part of the
value 150. Rather than persisting a different core.gvfs value (which
would disable prefetching forever), --no-prefetch only clears that bit
for the single git fetch invocation performed during the clone, by
passing -c core.gvfs=<value without the prefetch bit>.

The persisted core.gvfs is left untouched, so:

• the initial clone skips the prefetch and finishes sooner, and
• the next git fetch -- including the background maintenance
  prefetch task -- still performs the prefetch, hydrating the object
  cache shortly afterward.

The option has no effect when the GVFS Protocol is not in use.

Documentation

Documentation/scalar.adoc documents --[no-]prefetch, making clear that
it only affects the clone's initial fetch and that the prefetch data is
still downloaded by the next fetch.

Tests

t/t9210-scalar.sh gains a test against the GVFS-enabled test server which
asserts that:

• a normal clone emits a prefetch/since trace event,
• a --no-prefetch clone does not,
• the persisted core.gvfs remains 150, and
• a subsequent git fetch performs the deferred prefetch.

[Matthew John Cheetham (mjcheetham)]

This seems like a reasonable addition! Looks good to me.

[tyrielv]

GVFS has had --no-prefetch for a long time, and I recently changed the background maintenance to continue honoring it - ie if a commit-graph prefetch has not been done before, don't do one in the background. A manual gvfs prefetch -c OR a git fetch will both trigger prefetch.

The reasoning was that our primary case for using --no-prefetch is in pipelines that won't ever need the history - they'll throw the enlistment away or even recycle the VM, so doing the prefetch in the background at a later point just slows down the pipeline work they're trying to do for no gain.

I also have work pending to make the initial prefetch be asynchronous to clone completion for gvfs, even if --no-prefetch is not used (in tandem with the trust boundary realignment to index all received pack files instead of trusting the indexes from the cache server, since that adds 15+ minutes to first prefetch).

Just food for thought, I don't know if similar reasoning would apply to your use-cases for scalar clones.

[Derrick Stolee (derrickstolee)]

GVFS has had --no-prefetch for a long time, and I recently changed the background maintenance to continue honoring it - ie if a commit-graph prefetch has not been done before, don't do one in the background. A manual gvfs prefetch -c OR a git fetch will both trigger prefetch.

If someone wants a similar setup, then they can do scalar clone --no-prefetch --no-maintenance or long-term modify core.gvfs to the appropriate value (I think 134 instead of 150). I don't want to have users who run git fetch to operate without running the prefetch operation, at least right now. In the situation of a local developer environment, I either want them to be explicit to say "I know the impact of skipping prefetch long-term" and otherwise have folks get the data in the background or their first foreground git fetch.