Skip to content

gvfs-helper: parallelize POST requests #980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Derrick Stolee (derrickstolee) wants to merge 3 commits into
base: vfs-2.55.0
Choose a base branch
from
Open

gvfs-helper: parallelize POST requests #980

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3ace078
gvfs-helper: add gvfs.postThreads config option
derrickstolee Aug 24, 2026
69ec523
gvfs-helper: parallelize POST object requests
derrickstolee Aug 24, 2026
6b263d8
t5798: test parallel POST object requests
derrickstolee Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Documentation/config/gvfs.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,13 @@ gvfs.prefetchThreads::
index-pack execution, which can significantly speed up the
installation of multiple prefetch packs. Values less than `1` are
treated as `1`.

gvfs.postThreads::

@tyrielv tyrielv Aug 24, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this feature ever tend to bottleneck on CPU (ie when indexing pack files) or only on network/remote?

I'm wondering if there'd be any benefit or downside to supporting "values less than 1 are treated as NUMBER_OF_PROCESSORS" like checkout.workers does.

@tyrielv tyrielv Aug 24, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, GVFS uses Environment.ProcessorCount as the default parallelism value for its analogous workflow (gvfs prefetch --files or --folders)

@derrickstolee Derrick Stolee (derrickstolee) Aug 24, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could consider the "values less than one" option. I worry that the network will saturate at a lower parallelism than the CPU doing pack-indexing.

Set the number of parallel workers used when fetching objects
via HTTP POST requests. Each worker creates its own HTTP
connection and streams the response directly into an
`index-pack --stdin` child process. The default value is `1`,
which processes POST requests sequentially using the existing
code path. Setting this to a higher value (for example `4`)
downloads multiple batches of objects concurrently. Values less
than `1` are treated as `1`.
Loading
Loading