enhance: implement fast path for repository local change status update when core.fsmonitor and core.untrackedcache are enabled
#2012
Conversation
|
Oops, looks like I got the branch wrong and pulled in lots of commits. Fixing |
|
I don't like to do this, but I prefer to predict the changes of ‘Local Changes’ after the ‘add/reset’ operation is successful, so that even if there is something wrong, such as we cannot predict the renames, the list will be updated to the correct state after the background ‘git status’ command returns.
|
|
Maintaining consistency can be hard. Afaik predicting what's changed is how UGit works. Since the background operation is slow, UGit needs to run in the background (minimized to tray) until the operation is done. If you try to stage something, it shows its staged, then if you exit UGit immediately and reopen it, you see your changes are not staged. |
|
The real consistency issue emerges, when the user performs multiple ops in serial. Its like UE network prediction and rollback - if git status fails to catch up timely, you will need to design an algorithm to determine which user ops have failed and rollback to the specific point. If you don't allow such operation queues, the status update will be as slow as before once your user launched the second operation. |
2 participants
Finally this experiment has come to a stage where it's reviewable. It is still very early. Opinions are welcome. Binaries for testing (win-x64 only) can be downloaded here: https://github.com/AlanIWBFT/sourcegit/releases
Problem it aims to solve: #1998
To accurately report and let the user pick each individual untracked file, SourceGit uses
--untracked-files=all(-uall) when doingQueryLocalChanges, which is unfortunately very slow on large repositories.core.fsmonitorandcore.untrackedcachecannot help this case if-uallis specified as it makes the untracked cache bypassed, yielding no performance improvement.This pull request detects if
core.fsmonitorandcore.untrackedcacheare enabled and uses a two pass algorithm: it runs a normalgit statusfirst to retrive a list of untracked directories (which is fast as untracked cache is functioning here), then rungit status -uallonly on these dirs. The time it takes to traverse these dirs are much faster than running-uallon the entire repository.To use this experimental feature, the user must deliberately enable
core.fsmonitorandcore.untrackedcache, or not settingcore.untrackedcachebut settingfeature.manyfilestotrue.Components
First, to robustly detect if the fast path is possible,
Commands.Config.GetBoolAsyncis added which leverages git itself to canonicalize possible weird values in config files, like "000" will be intepreted as "false", "010" -> "true".Every time you open a repository the above detection is performed. If they are enabled, a normal, index-locking 'git status' will be run to populate/update the untracked cache. The first time will be slow as the cache needs to be populated entirely from empty. Subsequent runs will be much faster, in fact, negligible. (~0.4s on UE5-main). Thus the slow-down on UI start-up isn't very noticable expect for the first time. Since SourceGit always runs
git statuswith--no-optional-lockswhich means the index will not be modified, the untracked cache thus cannot be updated either, once SourceGit is up. So we need a place to insert the update, and it is scheduled when you open the repository, as said above.Performance results
On large repos like UE5, a consistent ~5x - ~6x speedup is observed whenever a status update is needed and the responsiveness is greatly improved.
Other fixes
As we've discussed in #1998 (comment),
Stash.PushAsyncbool includeUntracked's default value has been changed tofalse. The intention is to let git handle the untracked instead of manually stash them (which was performed with--included-untracked, very slow (2x slower than the slow status update)).