Skip to content

Improve resource file checksum performance - #5311

Open
MohabCodeX wants to merge 2 commits into
multitheftauto:masterfrom
MohabCodeX:perf/resource-checksum-scalability
Open

Improve resource file checksum performance#5311
MohabCodeX wants to merge 2 commits into
multitheftauto:masterfrom
MohabCodeX:perf/resource-checksum-scalability

Conversation

@MohabCodeX

@MohabCodeX MohabCodeX commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #5286

Optimizes resource loading and client join scaling for resources with large file sets

  • Client (CDownloadableResource): Defer checksum calculation for download="false" files until requested, eliminating the 5-second join freeze and "Network Trouble" dialog.
  • Server (CResource): Replace linear $O(N^2)$ duplicate checks in meta.xml parsing with hash sets to prevent server startup stalls.
  • HTTP Cache: Skip copying download="false" files to the server HTTP cache.
  • Reloads: Skip server re-checksumming on restart/refresh if file timestamp and size remain unchanged.

Testing

  • Tested with cef_checksum.zip (14,000 files). Client join is instant with no Network Trouble freeze (only a momentary <1s frame drop on initial load as expected).

Copilot AI lite review requested due to automatic review settings September 2, 2026 21:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@FileEX FileEX added refactor bugfix Solution to a bug of any kind labels Sep 2, 2026

if (pResourceFile->GetLastChecksum() != checksum)
return true;
if (!ec && currentWriteTime == pResourceFile->GetLastWriteTime() && currentFileSize == pResourceFile->GetLastFileSize())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sets a std::error_code& parameter to the OS API error code if an OS API call fails, and executes ec.clear() if no errors occur.

Functions from the std::filesystem library that take a std::error_code& overwrite its state. If the first function (last_write_time) reports an error, but the second one (file_size) succeeds, the latter will clear ec to the success state (ec.clear()). As a result, the if (!ec) check will ignore the fact that the first operation failed.

Additionally, I think it would be worth adding some logging when an error occurs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Solution to a bug of any kind refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Server freezes when a resource contains a large number of files in meta.xml

3 participants