Perf | Reduce allocations when sending large strings#4072
Open
edwardneal wants to merge 1 commit intodotnet:mainfrom
Open
Perf | Reduce allocations when sending large strings#4072edwardneal wants to merge 1 commit intodotnet:mainfrom
edwardneal wants to merge 1 commit intodotnet:mainfrom
Conversation
Shim GetByteCount and GetBytes methods, and make sure these are used more widely. Improves performance on netfx and netcore.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
At present, SqlClient calculates the number of bytes in a character string (and performs the actual conversions) by calling
string.ToCharArrayandEncoding.GetByteCountandEncoding.GetBytes. netfx also lacks some of the newer Encoding methods which allow GetByteCount and GetBytes to operate over a specific range of characters in the string.These two points mean that when sending a 50MB string using netcore, we perform 150MB of allocations. This PR resolves this. It introduces two new netfx-only extension methods in EncodingExtensions which match the netcore method signatures, so we can remove some conditional compilation in TdsParser and use the more performant paths.
This highlighted a second pattern: calling
string.ToCharArray(offset, length), then calling GetByteCount or GetBytes on the result. I replaced three instances of this pattern with the more efficientGetByteCount/GetBytes(string, offset, length)call.One noteworthy instance in the second pattern is in
GetEncodingCharLength. It looks like someone else had thought of this before, but it was undone - I don't see anything in the commit history to indicate why.Benchmarking highlights an across-the-board reduction in memory usage by 66%. Execution time is also cut - about 3-5% for <5MB strings, and about 20-30% for 10MB strings and above.
Issues
None.
Testing
Automated tests should continue to pass. Benchmark results are below.
Benchmarks