V4 core spike: RESP write path, client-side cache, and the context surface

.github/actions/run-tests/action.yml

@@ -81,6 +81,31 @@ runs:
         --results-directory ./test-results/
         /p:CI=true
 
+    # Neither of these needs a server, and between them they took about four seconds - but nothing ran
+    # them until now, so a namespace rename silently broke all 18 SER309 tests and nobody found out.
+    - name: RESPite.Tests
+      shell: bash
+      run: >-
+        dotnet test tests/RESPite.Tests/RESPite.Tests.csproj
+        --no-build
+        -c Release
+        -f net10.0
+        --logger trx
+        --logger "GitHubActions;summary-include-passed=false;summary-include-skipped=false"
+        --results-directory ./test-results/
+        /p:CI=true
+
+    - name: StackExchange.Redis.Build.Tests
+      shell: bash
+      run: >-
+        dotnet test tests/StackExchange.Redis.Build.Tests/StackExchange.Redis.Build.Tests.csproj
+        --no-build
+        -c Release
+        --logger trx
+        --logger "GitHubActions;summary-include-passed=false;summary-include-skipped=false"
+        --results-directory ./test-results/
+        /p:CI=true
+
     - uses: dorny/test-reporter@v3
       continue-on-error: true
       if: success() || failure()

.gitignore

@@ -34,3 +34,6 @@ BenchmarkDotNet.Artifacts/
 
 # local planning/design notes (not for publication)
 planning/
+
+# agent worktrees (see .claude/); these are separate checkouts, never repo content
+.claude/worktrees/

Directory.Build.props

@@ -1,4 +1,4 @@
-<Project>
+<Project>
   <PropertyGroup>
     <VersionPrefix>2.0.0</VersionPrefix>
     <Copyright>2014 - $([System.DateTime]::Now.Year) Stack Exchange, Inc.</Copyright>
@@ -10,9 +10,11 @@
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <CodeAnalysisRuleset>$(MSBuildThisFileDirectory)Shared.ruleset</CodeAnalysisRuleset>
     <MSBuildWarningsAsMessages>NETSDK1069</MSBuildWarningsAsMessages>
-    <!-- SER001/SER002/SER003/SER006/SER007/SER008 retired (vector sets, Redis 8.4/8.6/8.8/8.10 features,
-         and geo-redundant failover, no longer experimental); IDs reserved, see Experiments.cs -->
-    <NoWarn>$(NoWarn);NU5105;NU1507;SER004;SER005;SER009;SER010</NoWarn>
+    <!-- SER001/SER002/SER003/SER006/SER007/SER008/SER013 retired (vector sets, Redis 8.4/8.6/8.8/8.10
+         features, geo-redundant failover, and the interpolated writer, no longer experimental); IDs
+         reserved, see Experiments.cs.
+         SER011 is deliberately absent: hand-written RespFragment is meant to be seen at every call site. -->
+    <NoWarn>$(NoWarn);NU5105;NU1507;SER004;SER005;SER009;SER010;SER012</NoWarn>
     <PackageReleaseNotes>https://github.com/StackExchange/StackExchange.Redis/releases</PackageReleaseNotes>
     <PackageProjectUrl>https://seredis.dev/</PackageProjectUrl>
     <PackageLicenseExpression>MIT</PackageLicenseExpression>

docs/Arrays.md

@@ -14,9 +14,9 @@ Use `ArraySetAsync` and `ArrayGetAsync` to write and read individual cells:
 var db = conn.GetDatabase();
 RedisKey key = "events";
 
-bool inserted = await db.ArraySetAsync(key, 0, "created");
-RedisValue value = await db.ArrayGetAsync(key, 0);
-RedisValue missing = await db.ArrayGetAsync(key, 1);
+bool inserted = await db.Arrays.SetAsync(key, 0, "created");
+RedisValue value = await db.Arrays.GetAsync(key, 0);
+RedisValue missing = await db.Arrays.GetAsync(key, 1);
 
 Console.WriteLine(inserted); // True when the cell did not previously have a value
 Console.WriteLine(value);    // created
@@ -26,36 +26,36 @@ Console.WriteLine(missing.IsNull); // True
 Array indexes use `RedisArrayIndex`, with implicit conversions from `int`, `long`, and `ulong`. This allows normal small indexes to be used directly, while still allowing the full unsigned index range when needed.
 
 ```csharp
-await db.ArraySetAsync(key, 42, "answer");
-await db.ArraySetAsync(key, new RedisArrayIndex(10_000_000UL), "large index");
+await db.Arrays.SetAsync(key, 42, "answer");
+await db.Arrays.SetAsync(key, new RedisArrayIndex(10_000_000UL), "large index");
 ```
 
 ## Sparse Arrays
 
 Arrays are sparse: unset cells do not have values. `ArrayLengthAsync` reports the notional length, which is the highest used index plus one. `ArrayCountAsync` reports only cells that currently have values.
 
 ```csharp
-await db.KeyDeleteAsync(key);
+await db.Keys.DeleteAsync(key);
 
-await db.ArraySetAsync(key, 0, "a");
-await db.ArraySetAsync(key, 10, "b");
+await db.Arrays.SetAsync(key, 0, "a");
+await db.Arrays.SetAsync(key, 10, "b");
 
-RedisArrayIndex length = await db.ArrayLengthAsync(key); // 11
-RedisArrayIndex count = await db.ArrayCountAsync(key);   // 2
+RedisArrayIndex length = await db.Arrays.LengthAsync(key); // 11
+RedisArrayIndex count = await db.Arrays.CountAsync(key);   // 2
 ```
 
 ## Setting Multiple Values
 
 To write a contiguous range, pass the first index and the values:
 
 ```csharp
-int inserted = await db.ArraySetAsync(key, 0, ["a", "b", "c"]);
+int inserted = await db.Arrays.SetAsync(key, 0, ["a", "b", "c"]);
 ```
 
 To write multiple specific indexes, use `RedisArrayEntry` values:
 
 ```csharp
-await db.ArraySetAsync(key,
+await db.Arrays.SetAsync(key,
 [
     new RedisArrayEntry(0, "alpha"),
     new RedisArrayEntry(5, "bravo"),
@@ -70,20 +70,20 @@ The returned `int` is the number of cells that were newly filled.
 Read selected indexes with `ArrayGetAsync`:
 
 ```csharp
-RedisValue[] values = await db.ArrayGetAsync(key, [0, 5, 6, 100]);
+RedisValue[] values = await db.Arrays.GetAsync(key, [0, 5, 6, 100]);
 ```
 
 Read a range with `ArrayGetRangeAsync`. Ranges can be read forward or backward:
 
 ```csharp
-RedisValue[] forward = await db.ArrayGetRangeAsync(key, 0, 5);
-RedisValue[] reverse = await db.ArrayGetRangeAsync(key, 5, 0);
+RedisValue[] forward = await db.Arrays.GetRangeAsync(key, 0, 5);
+RedisValue[] reverse = await db.Arrays.GetRangeAsync(key, 5, 0);
 ```
 
 For sparse arrays, use `ArrayScanAsync` to return only populated cells in a range:
 
 ```csharp
-RedisArrayEntry[] entries = await db.ArrayScanAsync(key, 0, 100, limit: 50);
+RedisArrayEntry[] entries = await db.Arrays.ScanAsync(key, 0, 100, limit: 50);
 
 foreach (var entry in entries)
 {
@@ -96,21 +96,21 @@ foreach (var entry in entries)
 Delete a single cell with `ArrayDeleteAsync`:
 
 ```csharp
-bool removed = await db.ArrayDeleteAsync(key, 5);
+bool removed = await db.Arrays.DeleteAsync(key, 5);
 ```
 
 Delete multiple specific cells by index:
 
 ```csharp
-int removedCount = await db.ArrayDeleteAsync(key, [0, 5, 100]);
+int removedCount = await db.Arrays.DeleteAsync(key, [0, 5, 100]);
 ```
 
 Delete one or more ranges:
 
 ```csharp
-await db.ArrayDeleteRangeAsync(key, 10, 20);
+await db.Arrays.DeleteRangeAsync(key, 10, 20);
 
-await db.ArrayDeleteRangeAsync(key,
+await db.Arrays.DeleteRangeAsync(key,
 [
     new RedisArrayRange(100, 199),
     new RedisArrayRange(500, 599),
@@ -171,17 +171,17 @@ RedisArrayEntry[] matches = await db.ArrayGrepAsync(key, request);
 Arrays have a write head used by insert operations. `ArrayInsertAsync` writes at the current write head and advances it.
 
 ```csharp
-RedisArrayIndex first = await db.ArrayInsertAsync(key, "first");
-RedisArrayIndex second = await db.ArrayInsertAsync(key, "second");
+RedisArrayIndex first = await db.Arrays.InsertAsync(key, "first");
+RedisArrayIndex second = await db.Arrays.InsertAsync(key, "second");
 
-RedisArrayIndex? next = await db.ArrayNextAsync(key);
+RedisArrayIndex? next = await db.Arrays.NextAsync(key);
 ```
 
 Move the write head with `ArraySeekAsync`:
 
 ```csharp
-bool moved = await db.ArraySeekAsync(key, 1_000);
-RedisArrayIndex written = await db.ArrayInsertAsync(key, "later");
+bool moved = await db.Arrays.SeekAsync(key, 1_000);
+RedisArrayIndex written = await db.Arrays.InsertAsync(key, "later");
 ```
 
 ## Ring Buffers
@@ -191,34 +191,34 @@ Use `ArrayRingAsync` to keep at most a fixed number of cells and wrap writes aro
 ```csharp
 for (int i = 0; i < 10; i++)
 {
-    await db.ArrayRingAsync(key, maxLength: 5, value: i);
+    await db.Arrays.RingAsync(key, maxLength: 5, value: i);
 }
 
-RedisArrayIndex count = await db.ArrayCountAsync(key); // 5
+RedisArrayIndex count = await db.Arrays.CountAsync(key); // 5
 ```
 
 `ArrayLastItemsAsync` is intended for this capped ring-buffer model. It reads the last values in the ring-buffer sense, where "last" relates to the retained values after wrap-around and trimming:
 
 ```csharp
-RedisValue[] last = await db.ArrayLastItemsAsync(key, count: 10);
-RedisValue[] lastReversed = await db.ArrayLastItemsAsync(key, count: 10, reverse: true);
+RedisValue[] last = await db.Arrays.LastItemsAsync(key, count: 10);
+RedisValue[] lastReversed = await db.Arrays.LastItemsAsync(key, count: 10, reverse: true);
 ```
 
 ## Operations and Info
 
 Use `ArrayOperationAsync` for simple server-side operations over a range:
 
 ```csharp
-RedisValue sum = await db.ArrayOperationAsync(key, 0, 10, ArrayOperation.Sum);
-RedisValue used = await db.ArrayOperationAsync(key, 0, 10, ArrayOperation.Used);
-RedisValue matches = await db.ArrayOperationAsync(key, 0, 10, ArrayOperation.Match, "error");
+RedisValue sum = await db.Arrays.OperationAsync(key, 0, 10, ArrayOperation.Sum);
+RedisValue used = await db.Arrays.OperationAsync(key, 0, 10, ArrayOperation.Used);
+RedisValue matches = await db.Arrays.OperationAsync(key, 0, 10, ArrayOperation.Match, "error");
 ```
 
 Use `ArrayInfoAsync` for metadata:
 
 ```csharp
-ArrayInfo info = await db.ArrayInfoAsync(key);
-ArrayInfo fullInfo = await db.ArrayInfoAsync(key, full: true);
+ArrayInfo info = await db.Arrays.InfoAsync(key);
+ArrayInfo fullInfo = await db.Arrays.InfoAsync(key, full: true);
 
 Console.WriteLine($"Count: {info.Count}");
 Console.WriteLine($"Length: {info.Length}");

docs/Basics.md

@@ -43,29 +43,32 @@ object asyncState = ...
 IDatabase db = redis.GetDatabase(databaseNumber, asyncState);
 ```
 
-Once you have the `IDatabase`, it is simply a case of using the [redis API](https://redis.io/commands). Note that all methods have both synchronous and asynchronous implementations. In line with Microsoft's naming guidance, the asynchronous methods all end `...Async(...)`, and are fully `await`-able etc.
+Once you have the `IDatabase`, it is simply a case of using the [redis API](https://redis.io/commands). Commands are grouped by the redis data type they belong to - `db.Strings`, `db.Hashes`, `db.Lists`, `db.Sets`, `db.SortedSets`, `db.Keys` and so on - and every command is asynchronous, ending `...Async(...)` in line with Microsoft's naming guidance.
 
 The simplest operation would be to store and retrieve a value:
 
 ```csharp
 string value = "abcdefg";
-db.StringSet("mykey", value);
+await db.Strings.SetAsync("mykey", value);
 ...
-string value = db.StringGet("mykey");
+RedisValue value = await db.Strings.GetAsync("mykey");
 Console.WriteLine(value); // writes: "abcdefg"
 ```
 
-Note that the `String...` prefix here denotes the [String redis type](https://redis.io/topics/data-types), and is largely separate to the [.NET String type][3], although both can store text data. However, redis allows raw binary data for both keys and values - the usage is identical:
+Note that the `Strings` group here denotes the [String redis type](https://redis.io/topics/data-types), and is largely separate to the [.NET String type][3], although both can store text data. However, redis allows raw binary data for both keys and values - the usage is identical:
 
 ```csharp
 byte[] key = ..., value = ...;
-db.StringSet(key, value);
+await db.Strings.SetAsync(key, value);
 ...
-byte[] value = db.StringGet(key);
+RedisValue value = await db.Strings.GetAsync(key);
+byte[] bytes = value!;
 ```
 
 The entire range of [redis database commands](https://redis.io/commands) covering all redis data types is available for use.
 
+> **Coming from an earlier version?** These were `db.StringSet` and `db.StringGet`, on one flat interface. That spelling still works and is still supported - see [The original `IDatabase` API](LegacyApi) for what changed, the full mapping, and the handful of commands that are not on the groups yet.
+
 Using redis pub/sub
 ----
 
@@ -131,36 +134,30 @@ DateTime lastSave = server.LastSave();
 ClientInfo[] clients = server.ClientList();
 ```
 
-Sync vs Async vs Fire-and-Forget
+Async vs Fire-and-Forget
 ---
 
-There are 3 primary usage mechanisms with StackExchange.Redis:
+There are 2 primary usage mechanisms with StackExchange.Redis:
 
-- Synchronous - where the operation completes before the methods returns to the caller (note that while this may block the caller, it absolutely **does not** block other threads: the key idea in StackExchange.Redis is that it aggressively shares the connection between concurrent callers)
-- Asynchronous - where the operation completes some time in the future, and a `Task` or `Task<T>` is returned immediately, which can later:
-    - be `.Wait()`ed (blocking the current thread until the response is available)
-    - have a continuation callback added ([`ContinueWith`](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.continuewith) in the TPL)
-    - be *awaited* (which is a language-level feature that simplifies the latter, while also continuing immediately if the reply is already known)
+- Asynchronous - where the operation completes some time in the future, and a `ValueTask` or `ValueTask<T>` is returned immediately, which is then *awaited*. This is how the command groups work, and it is what you should reach for: it continues immediately if the reply is already known, and it never ties up a thread waiting for the network.
 - Fire-and-Forget - where you really aren't interested in the reply, and are happy to continue irrespective of the response
 
-The synchronous usage is already shown in the examples above. This is the simplest usage, and does not involve the [TPL][1].
-
-For asynchronous usage, the key difference is the `Async` suffix on methods, and (typically) the use of the `await` language feature. For example:
-
 ```csharp
 string value = "abcdefg";
-await db.StringSetAsync("mykey", value);
+await db.Strings.SetAsync("mykey", value);
 ...
-string value = await db.StringGetAsync("mykey");
+RedisValue value = await db.Strings.GetAsync("mykey");
 Console.WriteLine(value); // writes: "abcdefg"
 ```
 
 The fire-and-forget usage is accessed by the optional `CommandFlags flags` parameter on all methods (defaults to none). In this usage, the method returns the default value immediately (so a method that normally returns a `String` will always return `null`, and a method that normally returns an `Int64` will always return `0`). The operation will continue in the background. A typical use-case of this might be to increment page-view counts:
 
 ```csharp
-db.StringIncrement(pageKey, flags: CommandFlags.FireAndForget);
+await db.Strings.IncrementAsync(pageKey, flags: CommandFlags.FireAndForget);
 ```
 
+A third mechanism exists and is deliberately not shown here: the **synchronous** members of the original `IDatabase` API - `await db.Strings.GetAsync(key)` and friends - which block the calling thread until the reply arrives. They still work, and for a console tool or a startup path they are fine. On a server they are the single most common cause of the timeouts this library gets reported, because the thread you block is one the reply may need: see [Sync over async](SyncOverAsync), and [The original `IDatabase` API](LegacyApi) for the mapping.
+
   [1]: https://docs.microsoft.com/en-us/dotnet/standard/parallel-programming/task-parallel-library-tpl
   [2]: https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.asyncstate
   [3]: https://docs.microsoft.com/en-us/dotnet/api/system.string