chore(spanner): implement cache updater for location-aware routing

[olavloite]

Implement CacheUpdater to coordinate between Spanner CacheUpdate wire messages and the client's in-memory routing table (KeyRangeCache) and connection pool (ConnectionCache).

[gemini-code-assist]

Code Review

This pull request introduces the CacheUpdater module to handle location-aware routing cache updates by applying CacheUpdate payloads to the KeyRangeCache and pre-warming server connections in the ConnectionCache. The review feedback recommends using a safer, error-returning method like try_get instead of get to avoid potential panics when pre-warming connections, as well as logging errors instead of swallowing them. Additionally, it suggests replacing flaky tokio::task::yield_now() calls in unit tests with a robust polling mechanism with a timeout.

[gemini-code-assist]

In prewarm_server_connections, the background task uses connection_cache.get which can panic. We should prefer using try_get (or other safe, error-returning methods) over get to ensure robust error handling and prevent unexpected panics. Additionally, instead of completely swallowing the error with let _ =, we should log it using tracing::warn! to ensure visibility and aid in debugging connection issues.

            handle.spawn(async move {
                // Calling `try_get` asynchronously initializes the server connection in the cache
                // if it does not already exist, ensuring foreground RPCs don't incur connection
                // handshake latency.
                if let Err(err) = connection_cache.try_get(&address_string, &config).await {
                    tracing::warn!(
                        ?err,
                        address = %address_string,
                        "Failed to pre-warm connection to Spanner server"
                    );
                }
            });

References

1. Demand Explosive Correctness: Never swallow errors or ignore Result types. Fail loudly and explicitly when appropriate. ^(link)
2. Prefer using try_get (or other safe, error-returning methods) over get (which can panic) in Rust to ensure robust error handling and prevent unexpected panics.

[gemini-code-assist]

Using tokio::task::yield_now().await; to wait for background tasks to complete is highly prone to flakiness in unit tests, especially when the background task involves multiple yield points (such as awaiting Channel::create inside connection_cache.get). To ensure robust and reliable tests, we should poll the cache with a short sleep and a timeout instead of yielding once. This applies to all other tests in this file that use yield_now().

        let start = std::time::Instant::now();
        while updater.connection_cache().get_if_present("10.0.0.1:15000").is_none() {
            if start.elapsed() > std::time::Duration::from_secs(1) {
                panic!("timed out waiting for connection to be pre-warmed");
            }
            tokio::time::sleep(std::time::Duration::from_millis(1)).await;
        }