Unclear docs: Customization of `get` behavior in non-blocking mode

[tasdflkjweio]

Hi, I am using a CacheableMemory L1 cache and createKeyvNonBlocking to create my redis connection for L2. The migration docs say:

All get related functions now support nonBlocking which means if nonBlocking: true the primary store will return what it has and then in the background will work to sync from secondary storage for any misses. You can disable this by setting at the get function level the option nonBlocking: false which will look for any missing keys in the secondary.

I am using getOrSet and this is the signature:

    /**
     * Retrieves the value associated with the given key from the cache. If the key is not found,
     * invokes the provided function to calculate the value, stores it in the cache, and then returns it.
     *
     * @param {GetOrSetKey} key - The key to retrieve or set in the cache. This can also be a function that returns a string key.
     * If a function is provided, it will be called with the cache options to generate the key.
     * @param {() => Promise<T>} function_ - The asynchronous function that computes the value to be cached if the key does not exist.
     * @param {GetOrSetFunctionOptions} [options] - Optional settings for caching, such as the time to live (TTL) or whether to cache errors.
     * @return {Promise<T | undefined>} - A promise that resolves to the cached or newly computed value, or undefined if an error occurs and caching is not configured for errors.
     */
    getOrSet<T>(key: GetOrSetKey, function_: () => Promise<T>, options?: GetOrSetFunctionOptions): Promise<T | undefined>;

type GetOrSetFunctionOptions = {
    ttl?: number | string;
    cacheErrors?: boolean;
    throwErrors?: boolean;
};

If you want your layer 2 (secondary) store to be non-blocking you can set the nonBlocking property to true in the options. This will make the secondary store non-blocking and will not wait for the secondary store to respond on setting data, deleting data, or clearing data. This is useful if you want to have a faster response time and not wait for the secondary store to respond. Here is a full list of what each method does in nonBlocking mode:

set - in non-blocking mode it will set at the primary storage and then in the background update secondary

get - in non-blocking mode it will only check the primary storage but then in the background look to see if there is a value in the secondary and update the primary

getMany - in non-blocking mode it will only check the primary storage but then in the background look to see if there is a value in the secondary and update the primary

getRaw - in non-blocking mode it will only check the primary storage but then in the background look to see if there is a value in the secondary and update the primary

getManyRaw - in non-blocking mode it will only check the primary storage but then in the background look to see if there is a value in the secondary and update the primary

Questions

1. The behavior I want is: a) best-effort writes in the background to the L2 cache. If it throws an error or the connection is dead, then we should log and swallow the error. b) read from L1 cache and, if miss, read from L2 cache in the foreground. Reads against L1 should be best-effort. The doc that says you can configure this on gets is unclear how actually to do so.
2. It's unclear in the docs what happens if you use a non-blocking secondary (createKeyvNonBlocking) but leave nonBlocking=false.
3. I'm assuming getOrSet uses the same behavior as the get reads in non-blocking mode? If that's true, then a miss against L1 will sync from L2 in the background AND do the heavy call against the primary system (db/external service)? It seems like we'd be better off doing the lightweight check against L2 first.

[jaredwray]

#1580