Skip to content

Make semantic cache observability easier out of the box #684

Description

@kaizer113

Problem

RedisVL semantic caching can reduce LLM latency and cost, but customers do not currently have an easy, built-in way to understand how effectively a cache is working.

RedisVL exposes useful search index statistics, but those do not answer common semantic cache questions such as:

  • What is the cache hit ratio?
  • How many checks result in hits, misses, or errors?
  • How long do cache lookups take?
  • Is cache effectiveness improving or regressing over time?

Today, customers need to wrap cache calls with their own instrumentation. This adds setup work, leads to inconsistent metric definitions, and makes it harder to demonstrate the value of semantic caching. Redis server keyspace hit and miss statistics are not a substitute because they do not represent whether a semantic search produced an acceptable cache match.

Customer value

A more plug-and-play observability experience would help customers:

  • Quantify latency and LLM cost savings from semantic caching.
  • Tune thresholds, TTLs, and cache strategy using real workload data.
  • Detect drops in cache effectiveness or increases in lookup errors and latency.
  • Compare behavior across environments and releases.
  • Connect semantic cache signals to the monitoring tools they already use with minimal application code.

Desired experience

Customers should be able to opt into semantic cache observability with very little setup and get clearly defined, documented signals for each cache. The experience should be consistent across synchronous and asynchronous usage and, where practical, across RedisVL semantic cache backends.

Useful signals may include cache checks, hits, misses, hit ratio, lookup latency, errors, stores, cache size, and match-distance information. This list is intended to describe customer needs rather than prescribe a particular API, metrics library, or implementation.

Observability should add minimal overhead, avoid exposing prompt or response content, and have metric semantics that customers can rely on in dashboards and alerts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions