Skip to content

SqliteMemoryService with FTS5 finds nothing for a Chinese or Japanese keyword inside a sentence #7959

Closed

Description

@Fruiterer333

🔴 Required Information

Describe the Bug:
SqliteMemoryService cannot find a stored Chinese or Japanese memory by a keyword when FTS5 is used, which is the default (fts="auto").

The memory 我想买这辆车,价格是二十万。 ("I want to buy this car, the price is 200k.") is not found by searching 价格 ("price") or 二十万 ("200k"). The same search finds it with fts="off" and with InMemoryMemoryService. English keywords work in all three.

Steps to Reproduce:

  1. Install ADK from main (reproduced at 3565ad3dd)
  2. Save the script below as repro.py
  3. Run python repro.py

Expected Behavior:
价格 and 二十万 each return 1 memory in every service and mode, like price does.

Observed Behavior:

InMemoryMemoryService              query='price'    -> 1 memories
InMemoryMemoryService              query='价格'       -> 1 memories
InMemoryMemoryService              query='二十万'      -> 1 memories
SqliteMemoryService(fts='auto')    query='price'    -> 1 memories
SqliteMemoryService(fts='auto')    query='价格'       -> 0 memories
SqliteMemoryService(fts='auto')    query='二十万'      -> 0 memories
SqliteMemoryService(fts='off')     query='price'    -> 1 memories
SqliteMemoryService(fts='off')     query='价格'       -> 1 memories
SqliteMemoryService(fts='off')     query='二十万'      -> 1 memories

Environment Details:

  • ADK Library Version (pip show google-adk): main at 3565ad3dd (version string 2.13.0)
  • Desktop OS: macOS 26.6.2
  • Python Version (python -V): 3.12.7

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: N/A (no model call)

🟡 Optional Information

Additional Context:
The FTS5 table uses tokenize='unicode61'. Chinese and Japanese are written without spaces, so unicode61 stores a whole run of characters between punctuation (e.g. 价格是二十万) as one token, and a keyword inside it never matches. The LIKE path (fts="off") does substring matching, so the two modes give different results for the same data.

The existing test test_search_cjk_mixed_latin_query only searches for the Latin word Python inside 私はPythonを使う; searching 使う in the same text also returns nothing with FTS. InMemoryMemoryService already covers this case (('我喜欢机器学习', '机器学习', 1) in its tests).

I found this while building a Chinese-language voice agent. I'd be happy to send a PR.

Minimal Reproduction Code:

import asyncio

from google.adk.events.event import Event
from google.adk.memory import InMemoryMemoryService, SqliteMemoryService
from google.adk.sessions.session import Session
from google.genai import types

session = Session(
    id="s1",
    app_name="app",
    user_id="u1",
    events=[
        Event(
            id="e1",
            author="user",
            timestamp=1.0,
            content=types.Content(
                role="user",
                parts=[types.Part(text="我想买这辆车,价格是二十万。")],
            ),
        ),
        Event(
            id="e2",
            author="user",
            timestamp=2.0,
            content=types.Content(
                role="user", parts=[types.Part(text="The price is 200k.")]
            ),
        ),
    ],
)


async def main():
  services = {
      "InMemoryMemoryService": InMemoryMemoryService(),
      "SqliteMemoryService(fts='auto')": SqliteMemoryService(":memory:"),
      "SqliteMemoryService(fts='off')": SqliteMemoryService(
          ":memory:", fts="off"
      ),
  }
  for name, service in services.items():
    await service.add_session_to_memory(session)
    for query in ["price", "价格", "二十万"]:
      response = await service.search_memory(
          app_name="app", user_id="u1", query=query
      )
      print(f"{name:34} query={query!r:10} -> {len(response.memories)} memories")
    if hasattr(service, "close"):
      await service.close()


asyncio.run(main())

How often has this issue occurred?:

  • Always (100%)

Activity

  1. added a commit that references this issue on Nov 4, 2026
    f8ced6f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions