Repository navigation
Description
🔴 Required Information
Describe the Bug:
SqliteMemoryService cannot find a stored Chinese or Japanese memory by a keyword when FTS5 is used, which is the default (fts="auto").
The memory 我想买这辆车,价格是二十万。 ("I want to buy this car, the price is 200k.") is not found by searching 价格 ("price") or 二十万 ("200k"). The same search finds it with fts="off" and with InMemoryMemoryService. English keywords work in all three.
Steps to Reproduce:
- Install ADK from
main(reproduced at3565ad3dd) - Save the script below as
repro.py - Run
python repro.py
Expected Behavior:
价格 and 二十万 each return 1 memory in every service and mode, like price does.
Observed Behavior:
InMemoryMemoryService query='price' -> 1 memories
InMemoryMemoryService query='价格' -> 1 memories
InMemoryMemoryService query='二十万' -> 1 memories
SqliteMemoryService(fts='auto') query='price' -> 1 memories
SqliteMemoryService(fts='auto') query='价格' -> 0 memories
SqliteMemoryService(fts='auto') query='二十万' -> 0 memories
SqliteMemoryService(fts='off') query='price' -> 1 memories
SqliteMemoryService(fts='off') query='价格' -> 1 memories
SqliteMemoryService(fts='off') query='二十万' -> 1 memories
Environment Details:
- ADK Library Version (pip show google-adk):
mainat3565ad3dd(version string 2.13.0) - Desktop OS: macOS 26.6.2
- Python Version (python -V): 3.12.7
Model Information:
- Are you using LiteLLM: No
- Which model is being used: N/A (no model call)
🟡 Optional Information
Additional Context:
The FTS5 table uses tokenize='unicode61'. Chinese and Japanese are written without spaces, so unicode61 stores a whole run of characters between punctuation (e.g. 价格是二十万) as one token, and a keyword inside it never matches. The LIKE path (fts="off") does substring matching, so the two modes give different results for the same data.
The existing test test_search_cjk_mixed_latin_query only searches for the Latin word Python inside 私はPythonを使う; searching 使う in the same text also returns nothing with FTS. InMemoryMemoryService already covers this case (('我喜欢机器学习', '机器学习', 1) in its tests).
I found this while building a Chinese-language voice agent. I'd be happy to send a PR.
Minimal Reproduction Code:
import asyncio
from google.adk.events.event import Event
from google.adk.memory import InMemoryMemoryService, SqliteMemoryService
from google.adk.sessions.session import Session
from google.genai import types
session = Session(
id="s1",
app_name="app",
user_id="u1",
events=[
Event(
id="e1",
author="user",
timestamp=1.0,
content=types.Content(
role="user",
parts=[types.Part(text="我想买这辆车,价格是二十万。")],
),
),
Event(
id="e2",
author="user",
timestamp=2.0,
content=types.Content(
role="user", parts=[types.Part(text="The price is 200k.")]
),
),
],
)
async def main():
services = {
"InMemoryMemoryService": InMemoryMemoryService(),
"SqliteMemoryService(fts='auto')": SqliteMemoryService(":memory:"),
"SqliteMemoryService(fts='off')": SqliteMemoryService(
":memory:", fts="off"
),
}
for name, service in services.items():
await service.add_session_to_memory(session)
for query in ["price", "价格", "二十万"]:
response = await service.search_memory(
app_name="app", user_id="u1", query=query
)
print(f"{name:34} query={query!r:10} -> {len(response.memories)} memories")
if hasattr(service, "close"):
await service.close()
asyncio.run(main())How often has this issue occurred?:
- Always (100%)
Activity
fix: find CJK and Thai words inside a sentence in sqlite memory