[Bug]: LocalEvalSampler treats explicit empty eval case ID lists as all cases

[boshilin123]

🔴 Required Information

Describe the Bug:

LocalEvalSamplerConfig documents train_eval_case_ids and validation_eval_case_ids as optional lists where all cases are selected when a field is not provided. However, LocalEvalSampler.__init__ uses truthiness checks, so explicitly passing an empty list is indistinguishable from passing None.

As a result, train_eval_case_ids=[] expands to every training case, and validation_eval_case_ids=[] expands to every validation case (or inherits the training cases). This can unexpectedly turn a no-case selection into a full evaluation run.

Steps to Reproduce:

1. Install the current google-adk repository in an isolated environment.
2. Construct LocalEvalSamplerConfig with explicit empty training and validation case ID lists.
3. Construct LocalEvalSampler while returning known IDs from _get_eval_case_ids.
4. Inspect get_train_example_ids() and get_validation_example_ids().

Minimal code is included below.

Expected Behavior:

Only None / an omitted field should trigger the documented “all eval cases” fallback. An explicit empty list should either remain empty, consistent with sample_and_score(batch=[]), or be rejected with a clear validation error if empty datasets are unsupported. It should not silently expand to all cases.

Observed Behavior:

The explicit empty lists are replaced with every ID returned for their respective eval sets:

configured train: []
actual train: ['train-1', 'train-2']
configured validation: []
actual validation: ['validation-1']
lookup calls: [('train',), ('validation',)]

Environment Details:

• ADK Library Version: editable install from commit 3f24d2036a3434b755f6337026ac19a737041f85 (google-adk 2.9.0)
• Desktop OS: Windows 10.0.26200
• Python Version: Python 3.12 (64-bit)

Model Information:

• Are you using LiteLLM: No
• Which model is being used: N/A — the issue occurs during sampler construction before any model call

---

🟡 Optional Information

Regression:

Unknown. The truthiness behavior is present in the commit that originally introduced LocalEvalSampler.

Logs:

N/A — no external service or model call is required.

Screenshots / Video:

N/A.

Additional Context:

The constructor currently uses:

self._config.train_eval_case_ids or self._get_eval_case_ids(...)

and:

if self._config.validation_eval_case_ids:

Both checks collapse [] and None even though the Pydantic model and field descriptions distinguish them. The existing parameterized constructor test covers omitted and non-empty lists, but not explicit empty lists.

I searched the repository issues and pull requests for train_eval_case_ids and validation_eval_case_ids and found no existing report or implementation. I would be happy to contribute a focused fix and regression tests once the intended empty-list behavior is confirmed.

Minimal Reproduction Code:

from unittest.mock import MagicMock, patch

from google.adk.evaluation.eval_config import EvalConfig
from google.adk.evaluation.eval_sets_manager import EvalSetsManager
from google.adk.optimization.local_eval_sampler import LocalEvalSampler
from google.adk.optimization.local_eval_sampler import LocalEvalSamplerConfig

config = LocalEvalSamplerConfig(
    eval_config=EvalConfig(),
    app_name="app",
    train_eval_set="train",
    train_eval_case_ids=[],
    validation_eval_set="validation",
    validation_eval_case_ids=[],
)

with patch.object(
    LocalEvalSampler,
    "_get_eval_case_ids",
    side_effect=lambda eval_set_id: (
        ["train-1", "train-2"]
        if eval_set_id == "train"
        else ["validation-1"]
    ),
):
  sampler = LocalEvalSampler(config, MagicMock(spec=EvalSetsManager))

print("configured train:", config.train_eval_case_ids)
print("actual train:", sampler.get_train_example_ids())
print("configured validation:", config.validation_eval_case_ids)
print("actual validation:", sampler.get_validation_example_ids())

How often has this issue occurred?:

• Always (100%)

[chelsealong]

Picking this one up now — opening a PR shortly. Flagging it here so nobody duplicates the work; if someone is already on it, say so and I will drop mine.

[surajksharma07]

@boshilin123 The empty lists really do expand to every case since both checks use truthiness instead of an is not None check. chelsealong's #7170 swaps those two checks and keeps [] as a genuine empty selection matching the documented behavior. Could you pull that branch and run it against your actual eval-case setup to confirm it holds before it merges?

[boshilin123]

Thanks @surajksharma07. I applied #7170 on top of commit 3f24d203 and reran the same LocalEvalSampler setup from my report. Both explicit empty selections are now preserved:

configured train: []
actual train: []
configured validation: []
actual validation: []
lookup calls: []

This confirms that the patch fixes both the training and validation paths and avoids unnecessary eval-case ID lookups. The behavior now matches the documented distinction between None and an explicit empty list.