Describe the bug
GridPatchDataset(cache=True) does not preserve dataset contents for several documented cache configurations:
- With a partial cache (
0 < cache_rate < 1), an uncached item following a cached item reuses the previous item's cache index and emits the cached patches again.
- With
with_coordinates=False, cached patches are zipped with an empty coordinate cache, so no cached patches are emitted.
- With
cache_rate=0 (or cache_num=0), initialization tries to unpack an empty cache and raises ValueError.
- With a transform pipeline containing no random transform, cached iteration passes
start=None to Compose and raises ValueError.
These are data-correctness issues: caching can silently replace or drop training samples, or reject valid cache configurations.
To Reproduce
from monai.data import GridPatchDataset
from monai.transforms import Lambda
def patches(image):
for item in image:
yield item, item * 10
partial = GridPatchDataset(
[[1], [2]], patches, cache=True, cache_rate=0.5, progress=False
)
print(list(partial))
# Actual: [(1, 10), (1, 10)]
# Expected: [(1, 10), (2, 20)]
no_coordinates = GridPatchDataset(
[[1, 2]], patches, with_coordinates=False, cache=True, progress=False
)
print(list(no_coordinates))
# Actual: []
# Expected: [1, 2]
zero_cache = GridPatchDataset(
[[1], [2]], patches, cache=True, cache_rate=0, progress=False
)
# Actual: ValueError: not enough values to unpack (expected 2, got 0)
# Expected: construct successfully and iterate without using a cache
deterministic = GridPatchDataset(
[[1]], patches, transform=Lambda(lambda x: x + 100), cache=True, progress=False
)
print(list(deterministic))
# Actual: ValueError: 'start' (None) cannot be None
# Expected: [(101, 10)]
Expected behavior
Enabling caching must not change which patches are yielded. cache_rate should only select how many source items are cached, and with_coordinates should only control whether coordinates are included in each yielded item. Deterministic transforms should be computed while populating the cache, and a cache hit should resume at the end of that transform pipeline.
Environment
MONAI version: 0+untagged.3487.gd1306f6
MONAI rev id: d1306f6d1996cffeb9d10984dd1c056b7fe2ed1d
Python version: 3.12.0
NumPy version: 2.5.2
PyTorch version: 2.14.0+cpu
OS: Windows
Additional context
The partial-cache issue comes from cache_index being initialized before the image loop and not reset for cache misses. The coordinate-free path always calls zip(data, other) even though _cache_other is intentionally empty when with_coordinates=False. set_data() unconditionally unpacks zip(*self._fill_cache(...)), including when the configured cache size is zero. Finally, Compose.get_index_of_first(...) returns None when every transform is deterministic, but the cache-read path uses that value as the start index.
The cache implementation was introduced in #7180. Existing coverage exercises a full cache with coordinates enabled and a pipeline containing a random transform, so these paths are currently untested.
Describe the bug
GridPatchDataset(cache=True)does not preserve dataset contents for several documented cache configurations:0 < cache_rate < 1), an uncached item following a cached item reuses the previous item's cache index and emits the cached patches again.with_coordinates=False, cached patches are zipped with an empty coordinate cache, so no cached patches are emitted.cache_rate=0(orcache_num=0), initialization tries to unpack an empty cache and raisesValueError.start=NonetoComposeand raisesValueError.These are data-correctness issues: caching can silently replace or drop training samples, or reject valid cache configurations.
To Reproduce
Expected behavior
Enabling caching must not change which patches are yielded.
cache_rateshould only select how many source items are cached, andwith_coordinatesshould only control whether coordinates are included in each yielded item. Deterministic transforms should be computed while populating the cache, and a cache hit should resume at the end of that transform pipeline.Environment
Additional context
The partial-cache issue comes from
cache_indexbeing initialized before the image loop and not reset for cache misses. The coordinate-free path always callszip(data, other)even though_cache_otheris intentionally empty whenwith_coordinates=False.set_data()unconditionally unpackszip(*self._fill_cache(...)), including when the configured cache size is zero. Finally,Compose.get_index_of_first(...)returnsNonewhen every transform is deterministic, but the cache-read path uses that value as thestartindex.The cache implementation was introduced in #7180. Existing coverage exercises a full cache with coordinates enabled and a pipeline containing a random transform, so these paths are currently untested.