Cache manifest and manifest-list bytes by location, as Iceberg Java's io.manifest.cache-* does #4300

Feature Request / Improvement

Planning the same table repeatedly reads the same immutable files from object storage every time, and fails while the store is unreachable, even for a snapshot just planned.

Per load_table plus plan_files, pyiceberg 0.12.0 (and main) reads:

1. metadata.json on every load, on non-REST catalogs (REST returns it inline).
2. The manifest list on every plan. _manifests re-reads it on purpose, to avoid the O(N²) memory of caching overlapping ManifestFile tuples (Use cachetools's LRUCache to cache manifest list  #1187).
3. Each manifest on every plan, via fetch_manifest_entry. _manifest_cache keeps the ManifestFile records, not the manifests' contents.

These files are never rewritten, so the bytes at a location never change.

Iceberg Java caches these bytes. ManifestFiles and ManifestLists read through a per-FileIO ContentCache, set by these properties (CatalogProperties):

• io.manifest.cache-enabled (default false)
• io.manifest.cache.max-total-bytes (default 100 MiB)
• io.manifest.cache.max-content-length (default 8 MiB)
• io.manifest.cache.expiration-interval-ms (default 60 s)

Proposal

1. Cache manifest and manifest-list bytes by location, off by default and bounded in bytes, as in Java. Parsing stays as it is.

   Caching bytes avoids the problem in Process-global manifest cache is keyed only on manifest_path #3978 / fix(manifest): verify cached ManifestFile content before reusing it #3991. A ManifestFile record carries values from the manifest list that names it, so a parsed-object cache keyed by path can serve one reference's values to another. (iceberg-rust adds added_snapshot_id, sequence_number and first_row_id to its key for this reason.) Bytes are the same for every reference.

2. Let the caller supply the cache, e.g. get(location) -> bytes | None and put(location, data), with the in-memory one as the default. A disk cache behind it would survive restarts and keep planning working through a store outage for snapshots already read. iceberg-rust has a similar hook (ObjectCacheProvide) for parsed objects.

metadata.json could use the same cache. For REST, #4075 / #4094 cover it with If-None-Match.

Java configures this with FileIO properties; #2952 used Config and an env var for manifest-cache-size. Either is fine.

Related: #4007 and #3662 make each read cheaper; this removes repeated reads.

What we do today

In Strata (a materialization layer that, among other things, caches Iceberg scans) we set py-io-impl to a FileIO that wraps the one load_file_io would pick. Its new_input serves *.metadata.json and *.avro on remote schemes from a bounded disk cache, keyed by location plus the endpoint properties, and passes everything else through.

Drawbacks:

• It identifies immutable files by suffix.
• It takes over py-io-impl, so it must carry a user's own py-io-impl under another property.
• It builds an inner FileIO per scheme, since a FileIO does not know which locations it will serve.

A cache hook at the read points above would remove the need for the wrapper.