For massive stores without manifest splitting, it is currently possible to create a manifest so large that IC cannot read it back. e.g. for GOES-16
earthmover/goes-16 — icechunk can't deserialize its largest manifest. GC fails deterministically inside find_retained → fetch_manifest on the 159MB manifest with Invalid Icechunk metadata file: Apparent size too large. That string sits in the icechunk 2.1.0 binary right next to "Too many tables" and "Nested table depth limit reached", so this is the flatbuffers verifier refusing an oversized buffer, not a memory budget your GC knobs control. Importantly, goes-16 was already dead-lettering on July 11, so the tuning didn't cause this — the OOM crashes were just masking it. This needs an upstream icechunk fix.
If the manifest is part of the commit history then this prevents reading the repo. If the manifest is not part of the history (i.e. it was expired or branch was reset) then the repo can be read from, but not GC'ed. This appears to be what happened with GOES-16.
@li-em did some work on flatbuffers limits, but they may need to be made stricter to prevent this.
For massive stores without manifest splitting, it is currently possible to create a manifest so large that IC cannot read it back. e.g. for GOES-16
If the manifest is part of the commit history then this prevents reading the repo. If the manifest is not part of the history (i.e. it was expired or branch was reset) then the repo can be read from, but not GC'ed. This appears to be what happened with GOES-16.
@li-em did some work on flatbuffers limits, but they may need to be made stricter to prevent this.