Dynamic Preload cache for precompiled MXRs#201

Draft

TedThemistokleous wants to merge 9 commits intoadd_dynamic_batchfrom

debug_batch_size

Commits on Dec 30, 2025

add cache of preloaded models and use max_dynamic batch size for multiple batch models created then loaded into memory on initialization
TedThemistokleous
committed
Update to use session batch instead of previous stored batch value for lookup
TedThemistokleous
committed

Commits on Jan 2, 2026

Put precompiled mxrs into lookup to allow for lookup based compiled batch sizes
TedThemistokleous
committed
[AI Generated] Disable output shape verification unless verbose logging is enabled
TedThemistokleous
committed
[AI Generated] Ensure all batch sizes are pre-compiled before compute threads start
TedThemistokleous
committed
[AI Generated] Restore CompileProgramWithBatch and improve batch compilation logic
TedThemistokleous
committed
[AI Generated] Revert hash calculation to use first input shape only
TedThemistokleous
committed
[AI Generated] Fix batch cache not being populated for compute threads
TedThemistokleous
committed
[AI Generated] Fix batch cache empty when model has dynamic/symbolic shapes
TedThemistokleous
committed