Crashes during middle of clear_cache due to weak pointers

[Boshen]

#817

[sapphi-red]

Moving to #872 for now, is this still a blocking issue?

I've updated to latest version but it still happens:
https://github.com/vitejs/rolldown-vite/actions/runs/19702965387/job/56443484444?pr=471#step:11:227