Should as.integer64() always clear the cache?

> because `as.integer64` now removes all attributes, which includes the cache as well. Maybe this should be checked overall or `as.integer64` should keep the cache attribute. WDYT?

_via_ #265 

I don't have a strong feel here. I think we could at least expose an option to keep/remove the cache.

Letting this percolate for a while to build intuition.