Reduce usage of unsafe constructs throughout codebase (#7426)

- Targets StreamUtils, ToByteArrayExtensions, and VectorUtils
- For VectorUtils, only targets areas where we looked at the codegen and confirmed the JIT properly elides bounds checks

Author: GrabYourPitchforks