s3torchconnector version
1.4.3
s3torchconnectorclient version
1.4.3
AWS Region
No response
Describe the running environment
Any environment
What happened?
chunk is PyBytes and chunk[:chunk_size] will create and copy this bytearray in https://github.com/awslabs/s3-connector-for-pytorch/blob/main/s3torchconnector/src/s3torchconnector/s3reader/ranged.py#L151
Instead we should use memoryview to use lightweight pointer arithmetic
Relevant log output
Code of Conduct