_s3dataset_common.parse_s3_uri method does not parse version IDs #367
Description
s3torchconnector version
s3torchconnector-1.4.3
s3torchconnectorclient version
s3torchconnectorclient-1.4.3
AWS Region
No response
Describe the running environment
Python 3 on Mac
What happened?
Attempting to load a prior checkpoint of the neural network state via
with s3_checkpoint.reader(uri) as reader:
return torch.load(reader)
URI used was s3://checkpoint/staged-model.pt?versionId=7e5a3665-6663-4242-a779-8fe58ccea8bf
When run without a versionId flag, code executes as expected. When run with versionId, result is:
S3Exception: Service error: The key does not exist
Relevant log output
---------------------------------------------------------------------------
S3Exception Traceback (most recent call last)
Cell In[10], line 3
1 net = Net()
2 #net.load_state_dict(torch.load(PATH, weights_only=True))
----> 3 checkpoint_state = dl.load_checkpoint(model_bucket, model_name, version_id="7e5a3665-6663-4242-a779-8fe58ccea8bf")
4 net.load_state_dict(checkpoint_state['model_state_dict'])
5 print(f"Epoch: {checkpoint_state["epoch"]}, Loss: {checkpoint_state["loss"]}")
File ~/Documents/PyTorch/dataset_loaders.py:137, in load_checkpoint(bucket_name, model_name, version_id)
134 logger.info(f"Load checkpoint from URI: {uri} in region: {region}")
136 with s3_checkpoint.reader(uri) as reader:
--> 137 return torch.load(reader)
File ~/Documents/PyTorch/.venv/lib/python3.13/site-packages/torch/serialization.py:1485, in load(f, map_location, pickle_module, weights_only, mmap, **pickle_load_args)
1482 pickle_load_args["encoding"] = "utf-8"
1484 with _open_file_like(f, "rb") as opened_file:
-> 1485 if _is_zipfile(opened_file):
1486 # The zipfile reader is going to advance the current file position.
1487 # If we want to actually tail call to torch.jit.load, we need to
1488 # reset back to the original position.
1489 orig_position = opened_file.tell()
1490 overall_storage = None
File ~/Documents/PyTorch/.venv/lib/python3.13/site-packages/torch/serialization.py:438, in _is_zipfile(f)
436 # Read the first few bytes and match against the ZIP file signature
437 local_header_magic_number = b"PK\x03\x04"
--> 438 read_bytes = f.read(len(local_header_magic_number))
439 f.seek(start)
440 return read_bytes == local_header_magic_number
File ~/Documents/PyTorch/.venv/lib/python3.13/site-packages/s3torchconnector/s3reader/sequential.py:116, in SequentialS3Reader.read(self, size)
111 if self._position_at_end():
112 # Invariant: if we're at EOF, it doesn't matter what `size` is, we'll always return no data and have no
113 # side effect.
114 return b""
--> 116 self.prefetch()
117 assert self._stream is not None
118 cur_pos = self._position
File ~/Documents/PyTorch/.venv/lib/python3.13/site-packages/s3torchconnector/s3reader/sequential.py:63, in SequentialS3Reader.prefetch(self)
56 """Start fetching data from S3.
57
58 Raises:
59 S3Exception: An error occurred accessing S3.
60 """
62 if self._stream is None:
---> 63 self._stream = self._get_stream()
File ~/Documents/PyTorch/.venv/lib/python3.13/site-packages/s3torchconnector/_s3client/_s3client.py:179, in S3Client._get_object_stream(self, bucket, key, start, end)
172 def _get_object_stream(
173 self,
174 bucket: str,
(...) 177 end: Optional[int] = None,
178 ) -> GetObjectStream:
--> 179 return self._client.get_object(bucket, key, start, end)
S3Exception: Service error: The key does not existCode of Conduct
- I agree to follow this project's Code of Conduct