Skip to content

ts.keys() only works when the prefix is separated by a dot? #89

@galrotem

Description

@galrotem

Hey experts,
I noticed some weird behavior which I wasn't sure whether is intentional or a bug. Here's the setup:

Assuming my keys are prefixed with PREFIX

Then, if the keys I store (with ts.put()) are of the template PREFIX/{key}, rather than {PREFIX}.key, when I call ts.keys(PREFIX) - the stored values aren't returned.

Is that expected? I assume not, because I couldn't find anywhere in the documentation that says that the prefix value that is passed to ts.keys() must be separated by a dot from the suffixes.

This small repro snippet can demonstrate the issue:

param_names = ["model.layers.0.weight", "model.layers.0.bias"]
test_tensor = torch.tensor([1.0, 2.0, 3.0])

slash_prefix = f"policy_state_dict_{version}"
slash_keys = [f"{slash_prefix}/{name}" for name in param_names]

# Store with "/" delimiter
for key in slash_keys:
    await ts.put(key, test_tensor.clone())

# Try to retrieve keys with prefix
slash_matching_keys = await ts.keys(slash_prefix) # nothing will be returned 

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions