Skip to content

Implement better key naming fn #8

@lbradstreet

Description

@lbradstreet

We need the key naming fn to randomize the key prefixes to ensure proper s3 sharding. This could be done by generating a timestamp and reversing the end of the timestamp to ensure randomness.

The input plugin would then need to be modified to make this more pleasing to use, and ensure order is maintained.

This would break prefixing, as it depends on keys sharing a prefix. This could be worked around by listing all keys and filtering from onyx, rather than having amazon's servers pre filter. If we go this route we should have a metric for the time it takes to filter the keys.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions