Commit 9bf4848
committed
feat: implement unified stores configuration system
- Remove legacy ExternalSettings and ObjectStorageSettings classes
- Update Config to use stores.default and stores.<name> structure
- Update get_store_spec() to support default store (store=None)
- Add partition_pattern and token_length support for stores
- Update secrets loading to support per-store credentials
- Update save_template() to generate unified stores config
- Update _update_from_flat_dict() to handle stores.<name>.<attr> pattern
- Remove external.* from ENV_VAR_MAPPING
Unified stores configuration supports both:
- Hash-addressed storage (<blob@>, <attach@>) via _hash section
- Schema-addressed storage (<object@>, <npy@>) via _schema section
Configuration structure:
stores.default - name of default store
stores.<name>.protocol - storage protocol (file, s3, gcs, azure)
stores.<name>.location - base path (includes project context)
stores.<name>.partition_pattern - schema-addressed partitioning
stores.<name>.token_length - random token length
stores.<name>.subfolding - hash-addressed subfolding1 parent 3f1d072 commit 9bf4848
1 file changed
+101
-240
lines changed
0 commit comments