Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
de65b44
[feat] Added Attrs with validation preventing special keys. Added att…
yhteoh Sep 23, 2025
47f6bce
[feat] Added indexing into datastore indexes into groups
yhteoh Sep 23, 2025
0a5445c
[feat] Strict field type requirements for subclasses of GroupBase
yhteoh Sep 23, 2025
6d66226
[feat] Added str, bytes and bool support for Datasets
yhteoh Sep 24, 2025
0bc7790
[fix] serialization of str to hdf5. str are serialized as bytes durin…
yhteoh Sep 24, 2025
850e872
[feat, refactor] refactor dtype map to use Enum DTypes. Implemented c…
yhteoh Sep 24, 2025
f5a9276
[feat] Implemented CastDataset that cast a numpy array automatically …
yhteoh Sep 24, 2025
ca8920c
[rename] renamed protected attrs in hdf5
yhteoh Sep 24, 2025
a5c7ad3
[feat] Allow dictionary of datasets as a field in a Group
yhteoh Sep 24, 2025
f422b43
[feat] Implemented flex shape support and refactored validators
yhteoh Sep 25, 2025
92a5719
[test] Added more test of datasets, datastores and groups
yhteoh Sep 26, 2025
11abee3
[refactor] refactor datastore group validation and added comments
yhteoh Sep 26, 2025
1d82e3e
[fix, feat] Added method to Datastore to add and update groups. fixed…
yhteoh Oct 2, 2025
7ae3459
[fix] bug with Datastore.add, it may partially update groups before t…
yhteoh Oct 2, 2025
b76239e
[gitignore] updated gitignore
yhteoh Oct 6, 2025
453bf96
[fix] Saving and loading of optional datasets
yhteoh Oct 6, 2025
3c42527
[fix] protected attrs should be _group_schema
yhteoh Oct 6, 2025
0c9ecbf
[test] Added test for unmatched flex shape dataset
yhteoh Oct 6, 2025
caa91af
[refactor, feat] refactored package layout and added support for Tabl…
yhteoh Oct 6, 2025
af44790
[feat] Table supports casting to and from pandas.DataFrame
yhteoh Oct 6, 2025
bed844a
[refactor,feat] constrained types moved to separate module, Added req…
yhteoh Oct 6, 2025
3fd87d9
[feat] Table uses np.recarray for data
yhteoh Oct 6, 2025
c0ced54
[fix] error message for GroupBase, datafield can be Dataset or Table
yhteoh Oct 6, 2025
a9a29be
[feat] Implemented Folder for document type datastore.
yhteoh Oct 6, 2025
d42791c
[refactor] rename pd property of Table to dataframe
yhteoh Oct 7, 2025
2ab7791
Merge branch 'table' of https://github.com/OpenQuantumDesign/oqd-data…
yhteoh Oct 7, 2025
868cb06
[format] formatted files
yhteoh Oct 7, 2025
9a70409
[refactor,feat] Implemented abstract superclass GroupField for (Datas…
yhteoh Oct 7, 2025
3bfcda3
[fix] Modify document schema None dtypes to concrete values when load…
yhteoh Oct 7, 2025
b6c69b4
[feat] Added function for constructing numpy datatype from Table and …
yhteoh Oct 7, 2025
ab22d7a
[feat] Implemented CastFolder automatic casting of structured numpy a…
yhteoh Oct 7, 2025
a0da48f
[feat] Added helper functions unstructured_to_structured and dict_to_…
yhteoh Oct 7, 2025
d75e9ed
[clean] remove debugging print statement
yhteoh Oct 8, 2025
8129d6e
[fix] unstructured_to_structured infers the size str dtype from the data
yhteoh Oct 8, 2025
fbb1dca
[fix] use append_fields in dict_to_structured and also fixed append_f…
yhteoh Oct 8, 2025
7cb8806
[format] Formatted code
yhteoh Oct 8, 2025
ba51d01
[gitignore] updated gitignore
yhteoh Oct 14, 2025
d261e2c
[docs, format] Updated documentation, docstrings and default field in…
yhteoh Oct 14, 2025
06f7e6f
[docs] Updated documentation and docstrings
yhteoh Oct 14, 2025
0314d1a
[clean] remove concrete group implementations
yhteoh Oct 14, 2025
358120b
[docs] updated documentation and docstrings
yhteoh Oct 14, 2025
fd1693e
[fix] edge case where only a single group registered
yhteoh Oct 14, 2025
4295898
Merge pull request #10 from OpenQuantumDesign/table-and-folder
yhteoh Oct 14, 2025
d507aed
[feat] Added pipe to Datastore
yhteoh Oct 15, 2025
bf1fba7
[fix] GroupBase allows attrs field
yhteoh Oct 15, 2025
7d18f6c
[packaging] update exports
yhteoh Oct 15, 2025
5bc133a
[docs] update tutorial on docs
yhteoh Oct 15, 2025
db71491
[feat] Add validation for func output
yhteoh Oct 15, 2025
916e71e
[docs] fix tutorial documentation
yhteoh Oct 15, 2025
ce9246a
[docs] updated tutorials
yhteoh Oct 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1 @@
postprocessing/** linguist-vendored
postprocessing/** linguist-vendored
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@
- Subsystem:


* **Other information** (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)
* **Other information** (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)
4 changes: 2 additions & 2 deletions .github/workflows/check_copyright.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ jobs:
- name: Check license & copyright headers
uses: viperproject/check-license-header@v2
with:
path:
path:
config: .github/workflows/check_copyright_config.json
# strict: true
# strict: true
2 changes: 1 addition & 1 deletion .github/workflows/check_mkdocs_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ jobs:
uses: astral-sh/setup-uv@v4
# - run: cp -r examples/ docs/examples/
- run: uv pip install .[docs] --system
- run: mkdocs build
- run: mkdocs build
2 changes: 1 addition & 1 deletion .github/workflows/copyright.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# limitations under the License.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,5 @@ cython_debug/
*.h5
*.code-workspace
.pre-commit-config.yaml
_scripts
.vscode
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The design goals are to have:
- Minimizes *a priori* knowledge that is needed of the internal heirarchical structure, reducing friction for users to load data.
- Transparently return both raw and processed data, where the levels of post-processing can be selected by the user.

To install,
To install,
```bash
pip install git+https://github.com/OpenQuantumDesign/oqd-dataschema.git
```
33 changes: 0 additions & 33 deletions docs/api.md

This file was deleted.

21 changes: 21 additions & 0 deletions docs/api/base.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## Attributes Types

<!-- prettier-ignore -->
::: oqd_dataschema.base
options:
heading_level: 3
members: [
"AttrKey",
"Attrs",
]

## Data Types

<!-- prettier-ignore -->
::: oqd_dataschema.base
options:
heading_level: 3
members: [
"DTypes",
"DTypeNames",
]
7 changes: 7 additions & 0 deletions docs/api/datastore.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<!-- prettier-ignore -->
::: oqd_dataschema.datastore
options:
heading_level: 3
members: [
"Datastore",
]
8 changes: 8 additions & 0 deletions docs/api/group.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<!-- prettier-ignore -->
::: oqd_dataschema.group
options:
heading_level: 3
members: [
"GroupBase",
"GroupRegistry",
]
53 changes: 53 additions & 0 deletions docs/api/groupfield.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
<!-- prettier-ignore -->
::: oqd_dataschema.base
options:
filters: []
heading_level: 3
members: [
"GroupField",
]

## Dataset

<!-- prettier-ignore -->
::: oqd_dataschema.dataset
options:
heading_level: 3
members: [
"Dataset",
"CastDataset",
]

## Table

<!-- prettier-ignore -->
::: oqd_dataschema.table
options:
heading_level: 3
members: [
"Table",
"CastTable",
]

## Folder

<!-- prettier-ignore -->
::: oqd_dataschema.folder
options:
heading_level: 3
members: [
"Folder",
"CastFolder",
]

## Constrained Group Fields

<!-- prettier-ignore -->
::: oqd_dataschema.constrained
options:
heading_level: 3
members: [
"condataset",
"contable",
"confolder",
]
10 changes: 10 additions & 0 deletions docs/api/utils.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## Dataset

<!-- prettier-ignore -->
::: oqd_dataschema.utils
options:
heading_level: 3
members: [
"dict_to_structured",
"unstructured_to_structured",
]
22 changes: 22 additions & 0 deletions docs/explanation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## Datastore

A [Datastore][oqd_dataschema.datastore.Datastore] represents a HDF5 file of a particular hierarchical structure.

### Hierarchy

```
/
├── group1/
│ └── dataset1
├── group2/
│ ├── dataset2
│ ├── table1
│ └── folder1
└── group3/
├── table2
└── dataset_dict1/
├── dataset5
└── dataset6
```

The top level of [Datastore][oqd_dataschema.datastore.Datastore] contains multiple [Groups](api/group.md)
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#
#

<p align="center">
<img src="img/oqd-logo-black.png#only-light" alt="Logo" style="max-height: 200px;">
Expand Down
2 changes: 1 addition & 1 deletion docs/stylesheets/admonition_template.css
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@
background-color: #FFFFFF;
-webkit-mask-image: var(--md-admonition-icon--template);
mask-image: var(--md-admonition-icon--template);
}
}
3 changes: 0 additions & 3 deletions docs/stylesheets/admonitions.css
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,3 @@
-webkit-mask-image: var(--md-admonition-icon--acknowledgement);
mask-image: var(--md-admonition-icon--acknowledgement);
}



18 changes: 9 additions & 9 deletions docs/stylesheets/brand.css
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,12 @@ h1, h2, h3, h4, h5, h6,


/* Apply Raleway to all navigation and sidebar elements */
.md-nav,
.md-nav__title,
.md-nav__link,
.md-header,
.md-tabs,
.md-sidebar,
.md-nav,
.md-nav__title,
.md-nav__link,
.md-header,
.md-tabs,
.md-sidebar,
.md-sidebar__inner,
.md-nav__item,
.md-footer,
Expand Down Expand Up @@ -79,7 +79,7 @@ h1, h2, h3, h4, h5, h6,


/* Light mode nav/ToC font color */
[data-md-color-scheme="default"] .md-nav,
[data-md-color-scheme="default"] .md-nav,
[data-md-color-scheme="default"] .md-nav__link,
[data-md-color-scheme="default"] .md-header,
[data-md-color-scheme="default"] .md-tabs {
Expand All @@ -88,7 +88,7 @@ h1, h2, h3, h4, h5, h6,
}

/* Dark mode nav/ToC font color */
[data-md-color-scheme="slate"] .md-nav,
[data-md-color-scheme="slate"] .md-nav,
[data-md-color-scheme="slate"] .md-nav__link,
[data-md-color-scheme="slate"] .md-header,
[data-md-color-scheme="slate"] .md-tabs {
Expand All @@ -113,4 +113,4 @@ h1, h2, h3, h4, h5, h6,
.md-header .md-tabs__link:hover {
color: #ffffff !important;
text-decoration: underline;
}
}
Loading
Loading