-
Notifications
You must be signed in to change notification settings - Fork 1
Data Search Command #12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
wigwam/data_commands.py
Outdated
|
||
|
||
def data_search( | ||
data_file: Path, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why Path
specifically here instead of str | os.PathLike
, which is more abstract. Usually we prefer more abstract types as inputs in order to avoid overly constraining users.
wigwam/data_commands.py
Outdated
names: Iterable[str], | ||
fields: Iterable[str], | ||
all: bool = False, | ||
print_output: bool = True, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
print_output
is missing a description in the docstring
workflowdata.json
Outdated
{ | ||
"data": [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure there's any reason for this "data" key. How about removing this outer layer? i.e. basically replace this:
{"data": [<actual contents>]}
with just this:
[<actual contents>]
{ | |
"data": [ | |
[ |
wigwam/search.py
Outdated
json_dict = json.load(file) | ||
|
||
# Get everything held underneath the "data" key on the dictionary. | ||
data: List[Dict[str, str | Dict[str, str]]] = json_dict["data"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
List[Dict[str, str | Dict[str, str]]]
This type description is kind of a mouthful and it doesn't really tell us much about the actual structure of the data contained within data
.
Instead of passing around objects with type annotations that look like this, let's create a dataclass to represent the contents of the JSON file in a more structured way, e.g.
from dataclasses import dataclass
@dataclass
class TestFile:
name: str
checksum: str
@dataclass
class TestDataset:
name: str
tags: list[str]
url: str
files: list[TestFile]
Then we could convert data
to a list[TestDataset]
.
Added the
data search
command line argument towigwam
. This command searches a file (defaultworkflowdata.json
in the root of the project) for all of the data items that fit the names or tags given as search terms.