-
Notifications
You must be signed in to change notification settings - Fork 2
Return file paths for downloaded files #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR modifies the download function to return a dictionary mapping IDs to lists of downloaded file paths, making it easier for users to programmatically access the downloaded files without having to manually parse directory contents or match table metadata.
- Return file paths from the
downloadmethod as a dictionary mapping ID to file paths - Update internal URL building to use ID-to-URL mapping instead of just URL lists
- Add test verification that returned file paths actually exist
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| hepdata_cli/api.py | Modified download method to return file paths, updated _build_urls to return ID-to-URL mapping, enhanced download_url to track extracted files |
| tests/test_download.py | Updated test to verify returned file paths exist on filesystem |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
@codecov-ai-reviewer review |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@codecov-ai-reviewer review |
This comment has been minimized.
This comment has been minimized.
|
@runtingt : thanks for the PR! This seems like a useful addition and I'd be happy to merge. Please check the AI-generated review comments above and address them where relevant. Also, please update the |
|
@GraemeWatt Thanks for the review! I've made some changes based on the suggestions above |
|
@codecov-ai-reviewer test |
|
On it! Codecov is generating unit tests for this PR. |
|
@codecov-ai-reviewer test |
|
On it! Codecov is generating unit tests for this PR. |
GraemeWatt
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@runtingt : thanks for making the changes and extending the tests. Testing of the more robust tar file extraction is not straightforward because the tar files downloaded from hepdata.net should all be well formed, but your approach of mocking the response looks good. An alternative approach would be to refactor the download_url function so that the tar file extraction is handled by a separate function and is therefore easier to test.
I tried to use the @codecov-ai-reviewer test command (twice) to generate unit tests automatically, and commits c1a94ef and 5670820 were added to new branches, but the tests fail and PRs were not automatically opened, so I'll stick to your version of the tests. Thanks again for your contribution. I'll tag the new release 0.3.0 today.
It's not immediately obvious to me how to determine the path of the downloaded file(s), beyond the directory specified in
download_dir. I suppose one could read thejsonfor the table information and then match eachnameto theprocessed_name, but this feels cumbersomeThis PR modifies the
downloadfunction to return these paths and updates the unit tests accordingly.Apologies if there's already an easy way to do this!