Skip to content

Conversation

@runtingt
Copy link
Contributor

It's not immediately obvious to me how to determine the path of the downloaded file(s), beyond the directory specified in download_dir. I suppose one could read the json for the table information and then match each name to the processed_name, but this feels cumbersome

This PR modifies the download function to return these paths and updates the unit tests accordingly.

Apologies if there's already an easy way to do this!

@GraemeWatt GraemeWatt requested a review from Copilot August 29, 2025 20:14
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR modifies the download function to return a dictionary mapping IDs to lists of downloaded file paths, making it easier for users to programmatically access the downloaded files without having to manually parse directory contents or match table metadata.

  • Return file paths from the download method as a dictionary mapping ID to file paths
  • Update internal URL building to use ID-to-URL mapping instead of just URL lists
  • Add test verification that returned file paths actually exist

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
hepdata_cli/api.py Modified download method to return file paths, updated _build_urls to return ID-to-URL mapping, enhanced download_url to track extracted files
tests/test_download.py Updated test to verify returned file paths exist on filesystem

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@GraemeWatt
Copy link
Member

@codecov-ai-reviewer review

@codecov-ai

This comment has been minimized.

@codecov-ai

This comment has been minimized.

@GraemeWatt
Copy link
Member

@codecov-ai-reviewer review

@codecov-ai

This comment has been minimized.

@GraemeWatt
Copy link
Member

@runtingt : thanks for the PR! This seems like a useful addition and I'd be happy to merge. Please check the AI-generated review comments above and address them where relevant. Also, please update the README.md file to include the dictionary now returned by client.download, e.g. by modifying Example 4.

@runtingt
Copy link
Contributor Author

runtingt commented Sep 1, 2025

@GraemeWatt Thanks for the review! I've made some changes based on the suggestions above

@GraemeWatt
Copy link
Member

@codecov-ai-reviewer test

@codecov-ai
Copy link

codecov-ai bot commented Sep 1, 2025

On it! Codecov is generating unit tests for this PR.

@GraemeWatt
Copy link
Member

@codecov-ai-reviewer test

@codecov-ai
Copy link

codecov-ai bot commented Sep 1, 2025

On it! Codecov is generating unit tests for this PR.

Copy link
Member

@GraemeWatt GraemeWatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@runtingt : thanks for making the changes and extending the tests. Testing of the more robust tar file extraction is not straightforward because the tar files downloaded from hepdata.net should all be well formed, but your approach of mocking the response looks good. An alternative approach would be to refactor the download_url function so that the tar file extraction is handled by a separate function and is therefore easier to test.

I tried to use the @codecov-ai-reviewer test command (twice) to generate unit tests automatically, and commits c1a94ef and 5670820 were added to new branches, but the tests fail and PRs were not automatically opened, so I'll stick to your version of the tests. Thanks again for your contribution. I'll tag the new release 0.3.0 today.

@GraemeWatt GraemeWatt merged commit cbfe81c into HEPData:main Sep 1, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants