Backport downloader features from red-datasets #10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR backports downloader features from red-datasets, introducing support for fallback URLs, caching improvements, and enhanced HTTP functionality. The changes address multiple issues (#4, #6, #7) by extending the downloader's capabilities while maintaining backward compatibility.
- Adds support for fallback URLs to handle download failures
- Implements HTTP POST method support with parameters and GitHub token authentication
- Refactors caching logic and improves error handling
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| lib/remote_input/downloader.rb | Core implementation adding fallback URL support, HTTP POST functionality, improved caching logic, and GitHub token authentication |
| test/test-downloader.rb | Comprehensive test suite covering new initialization options, caching behavior, and fallback URL functionality |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Co-authored-by: Copilot <[email protected]>
kou
pushed a commit
to red-data-tools/red-datasets
that referenced
this pull request
Aug 15, 2025
Correct the elapsed time comparison for lock file validation. Lock files should be considered valid when they are recent (< 10 seconds), not when they are old (> 10 seconds). - Before: Lock files older than 10 seconds were considered valid - After: Lock files newer than 10 seconds are considered valid Originally suggested in a Copilot [review](red-data-tools/red-remote-input#10 (comment))
kou
reviewed
Aug 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #4
Fix #6
Fix #7