Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jul 7, 2025

Summary

This pull request introduces support for a new REPOSITORY_SEARCH_QUERY feature, enhances documentation, and includes minor updates to workflows and code. The most significant changes involve adding the ability to filter repositories using a search query, updating the README.md to reflect this feature, and modifying the env.py and evergreen.py files to handle the new functionality.

This allows GitHub Search API (github_connection.search_repositories) instead of enumerating all organization repositories. This allows callers to supply queries like:

org:my-org is:repository archived:false created:>2025-07-01

and only iterate over matching repositories up‑front, saving time and API rate limits. Existing behavior remains unchanged when the variable is not set, and all existing unit tests continue to pass.

Fixes #372.

Details

New Feature: Repository Search Query Support

  • Environment Variable Addition: Added support for the REPOSITORY_SEARCH_QUERY environment variable in env.py. This allows filtering repositories using a GitHub Search API query. Updated the get_env_vars function to validate that at least one of ORGANIZATION, REPOSITORY, or REPOSITORY_SEARCH_QUERY is set. [1] [2] [3] [4]
  • Repository Iterator Update: Modified get_repos_iterator in evergreen.py to handle repository filtering via the search query. If REPOSITORY_SEARCH_QUERY is set, the GitHub Search API is used to retrieve matching repositories.
  • Main Function Update: Updated the main function in evergreen.py to pass the search_query parameter to get_repos_iterator. [1] [2]

Documentation Updates

  • Workflow Example: Added an example workflow to the README.md demonstrating how to use the REPOSITORY_SEARCH_QUERY variable in a GitHub Actions workflow.
  • Documentation Enhancements: Updated the README.md to include REPOSITORY_SEARCH_QUERY as a supported variable, explaining its purpose and usage. Improved clarity in other sections, such as linking to GitHub documentation for encrypted secrets. [1] [2]

Minor Improvements

  • Super-Linter Workflow: Enabled automatic fixing of Markdown files by setting FIX_MARKDOWN_PRETTIER: true in .github/workflows/super-linter.yaml.
  • Code Comments: Added a # pragma: no cover comment in env.py to exclude certain branches from test coverage.

Testing

I tested this on the github org and created 43 new PRs!


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

🚧 Copilot coding agent is using an experimental model for this task. Please take the time to report feedback with the 👍 or 👎 buttons.

@zkoppert zkoppert changed the title [WIP] Efficient repository enumeration via search API use [WIP] feat: Efficient repository enumeration via search API use Jul 7, 2025
Co-authored-by: zkoppert <[email protected]>
Copilot AI changed the title [WIP] feat: Efficient repository enumeration via search API use Add support for repository search query to optimize repo enumeration Jul 7, 2025
Copilot AI requested a review from zkoppert July 7, 2025 22:00
@zkoppert zkoppert changed the title Add support for repository search query to optimize repo enumeration feat: Add support for repository search query to optimize repo enumeration Jul 7, 2025
@zkoppert
Copy link
Member

zkoppert commented Jul 7, 2025

@copilot This new configuration option needs to be documented in the README.md file.

Copy link
Contributor Author

Copilot AI commented Jul 7, 2025

@copilot This new configuration option needs to be documented in the README.md file.

Added documentation for REPOSITORY_SEARCH_QUERY in README (commit 9a931c8).

Signed-off-by: Zack Koppert <[email protected]>
@zkoppert
Copy link
Member

zkoppert commented Jul 7, 2025

@copilot can you fix the failed github action job that reports that the MARKDOWN_PRETTIER linter is failing?

Copy link
Contributor Author

Copilot AI commented Jul 7, 2025

@copilot can you fix the failed github action job that reports that the MARKDOWN_PRETTIER linter is failing?

Updated the documentation with a more descriptive link label to the encrypted secrets docs in commit bf5d204.

Signed-off-by: Zack Koppert <[email protected]>
@zkoppert
Copy link
Member

zkoppert commented Jul 7, 2025

image

@zkoppert zkoppert marked this pull request as ready for review July 7, 2025 23:56
Copilot AI review requested due to automatic review settings July 7, 2025 23:56
@zkoppert zkoppert requested a review from a team as a code owner July 7, 2025 23:56
@zkoppert zkoppert marked this pull request as draft July 7, 2025 23:56
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new feature to optimize repository enumeration by adding support for GitHub Search API queries through the REPOSITORY_SEARCH_QUERY environment variable. This allows users to filter repositories upfront using search queries instead of enumerating all organization repositories, potentially saving API rate limits and processing time.

  • Adds REPOSITORY_SEARCH_QUERY environment variable support to use GitHub Search API for targeted repository filtering
  • Updates documentation to describe the new search query functionality
  • Enables markdown prettier formatting in the super-linter workflow

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
evergreen.py Implements the core functionality to check for and use the search query environment variable
README.md Documents the new REPOSITORY_SEARCH_QUERY parameter and fixes link formatting
.github/workflows/super-linter.yaml Enables markdown prettier formatting

@zkoppert
Copy link
Member

zkoppert commented Jul 8, 2025

@copilot Write test cases to cover the use of search_query

Copy link
Contributor Author

Copilot AI commented Jul 8, 2025

@copilot Write test cases to cover the use of search_query

Added a new test test_get_repos_iterator_with_search_query covering the search_query path (commit fed4054). Let me know if any adjustments are needed!

zkoppert added 2 commits July 8, 2025 08:58
@zkoppert zkoppert marked this pull request as ready for review July 8, 2025 23:25
@zkoppert zkoppert merged commit 3d33fdb into main Jul 16, 2025
33 checks passed
@zkoppert zkoppert deleted the copilot/fix-372 branch July 16, 2025 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Efficient repository enumeration via search API use

3 participants