diff --git a/.github/workflows/super-linter.yaml b/.github/workflows/super-linter.yaml index c30d10d..2625273 100644 --- a/.github/workflows/super-linter.yaml +++ b/.github/workflows/super-linter.yaml @@ -35,3 +35,4 @@ jobs: DEFAULT_BRANCH: main GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_ACTIONS_COMMAND_ARGS: -shellcheck= + FIX_MARKDOWN_PRETTIER: true diff --git a/README.md b/README.md index ce1e0c5..c66cfe8 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,6 @@ All feedback regarding our GitHub Actions, as a whole, should be communicated th 1. Select a best fit workflow file from the [examples below](#example-workflows). 1. Copy that example into your repository (from step 1) and into the proper directory for GitHub Actions: `.github/workflows/` directory with the file extension `.yml` (ie. `.github/workflows/evergreen.yml`) 1. Edit the values below from the sample workflow with your information: - - `ORGANIZATION` - `TEAM_NAME` - `REPOSITORY` @@ -51,7 +50,6 @@ All feedback regarding our GitHub Actions, as a whole, should be communicated th 1. Also edit the value for `GH_ENTERPRISE_URL` if you are using a GitHub Server and not using github.com. For github.com users, leave it empty. 1. Update the value of `GH_TOKEN`. Do this by creating a [GitHub API token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) with the following permissions: - - If using **classic tokens**: - `workflow`, this will set also all permissions for `repo` - under `admin`, `read:org` and `write:org` @@ -65,7 +63,7 @@ All feedback regarding our GitHub Actions, as a whole, should be communicated th Then finally update the workflow file to use that repository secret by changing `GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}` to `GH_TOKEN: ${{ secrets.GH_TOKEN }}`. The name of the secret can really be anything, it just needs to match between when you create the secret name and when you refer to it in the workflow file. -1. If you want the resulting issue with the output to appear in a different repository other than the one the workflow file runs in, update the line `token: ${{ secrets.GITHUB_TOKEN }}` with your own GitHub API token stored as a repository secret. This process is the same as described in the step above. More info on creating secrets can be found [here](https://docs.github.com/en/actions/security-guides/encrypted-secrets). +1. If you want the resulting issue with the output to appear in a different repository other than the one the workflow file runs in, update the line `token: ${{ secrets.GITHUB_TOKEN }}` with your own GitHub API token stored as a repository secret. This process is the same as described in the step above. More info on creating secrets can be found in the [GitHub documentation on encrypted secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets). 1. Commit the workflow file to the default branch (often `master` or `main`) 1. Wait for the action to trigger based on the `schedule` entry or manually trigger the workflow as shown in the [documentation](https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow). @@ -105,8 +103,9 @@ The needed GitHub app permissions are the following under `Repository permission | field | required | default | description | | -------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `GH_ENTERPRISE_URL` | False | "" | The `GH_ENTERPRISE_URL` is used to connect to an enterprise server instance of GitHub, ex: `https://yourgheserver.com`.
github.com users should not enter anything here. | -| `ORGANIZATION` | Required to have `ORGANIZATION` or `REPOSITORY` | | The name of the GitHub organization which you want this action to work from. ie. github.com/github would be `github` | -| `REPOSITORY` | Required to have `ORGANIZATION` or `REPOSITORY` | | The name of the repository and organization which you want this action to work from. ie. `github/evergreen` or a comma separated list of multiple repositories `github/evergreen,super-linter/super-linter` | +| `ORGANIZATION` | Required to have `ORGANIZATION` or `REPOSITORY` or `REPOSITORY_SEARCH_QUERY` | | The name of the GitHub organization which you want this action to work from. ie. github.com/github would be `github` | +| `REPOSITORY` | Required to have `ORGANIZATION` or `REPOSITORY` or `REPOSITORY_SEARCH_QUERY` | | The name of the repository and organization which you want this action to work from. ie. `github/evergreen` or a comma separated list of multiple repositories `github/evergreen,super-linter/super-linter` | +| `REPOSITORY_SEARCH_QUERY` | Required to have `ORGANIZATION` or `REPOSITORY` or `REPOSITORY_SEARCH_QUERY` | "" | When set, directs the action to use the GitHub Search API to search repositories matching this query instead of enumerating all organization repositories. This overrides anything set in the `REPOSITORY` and `ORGANIZATION` variables. Example: `org:my-org is:repository archived:false created:>2025-07-01`. | | `EXEMPT_REPOS` | False | "" | These repositories will be exempt from this action considering them for dependabot enablement. ex: If my org is set to `github` then I might want to exempt a few of the repos but get the rest by setting `EXEMPT_REPOS` to `github/evergreen,github/contributors` | | `TYPE` | False | pull | Type refers to the type of action you want taken if this workflow determines that dependabot could be enabled. Valid values are `pull` or `issue`. | | `TITLE` | False | "Enable Dependabot" | The title of the issue or pull request that will be created if dependabot could be enabled. | @@ -257,6 +256,50 @@ jobs: run: cat summary.md >> $GITHUB_STEP_SUMMARY ``` +#### Using REPOSITORY_SEARCH_QUERY + +```yaml +--- +name: Weekly dependabot checks +on: + workflow_dispatch: + schedule: + - cron: "3 2 * * 6" + +permissions: + contents: read + +jobs: + evergreen: + name: evergreen + runs-on: ubuntu-latest + permissions: + issues: write + + steps: + - shell: bash + run: | + # Get the current date + current_date=$(date +'%Y-%m-%d') + + # Calculate the previous month + previous_date=$(date -d "$current_date -7 day" +'%Y-%m-%d') + + echo "$previous_date..$current_date" + echo "one_week_ago=$previous_date" >> "$GITHUB_ENV" + + - name: Run evergreen action + uses: github/evergreen@v1 + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + REPOSITORY_SEARCH_QUERY: "org:your_organization is:repository is:public archived:false created:>${{ env.one_week_ago }}" + TITLE: "Add dependabot configuration" + BODY: "Please add this dependabot configuration so that we can keep the dependencies in this repo up to date and secure. for help, contact XXX" + + - name: Post evergreen job summary + run: cat summary.md >> $GITHUB_STEP_SUMMARY +``` + #### Using GitHub app ```yaml diff --git a/env.py b/env.py index a5e5396..4c6c0bc 100644 --- a/env.py +++ b/env.py @@ -99,6 +99,7 @@ def get_env_vars( ) -> tuple[ str | None, list[str], + str | None, int | None, int | None, bytes, @@ -135,6 +136,7 @@ def get_env_vars( Returns: organization (str): The organization to search for repositories in repository_list (list[str]): A list of repositories to search for + search_query (str): A search query string to filter repositories by gh_app_id (int | None): The GitHub App ID to use for authentication gh_app_installation_id (int | None): The GitHub App Installation ID to use for authentication gh_app_private_key_bytes (bytes): The GitHub App Private Key as bytes to use for authentication @@ -162,18 +164,19 @@ def get_env_vars( dependabot_config_file (str): Dependabot extra configuration file location path """ - if not test: + if not test: # pragma: no cover # Load from .env file if it exists and not testing dotenv_path = join(dirname(__file__), ".env") load_dotenv(dotenv_path) organization = os.getenv("ORGANIZATION") repositories_str = os.getenv("REPOSITORY") + search_query = os.getenv("REPOSITORY_SEARCH_QUERY", "").strip() team_name = os.getenv("TEAM_NAME") - # Either organization or repository must be set - if not organization and not repositories_str: + # Either organization, repository, or search_query must be set + if not organization and not repositories_str and not search_query: raise ValueError( - "ORGANIZATION and REPOSITORY environment variables were not set. Please set one" + "ORGANIZATION, REPOSITORY, and REPOSITORY_SEARCH_QUERY environment variables were not set. Please set one" ) # Team name and repository are mutually exclusive if repositories_str and team_name: @@ -352,6 +355,7 @@ def get_env_vars( return ( organization, repositories_list, + search_query, gh_app_id, gh_app_installation_id, gh_app_private_key_bytes, diff --git a/evergreen.py b/evergreen.py index 15e80e3..34bb02f 100644 --- a/evergreen.py +++ b/evergreen.py @@ -20,6 +20,7 @@ def main(): # pragma: no cover ( organization, repository_list, + search_query, gh_app_id, gh_app_installation_id, gh_app_private_key, @@ -77,7 +78,7 @@ def main(): # pragma: no cover # Get the repositories from the organization, team name, or list of repositories repos = get_repos_iterator( - organization, team_name, repository_list, github_connection + organization, team_name, repository_list, search_query, github_connection ) # Setting up the action summary content @@ -341,9 +342,21 @@ def enable_dependabot_security_updates(ghe, owner, repo, access_token): print("\tFailed to enable Dependabot security updates.") -def get_repos_iterator(organization, team_name, repository_list, github_connection): - """Get the repositories from the organization, team_name, or list of repositories""" +def get_repos_iterator( + organization, team_name, repository_list, search_query, github_connection +): + """Get the repositories from the organization, team_name, repository_list, or via search query""" + # Use GitHub search API if REPOSITORY_SEARCH_QUERY is set + if search_query: + # Return repositories matching the search query + repos = [] + # Search results need to be converted to a list of repositories since they are returned as a search iterator + for repo in github_connection.search_repositories(search_query): + repos.append(repo.repository) + return repos + repos = [] + # Default behavior: list all organization/team repositories or specific repository list if organization and not repository_list and not team_name: repos = github_connection.organization(organization).repositories() elif team_name and organization: diff --git a/test_env.py b/test_env.py index 95fc9f1..1ab77cd 100644 --- a/test_env.py +++ b/test_env.py @@ -39,6 +39,7 @@ def setUp(self): "TYPE", "UPDATE_EXISTING", "REPO_SPECIFIC_EXEMPTIONS", + "REPOSITORY_SEARCH_QUERY", "SCHEDULE", "SCHEDULE_DAY", "LABELS", @@ -68,6 +69,7 @@ def test_get_env_vars_with_org(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -120,6 +122,7 @@ def test_get_env_vars_with_org_and_repo_specific_exemptions(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -232,6 +235,7 @@ def test_get_env_vars_with_repos(self): expected_result = ( None, ["org/repo1", "org2/repo2"], + "", # search_query None, None, b"", @@ -289,6 +293,7 @@ def test_get_env_vars_with_team(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -364,6 +369,7 @@ def test_get_env_vars_optional_values(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -410,6 +416,7 @@ def test_get_env_vars_with_update_existing(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -450,7 +457,7 @@ def test_get_env_vars_missing_org_or_repo(self): the_exception = cm.exception self.assertEqual( str(the_exception), - "ORGANIZATION and REPOSITORY environment variables were not set. Please set one", + "ORGANIZATION, REPOSITORY, and REPOSITORY_SEARCH_QUERY environment variables were not set. Please set one", ) @patch.dict( @@ -470,6 +477,7 @@ def test_get_env_vars_auth_with_github_app_installation(self): expected_result = ( "my_organization", [], + "", # search_query 12345, 678910, b"hello", @@ -559,6 +567,7 @@ def test_get_env_vars_with_repos_no_dry_run(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -605,6 +614,7 @@ def test_get_env_vars_with_repos_disabled_security_updates(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -652,6 +662,7 @@ def test_get_env_vars_with_repos_filter_visibility_multiple_values(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -699,6 +710,7 @@ def test_get_env_vars_with_repos_filter_visibility_single_value(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -776,6 +788,7 @@ def test_get_env_vars_with_repos_filter_visibility_no_duplicates(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -824,6 +837,7 @@ def test_get_env_vars_with_repos_exempt_ecosystems(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -871,6 +885,7 @@ def test_get_env_vars_with_no_batch_size(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -919,6 +934,7 @@ def test_get_env_vars_with_batch_size(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -1056,6 +1072,7 @@ def test_get_env_vars_with_valid_schedule_and_schedule_day(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -1141,6 +1158,7 @@ def test_get_env_vars_with_a_valid_label(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", @@ -1187,6 +1205,7 @@ def test_get_env_vars_with_valid_labels_containing_spaces(self): expected_result = ( "my_organization", [], + "", # search_query None, None, b"", diff --git a/test_evergreen.py b/test_evergreen.py index 298e81b..b0a14a2 100644 --- a/test_evergreen.py +++ b/test_evergreen.py @@ -320,6 +320,7 @@ def test_get_repos_iterator_with_organization(self, mock_github): """Test the get_repos_iterator function with an organization""" organization = "my_organization" repository_list = [] + search_query = "" github_connection = mock_github.return_value mock_organization = MagicMock() @@ -328,7 +329,7 @@ def test_get_repos_iterator_with_organization(self, mock_github): github_connection.organization.return_value = mock_organization result = get_repos_iterator( - organization, None, repository_list, github_connection + organization, None, repository_list, search_query, github_connection ) # Assert that the organization method was called with the correct argument @@ -345,6 +346,7 @@ def test_get_repos_iterator_with_repository_list(self, mock_github): """Test the get_repos_iterator function with a repository list""" organization = None repository_list = ["org/repo1", "org/repo2"] + search_query = "" github_connection = mock_github.return_value mock_repository = MagicMock() @@ -352,7 +354,7 @@ def test_get_repos_iterator_with_repository_list(self, mock_github): github_connection.repository.side_effect = mock_repository_list result = get_repos_iterator( - organization, None, repository_list, github_connection + organization, None, repository_list, search_query, github_connection ) # Assert that the repository method was called with the correct arguments for each repository in the list @@ -371,6 +373,7 @@ def test_get_repos_iterator_with_team(self, mock_github): organization = "my_organization" repository_list = [] team_name = "my_team" + search_query = "" github_connection = mock_github.return_value mock_team_repositories = MagicMock() @@ -382,6 +385,7 @@ def test_get_repos_iterator_with_team(self, mock_github): organization, team_name, repository_list, + search_query, github_connection, ) @@ -399,6 +403,31 @@ def test_get_repos_iterator_with_team(self, mock_github): # Assert that the function returned the expected result self.assertEqual(result, mock_team_repositories) + @patch("github3.login") + def test_get_repos_iterator_with_search_query(self, mock_github): + """Test the get_repos_iterator function with a search query""" + organization = "my_organization" + repository_list = [] + team_name = None + search_query = "org:my-org is:repository archived:false" + github_connection = mock_github.return_value + repo1 = MagicMock() + repo2 = MagicMock() + + # Mock the search_repositories method to return an iterator of repositories + github_connection.search_repositories.return_value = [repo1, repo2] + + get_repos_iterator( + organization, + team_name, + repository_list, + search_query, + github_connection, + ) + + # Assert that the search_repositories method was called with the correct argument + github_connection.search_repositories.assert_called_with(search_query) + class TestGetGlobalProjectId(unittest.TestCase): """Test the get_global_project_id function in evergreen.py"""