Skip to content

Commit d5af164

Browse files
ricardojdsilva87JM (Jason Meridth)
ricardojdsilva87
and
JM (Jason Meridth)
authored
feat: support private repository configuration (#265)
* feat: support private repository configuration * feat: add tests to dependabot_file.py * fix: remove prettier from Makefile * fix: README lint * fix: README lint * fix: add suggestions --------- Co-authored-by: JM (Jason Meridth) <[email protected]>
1 parent 35040bd commit d5af164

File tree

10 files changed

+586
-187
lines changed

10 files changed

+586
-187
lines changed

‎.env-example‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ BATCH_SIZE = ""
22
BODY = ""
33
COMMIT_MESSAGE = ""
44
CREATED_AFTER_DATE = ""
5+
DEPENDABOT_CONFIG_FILE = ""
56
DRY_RUN = ""
67
ENABLE_SECURITY_UPDATES = ""
78
EXEMPT_ECOSYSTEMS = ""

‎.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,4 +153,7 @@ devenv.local.nix
153153
# devenv
154154
.envrc
155155
devenv.*
156-
.devenv*
156+
.devenv*
157+
158+
# Local testing files
159+
dependabot-output.yaml

‎README.md‎

Lines changed: 54 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ This action can be configured to authenticate with GitHub App Installation or Pe
8686
| `GH_APP_PRIVATE_KEY` | True | `""` | GitHub Application Private Key. See [documentation](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app) for more details. |
8787
| `GITHUB_APP_ENTERPRISE_ONLY` | False | false | Set this input to `true` if your app is created in GHE and communicates with GHE. |
8888

89-
The needed GitHub app permissions are the following:
89+
The needed GitHub app permissions are the following under `Repository permissions`:
9090

9191
- `Administration` - Read and Write (Needed to activate the [automated security updates](https://docs.github.com/en/code-security/dependabot/dependabot-security-updates/configuring-dependabot-security-updates#managing-dependabot-security-updates-for-your-repositories) )
9292
- `Pull Requests` - Read and Write (If `TYPE` input is set to `pull`)
@@ -125,6 +125,58 @@ The needed GitHub app permissions are the following:
125125
| `SCHEDULE` | False | `weekly` | Schedule interval by which to check for dependency updates via Dependabot. Allowed values are `daily`, `weekly`, or `monthly` |
126126
| `SCHEDULE_DAY` | False | '' | Scheduled day by which to check for dependency updates via Dependabot. Allowed values are days of the week full names (i.e., `monday`) |
127127
| `LABELS` | False | "" | A comma separated list of labels that should be added to pull requests opened by dependabot. |
128+
| `DEPENDABOT_CONFIG_FILE` | False | "" | Location of the configuration file for `dependabot.yml` configurations. If the file is present locally it takes precedence over the one in the repository. |
129+
130+
### Private repositories configuration
131+
132+
Dependabot allows the configuration of [private registries](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#configuration-options-for-private-registries) for dependabot to use.
133+
To add a private registry configuration to the dependabot file the `DEPENDABOT_CONFIG_FILE` needs to be set with the path of the configuration file.
134+
135+
This configuration file needs to exist on the repository where the action runs. It can also be created locally to test some configurations (if created locally it takes precedence over the file on the repository).
136+
137+
#### Usage
138+
139+
Set the input variable:
140+
141+
```yaml
142+
DEPENDABOT_CONFIG_FILE = "dependabot-config.yaml"
143+
```
144+
145+
Create a file on your repository in the same path:
146+
147+
```yaml
148+
npm:
149+
type: "npm"
150+
url: "https://yourprivateregistry/npm/"
151+
username: "${{secrets.username}}"
152+
password: "${{secrets.password}}"
153+
key: <used if necessary>
154+
token: <used if necessary>
155+
replaces-base: <used if necessary>
156+
maven:
157+
type: "maven"
158+
url: "https://yourprivateregistry/maven/"
159+
username: "${{secrets.username}}"
160+
password: "${{secrets.password}}"
161+
```
162+
163+
The principal key of each configuration need to match the package managers that the [script is looking for](https://github.com/github/evergreen/blob/main/dependabot_file.py#L78).
164+
165+
The `dependabot.yaml` created file will look like the following with the `registries:` key added:
166+
167+
```yaml
168+
updates:
169+
- package-ecosystem: "npm"
170+
directory: "/"
171+
registries: --> added configuration
172+
- 'npm' --> added configuration
173+
schedule:
174+
interval: "weekly"
175+
labels:
176+
- "test"
177+
- "dependabot"
178+
- "new"
179+
```
128180

129181
### Example workflows
130182

@@ -225,7 +277,7 @@ jobs:
225277
GH_APP_PRIVATE_KEY: ${{ secrets.GH_APP_PRIVATE_KEY }}
226278
# GITHUB_APP_ENTERPRISE_ONLY: True --> Set to true when created GHE App needs to communicate with GHE api
227279
GH_ENTERPRISE_URL: ${{ github.server_url }}
228-
# GH_TOKEN: ${{ steps.app-token.outputs.token }} --> the token input is not used if the github app inputs are set
280+
# GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} --> the token input is not used if the github app inputs are set
229281
ORGANIZATION: your_organization
230282
UPDATE_EXISTING: True
231283
GROUP_DEPENDENCIES: True

‎dependabot_file.py‎

Lines changed: 96 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,52 +1,103 @@
11
"""This module contains the function to build the dependabot.yml file for a repo"""
22

3+
import base64
4+
import copy
5+
import io
6+
37
import github3
4-
import yaml
8+
import ruamel.yaml
9+
from ruamel.yaml.scalarstring import SingleQuotedScalarString
10+
11+
# Define data structure for dependabot.yaml
12+
data = {
13+
"version": 2,
14+
"registries": {},
15+
"updates": [],
16+
}
17+
18+
yaml = ruamel.yaml.YAML()
19+
stream = io.StringIO()
520

621

722
def make_dependabot_config(
8-
ecosystem, group_dependencies, indent, schedule, schedule_day, labels
23+
ecosystem,
24+
group_dependencies,
25+
schedule,
26+
schedule_day,
27+
labels,
28+
dependabot_config,
29+
extra_dependabot_config,
930
) -> str:
1031
"""
1132
Make the dependabot configuration for a specific package ecosystem
1233
1334
Args:
1435
ecosystem: the package ecosystem to make the dependabot configuration for
1536
group_dependencies: whether to group dependencies in the dependabot.yml file
16-
indent: the number of spaces to indent the dependabot configuration ex: " "
1737
schedule: the schedule to run dependabot ex: "daily"
1838
schedule_day: the day of the week to run dependabot ex: "monday" if schedule is "weekly"
1939
labels: the list of labels to be added to dependabot configuration
40+
dependabot_config: extra dependabot configs
41+
extra_dependabot_config: File with the configuration to add dependabot configs (ex: private registries)
2042
2143
Returns:
2244
str: the dependabot configuration for the package ecosystem
2345
"""
24-
schedule_day_line = ""
25-
if schedule_day:
26-
schedule_day_line += f"""
27-
{indent}{indent}{indent}day: '{schedule_day}'"""
2846

29-
dependabot_config = f"""{indent}- package-ecosystem: '{ecosystem}'
30-
{indent}{indent}directory: '/'
31-
{indent}{indent}schedule:
32-
{indent}{indent}{indent}interval: '{schedule}'{schedule_day_line}
33-
"""
47+
dependabot_config["updates"].append(
48+
{
49+
"package-ecosystem": SingleQuotedScalarString(ecosystem),
50+
"directory": SingleQuotedScalarString("/"),
51+
}
52+
)
53+
54+
if extra_dependabot_config:
55+
ecosystem_config = extra_dependabot_config.get(ecosystem)
56+
if ecosystem_config:
57+
dependabot_config["registries"][ecosystem] = ecosystem_config
58+
dependabot_config["updates"][-1].update(
59+
{"registries": [SingleQuotedScalarString(ecosystem)]}
60+
)
61+
else:
62+
dependabot_config.pop("registries", None)
63+
64+
if schedule_day:
65+
dependabot_config["updates"][-1].update(
66+
{
67+
"schedule": {
68+
"interval": SingleQuotedScalarString(schedule),
69+
"day": SingleQuotedScalarString(schedule_day),
70+
},
71+
}
72+
)
73+
else:
74+
dependabot_config["updates"][-1].update(
75+
{
76+
"schedule": {"interval": SingleQuotedScalarString(schedule)},
77+
}
78+
)
3479

3580
if labels:
36-
dependabot_config += f"""{indent}{indent}labels:
37-
"""
81+
quoted_labels = []
3882
for label in labels:
39-
dependabot_config += f"""{indent}{indent}{indent}- \"{label}\"
40-
"""
83+
quoted_labels.append(SingleQuotedScalarString(label))
84+
dependabot_config["updates"][-1].update({"labels": quoted_labels})
4185

4286
if group_dependencies:
43-
dependabot_config += f"""{indent}{indent}groups:
44-
{indent}{indent}{indent}production-dependencies:
45-
{indent}{indent}{indent}{indent}dependency-type: 'production'
46-
{indent}{indent}{indent}development-dependencies:
47-
{indent}{indent}{indent}{indent}dependency-type: 'development'
48-
"""
49-
return dependabot_config
87+
dependabot_config["updates"][-1].update(
88+
{
89+
"groups": {
90+
"production-dependencies": {
91+
"dependency-type": SingleQuotedScalarString("production")
92+
},
93+
"development-dependencies": {
94+
"dependency-type": SingleQuotedScalarString("development")
95+
},
96+
}
97+
}
98+
)
99+
100+
return yaml.dump(dependabot_config, stream)
50101

51102

52103
def build_dependabot_file(
@@ -58,6 +109,7 @@ def build_dependabot_file(
58109
schedule,
59110
schedule_day,
60111
labels,
112+
extra_dependabot_config,
61113
) -> str | None:
62114
"""
63115
Build the dependabot.yml file for a repo based on the repo contents
@@ -71,6 +123,7 @@ def build_dependabot_file(
71123
schedule: the schedule to run dependabot ex: "daily"
72124
schedule_day: the day of the week to run dependabot ex: "monday" if schedule is "daily"
73125
labels: the list of labels to be added to dependabot configuration
126+
extra_dependabot_config: File with the configuration to add dependabot configs (ex: private registries)
74127
75128
Returns:
76129
str: the dependabot.yml file for the repo
@@ -89,30 +142,20 @@ def build_dependabot_file(
89142
"github-actions": False,
90143
"maven": False,
91144
}
92-
DEFAULT_INDENT = 2 # pylint: disable=invalid-name
145+
93146
# create a local copy in order to avoid overwriting the global exemption list
94147
exempt_ecosystems_list = exempt_ecosystems.copy()
95148
if existing_config:
96-
dependabot_file = existing_config.decoded.decode("utf-8")
97-
ecosystem_line = next(
98-
line
99-
for line in dependabot_file.splitlines()
100-
if "- package-ecosystem:" in line
101-
)
102-
indent = " " * (len(ecosystem_line) - len(ecosystem_line.lstrip()))
103-
if len(indent) < DEFAULT_INDENT:
104-
print(
105-
f"Invalid dependabot.yml file. No indentation found. Skipping {repo.full_name}"
106-
)
107-
return None
149+
yaml.preserve_quotes = True
150+
try:
151+
dependabot_file = yaml.load(base64.b64decode(existing_config.content))
152+
except ruamel.yaml.YAMLError as e:
153+
print(f"YAML indentation error: {e}")
154+
raise
108155
else:
109-
indent = " " * DEFAULT_INDENT
110-
dependabot_file = """---
111-
version: 2
112-
updates:
113-
"""
156+
dependabot_file = copy.deepcopy(data)
114157

115-
add_existing_ecosystem_to_exempt_list(exempt_ecosystems_list, existing_config)
158+
add_existing_ecosystem_to_exempt_list(exempt_ecosystems_list, dependabot_file)
116159

117160
# If there are repository specific exemptions,
118161
# overwrite the global exemptions for this repo only
@@ -151,17 +194,14 @@ def build_dependabot_file(
151194
try:
152195
if repo.file_contents(file):
153196
package_managers_found[manager] = True
154-
# If the last thing in the file is not a newline,
155-
# add one before adding a new language config to the file
156-
if dependabot_file and dependabot_file[-1] != "\n":
157-
dependabot_file += "\n"
158-
dependabot_file += make_dependabot_config(
197+
make_dependabot_config(
159198
manager,
160199
group_dependencies,
161-
indent,
162200
schedule,
163201
schedule_day,
164202
labels,
203+
dependabot_file,
204+
extra_dependabot_config,
165205
)
166206
break
167207
except github3.exceptions.NotFoundError:
@@ -173,13 +213,14 @@ def build_dependabot_file(
173213
for file in repo.directory_contents("/"):
174214
if file[0].endswith(".tf"):
175215
package_managers_found["terraform"] = True
176-
dependabot_file += make_dependabot_config(
216+
make_dependabot_config(
177217
"terraform",
178218
group_dependencies,
179-
indent,
180219
schedule,
181220
schedule_day,
182221
labels,
222+
dependabot_file,
223+
extra_dependabot_config,
183224
)
184225
break
185226
except github3.exceptions.NotFoundError:
@@ -189,13 +230,14 @@ def build_dependabot_file(
189230
for file in repo.directory_contents(".github/workflows"):
190231
if file[0].endswith(".yml") or file[0].endswith(".yaml"):
191232
package_managers_found["github-actions"] = True
192-
dependabot_file += make_dependabot_config(
233+
make_dependabot_config(
193234
"github-actions",
194235
group_dependencies,
195-
indent,
196236
schedule,
197237
schedule_day,
198238
labels,
239+
dependabot_file,
240+
extra_dependabot_config,
199241
)
200242
break
201243
except github3.exceptions.NotFoundError:
@@ -212,7 +254,5 @@ def add_existing_ecosystem_to_exempt_list(exempt_ecosystems, existing_config):
212254
to the exempt list so we don't get duplicate entries and maintain configuration settings
213255
"""
214256
if existing_config:
215-
existing_config_obj = yaml.safe_load(existing_config.decoded)
216-
if existing_config_obj:
217-
for entry in existing_config_obj.get("updates", []):
218-
exempt_ecosystems.append(entry["package-ecosystem"])
257+
for entry in existing_config.get("updates", []):
258+
exempt_ecosystems.append(entry["package-ecosystem"])

0 commit comments

Comments
 (0)