Skip to content

Commit 6feeec8

Browse files
escapewindowescapewindow
authored
Merge branch 'master' into checks
2 parents 20cbc92 + 113a83a commit 6feeec8

21 files changed

+280
-180
lines changed

‎HISTORY.rst‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,50 @@ Change Log
44
All notable changes to this project will be documented in this file.
55
This project adheres to `Semantic Versioning <http://semver.org/>`__.
66

7+
[42.2.0] - 2022-02-16
8+
---------------------
9+
Added
10+
~~~~~
11+
- Mozilla VPN to `cot_restricted_scopes`
12+
13+
[42.1.0] - 2022-02-16
14+
---------------------
15+
Added
16+
~~~~~
17+
- Support for cedar project
18+
- Support for log rotation with RotatingFileHandler
19+
20+
[42.0.0] - 2021-12-02
21+
---------------------
22+
Removed
23+
~~~~~~~
24+
- Removed obsolete Focus scopes
25+
- Remove Pushsnap support
26+
27+
Added
28+
~~~~~~~
29+
- Added Focus for android scopes for github-script, signing-script and pushapk-script
30+
- Added xpi scopes for Beetmover and Balrog
31+
- Added microsoft store scopes for Firefox
32+
- Test coverage for utils and cleanup
33+
34+
[41.0.0] - 2021-09-02
35+
---------------------
36+
Removed
37+
~~~~~~~
38+
- Support for non-hook actions in CoT
39+
40+
Changed
41+
~~~~~~~
42+
- CoT bumped to version 7
43+
44+
[40.0.0] - 2021-08-24
45+
---------------------
46+
Changed
47+
~~~~~~~
48+
- Removed unused `mpd001` trust domain
49+
- Added new `mozillavpn` trust domain
50+
751
[39.0.0] - 2021-08-05
852
---------------------
953
Changed

‎README.rst‎

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Usage
3030
.. _example config file: https://github.com/mozilla-releng/scriptworker/blob/master/scriptworker.yaml.tmpl
3131
.. _scriptworker.constants.DEFAULT_CONFIG: https://github.com/mozilla-releng/scriptworker/blob/master/src/scriptworker/constants.py
3232

33-
Credentials can live in ``./scriptworker.yaml``, ``./secrets.json``, ``~/.scriptworker``, or in environment variables: ``TASKCLUSTER_ACCESS_TOKEN``, ``TASKCLUSTER_CLIENT_ID``, and ``TASKCLUSTER_CERTIFICATE``.
33+
Credentials can live in ``./scriptworker.yaml``, ``./secrets.json``, ``~/.scriptworker``.
3434

3535
* Launch: ``scriptworker [config_path]``
3636

@@ -42,11 +42,9 @@ Without integration tests install tox, then
4242

4343
``NO_CREDENTIALS_TESTS=1 tox -e py36``
4444

45-
Without any tests connecting ot the net, then
45+
Without any tests connecting to the net, then ``NO_TESTS_OVER_WIRE=1 tox -e py36``
4646

47-
``NO_TESTS_OVER_WIRE=1 tox -e py36``
48-
49-
With integration tests, first create a client with the scopes::
47+
With integration tests, first create a client in the Taskcluster UI with the scopes::
5048

5149
queue:cancel-task:test-dummy-scheduler/*
5250
queue:claim-work:test-dummy-provisioner/dummy-worker-*
@@ -58,20 +56,15 @@ With integration tests, first create a client with the scopes::
5856
queue:task-group-id:test-dummy-scheduler/*
5957
queue:worker-id:test-dummy-workers/dummy-worker-*
6058

61-
Then create a ``./secrets.json`` or ``~/.scriptworker`` that looks like::
59+
Then generate a no priviledge personal access token in Github for the scriptworker_github_token (to avoid rate limiting) and create a ``./secrets.json`` or ``~/.scriptworker`` that looks like::
6260

6361
{
6462
"integration_credentials": {
6563
"clientId": "...",
6664
"accessToken": "...",
67-
"certificate": "..."
6865
}
66+
"scriptworker_github_token": "..."
6967
}
7068

7169

72-
(``certificate`` is only specified if using temp creds)
73-
74-
75-
then
76-
77-
``tox``
70+
then to run all tests: ``tox``

‎docs/cot_key_management.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,3 +27,8 @@ There is also a ``verify_ed25519_signature`` commandline tool. This takes
2727
a file path and a signature path, and verifies if the file was validly signed
2828
by a known valid level 3 key. It also takes an optional ``--pubkey PUBKEY``
2929
argument, which allows you to verify if the file was signed by that pubkey.
30+
31+
Rotating the FirefoxCI CoT keys
32+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
33+
34+
See `this mana page <https://mana.mozilla.org/wiki/pages/viewpage.action?spaceKey=RelEng&title=Chain+of+Trust+key+rotation>`__.

‎docs/cot_overview.rst‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,10 @@ In conjunction with other best practices, like `separation of roles <https://en.
2929
Chain of Trust Versions
3030
^^^^^^^^^^^^^^^^^^^^^^^
3131

32-
1. Initial Chain of Trust implementation with GPG signatures: Initial `1.0.0b1 on 2016-11-14 <https://github.com/mozilla-releng/scriptworker/blob/master/CHANGELOG.md#100b1---2016-11-14>`_
33-
2. CoT v2: rebuild task definitions via json-e. `7.0.0 on 2018-01-18 <https://github.com/mozilla-releng/scriptworker/blob/master/CHANGELOG.md#700---2018-01-18>`_
34-
3. Generic action hook support. `12.0.0 on 2018-05-29 <https://github.com/mozilla-releng/scriptworker/blob/master/CHANGELOG.md#1200---2018-05-29>`_
35-
4. Release promotion action hook support. `17.1.0 on 2018-12-28 <https://github.com/mozilla-releng/scriptworker/blob/master/CHANGELOG.md#1710---2018-12-28>`_
36-
5. ed25519 support; deprecate GPG support. `22.0.0 on 2019-03-07 <https://github.com/mozilla-releng/scriptworker/blob/master/CHANGELOG.md#2200---2019-03-07>`_
37-
6. drop support for gpg `23.0.0 on 2019-03-27 <https://github.com/mozilla-releng/scriptworker/blob/master/CHANGELOG.md#2300---2019-03-27>`_
38-
39-
Planned future versions:
40-
41-
* drop support for non-hook actions
32+
1. Initial Chain of Trust implementation with GPG signatures: Initial `1.0.0b1 on 2016-11-14 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#100b1---2016-11-14>`_
33+
2. CoT v2: rebuild task definitions via json-e. `7.0.0 on 2018-01-18 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#700---2018-01-18>`_
34+
3. Generic action hook support. `12.0.0 on 2018-05-29 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#1200---2018-05-29>`_
35+
4. Release promotion action hook support. `17.1.0 on 2018-12-28 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#1710---2018-12-28>`_
36+
5. ed25519 support; deprecate GPG support. `22.0.0 on 2019-03-07 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#2200---2019-03-07>`_
37+
6. drop support for gpg `23.0.0 on 2019-03-27 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#2300---2019-03-27>`_
38+
7. drop support for non-hook actions `41.0.0 on 2021-09-02 <https://github.com/mozilla-releng/scriptworker/blob/master/HISTORY.rst#4100---2021-09-02>`_

‎docs/releases.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ If you're changing any dependencies, please update `setup.py`.
3232
If you add change the list of files that need to be packaged (either adding new files, or removing previous packaged files), modify `MANIFEST.in`.
3333

3434
### Versioning
35-
Modify `scriptworker/version.py` to set the `__version__` to the appropriate tuple. This is either a 3- or 4-part tuple, e.g.
35+
Modify `src/scriptworker/version.py` to set the `__version__` to the appropriate tuple. This is either a 3- or 4-part tuple, e.g.
3636

3737
```python
3838
# 0.10.0a1
@@ -49,7 +49,7 @@ Then run `version.py`:
4949

5050
```bash
5151
# Using the local venv python>=3.5,
52-
python scriptworker/version.py
52+
python src/scriptworker/version.py
5353
```
5454

5555
This will update `version.json`. Verify both files look correct.

‎requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,4 @@ immutabledict>=1.3.0
88
jsonschema
99
json-e>=2.5.0
1010
PyYAML
11-
taskcluster>39
11+
taskcluster>=40

‎src/scriptworker/constants.py‎

Lines changed: 28 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,10 @@
5656
"task_max_timeout_status": STATUSES["intermittent-task"],
5757
"invalid_reclaim_status": STATUSES["intermittent-task"],
5858
"task_script": ("bash", "-c", "echo foo && sleep 19 && exit 1"),
59+
# Logging settings
5960
"verbose": True,
61+
"log_max_bytes": 0,
62+
"log_max_backups": 10,
6063
# Task settings
6164
"work_dir": "...",
6265
"log_dir": "...",
@@ -116,7 +119,7 @@
116119
"firefox": "hg",
117120
"thunderbird": "hg",
118121
"mobile": "github",
119-
"mpd001": "github",
122+
"mozillavpn": "github",
120123
"app-services": "github",
121124
"glean": "github",
122125
"xpi": "github",
@@ -134,7 +137,7 @@
134137
"firefox": ("gecko-1/decision", "gecko-2/decision", "gecko-3/decision"),
135138
"thunderbird": ("comm-1/decision", "comm-2/decision", "comm-3/decision"),
136139
"mobile": ("mobile-1/decision", "mobile-3/decision"),
137-
"mpd001": ("mpd001-1/decision", "mpd001-3/decision"),
140+
"mozillavpn": ("mozillavpn-1/decision", "mozillavpn-3/decision"),
138141
"app-services": ("app-services-1/decision", "app-services-3/decision"),
139142
"glean": ("glean-1/decision", "glean-3/decision"),
140143
"xpi": ("xpi-1/decision", "xpi-3/decision"),
@@ -152,7 +155,7 @@
152155
"firefox": ("gecko-1/images", "gecko-2/images", "gecko-3/images"),
153156
"thunderbird": ("comm-1/images", "comm-2/images", "comm-3/images"),
154157
"mobile": ("mobile-1/images", "mobile-3/images"),
155-
"mpd001": ("mpd001-1/images", "mpd001-3/images"),
158+
"mozillavpn": ("mozillavpn-1/images", "mozillavpn-3/images"),
156159
"app-services": ("app-services-1/images", "app-services-3/images"),
157160
"glean": ("glean-1/images", "glean-3/images"),
158161
"xpi": ("xpi-1/images", "xpi-3/images"),
@@ -176,7 +179,7 @@
176179
r"^(?P<path>/mozilla-(central|unified))(/|$)",
177180
r"^(?P<path>/integration/(autoland|fx-team|mozilla-inbound))(/|$)",
178181
r"^(?P<path>/releases/mozilla-(beta|release|esr\d+))(/|$)",
179-
r"^(?P<path>/projects/(maple|oak|pine))(/|$)",
182+
r"^(?P<path>/projects/(maple|oak|cedar|pine))(/|$)",
180183
),
181184
}
182185
),
@@ -213,12 +216,12 @@
213216
}
214217
),
215218
),
216-
"mpd001": (
219+
"mozillavpn": (
217220
immutabledict(
218221
{
219222
"schemes": ("https", "ssh"),
220223
"netlocs": ("github.com",),
221-
"path_regexes": tuple([r"^(?P<path>/mozilla-services/(?:guardian-vpn-windows))(/|.git|$)"]),
224+
"path_regexes": tuple([r"^(?P<path>/mozilla-mobile/(?:mozilla-vpn-client))(/|.git|$)"]),
222225
}
223226
),
224227
),
@@ -275,7 +278,7 @@
275278
"github-push",
276279
"github-release",
277280
),
278-
"mpd001": ("cron", "github-pull-request", "github-push", "github-release"),
281+
"mozillavpn": ("cron", "github-pull-request", "github-push", "github-release"),
279282
"app-services": (
280283
"action",
281284
"cron",
@@ -308,7 +311,7 @@
308311
"firefox": "",
309312
"thunderbird": "",
310313
"mobile": "mozilla-mobile",
311-
"mpd001": "mozilla-services",
314+
"mozillavpn": "mozilla-mobile",
312315
"app-services": "mozilla",
313316
"glean": "mozilla",
314317
"xpi": "mozilla-extensions",
@@ -335,11 +338,10 @@
335338
"project:releng:signing:cert:release-signing": "all-release-branches",
336339
"project:releng:flathub:firefox:beta": "beta-or-release", # Needed on release for RCs
337340
"project:releng:flathub:firefox:stable": "release",
338-
"project:releng:snapcraft:firefox:beta": "beta-or-release", # Needed on release for RCs
339-
"project:releng:snapcraft:firefox:candidate": "release",
340-
"project:releng:snapcraft:firefox:esr": "esr",
341341
"project:releng:ship-it:production": "all-production-branches",
342342
"project:releng:treescript:action:push": "all-production-branches",
343+
"project:releng:microsoftstore:beta": "beta",
344+
"project:releng:microsoftstore:release": "release",
343345
}
344346
),
345347
"thunderbird": immutabledict(
@@ -372,16 +374,19 @@
372374
"project:mobile:fennec-profile-manager:releng:signing:cert:fennec-production-signing": "fennec-profile-manager-repo",
373375
"project:mobile:firefox-tv:releng:googleplay:product:firefox-tv": "firefox-tv-repo",
374376
"project:mobile:firefox-tv:releng:signing:cert:production-signing": "firefox-tv-repo",
375-
"project:mobile:focus:googleplay:product:focus": "focus-repo",
376-
"project:mobile:focus:releng:signing:cert:release-signing": "focus-repo",
377377
"project:mobile:reference-browser:releng:googleplay:product:reference-browser": "reference-browser-repo",
378378
"project:mobile:reference-browser:releng:signing:cert:release-signing": "reference-browser-repo",
379+
"project:mobile:focus-android:releng:github:project:focus-android": "focus-repo",
380+
"project:mobile:focus-android:releng:googleplay:product:focus-android": "focus-repo",
381+
# beta and nightly are signed with same key as production
382+
"project:mobile:focus-android:releng:signing:cert:production-signing": "focus-repo",
379383
}
380384
),
381-
"mpd001": immutabledict(
385+
"mozillavpn": immutabledict(
382386
{
383-
"project:mpd001:releng:signing:cert:nightly-signing": "mpd001-repo",
384-
"project:mpd001:releng:signing:cert:release-signing": "mpd001-repo",
387+
"project:mozillavpn:releng:signing:cert:nightly-signing": "mozillavpn-repo",
388+
"project:mozillavpn:releng:signing:cert:release-signing": "mozillavpn-repo",
389+
"project:mozillavpn:releng:googleplay:product:mozillavpn": "mozillavpn-repo",
385390
}
386391
),
387392
"app-services": immutabledict({"project:mozilla:app-services:releng:beetmover:bucket:maven-production": "app-services-repo"}),
@@ -391,6 +396,8 @@
391396
"project:xpi:signing:cert:release-signing": "xpi-manifest-repo",
392397
"project:xpi:releng:github:project:mozilla-extensions/*": "xpi-manifest-repo",
393398
"project:xpi:ship-it:production": "xpi-manifest-repo",
399+
"project:xpi:beetmover:bucket:release": "xpi-manifest-repo",
400+
"project:xpi:balrog:server:release": "xpi-manifest-repo",
394401
}
395402
),
396403
"adhoc": immutabledict({"project:adhoc:signing:cert:release-signing": "adhoc-signing-repos"}),
@@ -432,6 +439,7 @@
432439
# XXX remove /projects/maple when we have a
433440
# different prod signing testing solution
434441
# XXX remove /projects/oak when we no longer test updates against it
442+
# XXX remove /projects/cedar when we no longer need
435443
# XXX remove /projects/pine when we no longer need
436444
# nightly signing
437445
"all-nightly-branches": (
@@ -443,6 +451,7 @@
443451
"/releases/mozilla-esr91",
444452
"/projects/maple",
445453
"/projects/oak",
454+
"/projects/cedar",
446455
"/projects/pine",
447456
),
448457
"all-production-branches": (
@@ -476,7 +485,7 @@
476485
"reference-browser-repo": ("/mozilla-mobile/reference-browser",),
477486
}
478487
),
479-
"mpd001": immutabledict({"mpd001-repo": ("/mozilla-services/guardian-vpn-windows",)}),
488+
"mozillavpn": immutabledict({"mozillavpn-repo": ("/mozilla-mobile/mozilla-vpn-client",)}),
480489
"app-services": immutabledict({"app-services-repo": ("/mozilla/application-services",)}),
481490
"glean": immutabledict({"glean-repo": ("/mozilla/glean",)}),
482491
"xpi": immutabledict({"xpi-manifest-repo": ("/mozilla-extensions/xpi-manifest",)}),
@@ -497,7 +506,7 @@
497506
"thunderbird": ("decision", "action", "docker-image"),
498507
# XXX now that we're on taskgraph, we should limit these.
499508
"mobile": "any", # all allowed
500-
"mpd001": "any", # all allowed
509+
"mozillavpn": "any", # all allowed
501510
"app-services": "any", # all allowed
502511
"glean": "any", # all allowed
503512
"xpi": "any", # all allowed
@@ -512,7 +521,7 @@
512521
"firefox": "GECKO",
513522
"thunderbird": "COMM",
514523
"mobile": "MOBILE",
515-
"mpd001": "MPD001",
524+
"mozillavpn": "MOZILLAVPN",
516525
"app-services": "APPSERVICES",
517526
"glean": "GLEAN",
518527
"xpi": "XPI",

0 commit comments

Comments
 (0)