Skip to content

Commit 4cf2e7e

Browse files
lrlnaaddaleax
lrlna
and
addaleax
authored
chore(build): MONGOSH-122: upload linux packages to barque (#286)
- Introduces `curator` as a tool to upload packaged assets to barque, MongoDB's PPA service. - Uses an already bundled and packaged mongosh binary that was already uploaded to Evergreen's S3 bucket to send to barque. - Adds additional environmental variables to the build process: `BARQUE_USERNAME`, `BARQUE_API_KEY`, `NOTARY_TOKEN`, `NOTARY_KEY_NAME`. These are then used by `curator`. - Adds `repo-config.yml` that is necessary for a barque upload. The config specifies uploads for 3 different distros: `debian10`, `ubuntu1804`, `rhel80`. - We only upload the package to `org` MongoDB repository. - We only upload along with `4.4.0` MongoDB version. - These artifacts are only sent to `barque` when we do a public release (`shouldDoPublicRelease`), i.e. a tagged commit and on main branch. - We fetch the latest curator tarball before proceeding to upload to barque. Co-authored-by: Anna Henningsen <[email protected]>
1 parent 15dc515 commit 4cf2e7e

File tree

11 files changed

+749
-36
lines changed

11 files changed

+749
-36
lines changed

‎config/repo-config.yml‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
services:
2+
notary_url: "http://notary-service.build.10gen.cc:5000"
3+
4+
templates:
5+
deb:
6+
org: |
7+
Origin: mongodb
8+
Label: mongodb
9+
Suite: {{ .CodeName }}
10+
Codename: {{ .CodeName }}/mongodb-org
11+
Architectures: {{ .Architectures }}
12+
Components: {{ .Component }}
13+
Description: MongoDB packages
14+
enterprise: |
15+
Origin: mongodb
16+
Label: mongodb
17+
Suite: {{ .CodeName }}
18+
Codename: {{ .CodeName }}/mongodb-enterprise
19+
Architectures: {{ .Architectures }}
20+
Components: {{ .Component }}
21+
Description: MongoDB packages
22+
index_page: |
23+
<!DOCTYPE html>
24+
<html>
25+
<head>
26+
<title>{{ .Title }}</title>
27+
</head>
28+
<body>
29+
<table>
30+
<tr><td>
31+
<h1>{{ .Title }}</h1>
32+
</td></tr>
33+
<tr><td>
34+
<hr>
35+
</td></tr>
36+
<tr><td>
37+
<a href='..'>Parent Directory</a>
38+
</td></tr>
39+
{{ range $fn := .Files }}
40+
<tr><td>
41+
<a href='{{ $fn }}'>{{ $fn }}</a>
42+
</td></tr>
43+
{{ end }}
44+
<tr><td>
45+
<hr>
46+
</td></tr>
47+
<tr><td>
48+
<address>{{ .RepoName }}</address>
49+
</td></tr>
50+
</table>
51+
</body>
52+
</html>
53+
repos:
54+
55+
####################
56+
#
57+
# Community Repos:
58+
#
59+
####################
60+
61+
- name: debian10
62+
type: deb
63+
code_name: "buster"
64+
bucket: repo.mongodb.org
65+
edition: org
66+
component: main
67+
architectures:
68+
- amd64
69+
repos:
70+
- apt/debian/dists/buster/mongodb-org
71+
72+
- name: ubuntu1804
73+
type: deb
74+
code_name: "bionic"
75+
edition: org
76+
bucket: repo.mongodb.org
77+
component: multiverse
78+
architectures:
79+
- amd64
80+
repos:
81+
- apt/ubuntu/dists/bionic/mongodb-org
82+
83+
- name: rhel80
84+
type: rpm
85+
edition: org
86+
bucket: repo.mongodb.org
87+
repos:
88+
- yum/redhat/8/mongodb-org
89+
- yum/redhat/8Server/mongodb-org

‎package-lock.json‎

Lines changed: 116 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@
8585
"eslint-config-mongodb-js": "^5.0.3",
8686
"eslint-plugin-mocha": "^6.2.2",
8787
"fs-extra": "^8.1.0",
88+
"gunzip-maybe": "^1.4.2",
8889
"handlebars": "^4.7.6",
8990
"js-yaml": "^3.13.1",
9091
"karma": "^4.4.1",
@@ -98,6 +99,7 @@
9899
"mongodb-js-precommit": "^2.0.0",
99100
"mongodb-runner": "^4.7.5",
100101
"node-codesign": "github:addaleax/node-codesign",
102+
"node-fetch": "^2.6.1",
101103
"parcel-bundler": "^1.12.4",
102104
"pkg": "^4.4.3",
103105
"pkg-deb": "^1.1.1",
@@ -107,6 +109,8 @@
107109
"sinon": "^7.5.0",
108110
"sinon-chai": "^3.4.0",
109111
"tar": "^6.0.1",
112+
"tar-fs": "^2.1.0",
113+
"tmp-promise": "^3.0.2",
110114
"ts-node": "^8.5.2",
111115
"ts-sinon": "^1.2.0",
112116
"typescript": "^3.9.7",

‎packages/build/README.md‎

Lines changed: 27 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -10,25 +10,33 @@ root of the project.
1010

1111
Current build and release flow is as follows:
1212

13-
- `npm run evergreen-release package
14-
- A commit triggers an evergreen build based on currently available build
15-
variants: MacOS, Windows, Linux, Debian, and RedHat.
16-
- MacOS, Linux and Windows run three tasks: check, test, and release. Debian and
17-
Redhat run two tasks: check and release. Debian and Redhat also depend on
18-
tests to pass on Linux.
19-
- Identical bundle and binary are built on all five variants.
20-
- Each variant creates its own tarball (`.zip`, `.tgz`, `.deb`, `.rpm`). Type of
21-
tarball is determined by the current build variant.
22-
- Each variant uploads its own tarball to Evergreen’s AWS.
23-
- MacOS build variant uploads config file with information about the new version
24-
for each platform to Downloads Centre. This only happens on a tagged commit.
25-
- MacOS build variant creates a github release. This only happens on a tagged
26-
commit.
27-
- The five build variants run in parallel.
28-
- `npm run evergreen-release publish`
29-
- All the previous build steps succeeded.
30-
- A separate MacOS build variant (darwin_publish_release) uploads config file with information about the new version for each platform to Downloads Centre. This only happens on a tagged commit.
31-
- A separate MacOS build variant (darwin_publish_release) promotes the draft github release to public. This only happens on a tagged commit.
13+
### `npm run evergreen-release package`
14+
- A commit triggers an evergreen build based on currently available build
15+
variants: MacOS, Windows, Linux, Debian, and RedHat.
16+
- MacOS, Linux and Windows run three tasks: check, test, and release. Debian and
17+
Redhat run two tasks: check and release. Debian and Redhat also depend on
18+
tests to pass on Linux.
19+
- Identical bundle and binary are built on all five variants.
20+
- Each variant creates its own tarball (`.zip`, `.tgz`, `.deb`, `.rpm`). Type of
21+
tarball is determined by the current build variant.
22+
- Each variant uploads its own tarball to Evergreen’s AWS.
23+
- Linux build variants upload their artifacts to `barque` using
24+
[`curator`](https://github.com/mongodb/curator) to be used with MongoDB's PPA. The uploaded packages can be found under the following URLs:
25+
1. Ubuntu: https://repo.mongodb.org/apt/ubuntu/dists/bionic/mongodb-org/4.4/multiverse/binary-amd64/
26+
2. Redhat: https://repo.mongodb.org/yum/redhat/8Server/mongodb-org/4.4/x86_64/RPMS/
27+
3. Debian: https://repo.mongodb.org/apt/debian/dists/buster/mongodb-org/4.4/main/binary-amd64/
28+
- MacOS build variant uploads config file with information about the new version
29+
for each platform to Downloads Centre. This only happens on a tagged commit.
30+
- MacOS build variant creates a github release. This only happens on a tagged
31+
commit.
32+
- The five build variants run in parallel.
33+
### `npm run evergreen-release publish`
34+
- All the previous build steps succeeded.
35+
- A separate MacOS build variant (darwin_publish_release) uploads config file
36+
with information about the new version for each platform to Downloads Centre.
37+
This only happens on a tagged commit.
38+
- A separate MacOS build variant (darwin_publish_release) promotes the draft
39+
github release to public. This only happens on a tagged commit.
3240

3341
![build flow][build-img]
3442

0 commit comments

Comments
 (0)