You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: keps/sig-release/1732-artifact-management/README.md
+97-72Lines changed: 97 additions & 72 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,70 +14,88 @@
14
14
<!-- /toc -->
15
15
16
16
## Summary
17
-
This document describes how official artifacts (Container Images, Binaries) for the Kubernetes
18
-
project are managed and distributed.
19
17
18
+
This document describes how official artifacts (Container Images, Binaries) for
19
+
the Kubernetes project are managed and distributed.
20
20
21
21
## Motivation
22
22
23
-
The motivation for this KEP is to describe a process by which artifacts (container images, binaries)
24
-
can be distributed by the community. Currently the process by which images is both ad-hoc in nature
25
-
and limited to an arbitrary set of people who have the keys to the relevant repositories. Standardize
26
-
access will ensure that people around the world have access to the same artifacts by the same names
27
-
and that anyone in the project is capable (if given the right authority) to distribute images.
23
+
The motivation for this KEP is to describe a process by which artifacts
24
+
(container images, binaries) can be distributed by the community. Currently,
25
+
the process by which images is both ad hoc in nature and limited to an
26
+
arbitrary set of people who have the keys to the relevant repositories.
27
+
Standardized access will ensure that people around the world have access to the
28
+
same artifacts by the same names and that anyone in the project is capable (if
29
+
given the right authority) to distribute images.
28
30
29
31
### Goals
30
32
31
33
The goals of this process are to enable:
32
-
* Anyone in the community (with the right permissions) to manage the distribution of Kubernetes images and binaries.
33
-
* Fast, cost-efficient access to artifacts around the world through appropriate mirrors and distribution
34
34
35
-
This KEP will have succeeded when artifacts are all managed in the same manner and anyone in the community
36
-
(with the right permissions) can manage these artifacts.
35
+
- Anyone in the community (with the right permissions) to manage the
36
+
distribution of Kubernetes images and binaries
37
+
- Fast, cost-efficient access to artifacts around the world through appropriate
38
+
mirrors and distribution
39
+
40
+
This KEP will have succeeded when artifacts are all managed in the same manner
41
+
and anyone in the community (with the right permissions) can manage these
42
+
artifacts.
37
43
38
44
### Non-Goals
39
45
40
-
The actual process and tooling for promoting images, building packages or otherwise assembling artifacts
41
-
is beyond the scope of this KEP. This KEP deals with the infrastructure for serving these things via
42
-
HTTP as well as a generic description of how promotion will be accomplished.
46
+
The actual process and tooling for promoting images, building packages or
47
+
otherwise assembling artifacts is beyond the scope of this KEP. This KEP deals
48
+
with the infrastructure for serving these things via HTTP as well as a generic
49
+
description of how promotion will be accomplished.
43
50
44
51
## Proposal
45
52
46
-
The top level design will be to set up a global redirector HTTP service (`artifacts.k8s.io`)
47
-
which knows how to serve HTTP and redirect requests to an appropriate mirror. This redirector
48
-
will serve both binary and container image downloads. For container images, the HTTP redirector
49
-
will redirect users to the appropriate geo-located container registry. For binary artifacts,
50
-
the HTTP redirector will redirect to appropriate geo-located storage buckets.
51
-
52
-
To facilitate artifact promotion, each project, as necessary, will be given access to a
53
-
project staging area relevant to their particular artifacts (either storage bucket or image
54
-
registry). Each project is free to manage their assets in the staging area however they feel
55
-
it is best to do so. However, end-users are not expected to access artifacts through the
56
-
staging area.
57
-
58
-
For each artifact, there will be a configuration file checked into this repository. When a
59
-
project wants to promote an image, they will file a PR in this repository to update their
60
-
image promotion configuration to promote an artifact from staging to production. Once this
61
-
PR is approved, automation that is running in the k8s project infrastructure (e.g.
62
-
https://github.com/GoogleCloudPlatform/k8s-container-image-promoter) will pick up this new
63
-
configuration file and copy the relevant bits out to the production serving locations.
64
-
65
-
Importantly, if a project needs to roll-back or remove an artifact, the same process will
66
-
occur, so that the promotion tool needs to be capable of deleting images and artifacts as
67
-
well as promoting them.
53
+
The top level design will be to set up a global redirector HTTP service
54
+
(`artifacts.k8s.io`) which knows how to serve HTTP and redirect requests to an
55
+
appropriate mirror. This redirector will serve both binary and container image
56
+
downloads. For container images, the HTTP redirector will redirect users to the
57
+
appropriate geo-located container registry. For binary artifacts, the HTTP
58
+
redirector will redirect to appropriate geo-located storage buckets.
59
+
60
+
To facilitate artifact promotion, each project, as necessary, will be given
61
+
access to a project staging area relevant to their particular artifacts (either
62
+
storage bucket or image registry). Each project is free to manage their assets
63
+
in the staging area however they feel it is best to do so. However, end-users
64
+
are not expected to access artifacts through the staging area.
65
+
66
+
For each artifact, there will be a configuration file checked into this
67
+
repository. When a project wants to promote an image, they will file a PR in
68
+
this repository to update their image promotion configuration to promote an
69
+
artifact from staging to production. Once this PR is approved, automation that
70
+
is running in the k8s project infrastructure (built using our
71
+
[artifact promotion tooling][promo-tools]) will pick up this new configuration
72
+
file and copy the relevant bits out to the production serving locations.
73
+
74
+
Importantly, if a project needs to roll-back or remove an artifact, the same
75
+
process will occur, so that the promotion tool needs to be capable of deleting
76
+
images and artifacts as well as promoting them.
68
77
69
78
### HTTP Redirector Design
70
-
To facilitate world-wide distribution of artifacts from a single (virtual) location we will
71
-
ideally run a replicated redirector service in the United States, Europe and Asia.
72
-
Each of these redirectors
73
-
services will be deployed in a Kubernetes cluster and they will be exposed via a public IP
74
-
address and a dns record indicating their location (e.g. `europe.artifacts.k8s.io`).
75
79
76
-
We will use Geo DNS to route requests to `artifacts.k8s.io` to the correct redirector. This is necessary to ensure that we always route to a server which is accessible no matter what region we are in. We will need to extend or enhance the existing DNS synchronization tooling to handle creation of the GeoDNS records.
80
+
To facilitate world-wide distribution of artifacts from a single (virtual)
81
+
location we will ideally run a replicated redirector service in the United
82
+
States, Europe and Asia.
83
+
84
+
Each of these redirectors services will be deployed in a Kubernetes cluster and
85
+
they will be exposed via a public IP address and a dns record indicating their
86
+
location (e.g. `europe.artifacts.k8s.io`).
87
+
88
+
We will use Geo DNS to route requests to `artifacts.k8s.io` to the correct
89
+
redirector. This is necessary to ensure that we always route to a server which
90
+
is accessible no matter what region we are in. We will need to extend or
91
+
enhance the existing DNS synchronization tooling to handle creation of the
92
+
GeoDNS records.
77
93
78
94
#### Configuring the HTTP Redirector
79
-
THe HTTP Redirector service will be driven from a YAML configuration that specifies a path to mirror
80
-
mapping. For now the redirector will serve content based on continent, for example:
95
+
96
+
THe HTTP Redirector service will be driven from a YAML configuration that
97
+
specifies a path to mirror mapping. For now the redirector will serve content
98
+
based on continent, for example:
81
99
82
100
```yaml
83
101
/kops
@@ -86,45 +104,52 @@ mapping. For now the redirector will serve content based on continent, for examp
86
104
- default: americas.artificats.k8s.io
87
105
```
88
106
89
-
The redirector will use this data to redirect a request to the relevant mirror using HTTP 302 responses. The implementation of the mirrors themselves are details left to the service implementor and may be different depending on the artifacts being exposed (binaries vs. container images)
107
+
The redirector will use this data to redirect a request to the relevant mirror
108
+
using HTTP 302 responses. The implementation of the mirrors themselves are
109
+
details left to the service implementor and may be different depending on the
110
+
artifacts being exposed (binaries vs. container images).
90
111
91
112
## Graduation Criteria
92
113
93
-
This KEP will graduate when the process is implemented and has been successfully used to
94
-
manage the images for a Kubernetes release.
114
+
This KEP will graduate when the process is implemented and has been
115
+
successfully used to manage the images for a Kubernetes release.
95
116
96
117
## Implementation History
97
118
98
119
### Milestone 0 (MVP): In progress
99
120
100
-
(Described in terms of kops, our first candidate; other candidates welcome!)
121
+
(Described in terms of kOps, our first candidate; other candidates welcome!)
101
122
102
-
* k8s-infra creates a "staging" GCS bucket for each project
103
-
(e.g. `k8s-artifacts-staging-<project>`) and a "prod" GCS bucket for promoted
104
-
artifacts (e.g. `k8s-artifacts`, one bucket for all projects).
105
-
* We grant write-access to the staging GCS bucket to trusted jobs / people in
106
-
each project (e.g. kops OWNERS and prow jobs can push to
107
-
`k8s-artifacts-staging-kops`). We can encourage use of CI & reproducible
123
+
- k8s-infra creates a "staging" GCS bucket for each project (e.g.,
124
+
`k8s-artifacts-staging-<project>`) and a "prod" GCS bucket for promoted
125
+
artifacts (e.g., `k8s-artifacts`, one bucket for all projects)
126
+
-We grant writeaccess to the staging GCS bucket to trusted jobs/people in
127
+
each project (e.g., kOps OWNERS and prow jobs can push to
128
+
`k8s-artifacts-staging-kops`). We can encourage use of CI & reproducible
108
129
builds, but we do not block on it.
109
-
* We grant write-access to the prod bucket only to the infra-admins & the
110
-
promoter process.
111
-
* Promotion of artifacts to the "prod" GCS bucket is via a script / utility (as
112
-
we do today). For v1 we can promote based on a sha256sum file (only copy the
113
-
files listed), similarly to the image promoter. We will experiment to develop
114
-
that script / utility in this milestone, along with prow jobs (?) to publish
115
-
to the staging buckets, and to figure out how best to run the promoter.
116
-
Hopefully we can copy the image-promotion work closely.
117
-
* We create a bucket-backed GCLB for serving, with a single url-map entry for
130
+
-We grant writeaccess to the prod bucket only to the infra-admins & the
131
+
promoter process
132
+
-Promotion of artifacts to the "prod" GCS bucket is via a script/utility (as
133
+
we do today). For v1, we can promote based on a sha256sum file (only copy the
134
+
files listed), similarly to the image promoter. We will experiment to develop
135
+
that script/utility in this milestone, along with prow jobs (?) to publish to
136
+
the staging buckets, and to figure out how best to run the promoter.
137
+
Hopefully, we can copy the image-promotion work closely.
138
+
-We create a bucket-backed GCLB for serving, with a single url-map entry for
118
139
`binaries/`pointing to the prod bucket. (The URL prefix gives us some
119
140
flexibility to e.g. add dynamic content later)
120
-
* We create the artifacts.k8s.io DNS name pointing to the GCLB. (Unclear whether
121
-
we want one for staging, or just encourage pulling from GCS directly).
122
-
* Projects start using the mirrors e.g. kops adds the
141
+
-We create the artifacts.k8s.io DNS name pointing to the GCLB (unclear whether
142
+
we want one for staging, or just encourage pulling from GCS directly)
143
+
-Projects start using the mirrors e.g., kOps adds the
123
144
https://artifacts.k8s.io/binaries/kops mirror into the (upcoming) mirror-list
124
-
support, so that it will get real traffic but not break kops should this
145
+
support, so that it will get real traffic but not break kOps should this
125
146
infrastructure break
126
-
* We start to collect data from the GCLB logs. Questions we would like to
127
-
understand: What are the costs, and what would the costs be for localized
128
-
mirrors? What is the performance impact (latency, throughput) of serving
129
-
everything from GCLB? Is GCLB reachable from everywhere (including China)?
130
-
Can we support private mirrors (i.e. non-coordinated mirrors)?
147
+
- We start to collect data from the GCLB logs
148
+
- Questions we would like to understand:
149
+
- What are the costs and what would the costs be for localized mirrors?
150
+
- What is the performance impact (latency, throughput) of serving everything
151
+
from GCLB?
152
+
- Is GCLB reachable from everywhere (including China)?
153
+
- Can we support private (non-coordinated) mirrors?
0 commit comments