You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: keps/sig-release/3000-artifact-distribution/README.md
+16-15Lines changed: 16 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,4 @@
1
-
# KEP/MST-3000: Image Promotion and Distribution Policy
1
+
# KEP3000: Image Promotion and Distribution Policy
2
2
3
3
<!-- toc -->
4
4
@@ -47,17 +47,17 @@ For a few years now, we have been using k8s.gcr.io in all our repositories as de
47
47
48
48
The cost of distributing Kubernetes comes at great cost nearing $150kUSD/month (mostly egress) in donations.
49
49
50
-
Additionally some of our community members are unable to access the official release artifacts due to country level firewalls that do not them connect to Google services.
50
+
Additionally some of our community members are unable to access the official release container images due to country level firewalls that do not them connect to Google services.
51
51
52
-
Ideally we can dramatically reduce cost and allow everyone in the world to download the artifacts released by our community.
52
+
Ideally we can dramatically reduce cost and allow everyone in the world to download the container iamges released by our community.
53
53
54
54
We are now used to using the [image promoter process](https://github.com/kubernetes/enhancements/tree/master/keps/sig-release/1734-k8s-image-promoter) to promote images to the official kubernetes container registry using the infrastructure (GCR staging repos etc) provided by [sig-k8s-infra](https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io)
55
55
56
56
## Why a new domain?
57
57
58
58
So far we (all kubernetes project) are using GCP as our default infrastructure provider for all things like GCS, GCR, GKE based prow clusters etc. Google has graciously sponsored a lot of our infrastructure costs as well. However for about a year or so we are finding that our costs are sky-rocketing because the community usage of this infrastructure has been from other cloud providers like AWS, Azure etc. So in conjunction with CNCF staff we are trying to put together a plan to host copies of images and binaries nearer to where they are used rather than incur cross-cloud costs.
59
59
60
-
One part of this plan is to setup a proxy OCI service, that can identify where the traffic is coming from and redirect to the nearest image layer/repository. This is why we are setting up a new service using what we call an [oci-proxy](https://github.com/kubernetes-sigs/oci-proxy) for everyone to use. This proxy will identify traffic coming from, for example, a certain AWS region, then will setup a HTTP redirect to a source in that AWS region. If we get traffic from GKE/GCP or we don't know where the traffic is coming from, it will still redirect to the current infrastructure (k8s.gcr.io).
60
+
One part of this plan is to setup a redirecting web service, that can identify where the traffic is coming from and redirect to the nearest image layer/repository. This is why we are setting up a new service using what we call an [oci-proxy](https://github.com/kubernetes-sigs/oci-proxy) for everyone to use. This redirector will identify traffic coming from, for example, a certain AWS region, then will setup a HTTP redirect to a source in that AWS region. If we get traffic from GKE/GCP or we don't know where the traffic is coming from, it will still redirect to the current infrastructure (k8s.gcr.io).
61
61
62
62
## How can we help?
63
63
@@ -71,7 +71,7 @@ A solution to allow redirection to appropriate mirrors to lower cost and allow a
71
71
72
72
### Non-Goals
73
73
74
-
Anything related to creation of artifacts, bom, digital signatures, staging buckets.
74
+
Anything related to creation of artifacts, bom, staging buckets.
75
75
76
76
### What is not in scope
77
77
@@ -87,14 +87,14 @@ Anything related to creation of artifacts, bom, digital signatures, staging buck
87
87
88
88
There are two intertwined concepts that are part of this proposal.
89
89
90
-
First, the policy and procedures to promote/upload our artifacts to multiple providers. Our existing processes upload only to GCS buckets. Ideally we extend the existing software/promotion process to push directly to multiple providers. Alternatively we use a second process to synchronize artifacts from our existing production buckets to similar constructs at other providers.
90
+
First, the policy and procedures to promote/upload our container images to multiple providers. Our existing processes upload only to GCS buckets. Ideally we extend the existing software/promotion process to push directly to multiple providers. Alternatively we use a second process to synchronize container images from our existing production buckets to similar constructs at other providers.
91
91
92
92
Additionally we require a registry and artifact url-redirection solution to the local cloud provider or country.
93
93
94
94
## What exactly are you doing?
95
95
96
96
- We are setting up an AWS account with an IAM role and s3 buckets in AWS regions where we see a large percentage of source image pull traffic
97
-
- We will iterate on a sandbox url (registry.sandbox.k8s.io) for our experiments and ONLY promote things to (registry.k8s.io) when we have complete confidence
97
+
- We will iterate on a sandbox url (registry-sandbox.k8s.io) for our experiments and ONLY promote things to (registry.k8s.io) when we have complete confidence
98
98
- both registry and registry-sandbox are serving traffic using oci-proxy on google cloud run
99
99
- oci-proxy will be updated to identify incoming traffic from AWS regions based on IP ranges so we can route traffic to s3 buckets in that region. If a specific AWS region do not currently host s3 buckets, we will redirect to the nearest region which does have s3 buckets (tradeoff between storage and network costs)
100
100
- We will bulk sync existing image layers to these s3 layers as a starting point (from GCS/GCR)
@@ -109,8 +109,9 @@ Additionally we require a registry and artifact url-redirection solution to the
109
109
#### SIG Release - Image Promotion
110
110
111
111
```feature
112
-
As a SIG Release volunteer
113
-
I want to promote our binaries/images to multiple clouds
112
+
Scenario: images are promoted
113
+
As a SIG Release volunteer
114
+
I want to promote our binaries/images to multiple clouds
114
115
115
116
Given a promotion / manifest
116
117
When my PR is merged
@@ -120,12 +121,12 @@ Then the promotion process occurs
120
121
#### Cloud Customer - pulling an official container image
121
122
122
123
```feature
123
-
As a CLOUD end-user
124
-
I want to install Kubernetes
124
+
Scenario: use Kubernetes container images
125
+
I want to be able to pull and use Kubernetes container images
125
126
126
-
Given some compute resources at CLOUD
127
-
When I pull an official Kubernetes container image
128
-
Then I am redirected to a cloud provider backed bucket (set) or CDN otherwise fall back to k8s.gcr.io
127
+
Given some compute resources at cloud
128
+
When I pull an official Kubernetes container image from registry.k8s.io
129
+
Then I am redirected to a close-by cloud provider backed bucket (set) / CDN otherwise fall back to k8s.gcr.io
129
130
```
130
131
131
132
### Notes/Constraints/Caveats
@@ -156,7 +157,7 @@ Currently the promotion process is primarily driven by the CIP/[promo-tool#kprom
156
157
157
158
#### Process
158
159
159
-
Artifacts will be written to S3 style storage or CDNs provided by cloud providers through a tool in the promo-tools suite.
160
+
Container images will be written to S3 style storage or CDNs provided by cloud providers through a tool in the promo-tools suite.
0 commit comments