Skip to content

Commit dd1b4ad

Browse files
msumitmsumit
authored
Add README_RELEASE_CLIENT (#10)
1 parent 4080e25 commit dd1b4ad

File tree

5 files changed

+335
-30
lines changed

5 files changed

+335
-30
lines changed

‎.pre-commit-config.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,12 +58,12 @@ repos:
5858
- --fuzzy-match-generates-todo
5959
- id: insert-license
6060
name: Add license for all other files
61-
exclude: ^\.github/.*$
61+
exclude: ^\.github/.*$|\.*LICENSE.txt$
6262
args:
6363
- --comment-style
6464
- "|#|"
6565
- --license-filepath
6666
- license-templates/LICENSE.txt
6767
- --fuzzy-match-generates-todo
6868
files: >
69-
\.properties$|\.cfg$|\.conf$|\.ini$$
69+
\.properties$|\.cfg$|\.conf$|\.ini$|\.txt$$

‎dev/README_RELEASE_CLIENT.md‎

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,269 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
You can find the prerequisites to release Apache Airflow Python Client in [README.md](README.md).
21+
22+
# Prepare the Apache Airflow Python Client Package RC
23+
24+
## Build RC artifacts
25+
26+
The Release Candidate artifacts we vote upon should be the exact ones we vote against, without any modification than
27+
renaming – i.e. the contents of the files must be the same between voted release candidate and final release. Because
28+
of this the version in the built artifacts that will become the official Apache releases must not include the rcN suffix.
29+
30+
- Set environment variables
31+
32+
```shell script
33+
# Set Version
34+
export VERSION=2.0.0rc1
35+
36+
37+
# Example after cloning
38+
git clone https://github.com/apache/airflow-client-python.git airflow-client
39+
cd airflow-client
40+
export CLIENT_REPO_ROOT=$(pwd)
41+
```
42+
43+
- Set your version to 2.0.0 in `setup.py` (without the RC tag)
44+
- Commit the version change.
45+
46+
- Tag your release
47+
48+
```shell script
49+
git tag -s ${VERSION}
50+
```
51+
52+
- Clean the checkout: the sdist step below will
53+
54+
```shell script
55+
git clean -fxd
56+
```
57+
58+
- Tarball the repo
59+
60+
```shell script
61+
git archive --format=tar.gz ${VERSION} -o airflow-client-${VERSION}-source.tar.gz
62+
```
63+
64+
- Generate sdist
65+
66+
NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included
67+
in the file produced.
68+
69+
```shell script
70+
python setup.py sdist bdist_wheel
71+
```
72+
73+
- Rename the sdist
74+
75+
```shell script
76+
mv dist/airflow-client-${VERSION%rc?}.tar.gz airflow-client-${VERSION}-bin.tar.gz
77+
mv dist/airflow_client-${VERSION%rc?}-py3-none-any.whl airflow_client-${VERSION}-py3-none-any.whl
78+
```
79+
80+
- Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on
81+
http://www.apache.org/dev/openpgp.html#key-gen-generate-key)
82+
83+
```shell script
84+
${CLIENT_REPO_ROOT}/dev/sign.sh airflow-client-${VERSION}-source.tar.gz
85+
${CLIENT_REPO_ROOT}/dev/sign.sh airflow-client-${VERSION}-bin.tar.gz
86+
${CLIENT_REPO_ROOT}/dev/sign.sh airflow_client-${VERSION}-py3-none-any.whl
87+
```
88+
89+
- Push the artifacts to ASF dev dist repo
90+
91+
```
92+
# First clone the repo
93+
svn checkout https://dist.apache.org/repos/dist/dev/airflow airflow-dev
94+
95+
# Create new folder for the release
96+
cd airflow-dev/clients/python
97+
svn mkdir ${VERSION}
98+
99+
# Move the artifacts to svn folder & commit
100+
mv ${CLIENT_REPO_ROOT}/airflow{-,_}client-${VERSION}* ${VERSION}/
101+
cd ${VERSION}
102+
svn add *
103+
svn commit -m "Add artifacts for Airflow Python Client ${VERSION}"
104+
```
105+
106+
## Prepare PyPI convenience "snapshot" packages
107+
108+
At this point we have the artefact that we vote on, but as a convenience to developers we also want to
109+
publish "snapshots" of the RC builds to pypi for installing via pip.
110+
111+
To do this we need to
112+
113+
- Build the package:
114+
115+
```shell script
116+
cd ${CLIENT_REPO_ROOT}
117+
python setup.py egg_info --tag-build "$(sed -e "s/^[0-9.]*//" <<<"$VERSION")" sdist bdist_wheel
118+
```
119+
120+
- Verify the artifacts that would be uploaded:
121+
122+
```shell script
123+
twine check dist/*
124+
```
125+
126+
- Upload the package to PyPi's test environment:
127+
128+
```shell script
129+
twine upload --repository-url=https://test.pypi.org/legacy/ dist/*
130+
```
131+
132+
- Verify that the test package looks good by downloading it and installing it into a virtual environment. The package
133+
download link is available at:
134+
https://test.pypi.org/project/airflow-client/#files
135+
136+
Or via pypi
137+
pip install -i https://test.pypi.org/simple/ airflow-client==$(VERSION)
138+
139+
- Upload the package to PyPi's production environment:
140+
`twine upload -r pypi dist/*`
141+
142+
- Again, confirm that the package is available here:
143+
https://pypi.python.org/pypi/airflow-client
144+
145+
It is important to stress that this snapshot should not be named "release", and it
146+
is not supposed to be used by and advertised to the end-users who do not read the devlist.
147+
148+
- Push Tag for the release candidate
149+
150+
```shell script
151+
git push origin ${VERSION}
152+
```
153+
154+
## Prepare Vote email on the Airflow Client release candidate
155+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#prepare-vote-email-on-the-apache-airflow-release-candidate) (just replace Airflow with Airflow Client).
156+
157+
# Verify the release candidate by PMCs
158+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmcs).
159+
160+
## SVN check
161+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#svn-check) (just replace Airflow with Airflow Client).
162+
163+
## Licence check
164+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#licence-check).
165+
166+
## Signature check
167+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#signature-check).
168+
169+
170+
171+
# Verify release candidates by Contributors
172+
This can be done (and we encourage to) by any of the Contributors. In fact, it's best if the
173+
actual users of Airflow Client test it in their own staging/test installations. Each release candidate
174+
is available on PyPI apart from SVN packages, so everyone should be able to install
175+
the release candidate version of Airflow Client via simply (<VERSION> is 2.0.0 for example, and <X> is
176+
release candidate number 1,2,3,....).
177+
178+
```shell script
179+
pip install airflow-client==<VERSION>rc<X>
180+
```
181+
182+
Once you install and run Airflow Client, you should perform any verification you see as necessary to check
183+
that the client works as you expected.
184+
185+
# Publish the final Airflow client release
186+
187+
## Summarize the voting for the Airflow client release
188+
189+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#publish-the-final-apache-airflow-release) (just replace Airflow with Airflow Client).
190+
191+
## Publish release to SVN
192+
193+
```shell script
194+
# First clone the repo
195+
export RC=2.0.0rc1
196+
export VERSION=${RC/rc?/}
197+
svn checkout https://dist.apache.org/repos/dist/release/airflow airflow-release
198+
199+
# Create new folder for the release
200+
cd airflow-release/clients/python
201+
svn mkdir ${VERSION}
202+
cd ${VERSION}
203+
204+
# Move the artifacts to svn folder & commit
205+
for f in ../../../airflow-dev/clients/python/$RC/*; do svn cp $f ${$(basename $f)/rc?/}; done
206+
svn commit -m "Release Airflow Client ${VERSION} from ${RC}"
207+
208+
# Remove old release
209+
# http://www.apache.org/legal/release-policy.html#when-to-archive
210+
cd ..
211+
export PREVIOUS_VERSION=1.0.0
212+
svn rm ${PREVIOUS_VERSION}
213+
svn commit -m "Remove old release: ${PREVIOUS_VERSION}"
214+
```
215+
216+
Verify that the packages appear in [airflow](https://dist.apache.org/repos/dist/release/airflow/clients/python)
217+
218+
## Prepare PyPI "release" packages
219+
220+
At this point we release an official package:
221+
222+
- Build the package:
223+
224+
```shell script
225+
python setup.py sdist bdist_wheel
226+
```
227+
228+
- Verify the artifacts that would be uploaded:
229+
230+
```shell script
231+
twine check dist/*
232+
```
233+
234+
- Upload the package to PyPi's test environment:
235+
236+
```shell script
237+
twine upload --repository-url=https://test.pypi.org/legacy/ dist/*
238+
```
239+
240+
- Verify that the test package looks good by downloading it and installing it into a virtual environment.
241+
The package download link is available at: https://test.pypi.org/project/airflow-client/#files
242+
243+
- Upload the package to PyPi's production environment:
244+
245+
```shell script
246+
twine upload -r pypi dist/*
247+
```
248+
249+
- Again, confirm that the package is available here: https://pypi.python.org/pypi/airflow-client
250+
251+
## Update CHANGELOG.md
252+
253+
- Get a diff between the last version and the current version:
254+
255+
```shell script
256+
git log 1.0.0..2.0.0 --pretty=oneline
257+
```
258+
259+
- Update CHANGELOG.md with the details, and commit it.
260+
261+
- Push Tag for the final version
262+
263+
```shell script
264+
git push origin ${VERSION}
265+
```
266+
267+
## Notify developers of release
268+
269+
See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#notify-developers-of-release) (just replace Airflow with Airflow Client)

‎dev/sign.sh‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/usr/bin/env bash
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
set -euo pipefail
19+
20+
# Use this to sign the tar balls generated from
21+
# python setup.py sdist --formats=gztar
22+
# ie. sign.sh <my_tar_ball>
23+
# you will still be required to type in your signing key password
24+
# or it needs to be available in your keychain
25+
26+
# Which key to sign releases with? This can be a (partial) email address or a
27+
# key id. By default use any apache.org key
28+
SIGN_WITH="${SIGN_WITH:-apache.org}"
29+
30+
for name in "${@}"
31+
do
32+
gpg --armor --local-user "$SIGN_WITH" --output "${name}.asc" --detach-sig "${name}"
33+
shasum -a 512 "${name}" > "${name}.sha512"
34+
done

‎requirements.txt‎

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,20 @@
1-
# Licensed to the Apache Software Foundation (ASF) under one
2-
# or more contributor license agreements. See the NOTICE file
3-
# distributed with this work for additional information
4-
# regarding copyright ownership. The ASF licenses this file
5-
# to you under the Apache License, Version 2.0 (the
6-
# "License"); you may not use this file except in compliance
7-
# with the License. You may obtain a copy of the License at
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
88
#
9-
# http://www.apache.org/licenses/LICENSE-2.0
9+
# http://www.apache.org/licenses/LICENSE-2.0
1010
#
11-
# Unless required by applicable law or agreed to in writing,
12-
# software distributed under the License is distributed on an
13-
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14-
# KIND, either express or implied. See the License for the
15-
# specific language governing permissions and limitations
16-
# under the License.
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
1718
python_dateutil >= 2.5.3
1819
setuptools >= 21.0.0
1920
urllib3 >= 1.25.3

‎test-requirements.txt‎

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,20 @@
1-
# Licensed to the Apache Software Foundation (ASF) under one
2-
# or more contributor license agreements. See the NOTICE file
3-
# distributed with this work for additional information
4-
# regarding copyright ownership. The ASF licenses this file
5-
# to you under the Apache License, Version 2.0 (the
6-
# "License"); you may not use this file except in compliance
7-
# with the License. You may obtain a copy of the License at
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
88
#
9-
# http://www.apache.org/licenses/LICENSE-2.0
9+
# http://www.apache.org/licenses/LICENSE-2.0
1010
#
11-
# Unless required by applicable law or agreed to in writing,
12-
# software distributed under the License is distributed on an
13-
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14-
# KIND, either express or implied. See the License for the
15-
# specific language governing permissions and limitations
16-
# under the License.
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
1718
pytest-cov>=2.8.1
1819
python_dateutil >= 2.5.3
1920
urllib3 >= 1.25.3

0 commit comments

Comments
 (0)