Skip to content
This repository was archived by the owner on Feb 6, 2024. It is now read-only.

Update how to release instructions with more details #210

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
+118 −54
Open

Update how to release instructions with more details #210

Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
241f176
Update how to release instructions with more details
Mar 21, 2023
0e74ddc
add set upstream
Mar 21, 2023
d5479cc
add other steps
Mar 22, 2023
7d428af
add more details
Mar 22, 2023
82bf745
update doc inst
Mar 22, 2023
6a47851
fix more doc release inst
Mar 22, 2023
82307fc
reorder steps
Mar 23, 2023
0d4e7fa
update header sizes
Mar 23, 2023
2f211fd
Update landing-page/content/common/how-to-release.md
Fokko Jun 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 99 additions & 43 deletions landing-page/content/common/how-to-release.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,8 @@ Therefore, the release candidate is passed/rejected.

### Finishing the release

#### Apache release

After the release vote has passed, you need to release the last candidate's artifacts.

First, copy the source release directory to releases:
Expand All @@ -235,13 +237,40 @@ svn add apache-iceberg-<VERSION>
svn ci -m 'Iceberg: Add release <VERSION>'
```

#### GitHub release

Next, add a release tag to the git repository based on the passing candidate tag:

```bash
git tag -am 'Release Apache Iceberg <VERSION>' apache-iceberg-<VERSION> apache-iceberg-<VERSION>-rcN
git push apache apache-iceberg-<VERSION>
```

Then create a new GitHub release in https://github.com/apache/iceberg/releases from the release version tag.
Copy link
Member

@szehon-ho szehon-ho Jul 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we can be a bit more consistent with release name, how about specify to call it Apache Iceberg <VERSION>


Next, clean up each RC tag with:

```bash
git push --delete apache apache-iceberg-<VERSION>-rcN
```

For each major or minor version release,
publish the release branch with a `.x` in the end.
For example, for 1.2.0 release we do:

```bash
git checkout -b 1.2.x apache-iceberg-1.2.0
git push --set-upstream apache 1.2.x
```

Then release the candidate repository in [Nexus](https://repository.apache.org/#stagingRepositories).
#### Maven release

1. Go to [Nexus](https://repository.apache.org/) and log in
2. In the menu on the left, choose "Staging Repositories"
3. Select the Iceberg repository that was previously closed and passed vote
4. At the top, select "Release" and follow the instructions

#### Announcement email

To announce the release, wait until Maven central has mirrored the Apache binaries, then update the Iceberg site and send an announcement email:

Expand All @@ -263,64 +292,102 @@ Java artifacts are available from Maven Central.
Thanks to everyone for contributing!
```

### Documentation Release
#### Codebase updates

There are a few places in the codebase that reference the latest Iceberg release version number and need to be updated:
1. https://github.com/apache/iceberg/blob/master/.github/ISSUE_TEMPLATE/iceberg_bug_report.yml
2. https://github.com/apache/iceberg/blob/master/python/dev/Dockerfile
3. `oldVersion` in `revapi` in https://github.com/apache/iceberg/blob/master/build.gradle
Copy link
Member

@RussellSpitzer RussellSpitzer Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to modify all these values in the release commit?

Copy link
Author

@jackye1995 jackye1995 Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by release commit?

I think this can only be changed after the artifact is released and fully published, so there is a period of time that we have released but still cannot consume the artifact.

Copy link
Member

@RussellSpitzer RussellSpitzer Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I'm just confused about what these instructions are telling me to do, like after release am I making a new commit that updates these things?

Copy link
Contributor

@nastra nastra Mar 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usually we did those updates after a release (at least the revApi oldVersion bump you can only do after artifacts are available). The same applies for the docker file and the docker compose that have been upgraded in apache/iceberg#7164.

For the docker compose to actually work with the latest Iceberg version, I was updating databricks/docker-spark-iceberg#72 and databricks/iceberg-rest-image#7


### Documentation release

Documentation needs to be updated as a part of an Iceberg release after a release candidate is passed.
The commands described below assume you are in a directory containing a local clone of the `iceberg-docs`
repository and `iceberg` repository. Adjust the commands accordingly if it is not the case. Note that all
changes in `iceberg` need to happen against the `master` branch and changes in `iceberg-docs` need to happen
against the `main` branch.

#### iceberg repository preparations
#### Prerequisites

Similar to the `iceberg` repository, you also need to set up `https://github.com/apache/iceberg-docs.git` as a remote with name `apache`.

A PR needs to be published in the `iceberg` repository with the following changes:
The commands described below assume you have a directory structure of:

1. Create a new folder called `docs/releases/<VERSION NUMBER>` with an `_index.md` file. See the existing folders under `docs/releases` for more details.
```text
/some/path
├── iceberg
└── iceberg-docs
```

And you are:
1. in the `iceberg-docs` repository.
2. have `iceberg` repository on the specific version branch

#### Common documentation update
Adjust the commands below accordingly if it is not the case.

#### Update specs

Copy the latest format specifications to `landing-page/content/common`:

1. To start the release process, run the following steps in the `iceberg-docs` repository to copy docs over:
```shell
cp -r ../iceberg/format/* ../iceberg-docs/landing-page/content/common/
```
2. Change into the `iceberg-docs` repository and create a branch.
```shell
cd ../iceberg-docs
git checkout -b <BRANCH NAME>
```
3. Commit, push, and open a PR against the `iceberg-docs` repo (`<BRANCH NAME>` -> `main`)

#### Versioned documentation update
Raise a PR with the specific changes against `main` branch and merge.
Copy link
Member

@szehon-ho szehon-ho Jul 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this differ for patch release vs non-patch release?


Once the common docs changes have been merged into `main`, the next step is to update the versioned docs.
#### Copy versioned documentations
Copy link
Author

@jackye1995 jackye1995 Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also build a github action in Iceberg repo to sync latest docs and javadoc to here right after a new change. Not sure if there is any way for the github action to commit to this repo or create a PR though.

Copy link
Member

@RussellSpitzer RussellSpitzer Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently yes, https://github.com/marketplace/actions/create-pull-request

Copy link
Author

@jackye1995 jackye1995 Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me ping Infra team to see if this action can be used, and if it has required permissions


Copy the versioned docs into `docs/content`

1. In the `iceberg-docs` repository, cut a new branch using the version number as the branch name
```shell
cd ../iceberg-docs
git checkout -b <VERSION>
git push --set-upstream apache <VERSION>
```
2. Copy the versioned docs from the `iceberg` repo into the `iceberg-docs` repo
```shell
rm -rf ../iceberg-docs/docs/content
cp -r ../iceberg/docs ../iceberg-docs/docs/content
```
3. Commit the changes and open a PR against the `<VERSION>` branch in the `iceberg-docs` repo
Copy link
Author

@jackye1995 jackye1995 Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the new workflow described here, we do everything against main, and in the end cut the version branch and sync latest branch, this avoids unnecessary commit conflicts and need to do branch merges.


#### Javadoc update
Raise a PR with the specific changes against `main` branch and merge.

#### Copy versioned Javadoc

In the `iceberg` repository, generate the javadoc for your release and copy it to the `javadoc` folder:

In the `iceberg` repository, generate the javadoc for your release and copy it to the `javadoc` folder in `iceberg-docs` repo:
```shell
cd ../iceberg
echo "<VERSION>" > version.txt
Copy link
Author

@jackye1995 jackye1995 Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes sure the version number is correct for the javadoc folder

./gradlew refreshJavadoc
rm -rf ../iceberg-docs/javadoc
cp -r site/docs/javadoc/<VERSION NUMBER> ../iceberg-docs/javadoc
cp -r site/docs/javadoc/<VERSION> ../iceberg-docs/javadoc
```

This resulted changes in `iceberg-docs` should be approved in a separate PR.
Raise a PR with the specific changes against `main` branch and merge.

#### Set latest versions

The following fields need to be updated:
1. in `landing-page/config.toml`:
- update `latestVersions.iceberg`
- add one new row in `versions` for the latest version
2. in `docs/config.toml`:
- update `latestVersions.iceberg`
- update `versions.nessie` (check to the version of `org.projectnessie.nessie:*` from [versions.props](https://github.com/apache/iceberg/blob/master/versions.props))
- add one new row in `versions` for the latest version

Raise a PR with the specific changes against `main` branch and merge.

### update release notes

#### Update the latest branch
In page `landing-page/content/common/release-notes.md`:
1. Mark the current latest release notes to past releases
2. Add release notes for the new release version

Raise a PR with the specific changes against `main` branch and merge.

#### Create version branch

Create a branch with the specific version number:

```shell
git checkout -b <VERSION>
git push --set-upstream apache <VERSION>
```

#### Update the `latest` branch

Since `main` is currently the same as the version branch, one needs to rebase `latest` branch against `main`:

```shell
Expand All @@ -329,17 +396,6 @@ git rebase main
git push apache latest
```

#### Set latest version in iceberg-docs repo

The last step is to update the `main` branch in `iceberg-docs` to set the latest version.
A PR needs to be published in the `iceberg-docs` repository with the following changes:
1. Update variable `latestVersions.iceberg` to the new release version in `landing-page/config.toml`
2. Update variable `latestVersions.iceberg` to the new release version and
`versions.nessie` to the version of `org.projectnessie.nessie:*` from [versions.props](https://github.com/apache/iceberg/blob/master/versions.props) in `docs/config.toml`
3. Mark the current latest release notes to past releases under `landing-page/content/common/release-notes.md`
4. Add release notes for the new release version in `landing-page/content/common/release-notes.md`


# How to Verify a Release

Each Apache Iceberg release is validated by the community by holding a vote. A community release manager
Expand Down