Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 57 additions & 32 deletions BUILD_INFO.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ BUILD INFO

To build this project, three profiles must be explicitly deactivated.
These are associated with the standard build pom, but we need to make
sure that they aren't executed here.
sure that they aren't executed here.

Use the following command options:

Expand All @@ -14,8 +14,8 @@ $ mvn -P\!cfg-module-dist -P\!cfg-module-rpm -P\!module-test <phase>
```

Note that the three main phases that really make sense for the build tools are
`package`, `integration-test` and `install`. In particular the phase `test` is
generally expected to fail in `build-profile` if you have not run `install` before
`package`, `integration-test` and `install`. In particular the phase `test` is
generally expected to fail in `build-profile` if you have not run `install` before
as there is a chicken-and-egg issue: `build-profile` component requires the other
components built as part of this command. Use `integration-test` rather than `test`.

Expand Down Expand Up @@ -45,8 +45,8 @@ The version of the build tools used by other Quattor components is defined in th
pom.xml of the component (for Quattor configuration modules, in the pom.xml of
each configuration module), as part of the '<parent>' information.

To update the version used for a given component to the last release available,
check out its repository and in the top-level directory, execute the following
To update the version used for a given component to the last release available,
check out its repository and in the top-level directory, execute the following
command:

```bash
Expand Down Expand Up @@ -92,7 +92,7 @@ starting point is http://maven.apache.org/guides/getting-started/maven-in-five-m
Updating the pom files
----------------------

Pom files (`pom.xml`) are used to drive the building process. They are XML files. For some advanced
Pom files (`pom.xml`) are used to drive the building process. They are XML files. For some advanced
operations it is possible to edit them but for most of the operations, there are Maven plugins available
to do it. **It is strongly recommended** to use Maven when possible rather than editing the pom files.

Expand All @@ -111,12 +111,12 @@ warning if this is not the case. When this is not done in the parent pom, if any
Updating the Maven plugins used
-------------------------------

To build the build tools and the various Quattor components, we rely on many Maven plugins whose version is fixed in
To build the build tools and the various Quattor components, we rely on many Maven plugins whose version is fixed in
the main pom.xml file (the one in the top directory of the build tools). This is to ensure than when an old release is
rebuilt, this is done with the same version of the Maven plugins as the one originally used.

It remains important to update regurlarly the version used for every plugin. To help identifying the most recent of
a plugin compatible with the Maven version used, use the following command (for the build tools, use the same
a plugin compatible with the Maven version used, use the following command (for the build tools, use the same
Maven options as those used for building them, see above):

```
Expand All @@ -128,16 +128,16 @@ there is no reason to define the plugin version in the pom.xml file of a particu
it is always good to check if you suspect something is not working as expected.

Normally, this is enough to run this command for the build tools as all plugins should be declared here. But this is
possible to run it for any Quattor component: this may help to diagnose problems with specific components explicitly
possible to run it for any Quattor component: this may help to diagnose problems with specific components explicitly
defining the version of a plugin.

Producing a new release of the build tools
------------------------------------------

A release of a component (*artifact* in Maven terminology) managed by Maven is done with the Maven
A release of a component (*artifact* in Maven terminology) managed by Maven is done with the Maven
[release](http://maven.apache.org/maven-release/maven-release-plugin/) plugin. The main goals are:

* `release:prepare`: build everything to be released, update/tag the source repositories, update the pom files.
* `release:prepare`: build everything to be released, update/tag the source repositories, update the pom files.
It produces also a few files that allow to revert the release process.
* `release:perform`: publish the new release

Expand All @@ -148,23 +148,23 @@ In case of an error during of the goals above, they can be run again as many tim
until completion. At each run, Maven will guess what has already been done and do only the other
part.

*Note:* deploying a release requires to have GPG keys: create them if necessary before producing a release
and publish your public key on one GPG key server (they are all synchronised, see
http://central.sonatype.org/pages/working-with-pgp-signatures.html#distributing-your-public-key) and wait for
*Note:* deploying a release requires to have GPG keys: create them if necessary before producing a release
and publish your public key on one GPG key server (they are all synchronised, see
https://central.sonatype.org/publish/requirements/gpg/#distributing-your-public-key) and wait for
your key to be present on http://pgp.mit.edu:11371. Also, as for any `gpg` command,
you need to have a running `gpg-agent`. It also requires that you have the appropriate Maven configuration
you need to have a running `gpg-agent`. It also requires that you have the appropriate Maven configuration
in `~/.m2/settings.xml` (see below).

It is not possible to use `--batch-mode` at the moment because you need to enter your GPG key password during the release process.

If the userid used to push the package is incorrect, you will probably need to use the `-Dusername=XXX` property.

*Before deploying a release, in particular if you are not familiar with the process,
it is a good practice to deploy a snapshot. This is done with the `deploy` phase (don't forget to use the usual options
it is a good practice to deploy a snapshot. This is done with the `deploy` phase (don't forget to use the usual options
as specified at the beginning of this document).
To deploy a snapshot rather than a release, check that the artifact version in the pom file ends with -SNAPSHOT,
To deploy a snapshot rather than a release, check that the artifact version in the pom file ends with -SNAPSHOT,
before running the deploy phase. This allows the
basic configuration to be checked for correctness (in particular the parameters used to upload to Sonatype Nexus).
basic configuration to be checked for correctness (in particular the parameters used to upload to the central repository).

### Build release and push to nexus

Expand All @@ -176,7 +176,7 @@ an up-to-date version of `template-library-core` repository (required by tests).

For example:
1. `git clone https://github.com/quattor/template-library-core.git /tmp/template-library-core-master`
2. `export QUATTOR_TEST_TEMPLATE_LIBRARY_CORE=/tmp/template-library-core-master`
2. `export QUATTOR_TEST_TEMPLATE_LIBRARY_CORE=/tmp/template-library-core-master`

1. Clean the build area:

Expand All @@ -192,24 +192,49 @@ an up-to-date version of `template-library-core` repository (required by tests).

### Promoting from staging to release

1. After successfully executing `release:perform`, the new release will be in a staging area on Sonatype nexus server
(https://oss.sonatype.org). Before the release can be used, you must log in to the nexus server, close the staging
area and ask for the staging area being released.
1. After successfully executing `release:perform`, the new release will be held in the Portal OSSRH Staging API compatibility service.
Before the release can be used, you must make an API call to transfer the deployment from the compatibility service to the main Central Publisher Portal area.

1. Call `./ossrh-staging-api search` to verify that the new release exists, you should see something similar to the following:
```json
{
"repositories": [
{
"key": "ABCdef/198.51.100.119/org.quattor--default-repository",
"state": "open",
"description": null,
"portal_deployment_id": null
}
]
}

1. Select `Staging Repositories` in the left side menu (appearing
after you logged in) and search for [org.quattor.maven](https://oss.sonatype.org/#nexus-search;quick~org.quattor.maven).
```

1. Call `./ossrh-staging-api upload` to request transfer of all open repositories to the Central Publisher Portal.

1. Call `./ossrh-staging-api search` again to verify that the new release has been closed:
```json
{
"repositories": [
{
"key": "ABCdef/198.51.100.119/org.quattor--default-repository",
"state": "closed",
"description": null,
"portal_deployment_id": "6d9f74b9-ac49-487d-9bc3-7b8e448eb4fa"
}
]
}
```

1. Select the appropriate repository (you may have several if you did several attempts from different machines) and close it.
1. If successful go to https://central.sonatype.com/publishing/deployments and wait for the release to appear and pass validation. Once it does, press the "Publish" button to release it to Maven Central.

1. When the operation is successful (click on `Refresh`), click on the `Release` button. Once successfully released, you may have to wait a couple of hours before the release appears on the [Maven Central Repository](http://search.maven.org/#search%7Cga%7C1%7Corg.quattor.maven).

### Additional notes

* If something went wrong during `release:perform` (which pushes the release to the staging area), it is sometimes necessary to drop the staging directory after closing it to be able to run `release:perform` again (particularly if it complains that the packages are already present in the repository).
* A good documentation of the staging process can be found at http://books.sonatype.com/nexus-book/reference/staging-repositories.html
* Detailed information on using Maven to deploy releases at Sonatype is available at http://central.sonatype.org/pages/apache-maven.html.
* Documentation of the staging process can be found at https://central.sonatype.org/publish/publish-portal-ossrh-staging-api/

* If when the release becomes available on http://search.maven.org Maven complains that it cannot find the new release, add the `-U` option to the Maven command line. The execution of Maven may still result in an error but the local repository should be updated allowing the next execution of Maven to work.
* If when the release becomes available on https://central.sonatype.com/ Maven complains that it cannot find the new release, add the `-U` option to the Maven command line. The execution of Maven may still result in an error but the local repository should be updated allowing the next execution of Maven to work.

* If you really want to cancel the release process after doing `release:prepare` but before doing `release:perform` (it is preferable to fix the problem and rerun `release:prepare`), you need to issue the appropriate commands depending on whether `release:prepare` added some commits to the upstream branch whose commit message starts with `[maven-release-plugin]`:

Expand All @@ -231,7 +256,7 @@ after you logged in) and search for [org.quattor.maven](https://oss.sonatype.org
Connect to GitHub and delete the new tag if it has been created
```

* **Note: after a first execution of `release:perform` which pushed things to Sonatype Central Repository, it is not recommended that you attempt to revert a failed release. If the problems with the release in progress cannot be fixed, simply start a new release.**
* **Note: after a first execution of `release:perform` which pushed things to the Central Repository, it is not recommended that you attempt to revert a failed release. If the problems with the release in progress cannot be fixed, simply start a new release.**

Recommended Maven configuration
-------------------------------
Expand Down Expand Up @@ -269,5 +294,5 @@ Below is a typical Maven configuration file (`~/.m2/settings.xml`) to be able to
</settings>
```

In the above example `your_userid` and the password refer to your account at http://sonatype.org.
To get an account at Sonatype, follow the instructions at http://central.sonatype.org/pages/ossrh-guide.html.
In the above example `your_userid` and the password refer to your account at https://identity.sonatype.com.
To get an account, follow the instructions at https://central.sonatype.org/register/central-portal/#create-an-account
45 changes: 45 additions & 0 deletions ossrh-staging-api
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/bin/bash
# vim: noai:et:ts=4:sw=4

SETTINGS="$HOME/.m2/settings.xml"
SERVPATH="/settings/servers/server[id='sonatype-nexus-staging']"

if [[ $# -ne 1 || ! "$1" =~ ^(search|upload)$ ]]; then
echo "Usage: ossrh-staging-api [search | upload]"
exit 2
fi

action="$1"

if [[ ! -s "$SETTINGS" ]]; then
echo "Couldn't find $SETTINGS, cannot continue."
exit 1
fi

for util in curl jq xmllint; do
if ! which $util > /dev/null 2>&1; then
echo "Required utility $util not found"
exit 1
fi
done

user="$(xmllint --xpath "string($SERVPATH/username)" "$SETTINGS")"
pass="$(xmllint --xpath "string($SERVPATH/password)" "$SETTINGS")"
auth="Bearer $(base64 <(printf "%s:%s" "$user" "$pass"))"

function api {
curl \
--silent \
--request "$1" \
--header @<( echo "Authorization: $auth") \
https://ossrh-staging-api.central.sonatype.com/$2
}

case "$action" in
"search")
api 'GET' 'manual/search/repositories?ip=any&profile_id=org.quattor' | jq .
;;
"upload")
api 'POST' 'manual/upload/defaultRepository/org.quattor' | jq .
;;
esac