Skip to content

Commit af04273

Browse files
authored
ci: automatically update GA libraries in top-level README (#8463)
Use the top-level `BUILD.bazel` as the source of truth for GA libraries. That requires installing `bazelisk` in the `checkers` build image. Once that is working, we can automatically update the section of the README describing which libraries are GA: just iterate over the list, extract the service name / description from each library README, and then print out the right links. Note that I discovered incorrect links in the process.
1 parent 058c61f commit af04273

File tree

4 files changed

+38
-314
lines changed

4 files changed

+38
-314
lines changed

README.md

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,14 +28,16 @@
2828
This repository contains idiomatic C++ client libraries for the following
2929
[Google Cloud Platform](https://cloud.google.com/) services.
3030

31+
<!-- inject-GA-libraries-start -->
3132
* [Google Cloud BigQuery](google/cloud/bigquery/README.md) [[quickstart]](google/cloud/bigquery/quickstart/README.md)
3233
* [Google Cloud Bigtable](google/cloud/bigtable/README.md) [[quickstart]](google/cloud/bigtable/quickstart/README.md)
3334
* [Google Cloud IAM](google/cloud/iam/README.md) [[quickstart]](google/cloud/iam/quickstart/README.md)
34-
* [Google Cloud Spanner](google/cloud/spanner/README.md) [[quickstart]](google/cloud/spanner/quickstart/README.md)
3535
* [Google Cloud Pub/Sub](google/cloud/pubsub/README.md) [[quickstart]](google/cloud/pubsub/quickstart/README.md)
36+
* [Secret Manager API](google/cloud/secretmanager/README.md) [[quickstart]](google/cloud/secretmanager/quickstart/README.md)
37+
* [Google Cloud Spanner](google/cloud/spanner/README.md) [[quickstart]](google/cloud/spanner/quickstart/README.md)
3638
* [Google Cloud Storage](google/cloud/storage/README.md) [[quickstart]](google/cloud/storage/quickstart/README.md)
37-
* [Secret Manager API](google/cloud/secretmanager/README.md) [[quickstart]](google/cloud/secretmanager/README.md)
38-
* [Cloud Tasks API](google/cloud/tasks/README.md) [[quickstart]](google/cloud/tasks/README.md)
39+
* [Cloud Tasks API](google/cloud/tasks/README.md) [[quickstart]](google/cloud/tasks/quickstart/README.md)
40+
<!-- inject-GA-libraries-end -->
3941

4042
See each library's `README.md` file for more information about:
4143

@@ -98,17 +100,11 @@ intended to help you get up and running in a matter of minutes. This
98100
how to use the library, along with minimal build files for common build
99101
systems, such as CMake and Bazel.
100102

101-
* [Google Cloud BigQuery Quickstart](google/cloud/bigquery/quickstart/README.md)
102-
* [Google Cloud Bigtable Quickstart](google/cloud/bigtable/quickstart/README.md)
103-
* [Google Cloud IAM Quickstart](google/cloud/iam/quickstart/README.md)
104-
* [Google Cloud Spanner Quickstart](google/cloud/spanner/quickstart/README.md)
105-
* [Google Cloud Pub/Sub Quickstart](google/cloud/pubsub/quickstart/README.md)
106-
* [Google Cloud Storage Quickstart](google/cloud/storage/quickstart/README.md)
107-
108103
As an example, the following code snippet, taken from [Google Cloud
109104
Storage](google/cloud/storage/README.md), should give you a taste of what it's
110105
like to use one of these C++ libraries.
111106

107+
<!-- inject-quickstart-start -->
112108
```cc
113109
#include "google/cloud/storage/client.h"
114110
#include <iostream>
@@ -151,6 +147,7 @@ int main(int argc, char* argv[]) {
151147
return 0;
152148
}
153149
```
150+
<!-- inject-quickstart-end -->
154151
155152
## Support
156153

ci/cloudbuild/builds/checkers.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ time {
182182
printf "%-30s" "Running markdown generators:" >&2
183183
time {
184184
declare -A -r GENERATOR_MAP=(
185-
["ci/generate-markdown/generate-readme.sh"]="README.md"
185+
["ci/generate-markdown/generate-readme.sh"]="/dev/null"
186186
["ci/generate-markdown/generate-packaging.sh"]="doc/packaging.md"
187187
)
188188
for generator in "${!GENERATOR_MAP[@]}"; do

ci/cloudbuild/dockerfiles/checkers.Dockerfile

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,7 @@ RUN pip3 install cmake_format==0.6.8
4141
RUN pip3 install black==19.3b0
4242

4343
RUN cargo install typos-cli --version 1.3.9 --root /usr/local
44+
45+
RUN curl -o /usr/bin/bazelisk -sSL "https://github.com/bazelbuild/bazelisk/releases/download/v1.11.0/bazelisk-linux-amd64" && \
46+
chmod +x /usr/bin/bazelisk && \
47+
ln -s /usr/bin/bazelisk /usr/bin/bazel

ci/generate-markdown/generate-readme.sh

Lines changed: 26 additions & 303 deletions
Original file line numberDiff line numberDiff line change
@@ -16,306 +16,29 @@
1616

1717
set -euo pipefail
1818

19-
BINDIR=$(dirname "$0")
20-
readonly BINDIR
21-
22-
# The initial portion of the preamble needs variable expansion:
23-
cat <<EOF
24-
# Google Cloud Platform C++ Client Libraries
25-
26-
<!-- This file is automatically generated by ci/test-markdown/$(basename "$0") -->
27-
28-
EOF
29-
30-
cat <<"EOF"
31-
[![GCB CI status][gcb-clang-tidy-shield]][gcb-clang-tidy-link]
32-
[![Kokoro CI status][kokoro-windows-cmake-shield]][kokoro-windows-cmake-link]
33-
[![Kokoro CI status][kokoro-macos-cmake-shield]][kokoro-macos-cmake-link]
34-
[![Codecov Coverage status][codecov-shield]][codecov-link]<br>
35-
[![GCB CI status][gcb-asan-shield]][gcb-asan-link]
36-
[![Kokoro CI status][kokoro-windows-bazel-shield]][kokoro-windows-bazel-link]
37-
[![Kokoro CI status][kokoro-macos-bazel-shield]][kokoro-macos-bazel-link]
38-
39-
[kokoro-windows-cmake-shield]: https://storage.googleapis.com/cloud-cpp-kokoro-status/kokoro-windows-cmake.svg
40-
[kokoro-windows-cmake-link]: https://storage.googleapis.com/cloud-cpp-kokoro-status/kokoro-windows-cmake-link.html
41-
[kokoro-windows-bazel-shield]: https://storage.googleapis.com/cloud-cpp-kokoro-status/kokoro-windows-bazel.svg
42-
[kokoro-windows-bazel-link]: https://storage.googleapis.com/cloud-cpp-kokoro-status/kokoro-windows-bazel-link.html
43-
[kokoro-macos-cmake-shield]: https://storage.googleapis.com/cloud-cpp-kokoro-status/macos/kokoro-cmake-vcpkg.svg
44-
[kokoro-macos-cmake-link]: https://storage.googleapis.com/cloud-cpp-kokoro-status/macos/kokoro-cmake-vcpkg-link.html
45-
[kokoro-macos-bazel-shield]: https://storage.googleapis.com/cloud-cpp-kokoro-status/macos/kokoro-bazel.svg
46-
[kokoro-macos-bazel-link]: https://storage.googleapis.com/cloud-cpp-kokoro-status/macos/kokoro-bazel-link.html
47-
[codecov-shield]: https://codecov.io/gh/googleapis/google-cloud-cpp/branch/main/graph/badge.svg
48-
[codecov-link]: https://codecov.io/gh/googleapis/google-cloud-cpp
49-
[gcb-clang-tidy-shield]: https://storage.googleapis.com/cloud-cpp-community-publiclogs/badges/google-cloud-cpp/main/clang-tidy.svg
50-
[gcb-clang-tidy-link]: https://storage.googleapis.com/cloud-cpp-community-publiclogs/badges/google-cloud-cpp/main/clang-tidy.html
51-
[gcb-asan-shield]: https://storage.googleapis.com/cloud-cpp-community-publiclogs/badges/google-cloud-cpp/main/asan.svg
52-
[gcb-asan-link]: https://storage.googleapis.com/cloud-cpp-community-publiclogs/badges/google-cloud-cpp/main/asan.html
53-
54-
This repository contains idiomatic C++ client libraries for the following
55-
[Google Cloud Platform](https://cloud.google.com/) services.
56-
57-
* [Google Cloud BigQuery](google/cloud/bigquery/README.md) [[quickstart]](google/cloud/bigquery/quickstart/README.md)
58-
* [Google Cloud Bigtable](google/cloud/bigtable/README.md) [[quickstart]](google/cloud/bigtable/quickstart/README.md)
59-
* [Google Cloud IAM](google/cloud/iam/README.md) [[quickstart]](google/cloud/iam/quickstart/README.md)
60-
* [Google Cloud Spanner](google/cloud/spanner/README.md) [[quickstart]](google/cloud/spanner/quickstart/README.md)
61-
* [Google Cloud Pub/Sub](google/cloud/pubsub/README.md) [[quickstart]](google/cloud/pubsub/quickstart/README.md)
62-
* [Google Cloud Storage](google/cloud/storage/README.md) [[quickstart]](google/cloud/storage/quickstart/README.md)
63-
* [Secret Manager API](google/cloud/secretmanager/README.md) [[quickstart]](google/cloud/secretmanager/README.md)
64-
* [Cloud Tasks API](google/cloud/tasks/README.md) [[quickstart]](google/cloud/tasks/README.md)
65-
66-
See each library's `README.md` file for more information about:
67-
68-
* Where to find the documentation for the library and the service.
69-
* How to get started using the library.
70-
* How to incorporate the library into your build system.
71-
* The library's support status if not Generally Available (GA); unless noted in
72-
a library's `README.md`, these libraries are all GA and supported by Google.
73-
74-
## Building and Installing
75-
76-
This is a quickstart guide for developers wanting to compile the libraries and
77-
run the examples included with the libraries.
78-
79-
* Packaging maintainers or developers who prefer to install the library in a
80-
fixed directory (such as `/usr/local` or `/opt`) should consult the
81-
[packaging guide](/doc/packaging.md).
82-
* Developers wanting to use the libraries as part of a larger CMake or Bazel
83-
project should consult the [quickstart guides](#quickstart) for the library
84-
or libraries they want to use.
85-
* Developers wanting to compile the library just to run some of the examples or
86-
tests should read the current document.
87-
* Contributors and developers to `google-cloud-cpp` should consult the guide to
88-
[setup a development workstation][howto-setup-dev-workstation].
89-
90-
[howto-setup-dev-workstation]: /doc/contributor/howto-guide-setup-development-workstation.md
91-
92-
### Building with Bazel
93-
94-
This library requires Bazel >= 4.0. From the top-level directory, run the usual
95-
commands.
96-
97-
```shell
98-
bazel build //...
99-
```
100-
101-
### Building with CMake
102-
103-
This library requires CMake >= 3.5. If you are planning to install the libraries
104-
please consult the [packaging guide](/doc/packaging.md), these instructions will
105-
**NOT** produce artifacts that you can put in `/usr/local`, or share with your
106-
colleagues.
107-
108-
From the top-level directory of `google-cloud-cpp` run these commands:
109-
110-
```shell
111-
git -C $HOME clone https://github.com/microsoft/vcpkg.git
112-
env VCPKG_ROOT=$HOME/vcpkg $HOME/vcpkg/bootstrap-vcpkg.sh
113-
cmake -H. -Bcmake-out/ -DCMAKE_TOOLCHAIN_FILE=$HOME/vcpkg/scripts/buildsystems/vcpkg.cmake
114-
cmake --build cmake-out -- -j $(nproc)
115-
```
116-
117-
The binary artifacts, such as examples, will be placed in `cmake-out/`.
118-
119-
## Quickstart
120-
121-
Each library (linked above) contains a directory named `quickstart/` that's
122-
intended to help you get up and running in a matter of minutes. This
123-
`quickstart/` directory contains a minimal "Hello World" program demonstrating
124-
how to use the library, along with minimal build files for common build
125-
systems, such as CMake and Bazel.
126-
127-
* [Google Cloud BigQuery Quickstart](google/cloud/bigquery/quickstart/README.md)
128-
* [Google Cloud Bigtable Quickstart](google/cloud/bigtable/quickstart/README.md)
129-
* [Google Cloud IAM Quickstart](google/cloud/iam/quickstart/README.md)
130-
* [Google Cloud Spanner Quickstart](google/cloud/spanner/quickstart/README.md)
131-
* [Google Cloud Pub/Sub Quickstart](google/cloud/pubsub/quickstart/README.md)
132-
* [Google Cloud Storage Quickstart](google/cloud/storage/quickstart/README.md)
133-
134-
As an example, the following code snippet, taken from [Google Cloud
135-
Storage](google/cloud/storage/README.md), should give you a taste of what it's
136-
like to use one of these C++ libraries.
137-
138-
```cc
139-
EOF
140-
141-
# Dumps the contents of GCS's quickstart.cc starting at the first #include, so
142-
# we skip the license header comment.
143-
sed -n '/^#/,$p' "${BINDIR}/../../google/cloud/storage/quickstart/quickstart.cc"
144-
145-
cat <<"EOF"
146-
```
147-
148-
## Support
149-
150-
* This project supports Windows, macOS, Linux
151-
* This project supports C++11 (and higher) compilers (we test with GCC >= 6.3,
152-
Clang >= 6.0, and MSVC >= 2017)
153-
* This project supports Bazel and CMake builds. See the [Quickstart examples](https://github.com/googleapis/google-cloud-cpp#quickstart)
154-
* This project uses dependencies described in [doc/packaging.md](https://github.com/googleapis/google-cloud-cpp/blob/main/doc/packaging.md)
155-
* This project works with or without exceptions enabled
156-
* This project cuts [monthly releases](https://github.com/googleapis/google-cloud-cpp/releases) with detailed release notes
157-
158-
## Public API and API Breaking Changes
159-
160-
In general, we avoid making backwards incompatible changes to our C++ APIs (see
161-
below for the definition of "API"). Sometimes such changes yield benefits to
162-
our customers, in the form of better performance, easier-to-understand APIs,
163-
and/or more consistent APIs across services. When these benefits warrant it, we
164-
will announce these changes prominently in our `CHANGELOG.md` file and in the
165-
affected release's notes. Nevertheless, though we take commercially reasonable
166-
efforts to prevent this, it is possible that backwards incompatible changes go
167-
undetected and, therefore, undocumented. We apologize if this is the case and
168-
welcome feedback or bug reports to rectify the problem.
169-
170-
By "API" we mean the C++ API exposed by public header files in this repo. We
171-
are not talking about the gRPC or REST APIs exposed by Google Cloud servers. We
172-
are also talking only about A**P**I stability -- the ABI is subject to change
173-
without notice. You should not assume that binary artifacts (e.g. static
174-
libraries, shared objects, dynamically loaded libraries, object files) created
175-
with one version of the library are usable with newer/older versions of the
176-
library. The ABI may, and does, change on "minor revisions", and even patch
177-
releases.
178-
179-
We request that our customers adhere to the following guidelines to avoid
180-
accidentally depending on parts of the library we do not consider to be part of
181-
the public API and therefore may change (including removal) without notice:
182-
183-
Previous versions of the library will remain available on the [GitHub Releases
184-
page](https://github.com/googleapis/google-cloud-cpp/releases). In many cases,
185-
you will be able to use an older version even if a newer version has changes
186-
that you are unable (or do not have time) to adopt.
187-
188-
Note that this document has no bearing on the Google Cloud Platform deprecation
189-
policy described at https://cloud.google.com/terms.
190-
191-
### C++ Symbols and Files
192-
193-
* You should only include headers matching the `google/cloud/${library}/*.h`,
194-
`google/cloud/${library}/mock/*.h` or `google/cloud/*.h` patterns.
195-
* You should **NOT** directly include headers in any subdirectories, such as
196-
`google/cloud/${library}/internal`.
197-
* The files *included from* our public headers are **not part of our public
198-
API**. Depending on indirect includes may break your build in the future, as
199-
we may change a header `"foo.h"` to stop including `"bar.h"` if `"foo.h"` no
200-
longer needs the symbols in `"bar.h"`. To avoid having your code broken, you
201-
should directly include the public headers that define all the symbols you
202-
use (this is sometimes known as
203-
[include-what-you-use](https://include-what-you-use.org/)).
204-
* Any file or symbol that lives within a directory or namespace containing
205-
`internal`, `impl`, `test`, `detail`, `benchmark`, `sample`, or `example`, is
206-
explicitly **not part of our public API**.
207-
* Any file or symbol with `Impl` or `impl` in its name is **not part of our
208-
public API**.
209-
* Any symbol with `experimental` in its name is not part of the public API.
210-
* You should avoid naming our inline namespaces (even avoid spelling the
211-
preprocessor names like `GOOGLE_CLOUD_CPP_NS`) and instead rely on them being
212-
a transparent versioning mechanism that you almost certainly don't care
213-
about. If you do spell out specific inline namespace names, your code will be
214-
tightly coupled with that specific version and will likely break when
215-
upgrading to a new version of our library.
216-
217-
## Beyond the C++ API
218-
219-
Applications developers interact with a C++ library through more than just
220-
the C++ symbols and headers. They also need to reference the name of the
221-
library in their build scripts. Depending of the build system they use
222-
this may be a CMake target, a Bazel rule, a pkg-config module, or just the
223-
name of some object in the file system.
224-
225-
As with the C++ API, we try to avoid breaking changes to these interface
226-
points. Sometimes such changes yield benefits to our customers, in the form of
227-
easier-to-understand what names go with with services, or more consistency
228-
across services. When these benefits warrant it, we will announce these changes
229-
prominently in our `CHANGELOG.md` file and in the affected release's notes.
230-
Nevertheless, though we take commercially reasonable efforts to prevent this,
231-
it is possible that backwards incompatible changes go undetected and,
232-
therefore, undocumented. We apologize if this is the case and welcome feedback
233-
or bug reports to rectify the problem.
234-
235-
### Experimental Libraries
236-
237-
From time to time we add libraries to `google-cloud-cpp` to validate new
238-
designs, expose experimental (or otherwise not generally available) GCP
239-
features, or simply because a library is not yet complete. Such libraries
240-
will have `experimental` in their CMake target and Bazel rule names. The
241-
README file for these libraries will also document that they are experimental.
242-
Such libraries are subject to change, including removal, without notice.
243-
This includes, but it is not limited to, all their symbols, pre-processor
244-
macros, files, targets, rules, and installed artifacts.
245-
246-
### Bazel rules
247-
248-
Only the rules exported at the top-level directory are intended for customer
249-
use, e.g.,`//:spanner`. Experimental rules have `experimental` in their name,
250-
e.g. `//:experimental-logging`. As previously stated, experimental rules are
251-
subject to change or removal without notice.
252-
253-
### CMake targets and packages
254-
255-
Only CMake packages starting with the `google_cloud_cpp_` prefix are intended
256-
for customer use. Only targets starting with `google-cloud-cpp::`, are intended
257-
for customer use. Experimental targets have `experimental` in their name (e.g.
258-
`google-cloud-cpp::experimental-logging`). As previously stated, experimental
259-
targets are subject to change or removal without notice.
260-
261-
### pkg-config modules
262-
263-
Only modules starting with `google_cloud_cpp_` are intended for customer use.
264-
265-
### Unsupported use cases
266-
267-
We try to provide stable names for the previously described mechanisms:
268-
269-
* Bazel rules,
270-
* CMake targets loaded via `find_package()`,
271-
* pkg-config modules
272-
273-
It is certainly possible to use the library using other approaches. While
274-
these may work, we may accidentally break these from time to time. Examples of
275-
such, and the recommended alternatives, include:
276-
277-
* CMake's `FetchContent` and/or git submodules: in these approaches the
278-
`google-cloud-cpp` library becomes a sub-directory of a larger CMake build
279-
We do not test `google-cloud-cpp` in these configurations, and we find them
280-
brittle as **all** CMake targets become visible to the larger project.
281-
This is both prone to conflicts, and makes it impossible to enforce that
282-
some targets are only for testing or are implementation details.
283-
Applications may want to consider source package managers, such as
284-
`vcpkg`, or CMake super builds via `ExternalProject_Add()` as alternatives.
285-
286-
* Using library names directly: applications should not use the
287-
library names, e.g., by using `-lgoogle_cloud_cpp_bigtable`
288-
in build scripts. We may need to split or merge libraries over time,
289-
making such names unstable. Applications should use CMake targets,
290-
e.g., `google-cloud-cpp::bigtable`, or pkg-config modules, e.g.,
291-
`$(pkg-config google_cloud_cpp_bigtable --libs)` instead.
292-
293-
### Documentation and Comments
294-
295-
The documentation (and its links) is intended for human consumption and not
296-
third party websites, or automation (such as scripts scrapping the contents).
297-
The contents and links of our documentation may change without notice.
298-
299-
### Other Interface Points
300-
301-
We think this covers all interface points, if we missed something please
302-
file a [GitHub issue][github-issue].
303-
304-
## Contact us
305-
306-
* [GitHub Discussions] -- For questions and general comments
307-
* [GitHub Issues] -- For reporting bugs and feature requests
308-
309-
[GitHub Discussions]: https://github.com/googleapis/google-cloud-cpp/discussions
310-
[GitHub Issues]: https://github.com/googleapis/google-cloud-cpp/issues
311-
312-
## Contributing changes
313-
314-
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for details on how to contribute to
315-
this project, including how to build and test your changes as well as how to
316-
properly format your code.
317-
318-
## Licensing
319-
320-
Apache 2.0; see [`LICENSE`](LICENSE) for details.
321-
EOF
19+
(
20+
sed '/<!-- inject-quickstart-start -->/q' "README.md"
21+
echo '```cc'
22+
# Dumps the contents of quickstart.cc starting at the first #include, so we
23+
# skip the license header comment.
24+
sed -n -e '/END .*quickstart/,$d' -e '/^#/,$p' "google/cloud/storage/quickstart/quickstart.cc"
25+
echo '```'
26+
sed -n '/<!-- inject-quickstart-end -->/,$p' "README.md"
27+
) >"README.md.tmp"
28+
29+
(
30+
mapfile -t libraries < <(bazelisk --batch query \
31+
--noshow_progress --noshow_loading_progress \
32+
'kind(cc_library, //:all) except filter("experimental|mocks", kind(cc_library, //:all))' |
33+
sed -e 's;//:;;' |
34+
LC_ALL=C sort)
35+
sed '/<!-- inject-GA-libraries-start -->/q' "README.md.tmp"
36+
for library in "${libraries[@]}"; do
37+
description="$(sed -n '1 s/# \(.*\) C++ Client Library/\1/p' "google/cloud/${library}/README.md")"
38+
printf '* [%s](google/cloud/%s/README.md) [[quickstart]](google/cloud/%s/quickstart/README.md)\n' \
39+
"${description}" "${library}" "${library}"
40+
done
41+
sed -n '/<!-- inject-GA-libraries-end -->/,$p' "README.md.tmp"
42+
) >"README.md"
43+
44+
rm README.md.tmp

0 commit comments

Comments
 (0)