Skip to content

Commit e51c8be

Browse files
committed
chore: add readme for google-cloud-bigquery-jdbc
1 parent 2af2b2b commit e51c8be

File tree

1 file changed

+324
-0
lines changed

1 file changed

+324
-0
lines changed
Lines changed: 324 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,324 @@
1+
# Google BigQuery JDBC Client for Java
2+
3+
Java idiomatic client for [BigQuery JDBC][product-docs].
4+
5+
[![Maven][maven-version-image]][maven-version-link]
6+
![Stability][stability-image]
7+
8+
- [Product Documentation][product-docs]
9+
- [Client Library Documentation][javadocs]
10+
11+
12+
## Quickstart
13+
14+
15+
If you are using Maven, add this to your pom.xml file:
16+
17+
<!-- {x-version-update-start:google-cloud-bigquery-jdbc:released} -->
18+
19+
```xml
20+
<dependency>
21+
<groupId>com.google.cloud</groupId>
22+
<artifactId>google-cloud-bigquery-jdbc</artifactId>
23+
<version>0.1.0</version>
24+
</dependency>
25+
```
26+
27+
If you are using Gradle without BOM, add this to your dependencies:
28+
29+
```Groovy
30+
implementation 'com.google.cloud:google-cloud-bigquery-jdbc:0.1.0'
31+
```
32+
33+
If you are using SBT, add this to your dependencies:
34+
35+
```Scala
36+
libraryDependencies += "com.google.cloud" % "google-cloud-bigquery-jdbc" % "0.1.0"
37+
```
38+
<!-- {x-version-update-end} -->
39+
40+
## Authentication
41+
42+
See the [Authentication][authentication] section in the base directory's README.
43+
44+
## Authorization
45+
46+
The client application making API calls must be granted [authorization scopes][auth-scopes] required for the desired BigQuery JDBC APIs, and the authenticated principal must have the [IAM role(s)][predefined-iam-roles] required to access GCP resources using the BigQuery JDBC API calls.
47+
48+
## Developer Guide
49+
50+
### Prerequisites
51+
52+
You need to have either Java with Maven installed or Docker. You might want to install [`Make`](https://www.gnu.org/software/make/) to simplify running commands, otherwise please look into Makefile to check for specific configurations.
53+
54+
### Setup
55+
56+
`make install` primarily relies on `mvn install` command. All following commands are primarily applicable for the `google-cloud-bigquery-jdbc` project.
57+
58+
`bigquery-external-jdbc-tests` at this moment is indepdnent and has its own configuration.
59+
60+
### Running tests
61+
62+
#### Unittests
63+
64+
Run all unittests
65+
66+
`make unittest`
67+
68+
Run specific unittests
69+
70+
`make unittest test=<filter>`
71+
72+
Please reference [Maven documentation](https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html) for details about `<filter>`
73+
74+
Example: `make unittest test=BigQueryArrowStructTest`
75+
76+
#### Integration tests
77+
78+
IMPORTANT: Running integration tests will skip unit tests. Primary focus of this command is to run specific set of tests without a lot of overhead.
79+
80+
Majority of Integration tests rely on certain environment variables:
81+
82+
```
83+
# Default gcloud auth setup
84+
export GOOGLE_APPLICATION_CREDENTIALS=<path-to-service-account-json-config-file>
85+
86+
# Test specfic envs
87+
export SA_EMAIL=email@email.com
88+
export SA_SECRET=<path-to-service-account-json-config-file>
89+
# Alternatively it can be json content:
90+
export SA_SECRET=`cat <path-to-service-account-json-config-file>`
91+
```
92+
93+
Run all integration tests (currently takes 15+ minutes, so this is discouraged).
94+
95+
`make integration-test`
96+
97+
Run specific unitintegration test
98+
99+
`make integration-test test=<filter>`
100+
101+
Please reference [Maven documentation](https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html) for details about `<filter>`
102+
103+
Example: `make unittest test=ITBigQueryJDBCTest#testValidServiceAccountAuthenticationOAuthPvtKey`
104+
105+
### Dockerized environment
106+
107+
If you don't have Java or Maven, or if you want to test changes with a different Java version, you can leverage dockerized environment.
108+
109+
One-time run commands are similar to local development make commands:
110+
111+
```
112+
make docker-build
113+
make docker-unittest
114+
make docker-integration-test
115+
```
116+
117+
Please note that running unit of integration tests within docker doesn't leverage maven cache because it is not persisted.
118+
If you want to run multiple commands, you can start a session and treat this shell session as your local environment.
119+
120+
```
121+
make docker-session
122+
```
123+
124+
All Docker commands rely only on `GOOGLE_APPLICATION_CREDENTIALS` env being present, it will create rest of env vars as needed.
125+
126+
### Packaging
127+
128+
There are two ways to package Google JDBC Driver: thin jar and jar with all dependencies.
129+
130+
#### Thin jar
131+
132+
Thin jar can be generated by running `make docker-package`. It generates jar and list of dependencies in format compatible with copying into `pom.xml` file.
133+
134+
#### All dependencies
135+
136+
Jar with all dependencies can be generated by running `make docker-package-all-dependencies`. Produced jar includes all depenencies and can be used as a standalon jar with tools like R-Studio.
137+
138+
### Custom query tests
139+
140+
Thin client using JDBC driver can be found in 'bigquery-jdbc-test-client' folder. It allows you to run custom queries with custom connection string ot generate logs or measure performance.
141+
142+
It can be built for Google JDBC drivers or any other JDBC drivers. For other JDBC drivers, you need to specify a version and it will download it.
143+
144+
#### Google test
145+
146+
For Google, you need to build an uber jar following previous steps. When it is ready, you can use these
147+
sample commands to run it:
148+
149+
```
150+
cd bigquery-jdbc-test-client
151+
make docker-package-for-validation JDBC_JAR_PATH=<path-to-uber-jar>
152+
make run QUERY=small
153+
```
154+
155+
### Night builds
156+
157+
Nightly build is running full set of Integration tests and extended Nightly tests that includes some long-running tests (takes 20+ minutes to complete).
158+
159+
Nightly Integration Tests include a step to build a full and thin jars and upload them to [Google Storage](https://pantheon.corp.google.com/storage/browser/bq_devtools_release_private/drivers/jdbc;tab=objects?inv=1&invt=Ab4E6g&project=bigquery-devtools-drivers&prefix=&forceOnObjectsSortingFiltering=false).
160+
161+
They can be retrieved via following commands:
162+
163+
```
164+
gsutil cp gs://bq_devtools_release_private/drivers/jdbc/google-cloud-bigquery-jdbc-latest-full.jar .
165+
gsutil cp gs://bq_devtools_release_private/drivers/jdbc/google-cloud-bigquery-jdbc-latest.zip .
166+
```
167+
168+
#### Performance tests
169+
170+
[Cloud Build Pipeline](https://pantheon.corp.google.com/cloud-build/triggers;region=us-east1/edit/125214da-8603-449d-8336-b6dba7196bb4?project=bigquery-devtools-drivers)
171+
is uploading latest full jar to the internal location for perf tests once a week, at the moment scheduled to do this Sunday night.
172+
173+
### Code Coverage
174+
175+
We're using [JaCoCo](https://www.eclemma.org/jacoco/) to track Code Coverage. `Makefile` has 2 separate set of commands for unittests and integration tests reports.
176+
177+
You can run `make docker-coverage` and it will produce both results, you can find `jacoco-(unittest|full).zip` file in the root with results.
178+
179+
## Getting Started
180+
181+
### Prerequisites
182+
183+
You will need a [Google Cloud Platform Console][developer-console] project with the BigQuery JDBC [API enabled][enable-api].
184+
185+
[Follow these instructions][create-project] to get your project set up. You will also need to set up the local development environment by
186+
[installing the Google Cloud Command Line Interface][cloud-cli] and running the following commands in command line:
187+
`gcloud auth login` and `gcloud config set project [YOUR PROJECT ID]`.
188+
189+
### Installation and setup
190+
191+
You'll need to obtain the `google-cloud-bigquery-jdbc` library. See the [Quickstart](#quickstart) section
192+
to add `google-cloud-bigquery-jdbc` as a dependency in your code.
193+
194+
## About BigQuery JDBC
195+
196+
197+
[BigQuery JDBC][product-docs]
198+
199+
See the [BigQuery JDBC client library docs][javadocs] to learn how to
200+
use this BigQuery JDBC Client Library.
201+
202+
203+
204+
205+
206+
207+
## Troubleshooting
208+
209+
To get help, follow the instructions in the [shared Troubleshooting document][troubleshooting].
210+
211+
## Supported Java Versions
212+
213+
Java 8 or above is required for using this client.
214+
215+
Google's Java client libraries,
216+
[Google Cloud Client Libraries][cloudlibs]
217+
and
218+
[Google Cloud API Libraries][apilibs],
219+
follow the
220+
[Oracle Java SE support roadmap][oracle]
221+
(see the Oracle Java SE Product Releases section).
222+
223+
### For new development
224+
225+
In general, new feature development occurs with support for the lowest Java
226+
LTS version covered by Oracle's Premier Support (which typically lasts 5 years
227+
from initial General Availability). If the minimum required JVM for a given
228+
library is changed, it is accompanied by a [semver][semver] major release.
229+
230+
Java 11 and (in September 2021) Java 17 are the best choices for new
231+
development.
232+
233+
### Keeping production systems current
234+
235+
Google tests its client libraries with all current LTS versions covered by
236+
Oracle's Extended Support (which typically lasts 8 years from initial
237+
General Availability).
238+
239+
#### Legacy support
240+
241+
Google's client libraries support legacy versions of Java runtimes with long
242+
term stable libraries that don't receive feature updates on a best efforts basis
243+
as it may not be possible to backport all patches.
244+
245+
Google provides updates on a best efforts basis to apps that continue to use
246+
Java 7, though apps might need to upgrade to current versions of the library
247+
that supports their JVM.
248+
249+
#### Where to find specific information
250+
251+
The latest versions and the supported Java versions are identified on
252+
the individual GitHub repository `github.com/GoogleAPIs/java-SERVICENAME`
253+
and on [google-cloud-java][g-c-j].
254+
255+
## Versioning
256+
257+
258+
This library follows [Semantic Versioning](http://semver.org/).
259+
260+
261+
262+
## Contributing
263+
264+
265+
Contributions to this library are always welcome and highly encouraged.
266+
267+
See [CONTRIBUTING][contributing] for more information how to get started.
268+
269+
Please note that this project is released with a Contributor Code of Conduct. By participating in
270+
this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more
271+
information.
272+
273+
274+
## License
275+
276+
Apache 2.0 - See [LICENSE][license] for more information.
277+
278+
## CI Status
279+
280+
Java Version | Status
281+
------------ | ------
282+
Java 8 | [![Kokoro CI][kokoro-badge-image-2]][kokoro-badge-link-2]
283+
Java 8 OSX | [![Kokoro CI][kokoro-badge-image-3]][kokoro-badge-link-3]
284+
Java 8 Windows | [![Kokoro CI][kokoro-badge-image-4]][kokoro-badge-link-4]
285+
Java 11 | [![Kokoro CI][kokoro-badge-image-5]][kokoro-badge-link-5]
286+
287+
Java is a registered trademark of Oracle and/or its affiliates.
288+
289+
[product-docs]: https://cloud.google.com/bigquery
290+
[javadocs]: https://cloud.google.com/java/docs/reference/google-cloud-bigquery/latest/history
291+
[kokoro-badge-image-1]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java7.svg
292+
[kokoro-badge-link-1]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java7.html
293+
[kokoro-badge-image-2]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8.svg
294+
[kokoro-badge-link-2]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8.html
295+
[kokoro-badge-image-3]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-osx.svg
296+
[kokoro-badge-link-3]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-osx.html
297+
[kokoro-badge-image-4]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-win.svg
298+
[kokoro-badge-link-4]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-win.html
299+
[kokoro-badge-image-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java11.svg
300+
[kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java11.html
301+
[stability-image]: https://img.shields.io/badge/stability-unknown-red
302+
[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigquery-jdbc.svg
303+
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-bigquery-jdbc/0.0.0
304+
[authentication]: https://github.com/googleapis/google-cloud-java#authentication
305+
[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes
306+
[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
307+
[iam-policy]: https://cloud.google.com/iam/docs/overview#cloud-iam-policy
308+
[developer-console]: https://console.developers.google.com/
309+
[create-project]: https://cloud.google.com/resource-manager/docs/creating-managing-projects
310+
[cloud-cli]: https://cloud.google.com/cli
311+
[troubleshooting]: https://github.com/googleapis/google-cloud-java/blob/main/TROUBLESHOOTING.md
312+
[contributing]: https://github.com/googleapis/java-bigquery/blob/main/CONTRIBUTING.md
313+
[code-of-conduct]: https://github.com/googleapis/java-bigquery/blob/main/CODE_OF_CONDUCT.md
314+
[license]: https://github.com/googleapis/java-bigquery/blob/main/LICENSE
315+
316+
317+
[libraries-bom]: https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/The-Google-Cloud-Platform-Libraries-BOM
318+
[shell_img]: https://gstatic.com/cloudssh/images/open-btn.png
319+
320+
[semver]: https://semver.org/
321+
[cloudlibs]: https://cloud.google.com/apis/docs/client-libraries-explained
322+
[apilibs]: https://cloud.google.com/apis/docs/client-libraries-explained#google_api_client_libraries
323+
[oracle]: https://www.oracle.com/java/technologies/java-se-support-roadmap.html
324+
[g-c-j]: http://github.com/googleapis/google-cloud-java

0 commit comments

Comments
 (0)