Skip to content

Commit 0b5a5e2

Browse files
feat: add kafka connector (#1771)
* feat: add kafka connector * fix version
1 parent 3701cc2 commit 0b5a5e2

File tree

165 files changed

+33704
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

165 files changed

+33704
-0
lines changed
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* @confluentinc/connect
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Build products
2+
target/
3+
4+
# IntelliJ data
5+
*.iml
6+
.idea/
7+
.ipr
8+
9+
# Documentation build output
10+
/docs/_build
11+
12+
.DS_Store
13+
14+
derby.log
15+
16+
# Test databases
17+
__test_database_*
18+
test.sqlite
19+
*.db-journal
Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# Contributing
2+
3+
In order for us to consider merging a contribution, you will need to sign our
4+
**C**ontributor **L**icense **A**greement.
5+
6+
> The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
7+
> [Wikipedia](http://en.wikipedia.org/wiki/Contributor_License_Agreement)
8+
9+
You can read and sign our full Contributor License Agreement [here](http://clabot.confluent.io/cla).
10+
11+
## Reporting Bugs and Issues
12+
13+
Report bugs and issues by creating a new GitHub issue. Prior to creating an issue, please search
14+
through existing issues so that you are not creating duplicate ones. If a pull request exists that
15+
corresponds to the issue, mention this pull request on the GitHub issue.
16+
17+
## Guidelines for Contributing Code, Examples, Documentation
18+
19+
Code changes are submitted via a pull request (PR). When submitting a PR use the following
20+
guidelines:
21+
22+
* Follow the style guide below
23+
* Add/update documentation appropriately for the change you are making.
24+
* Non-trivial changes should include unit tests covering the new functionality and potentially integration tests.
25+
* Bug fixes should include unit tests and/or integration tests proving the issue is fixed.
26+
* Try to keep pull requests short and submit separate ones for unrelated features.
27+
* Keep formatting changes in separate commits to make code reviews easier and distinguish them from actual code changes.
28+
29+
### Code Style
30+
This connector is using a coding style that generally follows the [Google Java coding standard guide](https://google.github.io/styleguide/javaguide.html).
31+
32+
Some conventions worth mentioning are:
33+
34+
* Indentation (single tab) is 2 spaces.
35+
* All import statements are listed explicitly. The wildcard (*) is not used in imports.
36+
* Imports are groups as follows:
37+
```
38+
import all packages not listed below (all other imports)
39+
<blank line>
40+
import all javax.* packages
41+
import all java.* packages
42+
<blank line>
43+
import all io.confluent.* packages
44+
<blank line>
45+
import static packages
46+
```
47+
* Javadoc is highly recommended and often required during reviews in interfaces and public or protected classes and methods.
48+
49+
### Titles and changelogs
50+
51+
The title of a pull request is used as an entry on the release notes (aka changelogs) of the
52+
connector in every release.
53+
54+
For this reason, please use a brief but descriptive title for your pull request. If GitHub shortens
55+
your pull request title when you issue the pull request adding the excessive part to the pull
56+
request description, make sure that you correct the title before or after you issue the pull
57+
request.
58+
59+
If the fix is a minor fix you are encouraged to use the tag `MINOR:` followed by your pull request
60+
title. You may link the corresponding issue to the description of your pull request but adding it to
61+
the title will not be useful during changelog generation.
62+
63+
When reverting a previous commit, use the prefix `Revert ` on the pull request title (automatically
64+
added by GitHub when a pull request is created to revert an existing commit).
65+
66+
### Tests
67+
Every pull request should contain a sufficient amount of tests that assess your suggested code
68+
changes. It’s highly recommended that you also check the code coverage of the production code you
69+
are adding to make sure that your changes are covered sufficiently by the test code.
70+
71+
### Description
72+
Including a good description when you issue your pull requests helps significantly with reviews.
73+
Feel free to follow the template that is when issuing a pull request and mention how your changes
74+
are tested.
75+
76+
### Backporting Commits
77+
If your code changes are essentially bug fixes that make sense to backport to existing releases make sure to target the earliest release branch (e.g. 2.0.x) that should contain your changes. When selecting the release branch you should also consider how easy it will be to resolve any conflicts in newer release branches, including the `master` branch.
78+
79+
## Github Workflow
80+
81+
1. Fork the connector repository into your GitHub account: https://github.com/confluentinc/kafka-connect-jdbc/fork
82+
83+
2. Clone your fork of the GitHub repository, replacing `<username>` with your GitHub username.
84+
85+
Use ssh (recommended):
86+
87+
```bash
88+
git clone [email protected]:<username>/kafka-connect-jdbc.git
89+
```
90+
91+
Or https:
92+
93+
```bash
94+
git clone https://github.com/<username>/kafka-connect-jdbc.git
95+
```
96+
97+
3. Add a remote to keep up with upstream changes.
98+
99+
```bash
100+
git remote add upstream https://github.com/confluentinc/kafka-connect-jdbc.git
101+
```
102+
103+
If you already have a copy, fetch upstream changes.
104+
105+
```bash
106+
git fetch upstream
107+
```
108+
109+
or
110+
111+
```bash
112+
git remote update
113+
```
114+
115+
4. Create a feature branch to work in.
116+
117+
```bash
118+
git checkout -b feature-xyz upstream/master
119+
```
120+
121+
5. Work in your feature branch.
122+
123+
```bash
124+
git commit -a --verbose
125+
```
126+
127+
6. Periodically rebase your changes
128+
129+
```bash
130+
git pull --rebase
131+
```
132+
133+
7. When done, combine ("squash") related commits into a single one
134+
135+
```bash
136+
git rebase -i upstream/master
137+
```
138+
139+
This will open your editor and allow you to re-order commits and merge them:
140+
- Re-order the lines to change commit order (to the extent possible without creating conflicts)
141+
- Prefix commits using `s` (squash) or `f` (fixup) to merge extraneous commits.
142+
143+
8. Submit a pull-request
144+
145+
```bash
146+
git push origin feature-xyz
147+
```
148+
149+
Go to your fork main page
150+
151+
```bash
152+
https://github.com/<username>/kafka-connect-jdbc.git
153+
```
154+
155+
If you recently pushed your changes GitHub will automatically pop up a `Compare & pull request`
156+
button for any branches you recently pushed to. If you click that button it will automatically
157+
offer you to submit your pull-request to the `confluentinc` connector repository.
158+
159+
- Give your pull-request a meaningful title as described [above](#titles-and-changelogs).
160+
- In the description, explain your changes and the problem they are solving.
161+
162+
9. Addressing code review comments
163+
164+
Repeat steps 5. through 7. to address any code review comments and rebase your changes if necessary.
165+
166+
Push your updated changes to update the pull request
167+
168+
```bash
169+
git push origin [--force] feature-xyz
170+
```
171+
172+
`--force` may be necessary to overwrite your existing pull request in case your
173+
commit history was changed when performing the rebase.
174+
175+
Note: Be careful when using `--force` since you may lose data if you are not careful.
176+
177+
```bash
178+
git push origin --force feature-xyz
179+
```
180+
181+
## Useful Resources for Developers
182+
183+
1. Connector Developer Guide: https://docs.confluent.io/platform/current/connect/devguide.html
184+
2. A Guide to the Confluent Verified Integrations Program: https://www.confluent.io/blog/guide-to-confluent-verified-integrations-program/
185+
3. Verification Guide for Confluent Platform Integrations: https://cdn.confluent.io/wp-content/uploads/Verification-Guide-Confluent-Platform-Connectors-Integrations.pdf
186+
4. From Zero to Hero with Kafka Connect: https://www.confluent.io/kafka-summit-lon19/from-zero-to-hero-with-kafka-connect/
187+
5. 4 Steps to Creating Apache Kafka Connectors with the Kafka Connect API: https://www.confluent.io/blog/create-dynamic-kafka-connect-source-connectors/
188+
6. How to Write a Connector for Kafka Connect – Deep Dive into Configuration Handling: https://www.confluent.io/blog/write-a-kafka-connect-connector-with-configuration-handling/
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/usr/bin/env groovy
2+
common {
3+
slackChannel = '#connect-warn'
4+
nodeLabel = 'docker-debian-jdk8'
5+
upstreamProjects = 'confluentinc/common'
6+
pintMerge = true
7+
downStreamValidate = false
8+
}
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
Confluent Community License Agreement
2+
Version 1.0
3+
4+
This Confluent Community License Agreement Version 1.0 (the “Agreement”) sets
5+
forth the terms on which Confluent, Inc. (“Confluent”) makes available certain
6+
software made available by Confluent under this Agreement (the “Software”).  BY
7+
INSTALLING, DOWNLOADING, ACCESSING, USING OR DISTRIBUTING ANY OF THE SOFTWARE,
8+
YOU AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE TO
9+
SUCH TERMS AND CONDITIONS, YOU MUST NOT USE THE SOFTWARE.  IF YOU ARE RECEIVING
10+
THE SOFTWARE ON BEHALF OF A LEGAL ENTITY, YOU REPRESENT AND WARRANT THAT YOU
11+
HAVE THE ACTUAL AUTHORITY TO AGREE TO THE TERMS AND CONDITIONS OF THIS
12+
AGREEMENT ON BEHALF OF SUCH ENTITY. “Licensee” means you, an individual, or
13+
the entity on whose behalf you are receiving the Software.
14+
15+
1. LICENSE GRANT AND CONDITIONS.
16+
17+
1.1 License.  Subject to the terms and conditions of this Agreement,
18+
Confluent hereby grants to Licensee a non-exclusive, royalty-free,
19+
worldwide, non-transferable, non-sublicenseable license during the term
20+
of this Agreement to: (a) use the Software; (b) prepare modifications and
21+
derivative works of the Software; (c) distribute the Software (including
22+
without limitation in source code or object code form); and (d) reproduce
23+
copies of the Software (the “License”).  Licensee is not granted the
24+
right to, and Licensee shall not, exercise the License for an Excluded
25+
Purpose.  For purposes of this Agreement, “Excluded Purpose” means making
26+
available any software-as-a-service, platform-as-a-service,
27+
infrastructure-as-a-service or other similar online service that competes
28+
with Confluent products or services that provide the Software.
29+
30+
1.2 Conditions.  In consideration of the License, Licensee’s distribution
31+
of the Software is subject to the following conditions:
32+
33+
(a) Licensee must cause any Software modified by Licensee to carry
34+
prominent notices stating that Licensee modified the Software.
35+
36+
(b) On each Software copy, Licensee shall reproduce and not remove or
37+
alter all Confluent or third party copyright or other proprietary
38+
notices contained in the Software, and Licensee must provide the
39+
notice below with each copy.  
40+
41+
“This software is made available by Confluent, Inc., under the
42+
terms of the Confluent Community License Agreement, Version 1.0
43+
located at http://www.confluent.io/confluent-community-license.  BY
44+
INSTALLING, DOWNLOADING, ACCESSING, USING OR DISTRIBUTING ANY OF
45+
THE SOFTWARE, YOU AGREE TO THE TERMS OF SUCH LICENSE AGREEMENT.”
46+
47+
1.3 Licensee Modifications.  Licensee may add its own copyright notices
48+
to modifications made by Licensee and may provide additional or different
49+
license terms and conditions for use, reproduction, or distribution of
50+
Licensee’s modifications.  While redistributing the Software or
51+
modifications thereof, Licensee may choose to offer, for a fee or free of
52+
charge, support, warranty, indemnity, or other obligations. Licensee, and
53+
not Confluent, will be responsible for any such obligations.
54+
55+
1.4 No Sublicensing.  The License does not include the right to
56+
sublicense the Software, however, each recipient to which Licensee
57+
provides the Software may exercise the Licenses so long as such recipient
58+
agrees to the terms and conditions of this Agreement.  
59+
60+
2. TERM AND TERMINATION.  This Agreement will continue unless and until
61+
earlier terminated as set forth herein.  If Licensee breaches any of its
62+
conditions or obligations under this Agreement, this Agreement will
63+
terminate automatically and the License will terminate automatically and
64+
permanently.
65+
66+
3. INTELLECTUAL PROPERTY.  As between the parties, Confluent will retain all
67+
right, title, and interest in the Software, and all intellectual property
68+
rights therein.  Confluent hereby reserves all rights not expressly granted
69+
to Licensee in this Agreement. Confluent hereby reserves all rights in its
70+
trademarks and service marks, and no licenses therein are granted in this
71+
Agreement.
72+
73+
4. DISCLAIMER.  CONFLUENT HEREBY DISCLAIMS ANY AND ALL WARRANTIES AND
74+
CONDITIONS, EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE, AND SPECIFICALLY
75+
DISCLAIMS ANY WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
76+
PURPOSE, WITH RESPECT TO THE SOFTWARE.  
77+
78+
5. LIMITATION OF LIABILITY.  CONFLUENT WILL NOT BE LIABLE FOR ANY DAMAGES OF
79+
ANY KIND, INCLUDING BUT NOT LIMITED TO, LOST PROFITS OR ANY CONSEQUENTIAL,
80+
SPECIAL, INCIDENTAL, INDIRECT, OR DIRECT DAMAGES, HOWEVER CAUSED AND ON ANY
81+
THEORY OF LIABILITY, ARISING OUT OF THIS AGREEMENT.  THE FOREGOING SHALL
82+
APPLY TO THE EXTENT PERMITTED BY APPLICABLE LAW.
83+
84+
6.GENERAL.
85+
86+
6.1 Governing Law. This Agreement will be governed by and interpreted in
87+
accordance with the laws of the state of California, without reference to
88+
its conflict of laws principles.  If Licensee is located within the
89+
United States, all disputes arising out of this Agreement are subject to
90+
the exclusive jurisdiction of courts located in Santa Clara County,
91+
California. USA.  If Licensee is located outside of the United States,
92+
any dispute, controversy or claim arising out of or relating to this
93+
Agreement will be referred to and finally determined by arbitration in
94+
accordance with the JAMS International Arbitration Rules.  The tribunal
95+
will consist of one arbitrator. The place of arbitration will be Palo
96+
Alto, California. The language to be used in the arbitral proceedings
97+
will be English. Judgment upon the award rendered by the arbitrator may
98+
be entered in any court having jurisdiction thereof.
99+
100+
6.2 Assignment.  Licensee is not authorized to assign its rights under
101+
this Agreement to any third party. Confluent may freely assign its rights
102+
under this Agreement to any third party.
103+
104+
6.3 Other.  This Agreement is the entire agreement between the parties
105+
regarding the subject matter hereof.  No amendment or modification of
106+
this Agreement will be valid or binding upon the parties unless made in
107+
writing and signed by the duly authorized representatives of both
108+
parties.  In the event that any provision, including without limitation
109+
any condition, of this Agreement is held to be unenforceable, this
110+
Agreement and all licenses and rights granted hereunder will immediately
111+
terminate.  Waiver by Confluent of a breach of any provision of this
112+
Agreement or the failure by Confluent to exercise any right hereunder
113+
will not be construed as a waiver of any subsequent breach of that right
114+
or as a waiver of any other right.
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
kafka-connect-jdbc
2+
Copyright (c) 2015 Confluent Inc.
3+
4+
The following libraries are included in packaged versions of this project:
5+
6+
* SQLite JDBC Driver
7+
* COPYRIGHT: Copyright Taro L. Saito, David Crenshaw
8+
* LICENSE: licenses/LICENSE.apache2.txt
9+
* NOTICE: licenses/NOTICE.sqlite-jdbc.txt
10+
* HOMEPAGE: https://github.com/xerial/sqlite-jdbc
11+
12+
* PostgreSQL JDBC Driver
13+
* COPYRIGHT: Copyright 1997-2011, PostgreSQL Global Development Group
14+
* LICENSE: licenses/LICENSE.bsd.txt
15+
* HOMEPAGE: https://jdbc.postgresql.org/
16+
17+
* MariaDB JDBC Driver
18+
* COPYRIGHT: Copyright 2012 Monty Program Ab., 2009-2011, Marcus Eriksson
19+
* LICENSE: licenses/LICENSE.lgpl.txt
20+
* HOMEPAGE: https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Kafka Connect JDBC Connector
2+
3+
kafka-connect-jdbc is a [Kafka Connector](http://kafka.apache.org/documentation.html#connect)
4+
for loading data to and from any JDBC-compatible database.
5+
6+
Documentation for this connector can be found [here](http://docs.confluent.io/current/connect/connect-jdbc/docs/index.html).
7+
8+
# Development
9+
10+
To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the [FAQ](https://github.com/confluentinc/kafka-connect-jdbc/wiki/FAQ)
11+
for guidance on this process.
12+
13+
You can build kafka-connect-jdbc with Maven using the standard lifecycle phases.
14+
15+
# FAQ
16+
17+
Refer frequently asked questions on Kafka Connect JDBC here -
18+
https://github.com/confluentinc/kafka-connect-jdbc/wiki/FAQ
19+
20+
# Contribute
21+
22+
Contributions can only be accepted if they contain appropriate testing. For example, adding a new dialect of JDBC will require an integration test.
23+
24+
- Source Code: https://github.com/confluentinc/kafka-connect-jdbc
25+
- Issue Tracker: https://github.com/confluentinc/kafka-connect-jdbc/issues
26+
- Learn how to work with the connector's source code by reading our [Development and Contribution guidelines](CONTRIBUTING.md).
27+
28+
# Information
29+
30+
For more information, check the documentation for the JDBC connector on the [confluent.io](https://docs.confluent.io/current/connect/kafka-connect-jdbc/index.html) website. Questions related to the connector can be asked on [Community Slack](https://launchpass.com/confluentcommunity) or the [Confluent Platform Google Group](https://groups.google.com/forum/#!topic/confluent-platform/).
31+
32+
# License
33+
34+
This project is licensed under the [Confluent Community License](LICENSE).
35+

0 commit comments

Comments
 (0)