Skip to content

Commit aa3a102

Browse files
azure-cosmos-kafka-connect skeleton (Azure#37283)
* azure-cosmos-kafka-connect skeleton * Build fixes * Update .docsettings.yml * Update FaultInjectionWithAvailabilityStrategyTests.java * Fixing linting errors * Linting issues * Create package-info.java * Update sdk/cosmos/ci.yml Co-authored-by: Kushagra Thapar <[email protected]> * Reacting to PR review feedback * Update README.md * Adding shading * Update pom.xml * Update pom.xml * Addressing build analyzer issues * Update CHANGELOG.md --------- Co-authored-by: Kushagra Thapar <[email protected]>
1 parent 35ef589 commit aa3a102

File tree

14 files changed

+606
-0
lines changed

14 files changed

+606
-0
lines changed

eng/.docsettings.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ known_presence_issues:
8383
- ['sdk/cognitiveservices/ms-azure-cs-visualsearch/CHANGELOG.md', '#2847']
8484
- ['sdk/cognitiveservices/ms-azure-cs-websearch/CHANGELOG.md', '#2847']
8585
- ['sdk/cosmos/azure-cosmos-examples/CHANGELOG.md', '#2847']
86+
- ['sdk/cosmos/azure-cosmos-kafka-connect/CHANGELOG.md', '#2847']
8687
- ['sdk/cosmos/CHANGELOG.md', '#2847']
8788
- ['sdk/eventgrid/azure-messaging-eventgrid/CHANGELOG.md', '#2847']
8889
- ['sdk/eventgrid/microsoft-azure-eventgrid/CHANGELOG.md', '#2847']
@@ -114,6 +115,7 @@ known_content_issues:
114115
- ['sdk/cosmos/faq/README.md', '#3113']
115116
- ['sdk/cosmos/azure-cosmos-benchmark/README.md', '#3113']
116117
- ['sdk/cosmos/azure-cosmos-examples/README.md', '#3113']
118+
- ['sdk/cosmos/azure-cosmos-kafka-connect/README.md', '#3113']
117119
- ['sdk/cosmos/azure-cosmos/README.md', '#3113']
118120
- ['sdk/cosmos/azure-cosmos-encryption/README.md', '#3113']
119121
- ['sdk/cosmos/azure-cosmos-spark_3_2-12/README.md', '#3113']

eng/code-quality-reports/src/main/resources/checkstyle/checkstyle-suppressions.xml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -378,6 +378,7 @@ the main ServiceBusClientBuilder. -->
378378

379379
<!-- Cosmos sdk suppressions -->
380380
<suppress checks="[a-zA-Z0-9]*" files="[/\\]azure-cosmos-encryption[/\\]src[/\\]test[/\\]"/>
381+
<suppress checks="[a-zA-Z0-9]*" files="[/\\]azure-cosmos-kafka-connect[/\\]src[/\\]test[/\\]"/>
381382
<suppress checks="[a-zA-Z0-9]*"
382383
files=".*[/\\]azure-cosmos[/\\]((?!(BridgeInternal|CosmosBridgeInternal|CommonsBridgeInternal)).*)"/>
383384
<suppress checks="[a-zA-Z0-9]*"

eng/versioning/external_dependencies.txt

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -383,6 +383,13 @@ cosmos_org.scalatest:scalatest-maven-plugin;2.0.2
383383
cosmos_net.alchim31.maven:scala-maven-plugin;4.8.1
384384
cosmos_org.scalastyle:scalastyle-maven-plugin;1.0.0
385385

386+
## Cosmos Kafka connector under sdk\cosmos\azure-cosmos-kafka-connect\pom.xml
387+
# Cosmos Kafka connector runtime dependencies
388+
cosmos_org.apache.kafka:connect-api;3.6.0
389+
# Cosmos Kafka connector tests only
390+
# Maven Tools for Cosmos Kafka connector only
391+
cosmos_io.confluent:kafka-connect-maven-plugin;0.12.0
392+
386393
# sdk\core\azure-core-serializer-avro-jackson\pom.xml
387394
# This dependency is needed since Jackson Avro uses an older dependency on Apache Avro which is another library.
388395
jacksonavro_org.apache.avro:avro-maven-plugin;1.8.2

eng/versioning/version_client.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ com.azure.cosmos.spark:azure-cosmos-spark_3-4_2-12;4.23.0;4.24.0-beta.1
108108
com.azure:azure-cosmos-encryption;2.6.0;2.7.0-beta.1
109109
com.azure:azure-cosmos-test;1.0.0-beta.6;1.0.0-beta.7
110110
com.azure:azure-cosmos-tests;1.0.0-beta.1;1.0.0-beta.1
111+
com.azure.cosmos.kafka:azure-cosmos-kafka-connect;1.0.0-beta.1;1.0.0-beta.1
111112
com.azure:azure-data-appconfiguration;1.4.10;1.5.0-beta.3
112113
com.azure:azure-data-appconfiguration-perf;1.0.0-beta.1;1.0.0-beta.1
113114
com.azure:azure-data-schemaregistry;1.4.0;1.5.0-beta.1
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
## Release History
2+
3+
### 1.0.0-beta.1 (Unreleased)
4+
5+
#### Features Added
6+
7+
#### Breaking Changes
8+
9+
#### Bugs Fixed
10+
11+
#### Other Changes
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Apache Kafka Connect source and sink connectors for Azure Cosmos DB Java SDK for Java
2+
The Azure Cosmos DB connectors allow moving data between Azure Cosmos DB and Kafka. The Cosmos DB Sink connector writes data from a Kafka topic to a Cosmos DB container. The Cosmos DB Source connector writes changes from a Cosmos DB container to a Kafka topic.
3+
4+
[Source code][kafka_source_code] | [Package (Maven)][cosmos_kafka_maven] | [Product documentation][cosmos_docs]
5+
6+
## Getting started
7+
### Include the package
8+
9+
[//]: # ({x-version-update-start;com.azure.kafka-connect:azure-cosmos-kafka-connect;current})
10+
```xml
11+
<dependency>
12+
<groupId>com.azure.kafka.connect</groupId>
13+
<artifactId>azure-cosmos-kafka-connect</artifactId>
14+
<version>2.0.0-beta.1</version>
15+
</dependency>
16+
```
17+
[//]: # ({x-version-update-end})
18+
19+
Refer to maven central for previous [releases][cosmos_kafka_maven]
20+
21+
### Prerequisites
22+
23+
- [Java Development Kit (JDK) with version 8 or above][jdk]
24+
- An active Azure account. If you don't have one, you can sign up for a [free account][azure_subscription]. Alternatively, you can use the [Azure Cosmos DB Emulator](https://docs.microsoft.com/azure/cosmos-db/local-emulator) for development and testing. As emulator HTTPS certificate is self-signed, you need to import its certificate to java trusted cert store as [explained here](https://docs.microsoft.com/azure/cosmos-db/local-emulator-export-ssl-certificates)
25+
- (Optional) SLF4J is a logging facade.
26+
- (Optional) [SLF4J binding](https://www.slf4j.org/manual.html) is used to associate a specific logging framework with SLF4J.
27+
- (Optional) [Maven][maven]
28+
29+
SLF4J is only needed if you plan to use logging, please also download an SLF4J binding which will link the SLF4J API with the logging implementation of your choice. See the [SLF4J user manual](https://www.slf4j.org/manual.html) for more information.
30+
31+
The SDK provides Reactor Core-based async APIs. You can read more about Reactor Core and [Flux/Mono types here](https://projectreactor.io/docs/core/release/api/)
32+
33+
## Key concepts
34+
35+
TBD
36+
37+
## Examples
38+
TBD
39+
40+
## Troubleshooting
41+
42+
### General
43+
44+
Azure Cosmos DB is a fast and flexible distributed database that scales seamlessly with guaranteed latency and throughput.
45+
You do not have to make major architecture changes or write complex code to scale your database with Azure Cosmos DB.
46+
Scaling up and down is as easy as making a single API call or SDK method call.
47+
However, because Azure Cosmos DB is accessed via network calls there are client-side optimizations you can make to achieve peak performance when using Azure Cosmos DB Java SDK v4.
48+
49+
- [Performance][perf_guide] guide covers these client-side optimizations.
50+
51+
- [Troubleshooting Guide][troubleshooting] covers common issues, workarounds, diagnostic steps, and tools when you use Azure Cosmos DB Java SDK v4 with Azure Cosmos DB SQL API accounts.
52+
53+
## Next steps
54+
55+
- Quick start of Cosmos DB core java sdk [quickstart][quickstart] - Building a java app to manage Cosmos DB SQL API data
56+
- [Read more about Azure Cosmos DB Service][cosmos_docs]
57+
58+
## Contributing
59+
60+
This project welcomes contributions and suggestions. Most contributions require you to agree to a
61+
[Contributor License Agreement (CLA)][cla] declaring that you have the right to, and actually do, grant us the rights
62+
to use your contribution.
63+
64+
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate
65+
the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to
66+
do this once across all repos using our CLA.
67+
68+
This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information see the [Code of Conduct FAQ][coc_faq]
69+
or contact [[email protected]][coc_contact] with any additional questions or comments.
70+
71+
<!-- LINKS -->
72+
[kafka_source_code]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-kafka-connect/src
73+
[cosmos_introduction]: https://docs.microsoft.com/azure/cosmos-db/
74+
[cosmos_docs]: https://docs.microsoft.com/azure/cosmos-db/introduction
75+
[jdk]: https://docs.microsoft.com/java/azure/jdk/
76+
[maven]: https://maven.apache.org/
77+
[cosmos_kafka_maven]: https://central.sonatype.com/artifact/com.azure.kafka.connect/azure-cosmos-kafka-connect
78+
[cla]: https://cla.microsoft.com
79+
[coc]: https://opensource.microsoft.com/codeofconduct/
80+
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
81+
[coc_contact]: mailto:[email protected]
82+
[azure_subscription]: https://azure.microsoft.com/free/
83+
[troubleshooting]: https://docs.microsoft.com/azure/cosmos-db/troubleshoot-java-sdk-v4-sql
84+
[perf_guide]: https://docs.microsoft.com/azure/cosmos-db/performance-tips-java-sdk-v4-sql?tabs=api-async
85+
[sql_api_query]: https://docs.microsoft.com/azure/cosmos-db/sql-api-sql-query
86+
[quickstart]: https://docs.microsoft.com/azure/cosmos-db/create-sql-api-java?tabs=sync
87+
88+
![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-java%2Fsdk%2Fcosmos%2Fazure-cosmos-kafka-connect%2FREADME.png)
13.1 KB
Loading

0 commit comments

Comments
 (0)