Skip to content

Commit 09a43f4

Browse files
nikhil-ctdsnikhil-ctds
authored
Added Luna Streaming 4.0.3.2 (#198)
1 parent b42f92f commit 09a43f4

File tree

1 file changed

+218
-0
lines changed

1 file changed

+218
-0
lines changed

‎Luna_Streaming_4.0_Release_Notes.md

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# Release notes for DataStax Luna Streaming Distribution
2+
3+
## About DataStax Luna Streaming Distribution
4+
5+
Luna Streaming Distribution 4.0 is compatible with Apache Pulsar™ 4.0
6+
and subsequent releases.
7+
8+
### Components
9+
10+
- [DataStax Luna Streaming](https://github.com/datastax/pulsar) (Apache Pulsar 4.0 fork)
11+
- [DataStax BookKeeper](https://github.com/datastax/bookkeeper) (Apache BookKeeper fork)
12+
- DataStax Burnell
13+
- DataStax Pulsar Admin Console
14+
- DataStax Pulsar Heartbeat
15+
- DataStax Apache Pulsar Cassandra Sink
16+
- DataStax Apache Pulsar Cassandra Source
17+
- DataStax Starlight For Kafka
18+
- DataStax Starlight For RabbitMQ
19+
- DataStax Starlight For JMS Broker filter
20+
- DataStax Snowflake connector
21+
22+
This release adds these features to the original Apache Pulsar 4.0 release:
23+
24+
- Stability improvements
25+
- Third party libraries updates for security fixes
26+
- Dependency upgrades (for security, stability and performances)
27+
- An enhanced ElasticSearch Pulsar IO Sink
28+
- An enhanced version of Apache BookKeeper with security fixes
29+
30+
*Note:* The DataStax Luna Streaming Distribution is designed for Java 17.
31+
However, because the product releases Docker images, you do not need to install
32+
Java (8 or 17) in advance. Java 17 is bundled in the Docker image.
33+
34+
### Security
35+
36+
Luna Streaming is developed with great attention to security.
37+
38+
Since it is based on Apache Pulsar™, Luna Streaming Pulsar distribution
39+
has all the security processes and requirements required by the Apache Software
40+
Foundation™, such as:
41+
42+
- Peer to peer code reviews.
43+
- Clear process for handling possible security vulnerabilities.
44+
45+
On top of that, Luna Streaming development processes are enhanced with
46+
additional security best practices for all the components involved:
47+
48+
- Robust Continuous Integration and reproducible Release pipeline in a secure
49+
monitored environment. Such monitoring improves the overall security of the
50+
application.
51+
- Reactivity to third-party vulnerabilities. Every component is scanned and
52+
monitored daily, ensuring quick fixes in case of vulnerability disclosure.
53+
- Regular docker images vulnerability scans on latest Luna Streaming releases,
54+
with a well-defined release procedure.
55+
56+
57+
### Packaging
58+
59+
Luna Streaming Distribution comes with different packages tooled for different
60+
purposes. The distributions are available as Docker images and tarballs, and
61+
both methods follow the same packaging patterns.
62+
63+
#### Patterns
64+
65+
- **lunastreaming**: the basic Luna Streaming Distribution **including Pulsar
66+
SQL** feature.
67+
- **lunastreaming-core**: the basic Luna Streaming Distribution **without Pulsar
68+
SQL** feature. Pick this distribution if you don't need Pulsar SQL features.
69+
- **lunastreaming-all**: contains the Core Luna Streaming Distribution,
70+
including **Pulsar Offloaders** and the **DataStax Pulsar IO Connectors**
71+
listed above. Pick this distribution if you want the DataStax connectors or
72+
the offloading feature.
73+
74+
# Releases
75+
76+
## Luna Streaming Distribution 4.0 3.2
77+
This is the initial release of the DataStax Luna Streaming Distribution for 4.0. This release contains all the commits included in the Apache Pulsar v4.0.3 release plus the following changes specific to Luna Streaming:
78+
79+
### Most notable commits
80+
81+
* [96f25cd](https://github.com/datastax/pulsar/commit/96f25cd) [improve][meta] Change log level from error to warn for unknown notification types in OxiaMetadataStore (#24126)
82+
* [ba8037d](https://github.com/datastax/pulsar/commit/ba8037d) [fix][ml] Return 1 when bytes size is 0 or negative for entry count estimation (#24131)
83+
* [167ffea](https://github.com/datastax/pulsar/commit/167ffea) [improve][io] Enhance Kafka connector logging with focused bootstrap server information (#24128)
84+
* [5edc57c](https://github.com/datastax/pulsar/commit/5edc57c) [improve][client] Prevent NullPointException when closing ClientCredentialsFlow (#24123)
85+
* [21f0750](https://github.com/datastax/pulsar/commit/21f0750) [fix][ml] Don't estimate number of entries when ledgers are empty, return 1 instead (#24125)
86+
* [f8816e1](https://github.com/datastax/pulsar/commit/f8816e1) [improve][io] Remove sleep when sourceTask.poll of kafka return null (#24124)
87+
* [71330c4](https://github.com/datastax/pulsar/commit/71330c4) [improve][broker] Change topic exists log to warn (#24116)
88+
* [d699029](https://github.com/datastax/pulsar/commit/d699029) [improve][broker][branch-4.0] PIP-406: Introduce metrics related to dispatch throttled events (#24111)
89+
* [e53623d](https://github.com/datastax/pulsar/commit/e53623d) [fix][client] Pattern subscription regression when broker-side evaluation is disabled (#24104)
90+
* [3ffcbfc](https://github.com/datastax/pulsar/commit/3ffcbfc) [fix][client] Fix consumer leak when thread is interrupted before subscribe completes (#24100)
91+
* [83ba1ab](https://github.com/datastax/pulsar/commit/83ba1ab) [fix][ci] Bump dependency-check to 12.1.0 to fix OWASP Dependency Check job (#24083)
92+
* [2b933c7](https://github.com/datastax/pulsar/commit/2b933c7) [fix][broker] Restore the behavior to dispatch batch messages according to consumer permits (#24092)
93+
* [fef163c](https://github.com/datastax/pulsar/commit/fef163c) [improve][broker] Optimize message expiration rate repeated update issues (#24073)
94+
* [814781d](https://github.com/datastax/pulsar/commit/814781d) [fix][ml] Fix issues in estimateEntryCountBySize (#24089)
95+
* [dc0176e](https://github.com/datastax/pulsar/commit/dc0176e) [fix][broker] Fix Metadata Event Synchronizer producer creation retry so that the producer gets created eventually (#24081)
96+
* [4a5f398](https://github.com/datastax/pulsar/commit/4a5f398) [fix][broker] Fix Metadata event synchronizer should not fail with bad version (#24080)
97+
* [e9aedd2](https://github.com/datastax/pulsar/commit/e9aedd2) [fix][broker] Fix NPE while publishing Metadata-Event with not init producer (#24079)
98+
* [cdaec92](https://github.com/datastax/pulsar/commit/cdaec92) [clean][client] Clean code for the construction of retry/dead letter topic name (#24082)
99+
* [3c6b12a](https://github.com/datastax/pulsar/commit/3c6b12a) [fix][client] Fix building broken batched message when publishing (#24061)
100+
* [3efd9bb](https://github.com/datastax/pulsar/commit/3efd9bb) [fix][broker]Fix failed consumption after loaded up a terminated topic (#24063)
101+
* [f3dfaeb](https://github.com/datastax/pulsar/commit/f3dfaeb) [fix][test] Fix flaky PrometheusMetricsTest.testBrokerMetrics (#24042)
102+
* [aaa2ed1](https://github.com/datastax/pulsar/commit/aaa2ed1) [fix][broker] http metric endpoint get compaction latency stats always be 0 (#24067)
103+
* [00fedf9](https://github.com/datastax/pulsar/commit/00fedf9) [fix][client] PIP-409 retry/dead letter topic producer config don't take effect. (#24071)
104+
* [c4b8ca9](https://github.com/datastax/pulsar/commit/c4b8ca9) [fix][ml] Corrected pulsar_storage_size metric to not multiply offloaded storage by the write quorum (#24054)
105+
* [9ab3221](https://github.com/datastax/pulsar/commit/9ab3221) [fix][broker] Fix UnsupportedOperationException while setting subscription level dispatch rate policy (#24048)
106+
* [6dd61f1](https://github.com/datastax/pulsar/commit/6dd61f1) [improve][broker] Optimize ThresholdShedder with improved boundary checks and parameter reuse (#24064)
107+
* [5546d45](https://github.com/datastax/pulsar/commit/5546d45) [improve][client] PIP-409: support producer configuration for retry/dead letter topic producer (#24020)
108+
* [1b250e3](https://github.com/datastax/pulsar/commit/1b250e3) [fix][client] Copy eventTime to retry letter topic and DLQ messages (#24059)
109+
* [379a203](https://github.com/datastax/pulsar/commit/379a203) [improve][monitor] Add version=0.0.4 to /metrics content type for Prometheus 3.x compatibility (#24060)
110+
* [dd26654](https://github.com/datastax/pulsar/commit/dd26654) [fix][broker] Pattern subscription doesn't work when the pattern excludes the topic domain. (#24072)
111+
* [807a2bc](https://github.com/datastax/pulsar/commit/807a2bc) [fix] Avoid negative estimated entry count (#24055)
112+
* [6b0806a](https://github.com/datastax/pulsar/commit/6b0806a) [fix][doc] fix doc related to chunk message feature. (#24023)
113+
* [9feba1f](https://github.com/datastax/pulsar/commit/9feba1f) [fix][broker] Add expire check for replicator (#23975)
114+
* [a9ffcb7](https://github.com/datastax/pulsar/commit/a9ffcb7) [fix][broker] Fix missing validation when setting retention policy on topic level (#24032)
115+
* [55dfc00](https://github.com/datastax/pulsar/commit/55dfc00) [fix][broker] fix delay queue sequence issue. (#24035)
116+
* [2bb61b2](https://github.com/datastax/pulsar/commit/2bb61b2) [fix][doc] Workaround Go Yaml issue go-yaml/yaml#789 in docker-compose example (#24040)
117+
* [d9ab831](https://github.com/datastax/pulsar/commit/d9ab831) [improve] Upgrade Netty to 4.1.119.Final (#24049)
118+
119+
### `lunastreaming-all` distribution
120+
<details><summary>Sinks</summary>
121+
122+
| Name | Description | Version | File |
123+
|--------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------|---------|-----------------------------------------------|
124+
| [cassandra-enhanced](https://github.com/datastax/pulsar-sink) | A DataStax Pulsar Sink to load records from Pulsar topics to Apache Cassandra(R) or DataStax Enterprise(DSE) | 1.6.13 | cassandra-enhanced-pulsar-sink-1.6.13-nar.nar |
125+
| [cloud-storage](https://github.com/streamnative/pulsar-io-cloud-storage) | Writes data into cloud storage | 3.2.2 | pulsar-io-cloud-storage-3.2.2.nar |
126+
| [data-generator](https://pulsar.apache.org/docs/io-connectors) | Test data generator source | 4.0.3.2 | pulsar-io-data-generator-4.0.3.2.nar |
127+
| [elastic_search](https://pulsar.apache.org/docs/io-connectors) | Writes data into Elastic Search | 4.0.3.2 | pulsar-io-elastic-search-4.0.3.2.nar |
128+
| [http](https://pulsar.apache.org/docs/io-connectors) | Writes data to an HTTP server (Webhook) | 4.0.3.2 | pulsar-io-http-4.0.3.2.nar |
129+
| [jdbc-clickhouse](https://pulsar.apache.org/docs/io-connectors) | JDBC sink for ClickHouse | 4.0.3.2 | pulsar-io-jdbc-clickhouse-4.0.3.2.nar |
130+
| [jdbc-mariadb](https://pulsar.apache.org/docs/io-connectors) | JDBC sink for MariaDB | 4.0.3.2 | pulsar-io-jdbc-mariadb-4.0.3.2.nar |
131+
| [jdbc-openmldb](https://pulsar.apache.org/docs/io-connectors) | JDBC sink for OpenMLDB | 4.0.3.2 | pulsar-io-jdbc-openmldb-4.0.3.2.nar |
132+
| [jdbc-postgres](https://pulsar.apache.org/docs/io-connectors) | JDBC sink for PostgreSQL | 4.0.3.2 | pulsar-io-jdbc-postgres-4.0.3.2.nar |
133+
| [jdbc-sqlite](https://pulsar.apache.org/docs/io-connectors) | JDBC sink for SQLite | 4.0.3.2 | pulsar-io-jdbc-sqlite-4.0.3.2.nar |
134+
| [kafka](https://pulsar.apache.org/docs/io-connectors) | Kafka source and sink connector | 4.0.3.2 | pulsar-io-kafka-4.0.3.2.nar |
135+
| [kinesis](https://pulsar.apache.org/docs/io-connectors) | Kinesis connectors | 4.0.3.2 | pulsar-io-kinesis-4.0.3.2.nar |
136+
| [snowflake](https://github.com/datastax/snowflake-connector) | Snowflake Connector | 0.1.15 | pulsar-snowflake-connector-0.1.15.nar |
137+
| [lakehouse](https://github.com/streamnative/pulsar-io-lakehouse) | Lakehouse Connector | 3.3.3.1 | pulsar-io-lakehouse-3.3.3.1.nar |
138+
| [lakehouse-cloud](https://github.com/streamnative/pulsar-io-lakehouse) | Lakehouse Cloud Connector | 3.3.3.1 | pulsar-io-lakehouse-3.3.3.1-cloud.nar |
139+
</details>
140+
<details><summary>Sources</summary>
141+
142+
| Name | Description | Version | File |
143+
|--------------------------------------------------------------------------|--------------------------------------|---------|-----------------------------------------|
144+
| [cassandra-source](https://github.com/datastax/cdc-for-apache-cassandra) | Read data from Cassandra | 2.3.1 | pulsar-cassandra-source-2.3.1.nar |
145+
| [data-generator](https://pulsar.apache.org/docs/io-connectors) | Test data generator source | 4.0.3.2 | pulsar-io-data-generator-4.0.3.2.nar |
146+
| [debezium-mongodb](https://pulsar.apache.org/docs/io-connectors) | Debezium MongoDb Source | 4.0.3.2 | pulsar-io-debezium-mongodb-4.0.3.2.nar |
147+
| [debezium-mssql](https://pulsar.apache.org/docs/io-connectors) | Debezium Microsoft SQL Server Source | 4.0.3.2 | pulsar-io-debezium-mssql-4.0.3.2.nar |
148+
| [debezium-mysql](https://pulsar.apache.org/docs/io-connectors) | Debezium MySql Source | 4.0.3.2 | pulsar-io-debezium-mysql-4.0.3.2.nar |
149+
| [debezium-oracle](https://pulsar.apache.org/docs/io-connectors) | Debezium Oracle Source | 4.0.3.2 | pulsar-io-debezium-oracle-4.0.3.2.nar |
150+
| [debezium-postgres](https://pulsar.apache.org/docs/io-connectors) | Debezium Postgres Source | 4.0.3.2 | pulsar-io-debezium-postgres-4.0.3.2.nar |
151+
| [kafka](https://pulsar.apache.org/docs/io-connectors) | Kafka source and sink connector | 4.0.3.2 | pulsar-io-kafka-4.0.3.2.nar |
152+
| [kinesis](https://pulsar.apache.org/docs/io-connectors) | Kinesis connectors | 4.0.3.2 | pulsar-io-kinesis-4.0.3.2.nar |
153+
</details>
154+
<details><summary>Proxy extensions</summary>
155+
156+
| Name | Description | Version | File |
157+
|----------------------------------------------------------------|----------------------------------------|----------|---------------------------------|
158+
| [kafka](https://github.com/datastax/starlight-for-kafka) | Kafka Proxy Extension | 4.0.3.0 | pulsar-kafka-proxy-4.0.3.0.nar |
159+
| [rabbitmq](https://github.com/datastax/starlight-for-rabbitmq) | Starlight for RabbitMQ Proxy Extension | 2.10.1.0 | starlight-rabbitmq-2.10.1.0.nar |
160+
</details>
161+
<details><summary>Protocol handlers</summary>
162+
163+
| Name | Description | Version | File |
164+
|----------------------------------------------------------------|----------------------------------------|----------|-------------------------------------------|
165+
| [rabbitmq](https://github.com/datastax/starlight-for-rabbitmq) | Starlight for RabbitMQ Proxy Extension | 2.10.1.0 | starlight-rabbitmq-2.10.1.0.nar |
166+
| [kafka](https://github.com/datastax/starlight-for-kafka) | Kafka Protocol Handler | 4.0.3.0 | pulsar-protocol-handler-kafka-4.0.3.0.nar |
167+
</details>
168+
<details><summary>CLI extensions</summary>
169+
170+
| Name | Description | Version | File |
171+
|---------------------------------------------------------------|--------------------------------------------------|---------|--------------------------------------|
172+
| [cassandra-cdc](https://pulsar.apache.org/docs/io-connectors) | Cassandra CDC - Pulsar Admin Custom Commands | 2.3.1 | pulsar-cassandra-admin-2.3.1-nar.nar |
173+
| [jms](https://pulsar.apache.org/docs/io-connectors) | Starlight for JMS - Pulsar Admin Custom Commands | 6.0.1 | pulsar-jms-admin-6.0.1-nar.nar |
174+
</details>
175+
<details><summary>Filters</summary>
176+
177+
| Name | Description | Version | File |
178+
|-----------------------------------------------------|-----------------------------------------------------|---------|--------------------------|
179+
| [jms](https://pulsar.apache.org/docs/io-connectors) | Starlight for JMS - support for server side filters | 6.0.1 | pulsar-jms-6.0.1-nar.nar |
180+
</details>
181+
<details><summary>Functions</summary>
182+
183+
| Name | Description | Version | File |
184+
|------------------------------------------------------------|-------------------------|---------|----------------------------------|
185+
| [ai-tools](https://pulsar.apache.org/docs/io-connectors) | Generative AI tools | 3.2.0 | pulsar-ai-tools-3.2.0.nar |
186+
| [transforms](https://pulsar.apache.org/docs/io-connectors) | Transformation function | 3.2.0 | pulsar-transformations-3.2.0.nar |
187+
</details>
188+
189+
190+
## Environment (Connectors, Protocol Handlers, Proxy Extensions, extra libraries)
191+
See the [environment variables](https://github.com/riptano/pulsar-distro/blob/ls40_3.2/build.json) used for the build.
192+
193+
194+
### Builtin connectors
195+
196+
- cassandra-enhanced-pulsar-sink-1.6.13-nar.nar
197+
- pulsar-cassandra-source-2.3.1.nar
198+
- pulsar-io-cloud-storage-3.2.2.nar
199+
- pulsar-io-debezium-mongodb-4.0.3.2.nar
200+
- pulsar-io-jdbc-sqlite-4.0.3.2.nar
201+
- pulsar-io-kinesis-4.0.3.2.nar
202+
- pulsar-io-data-generator-4.0.3.2.nar
203+
- pulsar-io-elastic-search-4.0.3.2.nar
204+
- pulsar-io-jdbc-clickhouse-4.0.3.2.nar
205+
- pulsar-io-debezium-oracle-4.0.3.2.nar
206+
- pulsar-io-jdbc-openmldb-4.0.3.2.nar
207+
- pulsar-io-jdbc-postgres-4.0.3.2.nar
208+
- pulsar-io-debezium-mssql-4.0.3.2.nar
209+
- pulsar-io-debezium-mysql-4.0.3.2.nar
210+
- pulsar-io-jdbc-mariadb-4.0.3.2.nar
211+
- pulsar-io-debezium-postgres-4.0.3.2.nar
212+
- pulsar-io-http-4.0.3.2.nar
213+
- pulsar-io-kafka-4.0.3.2.nar
214+
- pulsar-snowflake-connector-0.1.15.nar
215+
- pulsar-protocol-handler-kafka-4.0.3.2.nar
216+
- starlight-rabbitmq-2.10.1.0.nar
217+
- pulsar-kafka-proxy-4.0.3.2.nar
218+
- pulsar-jms-6.0.1-nar.nar

0 commit comments

Comments
 (0)