v1.4.1

Changelog

[1.4.1] (2026-02-27)

Improvements

• Official Docker Image: Introduced official Docker images published to GitHub Container Registry (ghcr.io/lightstreamer/lightstreamer-kafka-connector), with automated builds via GitHub Actions on each release. Moved Docker resources from examples/docker to the new /docker folder with production-ready build scripts and Dockerfile. (#77)

• Absolute Path Support: Extended file path configuration parameters (such as logging.configuration.path, encryption.truststore.path, encryption.keystore.path, authentication.gssapi.key.tab.path, and record.*.evaluator.schema.path) to also accept absolute paths, in addition to paths relative to the deployment folder. Updated the factory adapters.xml and README.md files accordingly. (#79)

Examples and Documentation

• Added comprehensive docker/README.md with Quick Start guide, configuration examples, and instructions for pulling published images and building locally. (#77)

• Updated examples/vendors/confluent/README.md: Restructured the "Docker-based Deployment" section to prioritize the official Docker image from GitHub Container Registry. (#77)

• Updated all examples to use the new official Docker image. (#77)

v1.4.0

Changelog

[1.4.0] (2026-02-18)

Improvements

• Polling Lifecycle Optimization: Significantly enhanced the polling lifecycle to improve throughput and minimize record lag. Introduced an adaptive commit strategy that dynamically adjusts commit frequency based on message rate. This optimization also removes previous constraints that prevented concurrent processing when using compacted topics. (#76)

• Monitor Logger: Added a dedicated logger to track essential performance metrics including records received, records processed, and internal ring buffer utilization rates. (#76)

• Configurable Max Poll Records: Exposed the record.consume.max.poll.records parameter to configure the maximum number of records fetched in each polling cycle of the internal Kafka consumer. (#76)

• Microbenchmarks Consolidation: Refactored and consolidated the internal microbenchmarking suite to improve performance testing capabilities. (#76)

Examples and Documentation

• Significantly reorganized the structure of examples as follows:

  • Split Confluent QuickStart into separate quickstart-confluent-cloud/ and quickstart-confluent-platform/ directories. (#74)
  • Renamed quickstart-redpanda-self-hosted to quickstart-redpanda-self-managed. (#74)
  • Removed unused web client assets and QuickStartConfluentCloud configuration from factory files. (#74)
  • Updated internal links to reflect new folder locations. (#74)

• Updated README.md: Removed vendor-specific connection instructions (Confluent Cloud, Redpanda Cloud) and moved them to dedicated quickstarts. Expanded broker list with links to all vendor quickstarts. (#74)

• Updated examples/vendors/confluent/README.md: Improved section organization and clarity. (#74)

• Clarified adapters.xml descriptions in various QuickStart READMEs. (#74)

Third-Party Library Updates

• Upgraded the confluent-kafka dependency to version 8.1.1-ccs. (#76)

Bug Fixes

• Fixed outdated links to official Kafka documentation in the README.md, examples/vendors/confluent/README.md, and factory adapters.xml files. (#76)

• Fixed heading hierarchy, TOC indentation, typos, and trailing spaces in the README.md and examples/vendors/confluent/README.md files. (#74)

v1.3.2

Changelog

[1.3.2] (2026-01-26)

Improvements

• Deferred Deserialization: Refactored the deserialization pipeline to defer the conversion of raw bytes to deserialized objects. This architectural change makes the Kafka Connector open for further extension by allowing custom implementations to intercept and access raw bytes from Kafka records before or instead of automatic deserialization. (#75)

• Auto COMMAND Mode: Added support for the Auto COMMAND mode feature that generates command operations for Lightstreamer items without requiring explicit command fields in the Kafka records. (#75)

• Consolidated the script for running microbenchmarks. (#75)

• Upgraded Gradle to version 9.3.0. (#75)

Bug Fixes

• Null Value Deserialization: Fixed incorrect handling of null values during deserialization when using ProtoBuf, JSON, and Avro formats. The deserializers now properly process null payloads without triggering unexpected errors or data loss. (#75)

v1.3.1

Changelog

[1.3.1] (2025-12-19)

Improvements

• Data Extraction Language Enhancement and Dynamic Field Discovery: Extended the Data Extraction Language to support wildcard expressions (e.g., #{VALUE.*}, #{KEY.*}, #{HEADERS.*}), enabling the new Dynamic Field Discovery mechanism with the field.* configuration parameter. This allows automatic mapping of Kafka record fields to Lightstreamer fields at runtime, eliminating the need to explicitly configure each field individually – especially useful for records with numerous or dynamically varying fields (also available for the Sink connector). (#72)

• Upgraded Gradle to version 9.2.1. (#72)

Third-Party Library Updates

• Upgraded JUnit to version 6.0.1. (#72)
• Upgraded Truth to version 1.4.5. (#72)

Bug Fixes

• Fixed the examples/quickstart-schema-registry/README.md file to mention usage of Protobuf. (#72)

• Corrected broken anchor links in the Table of Contents of the README.md file. (#72)

v1.3.0-auto-field-mapping.1

Update javadoc resources

v1.3.0

Changelog

[1.3.0] (2025-10-21)

Breaking Changes

• KafkaConnectorMetadataAdapter Class Hierarchy Change: The com.lightstreamer.kafka.adapters.pub.KafkaConnectorMetadataAdapter class now extends com.lightstreamer.adapters.metadata.MetadataProviderAdapter directly instead of com.lightstreamer.adapters.metadata.LiteralBasedProvider. This change affects any custom implementations that relied on com.lightstreamer.adapters.metadata.LiteralBasedProvider-specific functionality. Additionally, a new remapItems method has been introduced that provides a hook for custom item resolution logic. Custom metadata adapters extending com.lightstreamer.kafka.adapters.pub.KafkaConnectorMetadataAdapter should override the new remapItems method to provide custom item resolution logic, rather than overriding getItems directly. Other methods that were previously provided by com.lightstreamer.adapters.metadata.LiteralBasedProvider, such as getSchema and other metadata resolution methods, may also need to be implemented. (#71)

Improvements

• Core Algorithm Optimizations: Enhanced overall system performance through optimizations in data processing, record mapping, and routing logic. These improvements result in faster record processing and reduced resource consumption during high-throughput scenarios. (#71)

• Data Access Pattern Optimization: Optimized record routing algorithm to use more efficient data access patterns, improving throughput when delivering messages to multiple subscribed clients. (#71)

• Comprehensive Benchmarking Suite: Added comprehensive performance benchmarking infrastructure to enable systematic performance monitoring and optimization of core components including data extraction, record mapping, and expression evaluation. (#71)

• Improved Logging: Refined logging levels to reduce verbosity in production environments while maintaining debugging capabilities when needed. (#71)

Bug Fixes

• Consumer Shutdown Robustness: Enhanced error handling during Kafka consumer shutdown to prevent hanging or incomplete disconnections. (#71)

• Configuration Cleanup: Removed group.id assignment from the factory adapters.xml file. (#71)

• Protobuf Deserialization: Improved robustness of Protobuf message deserialization by ensuring that the specified message type exists in the provided schema file. (#70)

Code Quality Improvements

• Import Organization: Cleaned up unused imports across multiple files. (#71)

• Documentation Enhancement: Improved inline documentation and method signatures. (#71)

• Code Structure: Removed commented-out code and unnecessary blank lines for cleaner codebase. (#71)

v1.2.10

Changelog

[1.2.10] (2025-09-18)

Improvements

• Added the Gradle-License-Report plugin to the project to automatically generate and include third-party license files in the distribution packages (including the Sink connector). (#68)

Bug Fixes

• Fixed examples/compose-templates/log4j.properties file to explicitly set the logging level and output for the KafkaConnectorMetadataAdapter class. (#67)

• Fixed regex pattern for semantic version matching in the com.lightstreamer.kafka.test_utils.VersionUtils class to ensure accurate handling of all version formats in unit tests. (#69)

v1.2.9

Changelog

[1.2.9] (2025-08-25)

Improvements

• Added a quickstart demo for AutoMQ, a cloud-native, S3-based Kafka distribution. (#62)

• Introduced support for connection inversion in the Sink connector, allowing it to accept incoming connections from the Lightstreamer Broker. (#65)

• Upgraded Gradle to version 9.0.0. (#63)

• Clarified the general format of map.TOPIC_NAME.to parameter in the factory adapters.xml. (#61)

Bug Fixes

• Corrected minor documentation issues in README.md files and improved code formatting in various source files. (#66)

• Added missing documentation for trust store and key store types in README.md and the factory adapters.xml files. (#64)

• Corrected the IAM role parameter examples in the factory adapters.xml file. (#60)

v1.3.0-alpha-3

Update Javadoc for kafka-connector to version 1.3.0-alpha-3

v1.2.8

Changelog

[1.2.8] (2025-07-10)

Improvements

• Extended the Data Extraction Language to support Headers. (#57)

• Enhanced Protobuf support by allowing to specify local binary descriptor files and a message type as an alternative to using the Schema Registry. (#59)

Bug Fixes

• Fixed unexpected behaviors that occurred when processing Kafka records with null payloads. (#58)