|
1 | 1 | --- |
2 | 2 | title: Kafka Consumer |
3 | 3 | description: Utility |
4 | | -status: new |
5 | 4 | --- |
6 | 5 |
|
7 | 6 | The Kafka Consumer utility transparently handles message deserialization, provides an intuitive developer experience, and integrates seamlessly with the rest of the Powertools for AWS Lambda ecosystem. |
@@ -38,7 +37,7 @@ The Kafka Consumer utility transparently handles message deserialization, provid |
38 | 37 | Lambda processes Kafka messages as discrete events rather than continuous streams, requiring a different approach to consumer development that Powertools for AWS helps standardize. |
39 | 38 |
|
40 | 39 | | Aspect | Traditional Kafka Consumers | Lambda Kafka Consumer | |
41 | | -|-----------------------|-------------------------------------|----------------------------------------------------------------| |
| 40 | +| --------------------- | ----------------------------------- | -------------------------------------------------------------- | |
42 | 41 | | **Model** | Pull-based (you poll for messages) | Push-based (Lambda invoked with messages) | |
43 | 42 | | **Scaling** | Manual scaling configuration | Automatic scaling to partition count | |
44 | 43 | | **State** | Long-running application with state | Stateless, ephemeral executions | |
@@ -201,15 +200,15 @@ The Kafka consumer utility supports multiple serialization formats to match your |
201 | 200 | === "Supported Formats" |
202 | 201 |
|
203 | 202 | | Format | Schema Type | Description | Required Parameters | |
204 | | - |----------------------|--------------|-----------------------------------|--------------------------------------| |
| 203 | + | -------------------- | ------------ | --------------------------------- | ------------------------------------ | |
205 | 204 | | **JSON** | `"JSON"` | Human-readable text format | None | |
206 | 205 | | **Avro** | `"AVRO"` | Compact binary format with schema | `value.schema` (Avro schema string) | |
207 | 206 | | **Protocol Buffers** | `"PROTOBUF"` | Efficient binary format | `value.schema` (Proto message class) | |
208 | 207 |
|
209 | 208 | === "Format Comparison" |
210 | 209 |
|
211 | 210 | | Feature | JSON | Avro | Protocol Buffers | |
212 | | - |-------------------------------|----------|----------------------|-------------------------| |
| 211 | + | ----------------------------- | -------- | -------------------- | ----------------------- | |
213 | 212 | | **Schema Definition** | Optional | Required JSON schema | Required Protobuf class | |
214 | 213 | | **Schema Evolution** | None | Strong support | Strong support | |
215 | 214 | | **Size Efficiency** | Low | High | Highest | |
@@ -241,7 +240,7 @@ For debugging purposes, you can also access the original key, value, and headers |
241 | 240 | #### Available metadata properties |
242 | 241 |
|
243 | 242 | | Property | Description | Example Use Case | |
244 | | -|-----------------------|------------------------------------------------------------------|---------------------------------------------------------------------| |
| 243 | +| --------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------- | |
245 | 244 | | `topic` | Topic name the record was published to | Routing logic in multi-topic consumers | |
246 | 245 | | `partition` | Kafka partition number | Tracking message distribution | |
247 | 246 | | `offset` | Position in the partition | De-duplication, exactly-once processing | |
@@ -304,7 +303,7 @@ Handle errors gracefully when processing Kafka messages to ensure your applicati |
304 | 303 | #### Error types |
305 | 304 |
|
306 | 305 | | Exception | Description | Common Causes | |
307 | | -|--------------------------------------|-----------------------------------------------|-----------------------------------------------------------------------------| |
| 306 | +| ------------------------------------ | --------------------------------------------- | --------------------------------------------------------------------------- | |
308 | 307 | | `KafkaConsumerError`. | Base class for all Kafka consumer errors | General unhandled errors | |
309 | 308 | | `KafkaConsumerDeserializationError` | Thrown when message deserialization fails | Corrupted message data, schema mismatch, or wrong schema type configuration | |
310 | 309 | | `KafkaConsumerMissingSchemaError` | Thrown when a required schema is not provided | Missing schema for AVRO or PROTOBUF formats (required parameter) | |
|
0 commit comments