You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
-44Lines changed: 0 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,50 +30,6 @@ Superstream Clients works as a Python import hook that intercepts Kafka producer
30
30
-**Intelligent optimization**: Identifies the most impactful topics to optimize
31
31
-**Graceful fallback**: Falls back to default settings if optimization fails
32
32
33
-
## Important: Producer Configuration Requirements
34
-
35
-
When initializing your Kafka producers, please ensure you pass the configuration as a mutable object. The Superstream library needs to modify the producer configuration to apply optimizations. The following initialization patterns are supported:
36
-
37
-
✅ **Supported (Recommended)**:
38
-
```python
39
-
# Using kafka-python
40
-
from kafka import KafkaProducer
41
-
producer = KafkaProducer(
42
-
bootstrap_servers=['localhost:9092'],
43
-
compression_type='snappy',
44
-
batch_size=16384
45
-
)
46
-
47
-
# Using aiokafka
48
-
from aiokafka import AIOKafkaProducer
49
-
producer = AIOKafkaProducer(
50
-
bootstrap_servers='localhost:9092',
51
-
compression_type='snappy',
52
-
batch_size=16384
53
-
)
54
-
55
-
# Using confluent-kafka
56
-
from confluent_kafka import Producer
57
-
producer = Producer({
58
-
'bootstrap.servers': 'localhost:9092',
59
-
'compression.type': 'snappy',
60
-
'batch.size': 16384
61
-
})
62
-
```
63
-
64
-
❌ **Not Supported**:
65
-
```python
66
-
# Using frozen dictionaries or immutable configurations
67
-
from types import MappingProxyType
68
-
config = MappingProxyType({
69
-
'bootstrap.servers': 'localhost:9092'
70
-
})
71
-
producer = KafkaProducer(**config)
72
-
```
73
-
74
-
### Why This Matters
75
-
The Superstream library needs to modify your producer's configuration to apply optimizations based on your cluster's characteristics. This includes adjusting settings like compression, batch size, and other performance parameters. When the configuration is immutable, these optimizations cannot be applied.
0 commit comments