Skip to content

Commit a828275

Browse files
autodidacticonrtyler
authored andcommitted
Update README.adoc
Updated to get examples working
1 parent 39fa51e commit a828275

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

README.adoc

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,15 +27,18 @@ The included docker-compose.yml contains link:https://github.com/wurstmeister/ka
2727
==== Starting Worker Processes
2828

2929
1. Launch test services - `docker-compose up setup`
30-
2. Compile: `cargo build`
31-
3. Run kafka-delta-ingest against the web_requests example topic and table (customize arguments as desired):
30+
1. Download and extract kafka: `curl -L https://dlcdn.apache.org/kafka/3.9.0/kafka_2.13-3.9.0.tgz | tar -xz`
31+
1. Create kafka topic: `./kafka_2.13-3.9.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic web_requests --create --if-not-exists`
32+
1. Ingest test messages into kafka: `./kafka_2.13-3.9.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic web_requests < tests/json/web_requests-100.json`
33+
1. Compile: `cargo build --features s3`, MacOS: `brew install librdkafka && cargo build --features s3,dynamic-linking`
34+
1. Run kafka-delta-ingest against the web_requests example topic and table (customize arguments as desired):
3235

3336
```bash
3437
export AWS_ENDPOINT_URL=http://0.0.0.0:4566
3538
export AWS_ACCESS_KEY_ID=test
3639
export AWS_SECRET_ACCESS_KEY=test
3740

38-
RUST_LOG=debug cargo run ingest web_requests ./tests/data/web_requests \
41+
RUST_LOG=debug cargo run --features s3,dynamic-linking ingest web_requests ./tests/data/web_requests \
3942
--allowed_latency 60 \
4043
--app_id web_requests \
4144
--transform 'date: substr(meta.producer.timestamp, `0`, `10`)' \

0 commit comments

Comments
 (0)