Iggy Connector Ecosystem Tracking

[amuldotexe]

A key ingredient for adoption for Iggy will be relevant connector coverage.

This is still a draft, but I want it to be useful as a live tracker before someone picks up new work.

Last updated: March 8, 2026

Connector Ecosystem Roadmap: tracking connector work, gaps, and active PRs

Overview

Apache Iggy's connector ecosystem is growing. The main purpose of this discussion is to help contributors avoid duplicate work and find the next useful thing to pick up.

The live status in the next two sections should be trusted first. The long target map below is the broader draft.

Shipped today

Connector	Type	Reference	Notes
PostgreSQL	Sink + Source (CDC)	#1849, #2579	Native Rust. Integration coverage expanded in #2579.
Elasticsearch	Sink + Source	#1851, #1872, #2592, #2593	Native Rust. Sink and source e2e coverage landed.
Apache Iceberg	Sink	#1850, #2191, #2595	Native Rust.
Quickwit	Sink	#2594	Native Rust. E2E coverage landed.
Stdout	Sink	--	Reference implementation for connector developers.
Apache Pinot	Sink	#2499, #2598	External Java connector exists. Proper Java integration tests are still open in #2598.

Active work right now

Please check both open issues and open PRs before starting something new. In a few cases, the issue is open and there is already active PR work behind it.

Area	Issue	Current signal
ClickHouse sink	#2539	Two open PRs: #2886 and #2891
InfluxDB sink + source	#2700	Open PR: #2801
MongoDB sink + source	#2739	Open sink PR: #2815. Source side is not merged yet.
Delta Lake connectors	#1852	Open sink PR: #2889
JDBC / MySQL sink + source	#2500	Open issue
Amazon Redshift sink	#2540	Open issue. Earlier PR #2557 was closed.
Avro codec	#1846	Open issue
BSON codec	#1847	Open issue
Flink transport	#2484	Open issue. Earlier PR #2560 was closed.
Restart connector with new config	#2417	Open PR: #2781
Pinot connector e2e tests	#2598	Open issue
Pareto source connector test suite	#2892	Open issue
Pareto sink connector test suite	#2893	Open issue

---

1. Relational Databases

Target	Type	Priority	Comp	Why It Matters	Rust Crate
MySQL / MariaDB	Sink+Source	P1	4/4	Most popular RDBMS globally. Covered by planned JDBC connector #2500. Every web application's default database.	sqlx (54.9M downloads) or mysql_async
Microsoft SQL Server	Sink+Source	P2	3/4	Enterprise RDBMS. Required for Fortune 500 adoption. Debezium CDC in Kafka ecosystem.	tiberius (native TDS protocol)
Oracle	Sink+Source	P3	3/4	Complex licensing (GoldenGate CDC). Enterprise-heavy. Required for legacy enterprise migration but licensing costs limit adoption.	No mature Rust crate; JDBC bridge or C OCI FFI
SQLite	Sink	P4	2/4	Embedded database. Niche use case for local/edge analytics sinks.	rusqlite
CockroachDB	Sink+Source	--	2/4	PostgreSQL wire protocol compatible. Works via existing Postgres connector automatically.	Existing PG connector
IBM Db2	Sink+Source	P4	1/4	Enterprise/mainframe only. Very niche.	JDBC bridge
Generic JDBC	Sink+Source	P1	3/4	Planned as #2500. Covers MySQL, MariaDB, and dozens of other databases through a single connector.	sqlx

2. NoSQL / Document Databases

Target	Type	Priority	Comp	Why It Matters	Rust Crate
MongoDB	Sink+Source	P0	4/4	Planned #2739. 28k GitHub stars, 50k customers. Most popular NoSQL DB. Change Streams enable real-time CDC without WAL complexity. Essential for IoT device metadata and telemetry storage.	mongodb (official, 9.5M downloads)
Cassandra	Sink	P2	3/4	9.6k stars. Wide-column store optimized for high-write workloads. Bloomberg serves 20B+ requests/day on 1,700+ nodes. Used by Netflix, Apple. Essential for time-series and IoT at massive scale.	scylla (Cassandra-compatible, pure Rust)
DynamoDB	Sink+Source	P2	3/4	AWS-native NoSQL. Handles 10T+ requests/day. DynamoDB Streams for CDC source. Required for AWS-centric architectures. Used by Lyft, Airbnb, Samsung.	aws-sdk-dynamodb (official AWS SDK)
Couchbase	Sink+Source	P3	2/4	Multi-model NoSQL (document + KV + search). Used in mobile/edge sync. Less common than MongoDB.	No mature Rust crate; REST API via reqwest
HBase	Sink	P3	2/4	Hadoop ecosystem wide-column store. Declining as Cassandra and cloud-native alternatives grow.	No Rust crate; Thrift/REST API
Azure Cosmos DB	Sink+Source	P3	1/4	Azure-native multi-model database. MongoDB API compatible, so MongoDB connector partially covers it.	azure_data_cosmos
ScyllaDB	Sink	P3	1/4	Cassandra-compatible but written in C++ for 10x performance. Growing adoption for low-latency workloads. Covered by Cassandra connector.	scylla (official Rust driver)
Apache CouchDB	Sink+Source	P4	1/4	Document DB with built-in replication. Niche.	reqwest (CouchDB REST API)

3. Search & Analytics Engines

Target	Type	Priority	Comp	Why It Matters	Rust Crate
OpenSearch	Sink	P1	2/4	12.4k GitHub stars. Elasticsearch fork, AWS-aligned. Linux Foundation project. May work as thin adapter over existing ES connector with compatibility layer.	opensearch (official Rust client)
Apache Solr	Sink	P3	2/4	Legacy search engine. Declining as Elasticsearch/OpenSearch dominate. Only for existing Solr shops.	reqwest (Solr REST API)
Splunk	Sink	P2	3/4	15k customers. 89 of Fortune 100. Acquired by Cisco for $28B. HTTP Event Collector (HEC) API makes integration straightforward. Enterprise observability standard.	reqwest (HEC is HTTP POST)
Meilisearch	Sink	P4	0/4	48k GitHub stars. Lightweight search engine with great DX. Growing alternative to Elasticsearch for smaller deployments.	meilisearch-sdk
Typesense	Sink	P4	0/4	22k GitHub stars. Another lightweight search alternative. REST API.	reqwest (Typesense REST API)

4. Cloud Object Storage

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Amazon S3	Sink+Source	P0	4/4	Universal data lake landing zone. 350T+ objects stored, 100M+ req/sec. 90%+ Fortune 100 via AWS. Enables Snowflake/Redshift S3 staging. Build with abstract storage interface so GCS/Azure/MinIO become thin adapters.	aws-sdk-s3 (official AWS SDK, v1.123)
Google Cloud Storage	Sink+Source	P1	3/4	GCP object storage. Thin adapter over S3 connector architecture. GCP holds ~13% cloud market.	cloud-storage or google-cloud-storage
Azure Blob Storage	Sink+Source	P1	3/4	Azure object storage. Thin adapter over S3 connector architecture. Azure holds ~20% cloud market.	azure_storage_blobs
MinIO	Sink+Source	P1	1/4	S3-compatible, self-hosted. 50k+ GitHub stars. Works via S3 connector automatically since it implements S3 API.	aws-sdk-s3 (S3-compatible)
DigitalOcean Spaces	Sink	P4	0/4	S3-compatible. Covered by S3 connector.	aws-sdk-s3 (S3-compatible)
Backblaze B2	Sink	P4	0/4	S3-compatible. Covered by S3 connector.	aws-sdk-s3 (S3-compatible)

5. Data Lakehouse Formats

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Delta Lake	Sink+Source	P2	2/4	Planned #1852. 8.6k GitHub stars. Databricks/Linux Foundation. ACID transactions on data lakes. Growing fast with Databricks adoption.	deltalake (Rust-native, delta-rs project)
Apache Hudi	Sink	P3	1/4	Uber's lakehouse format. 5.5k stars. Strong in incremental processing and CDC use cases. Less popular than Iceberg/Delta.	hudi (Rust-native, apache/hudi-rs)
Apache Paimon	Sink	P4	0/4	Flink-native lakehouse format. Emerging. Formerly Flink Table Store. Optimized for streaming writes.	No Rust crate; Java only

6. Data Warehouses

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Snowflake	Sink	P1	3/4	Leading cloud DW. 12k+ customers, $1.09B product revenue (Q2 FY26). Snowpipe Streaming API for real-time ingestion. 3/4 competitors have it.	reqwest (Snowpipe REST API); or S3 staging
ClickHouse	Sink	P1	2/4	Planned #2539. 45k GitHub stars. Real-time OLAP. $400M Series D (Jan 2026). Native HTTP interface. Popular in Rust community.	clickhouse; klickhouse (native protocol)
Amazon Redshift	Sink	P2	2/4	Planned #2540. AWS cloud DW. S3 staging + COPY pattern for efficient bulk loading.	aws-sdk-s3 + sqlx (PG wire protocol)
Google BigQuery	Sink	P2	3/4	GCP flagship analytics warehouse. 13.6% DW market share. 13k+ companies. Storage Write API for streaming.	gcp-bigquery-client
Databricks	Sink	P2	2/4	Via Delta Lake connector or SQL API. Databricks SQL Connector available. Spark ecosystem integration.	deltalake crate or REST API
Azure Synapse	Sink	P3	1/4	Microsoft cloud DW. Competes with Snowflake on Azure.	tiberius (TDS/SQL Server protocol)
StarRocks	Sink	P3	1/4	Real-time OLAP. Fast-growing fork of Apache Doris. Strong benchmark performance for complex JOINs.	reqwest (Stream Load HTTP API)
Apache Doris	Sink	P3	1/4	StarRocks parent project. 13k+ GitHub stars. Strong in China/Asia-Pacific.	reqwest (Stream Load HTTP API)
Firebolt	Sink	P4	0/4	Niche cloud DW for extreme query speed. Startup.	REST API via reqwest
DuckDB	Sink	P3	0/4	28k+ GitHub stars. Embedded OLAP. Huge Rust/data community adoption. "SQLite for analytics."	duckdb (official Rust bindings)

7. Time-Series Databases

Target	Type	Priority	Comp	Why It Matters	Rust Crate
InfluxDB	Sink+Source	P0	2/4	Planned #2700. 31k GitHub stars. #1 time-series DB by community. IoT standard. Line protocol write API is universal. InfluxDB 3 is built in Rust.	reqwest (HTTP API; no mature Rust client)
TimescaleDB	Sink+Source	--	2/4	PostgreSQL extension. Works via existing Postgres connector automatically. No additional work needed.	Existing PG connector
Prometheus	Sink	P2	1/4	63k GitHub stars. CNCF graduated. Industry-standard monitoring. remote_write API (protobuf/snappy over HTTP) for pushing metrics from Iggy.	prost + reqwest
QuestDB	Sink	P3	1/4	15k GitHub stars. Fast time-series DB. InfluxDB Line Protocol compatible, so InfluxDB connector could cover it.	InfluxDB Line Protocol or PG wire protocol
TDengine	Sink	P3	0/4	IoT-focused TSDB. Popular in Asia-Pacific. REST API available.	reqwest (REST API)
VictoriaMetrics	Sink	P3	0/4	13k+ GitHub stars. Prometheus-compatible. Covered by Prometheus remote_write connector.	Same as Prometheus (remote_write)
CrateDB	Sink	P4	0/4	SQL time-series DB. PostgreSQL wire protocol compatible. Works via PG connector.	Existing PG connector or sqlx

8. Graph Databases

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Neo4j	Sink	P3	3/4	$200M+ ARR. 84% Fortune 100. 44% graph DB market. Stream events into graph for relationship analysis, fraud detection, recommendation engines.	neo4rs (async Bolt protocol client)
ArangoDB	Sink	P4	0/4	Multi-model (graph + document + KV). 14k GitHub stars. Niche but versatile.	arangors
Amazon Neptune	Sink	P4	0/4	AWS-managed graph DB. Gremlin and SPARQL APIs.	reqwest (Neptune REST API)
JanusGraph	Sink	P4	0/4	Open-source, Hadoop-backed graph DB. Declining adoption.	Gremlin protocol via reqwest

9. Message Queues & Streaming Bridges

Target	Type	Priority	Comp	Why It Matters	Rust Crate
MQTT	Source+Sink	P0	3/4	THE IoT protocol. De facto standard with 50%+ enterprise IoT adoption. Billions of connected devices. Iggy's "runs on Raspberry Pi" story is incomplete without MQTT. The single most important missing connector.	rumqttc (237K downloads/month, async, MQTT v5)
RabbitMQ / AMQP	Source+Sink	P1	3/4	13k GitHub stars. Most popular traditional message broker. Migration path from legacy MQ to Iggy. AMQP 0.9.1 protocol.	lapin (async AMQP 0.9.1)
Apache Kafka	Source+Sink	P2	n/a	32k GitHub stars. 80%+ Fortune 100. Bridge connector for gradual Kafka-to-Iggy migration. Enables hybrid architectures during migration.	rdkafka (18.6M sys downloads)
Amazon Kinesis	Source+Sink	P2	3/4	AWS streaming service. Cross-cloud bridge for AWS-centric organizations moving to Iggy.	aws-sdk-kinesis (official AWS SDK)
Amazon SQS	Source+Sink	P3	2/4	AWS queue service. Simple point-to-point messaging. Bridge for SQS-to-Iggy migration.	aws-sdk-sqs
Google Pub/Sub	Source+Sink	P2	2/4	GCP streaming/messaging service. Cross-cloud bridge for GCP organizations.	google-cloud-pubsub
Apache Pulsar	Source+Sink	P3	1/4	Alternative streaming platform. Bridge connector for Pulsar-to-Iggy migration.	pulsar (async Rust client)
NATS / JetStream	Source+Sink	P3	1/4	Lightweight messaging. Growing in cloud-native/K8s. Bridge for NATS-to-Iggy migration.	async-nats (official async client)
NSQ	Source+Sink	P4	2/4	Simple distributed queue by Bitly. Declining.	No mature Rust crate
IBM MQ	Source+Sink	P4	1/4	Enterprise mainframe messaging. Only for large financial institutions with IBM infrastructure.	No native Rust crate; JMS bridge
Azure Service Bus	Source+Sink	P3	1/4	Azure messaging service. AMQP 1.0 protocol.	azure_messaging_servicebus; or fe2o3-amqp
Azure Event Hubs	Source	P3	1/4	Azure Kafka-compatible event streaming. Since it supports Kafka wire protocol, the Kafka bridge connector covers it.	rdkafka (Kafka-compatible protocol)

10. Caches & Key-Value Stores

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Redis	Sink+Source	P0	3/4	67k+ GitHub stars. Most popular KV store. Cache warming from streams, real-time state management, pub/sub bridge. 4B+ Docker pulls. 10k enterprise customers.	redis (official); fred (high-level async)
DragonflyDB	Sink+Source	P3	0/4	29.4k GitHub stars. Redis-compatible with 25x throughput. Covered by Redis connector automatically.	redis crate (Redis-compatible API)
Memcached	Sink	P4	1/4	Legacy cache. Declining in favor of Redis. Still used by Facebook, Wikipedia.	memcache
Aerospike	Sink	P4	2/4	High-performance KV for adtech, fraud detection. $1B+ valuation. Niche.	aerospike
Hazelcast	Source	P4	2/4	In-memory data grid. Java-centric. Low priority.	REST API via reqwest
etcd	Sink+Source	P4	0/4	48k+ GitHub stars. CNCF graduated. Kubernetes config store. Streaming config changes for K8s auditing.	etcd-client (async gRPC)

11. File Systems & Formats

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Local Filesystem	Sink+Source	P3	2/4	File tail source (like Filebeat) and file write sink. Basic but essential for log ingestion and data export.	tokio::fs; notify (file watcher)
FTP / SFTP	Sink+Source	P3	3/4	Legacy file transfer. Still required in regulated industries (healthcare, banking).	suppaftp; russh-sftp
HDFS	Sink	P3	3/4	Declining but still in legacy Hadoop shops. Being replaced by S3/GCS.	hdfs-native
Parquet (format)	Output fmt	P1	3/4	De facto columnar storage format. Essential for S3/GCS/HDFS sinks. Without Parquet, streaming data to object storage is inefficient.	parquet (Apache Arrow, very mature)
Avro (codec)	Codec	P0	4/4	Planned #1846. Kafka ecosystem standard. Without Avro, interop with Kafka-migrating users is blocked. Enables schema evolution.	apache-avro (official)
BSON (codec)	Codec	P2	0/4	Planned #1847. MongoDB native format. Enables zero-transformation pipelines between Iggy and MongoDB.	bson (official, 14.2M downloads)
ORC (format)	Output fmt	P4	1/4	Hadoop-specific columnar format. Declining as Parquet dominates.	No mature Rust crate
CSV (format)	Input/Output	P3	2/4	Universal data exchange. Human-readable. Batch imports/exports.	csv (BurntSushi's, very popular)
MessagePack (codec)	Codec	P4	1/4	Compact binary. More efficient than JSON. Useful for bandwidth-constrained IoT.	rmp-serde

12. HTTP & API Protocols

Target	Type	Priority	Comp	Why It Matters	Rust Crate
HTTP / Webhook	Source+Sink	P0	4/4	Universal integration. HTTP source receives webhooks from any service; HTTP sink pushes to any API. The "connect to anything" fallback. Every web service speaks HTTP.	reqwest (client); axum (server)
gRPC	Source+Sink	P1	1/4	44k GitHub stars. High-performance RPC for microservices. Pairs with OTLP (which uses gRPC). Essential for microservice architectures.	tonic (gRPC); prost (protobuf)
GraphQL	Sink	P4	0/4	Niche for streaming. GraphQL subscriptions could make Iggy a data source for GraphQL APIs. Specialized use case.	graphql_client
WebSocket	Source+Sink	P3	1/4	Real-time bidirectional. Ingest trading data, chat, live updates. Push events to browser dashboards.	tokio-tungstenite

13. Observability & Monitoring

Target	Type	Priority	Comp	Why It Matters	Rust Crate
OpenTelemetry (OTLP)	Source+Sink	P0	1/4	2nd largest CNCF project. 10k contributors. 1,200 companies. OTLP becoming universal observability standard. Makes Iggy an observability pipeline backbone for logs, traces, and metrics.	opentelemetry; opentelemetry-otlp
Splunk (HEC)	Sink	P2	3/4	46.8% SIEM market. 18k+ companies. Cisco-owned ($28B acquisition). HTTP Event Collector is simple HTTP POST.	reqwest
Datadog	Sink	P3	2/4	$827M Q2 2025 revenue. 47k+ customers. SaaS observability leader. HTTP API for metrics, logs, traces.	reqwest
Prometheus remote_write	Sink	P2	1/4	63k GitHub stars. De facto monitoring standard. One connector covers the entire Prometheus ecosystem (VictoriaMetrics, Cortex, Thanos, Mimir).	prost + reqwest (protobuf/snappy over HTTP)
Grafana Loki	Sink	P2	1/4	28k GitHub stars. 100k+ active clusters. 1B+ Docker pulls. Log aggregation that pairs with Grafana. HTTP push API.	reqwest
Jaeger	Sink	P3	0/4	22k+ GitHub stars. CNCF graduated. Being replaced by OTLP as ingestion standard, so OTLP connector covers it.	opentelemetry-otlp (Jaeger now prefers OTLP)
New Relic	Sink	P4	1/4	552k+ companies. Supports OTLP, so the OTLP connector covers it.	reqwest or OTLP connector

14. AI/ML & Vector Databases

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Qdrant	Sink	P3	0/4	27k+ GitHub stars. Rust-native vector DB. Natural ecosystem fit for Iggy. Stream embeddings for real-time semantic search, RAG, recommendations.	qdrant-client (official gRPC client)
Weaviate	Sink	P3	0/4	11k GitHub stars. 1M+ pulls/month. Best-in-class hybrid search. REST API.	reqwest (REST API)
Pinecone	Sink	P3	0/4	Leading SaaS vector DB. Simplest integration (REST API). Used by Gong, Vanguard.	pinecone-sdk or reqwest
Milvus	Sink	P3	0/4	40k+ GitHub stars. Most popular OSS vector DB. Used by NVIDIA, Salesforce, eBay. 72% memory reduction in v2.6.	milvus-sdk; or gRPC via tonic
MLflow	Sink	P4	0/4	23k GitHub stars. 30M+ monthly downloads. Most popular MLOps platform. Stream training metrics.	reqwest (REST API)
Feast	Source	P4	0/4	5k+ GitHub stars. Feature store. Linux Foundation. ML feature serving from Iggy.	reqwest (REST API)

15. SaaS & Enterprise

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Salesforce	Source+Sink	P4	1/4	#1 CRM. $34.9B revenue. 150k+ customers. Stream CRM events. Enterprise-heavy.	reqwest (REST/Bulk API)
ServiceNow	Source+Sink	P4	1/4	Leading ITSM. $10B+ revenue. 85% Fortune 500. Stream incident events.	reqwest (REST API)
Slack	Sink	P4	0/4	65M+ daily active users. Alert notifications from Iggy streams to channels. Simple webhook.	reqwest (Webhook HTTP POST)
Email (SMTP)	Sink	P4	1/4	Universal alert notifications. Stream events matching conditions as email alerts.	lettre
Stripe	Source	P4	0/4	Leading payment processor. $1T+ processed. Ingest payment events for real-time financial analytics.	reqwest (webhooks)
Twilio	Sink	P4	0/4	Leading CPaaS. 300k+ accounts. SMS alerts for IoT scenarios (equipment failure notifications).	reqwest (REST API)

16. Stream Processing Integration

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Apache Flink	Source+Sink	P1	2/4	26k GitHub stars. Leading stream processor. Used by Alibaba, Apple, Netflix, Uber. Needs Flink connector JAR. Enables complex event processing, windowing, joins. Already has #2484 for transport.	Java connector JAR; Iggy Kafka compat could help
Apache Spark Streaming	Source	P3	2/4	41k GitHub stars. Most popular data processing framework. Structured Streaming for batch+stream.	Java/Scala connector; or Kafka compat layer
Apache Beam	Source+Sink	P3	1/4	8k+ GitHub stars. Google-backed. Unified batch+stream. Runs on Flink, Spark, Dataflow.	Java SDK connector
Materialize	Source	P4	0/4	Streaming SQL DB. PostgreSQL-compatible. Incremental materialized views.	PG wire protocol or Kafka compat
RisingWave	Source	P4	0/4	8.6k GitHub stars. Rust-based streaming DB. Natural ecosystem partner. PG-compatible.	Native Rust integration or PG/Kafka compat
Apache Camel	Framework	P2	n/a	6.1k GitHub stars. 193+ connectors (Kamelets). An Iggy Camel component would give access to hundreds of systems. Requires JVM.	Java component (Camel is JVM-based)

17. Workflow & Orchestration

Target	Type	Priority	Comp	Why It Matters	Rust Crate
Apache Airflow	Trigger	P4	1/4	44k GitHub stars. Industry standard workflow orchestration. Iggy sensor/trigger for event-driven DAGs.	reqwest (REST API)
Temporal	Source+Sink	P4	0/4	12k+ GitHub stars. Durable execution platform. Bridge between streams and fault-tolerant workflows.	temporal-sdk-core (official Rust SDK)
Prefect	Trigger	P4	0/4	20k GitHub stars. 25k+ community. Modern Airflow alternative. Event-driven workflow triggering.	reqwest (REST API / webhooks)
Dagster	Source	P4	0/4	14k GitHub stars. Asset-based data orchestration. Growing adoption.	reqwest (REST API)

18. IoT Protocols

Target	Type	Priority	Comp	Why It Matters	Rust Crate
MQTT	Source+Sink	P0	3/4	(See #9.) De facto IoT standard. 50%+ enterprise adoption. $17.6B IoT protocol market. Billions of devices. The single most important missing connector.	rumqttc
CoAP	Source+Sink	P3	0/4	IETF RFC 7252. UDP-based, 4-byte header. For ultra-constrained sensors where MQTT is too heavy. Healthcare, robotics, smart cities.	coap; coap-lite
OPC-UA	Source	P2	1/4	IEC 62541 standard. 40%+ market share in industrial data acquisition. 1,000+ manufacturers. Industry 4.0 standard. Bridges factory floor into streaming.	opcua (pure Rust client/server)
Modbus	Source	P3	1/4	40%+ market share in industrial data. Hundreds of millions of devices. 45+ years old. Legacy factory equipment integration.	tokio-modbus (async RTU/TCP)
LwM2M	Source	P4	0/4	IoT device management protocol on CoAP. 400+ ready-to-use objects. Smart metering, smart cities. Specialized.	coap crate (LwM2M builds on CoAP)

---

Codec / Serialization Format Status

Format	Status	Issue	Notes
JSON	Done	--	Built-in to connector runtime
Protobuf	Done	--	Built-in to connector runtime
FlatBuffers	Done	--	Unique to Iggy -- zero-copy for IoT
Raw / Bytes	Done	--	Built-in
Text / String	Done	--	Built-in
Avro	In progress	#1846	Critical for Kafka ecosystem interop. apache-avro crate
BSON	In progress	#1847	MongoDB native format. bson crate (14.2M downloads)
Parquet	Proposed	--	Output format for S3/GCS/HDFS sinks. parquet crate
CSV	Proposed	--	Human-readable batch import/export. csv crate
MessagePack	Proposed	--	Compact binary for IoT. rmp-serde crate

---

Summary Counts

Category	Total	Done	In Progress	Proposed
Relational Databases	7	1 (PG)	1 (JDBC)	5
NoSQL / Document	8	0	1 (MongoDB)	7
Search & Analytics	5	3 (ES, Quickwit, Pinot)	0	2
Cloud Object Storage	6	0	0	6
Data Lakehouse	3	1 (Iceberg)	1 (Delta)	1
Data Warehouses	10	0	2 (CH, Redshift)	8
Time-Series DBs	7	0	1 (InfluxDB)	6
Graph DBs	4	0	0	4
Message Queues	12	0	0	12
Caches & KV	6	0	0	6
File Systems & Formats	9	0	2 (Avro, BSON)	7
HTTP & API	4	0	0	4
Observability	7	0	0	7
AI/ML & Vector DBs	6	0	0	6
SaaS & Enterprise	6	0	0	6
Stream Processing	6	0	1 (Flink)	5
Workflow	4	0	0	4
IoT Protocols	5	0	0	5
TOTAL	115	5	9	101

---

How to Contribute

1. Check this issue and the linked open issues above to avoid duplicating work
2. Comment here to claim a proposed connector
3. Reference the existing connectors (PostgreSQL, Elasticsearch) as implementation templates
4. See the connector SDK traits (Sink, Source, Transform) in the SDK crate
5. Each connector is a separate Rust cdylib crate under core/connectors/
6. For every P0-P2 target, a mature Rust crate exists on crates.io (linked above)

[kparisa]

@amuldotexe thanks for the initial draft/proposal. let's use this discussion thread to discuss the priorities as they should be derived based on the community needs and of course anyone can pickup anything and contribute if they are comfortable with a specific tool/integration.

[amuldotexe]

One thing that may help connector contributions is a very small behavior section in each connector PR which helps avoid errors

1. How do we know what this connector has already read or written?
2. At what exact point do we call a message successfully handled?
3. If the connector fails or restarts, what can be replayed, duplicated, or lost?
4. What is the smallest test that checks this?

What do you think? How should we proceed on something like this?
Also could the above list become more comprehensive?

[bio-nerd42]

I posted this under the issue but didn't know there was a discussion for this topic. So I be posting this question here instead.

Would using Apache OpenDAL be good for any of these connector since the point of Apache OpenDAL to have one layer with as many storage as possible? I'm reading more into docs and use cases, but I figured I would at least ask the question early on to see others view point.

[amuldotexe]

@bio-nerd42 yes I saw your separate discussion it is an interesting idea.

I think @kparisa @spetz already talked about it in this issue - #1419 ; If this is indeed the one, maybe a good idea to bump your action plan there coz some nuance is already discussed with Xuanwo who I think is PMC Chair for Open DAL;

cmiiw

[bio-nerd42]

@amuldotexe Thanks for pointing me to that issue, really helpful! I'll dig into it next weekend since I'm on-call this week and it's shaping up to be a long one.

My interest in OpenDAL actually comes from my day-to-day work with S3 and SQS. I've been using InterSystems' Cloud Storage APIs, which provide a nice unified layer across different storage platforms, and it got me thinking OpenDAL could serve a similar role here, giving us a more consistent abstraction across connectors. Looking forward to seeing what's already been discussed in that thread!