Skip to content

Commit 54e9a6f

Browse files
freeznetfreeznet
authored
Enhance documentation for features and tools (#18)
1 parent 25e8020 commit 54e9a6f

29 files changed

+928
-143
lines changed

‎README.md‎

Lines changed: 73 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,32 @@
11
# StreamNative MCP Server
22

3-
A Model Context Protocol (MCP) server for integrating AI agents with StreamNative Cloud resources and Apache Pulsar/Kafka messaging systems.
3+
A Model Context Protocol (MCP) server for integrating AI agents with StreamNative Cloud resources and Apache Kafka/Pulsar messaging systems.
44

55
## Overview
66

7-
StreamNative MCP Server provides a standard interface for LLMs (Large Language Models) and AI agents to interact with StreamNative Cloud services, Apache Pulsar, and Apache Kafka. This implementation follows the [Model Context Protocol](https://modelcontextprotocol.io/introduction) specification, enabling AI applications to access messaging services through a standardized interface.
7+
StreamNative MCP Server provides a standard interface for LLMs (Large Language Models) and AI agents to interact with StreamNative Cloud services, Apache Kafka, and Apache Pulsar. This implementation follows the [Model Context Protocol](https://modelcontextprotocol.io/introduction) specification, enabling AI applications to access messaging services through a standardized interface.
88

99
## Features
1010

1111
- **StreamNative Cloud Integration**:
1212
- Connect to StreamNative Cloud resources with authentication
1313
- Switch to clusters available in your organization
1414
- Describe the status of clusters resources
15-
- **Apache Pulsar Support**: Interact with Pulsar resources including:
16-
- Pulsar Admin operations (topics, namespaces, tenants, schemas, etc.)
17-
- Pulsar Client operations (producers, consumers)
18-
- Functions, Sources, and Sinks management
19-
- **Apache Kafka Support**: Interact with Kafka resources including:
15+
- **Apache Kafka Support**: Interact with Apache Kafka resources including:
2016
- Kafka Admin operations (topics, partitions, consumer groups)
2117
- Schema Registry operations
22-
- Kafka Connect operations
18+
- Kafka Connect operations (*)
2319
- Kafka Client operations (producers, consumers)
20+
- **Apache Pulsar Support**: Interact with Apache Pulsar resources including:
21+
- Pulsar Admin operations (topics, namespaces, tenants, schemas, etc.)
22+
- Pulsar Client operations (producers, consumers)
23+
- Functions, Sources, and Sinks management
2424
- **Multiple Connection Options**:
2525
- Connect to StreamNative Cloud with service account authentication
26-
- Connect directly to external Pulsar clusters
27-
- Connect directly to external Kafka clusters
26+
- Connect directly to external Apache Kafka clusters
27+
- Connect directly to external Apache Pulsar clusters
28+
29+
> *: The Kafka Connect operations are only tested and verfied on StreamNative Cloud.
2830
2931
## Installation
3032

@@ -124,45 +126,65 @@ The StreamNative MCP Server supports enabling or disabling specific groups of fu
124126

125127
### Available Features
126128

127-
The following sets of tools are available (all available by default on StreamNative Cloud)
128-
129-
| Features | Description |
130-
| ------|-------|
131-
| `all` | All tools, including StreamNative Cloud tools, Pulsar tools and Kafka tools |
132-
| `all-pulsar` | All Pulsar admin and Pulsar client tools |
133-
| `all-kafka` | All Kafka admin and Kafka client tools |
134-
| `pulsar-admin` | Pulsar administrative operations (including all `pulsar-admin-*`) |
135-
| `pulsar-client` | Pulsar client operations (produce and consume messages) |
136-
| `pulsar-admin-brokers` | Manage Pulsar brokers |
137-
| `pulsar-admin-broker-stats` | Access Pulsar broker statistics |
138-
| `pulsar-admin-clusters` | Manage Pulsar clusters |
139-
| `pulsar-admin-functions-worker` | Manage Pulsar Function workers |
140-
| `pulsar-admin-namespaces` | Manage Pulsar namespaces |
141-
| `pulsar-admin-namespace-policy` | Configure Pulsar namespace policies |
142-
| `pulsar-admin-isolation-policy` | Manage namespace isolation policies |
143-
| `pulsar-admin-packages` | Manage Pulsar packages |
144-
| `pulsar-admin-resource-quotas` | Configure resource quotas |
145-
| `pulsar-admin-schemas` | Manage Pulsar schemas |
146-
| `pulsar-admin-subscriptions` | Manage Pulsar subscriptions |
147-
| `pulsar-admin-tenants` | Manage Pulsar tenants |
148-
| `pulsar-admin-topics` | Manage Pulsar topics |
149-
| `pulsar-admin-sinks` | Manage Pulsar IO sinks |
150-
| `pulsar-admin-functions` | Manage Pulsar Functions |
151-
| `pulsar-admin-sources` | Manage Pulsar IO sources |
152-
| `pulsar-admin-topic-policy` | Configure Pulsar topic policies |
153-
| `kafka-admin` | Kafka administrative operations (including all `kafka-admin-*`) |
154-
| `kafka-client` | Kafka client operations (produce and consume messages) |
155-
| `kafka-admin-topics` | Manage Kafka partitions |
156-
| `kafka-admin-partitions` | Manage Kafka partitions |
157-
| `kafka-admin-groups` | Manage Kafka consumer groups |
158-
| `kafka-admin-schema-registry` | Interact with Kafka Schema Registry |
159-
| `kafka-admin-connect` | Manage Kafka Connect connectors |
160-
| `streamnative-cloud` | Manage the context, check resources logs of StreamNative Cloud |
161-
162-
### Usage Examples
163-
164-
To enable only specific feature sets:
165-
129+
The StreamNative MCP Server allows you to enable or disable specific groups of features using the `--features` flag. This helps you control which tools are available to your AI agents and can reduce context size for LLMs.
130+
131+
#### Combination Feature Sets
132+
133+
| Feature | Description |
134+
|---------------|-----------------------------------------------------------------------------|
135+
| `all` | Enables all features: StreamNative Cloud, Pulsar, and Kafka tools |
136+
137+
---
138+
139+
#### Kafka Features
140+
141+
| Feature | Description | Docs |
142+
|--------------------------|--------------------------------------------------|------|
143+
| `all-kafka` | Enables all Kafka admin and client tools, without Apache Pulsar and StreamNative Cloud tools |
144+
| `kafka-admin` | Kafka administrative operations (all admin tools) | |
145+
| `kafka-client` | Kafka client operations (produce/consume) |[kafka_client_consume.md](docs/tools/kafka_client_consume.md), [kafka_client_produce.md](docs/tools/kafka_client_produce.md) |
146+
| `kafka-admin-topics` | Manage Kafka topics | [kafka_admin_topics.md](docs/tools/kafka_admin_topics.md) |
147+
| `kafka-admin-partitions` | Manage Kafka partitions | [kafka_admin_partitions.md](docs/tools/kafka_admin_partitions.md) |
148+
| `kafka-admin-groups` | Manage Kafka consumer groups | [kafka_admin_groups.md](docs/tools/kafka_admin_groups.md) |
149+
| `kafka-admin-schema-registry` | Interact with Kafka Schema Registry | [kafka_admin_schema_registry.md](docs/tools/kafka_admin_schema_registry.md) |
150+
| `kafka-admin-connect` | Manage Kafka Connect connectors | [kafka_admin_connect.md](docs/tools/kafka_admin_connect.md) |
151+
152+
---
153+
154+
#### Pulsar Features
155+
156+
| Feature | Description | Docs |
157+
|--------------------------|--------------------------------------------------|------|
158+
| `all-pulsar` | Enables all Pulsar admin and client tools, without Apache Kafka and StreamNative Cloud tools | |
159+
| `pulsar-admin` | Pulsar administrative operations (all admin tools)| |
160+
| `pulsar-client` | Pulsar client operations (produce/consume) | [pulsar_client_consume.md](docs/tools/pulsar_client_consume.md), [pulsar_client_produce.md](docs/tools/pulsar_client_produce.md) |
161+
| `pulsar-admin-brokers` | Manage Pulsar brokers | [pulsar_admin_brokers.md](docs/tools/pulsar_admin_brokers.md) |
162+
| `pulsar-admin-broker-stats` | Access Pulsar broker statistics | [pulsar_admin_broker_stats.md](docs/tools/pulsar_admin_broker_stats.md) |
163+
| `pulsar-admin-clusters` | Manage Pulsar clusters | [pulsar_admin_clusters.md](docs/tools/pulsar_admin_clusters.md) |
164+
| `pulsar-admin-functions-worker`| Manage Pulsar Function workers | [pulsar_admin_functions_worker.md](docs/tools/pulsar_admin_functions_worker.md) |
165+
| `pulsar-admin-namespaces` | Manage Pulsar namespaces | [pulsar_admin_namespaces.md](docs/tools/pulsar_admin_namespaces.md) |
166+
| `pulsar-admin-namespace-policy`| Configure Pulsar namespace policies | [pulsar_admin_namespace_policy.md](docs/tools/pulsar_admin_namespace_policy.md) |
167+
| `pulsar-admin-isolation-policy`| Manage namespace isolation policies | [pulsar_admin_isolation_policy.md](docs/tools/pulsar_admin_isolation_policy.md) |
168+
| `pulsar-admin-packages` | Manage Pulsar packages | [pulsar_admin_packages.md](docs/tools/pulsar_admin_packages.md) |
169+
| `pulsar-admin-resource-quotas` | Configure resource quotas | [pulsar_admin_resource_quotas.md](docs/tools/pulsar_admin_resource_quotas.md) |
170+
| `pulsar-admin-schemas` | Manage Pulsar schemas | [pulsar_admin_schemas.md](docs/tools/pulsar_admin_schemas.md) |
171+
| `pulsar-admin-subscriptions` | Manage Pulsar subscriptions | [pulsar_admin_subscriptions.md](docs/tools/pulsar_admin_subscriptions.md) |
172+
| `pulsar-admin-tenants` | Manage Pulsar tenants | [pulsar_admin_tenants.md](docs/tools/pulsar_admin_tenants.md) |
173+
| `pulsar-admin-topics` | Manage Pulsar topics | [pulsar_admin_topics.md](docs/tools/pulsar_admin_topics.md) |
174+
| `pulsar-admin-sinks` | Manage Pulsar IO sinks | [pulsar_admin_sinks.md](docs/tools/pulsar_admin_sinks.md) |
175+
| `pulsar-admin-functions` | Manage Pulsar Functions | [pulsar_admin_functions.md](docs/tools/pulsar_admin_functions.md) |
176+
| `pulsar-admin-sources` | Manage Pulsar Sources | [pulsar_admin_sources.md](docs/tools/pulsar_admin_sources.md) |
177+
| `pulsar-admin-topic-policy` | Configure Pulsar topic policies | [pulsar_admin_topic_policy.md](docs/tools/pulsar_admin_topic_policy.md) |
178+
179+
---
180+
181+
#### StreamNative Cloud Features
182+
183+
| Feature | Description | Docs |
184+
|---------------------|------------------------------------------------------------------|------|
185+
| `streamnative-cloud`| Manage StreamNative Cloud context and check resource logs | [streamnative_cloud.md](docs/tools/streamnative_cloud.md) |
186+
187+
You can combine these features as needed using the `--features` flag. For example, to enable only Pulsar client features:
166188
```bash
167189
# Enable only Pulsar client features
168190
bin/snmcp stdio --organization my-org --key-file /path/to/key-file.json --features pulsar-client
@@ -244,3 +266,5 @@ This project uses [semver](https://semver.org/) semantics.
244266
## License
245267

246268
Licensed under the Apache License Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
269+
270+

‎docs/tools/kafka_admin_connect.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#### kafka-admin-connect
2+
3+
Kafka Connect is a framework for integrating Kafka with external systems. The following resources and operations are supported:
4+
5+
- **kafka-connect-cluster**
6+
- **get**: Get information about the Kafka Connect cluster
7+
- _Parameters_: None
8+
9+
- **connectors**
10+
- **list**: List all connectors in the cluster
11+
- _Parameters_: None
12+
13+
- **connector**
14+
- **get**: Get details of a specific connector
15+
- `name` (string, required): The connector name
16+
- **create**: Create a new connector
17+
- `name` (string, required): The connector name
18+
- `config` (object, required): Connector configuration
19+
- Must include at least `connector.class` and other required fields for the connector type
20+
- **update**: Update an existing connector
21+
- `name` (string, required): The connector name
22+
- `config` (object, required): Updated configuration
23+
- **delete**: Delete a connector
24+
- `name` (string, required): The connector name
25+
- **restart**: Restart a connector
26+
- `name` (string, required): The connector name
27+
- **pause**: Pause a connector
28+
- `name` (string, required): The connector name
29+
- **resume**: Resume a paused connector
30+
- `name` (string, required): The connector name
31+
32+
- **connector-plugins**
33+
- **list**: List all available connector plugins
34+
- _Parameters_: None

‎docs/tools/kafka_admin_groups.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#### kafka-admin-groups
2+
3+
This tool provides access to Kafka consumer group operations including listing, describing, and managing group membership.
4+
5+
- **groups**
6+
- **list**: List all Kafka Consumer Groups in the cluster
7+
- _Parameters_: None
8+
9+
- **group**
10+
- **describe**: Get detailed information about a specific Consumer Group
11+
- `group` (string, required): The name of the Kafka Consumer Group
12+
- **remove-members**: Remove specific members from a Consumer Group
13+
- `group` (string, required): The name of the Kafka Consumer Group
14+
- `members` (string, required): Comma-separated list of member instance IDs (e.g. "consumer-instance-1,consumer-instance-2")
15+
- **offsets**: Get offsets for a specific consumer group
16+
- `group` (string, required): The name of the Kafka Consumer Group
17+
- **delete-offset**: Delete a specific offset for a consumer group of a topic
18+
- `group` (string, required): The name of the Kafka Consumer Group
19+
- `topic` (string, required): The name of the Kafka topic
20+
- **set-offset**: Set a specific offset for a consumer group's topic-partition
21+
- `group` (string, required): The name of the Kafka Consumer Group
22+
- `topic` (string, required): The name of the Kafka topic
23+
- `partition` (number, required): The partition number
24+
- `offset` (number, required): The offset value to set (use -1 for earliest, -2 for latest, or a specific value)

‎docs/tools/kafka_admin_partitions.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#### kafka-admin-partitions
2+
3+
This tool provides access to Kafka partition operations, particularly adding partitions to existing topics.
4+
5+
- **partition**
6+
- **update**: Update the number of partitions for an existing Kafka topic (can only increase, not decrease)
7+
- `topic` (string, required): The name of the Kafka topic
8+
- `new-total` (number, required): The new total number of partitions (must be greater than current)

‎docs/tools/kafka_admin_schema_registry.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#### kafka-admin-schema-registry
2+
3+
This tool provides access to Kafka Schema Registry operations, including managing subjects, versions, and compatibility settings.
4+
5+
- **subjects**
6+
- **list**: List all schema subjects in the Schema Registry
7+
- _Parameters_: None
8+
9+
- **subject**
10+
- **get**: Get the latest schema for a subject
11+
- `subject` (string, required): The subject name
12+
- **create**: Register a new schema for a subject
13+
- `subject` (string, required): The subject name
14+
- `schema` (string, required): The schema definition (in AVRO/JSON/PROTOBUF, etc.)
15+
- `type` (string, optional): The schema type (e.g. AVRO, JSON, PROTOBUF)
16+
- **delete**: Delete a schema subject
17+
- `subject` (string, required): The subject name
18+
19+
- **versions**
20+
- **list**: List all versions for a specific subject
21+
- `subject` (string, required): The subject name
22+
- **get**: Get a specific version of a subject's schema
23+
- `subject` (string, required): The subject name
24+
- `version` (number, required): The version number
25+
- **delete**: Delete a specific version of a subject's schema
26+
- `subject` (string, required): The subject name
27+
- `version` (number, required): The version number
28+
29+
- **compatibility**
30+
- **get**: Get compatibility setting for a subject
31+
- `subject` (string, required): The subject name
32+
- **set**: Set compatibility level for a subject
33+
- `subject` (string, required): The subject name
34+
- `level` (string, required): The compatibility level (e.g. BACKWARD, FORWARD, FULL, NONE)
35+
36+
- **types**
37+
- **list**: List supported schema types (e.g. AVRO, JSON, PROTOBUF)
38+
- _Parameters_: None

‎docs/tools/kafka_admin_topics.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#### kafka-admin-topics
2+
3+
This tool provides access to various Kafka topic operations, including creation, deletion, listing, and configuration retrieval.
4+
5+
- **topics**
6+
- **list**: List all topics in the Kafka cluster
7+
- `include-internal` (boolean, optional): Whether to include internal Kafka topics (those starting with an underscore). Default: false
8+
9+
- **topic**
10+
- **get**: Get detailed configuration for a specific topic
11+
- `name` (string, required): The name of the Kafka topic
12+
- **create**: Create a new topic
13+
- `name` (string, required): The name of the Kafka topic
14+
- `partitions` (number, optional): Number of partitions. Default: 1
15+
- `replication-factor` (number, optional): Replication factor. Default: 1
16+
- `configs` (array of string, optional): Topic configuration overrides as key-value strings, e.g. ["cleanup.policy=compact", "retention.ms=604800000"]
17+
- **delete**: Delete an existing topic
18+
- `name` (string, required): The name of the Kafka topic

‎docs/tools/kafka_client_consume.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#### kafka-client-consume
2+
3+
Consume messages from a Kafka topic. This tool allows you to read messages from Kafka topics with various consumption options.
4+
5+
- **kafka_client_consume**
6+
- **Description**: Read messages from a Kafka topic, with support for consumer groups, offset control, and timeouts. If schema registry integration enabled, and the topic have schema with `topicName-value`, the consume tool will try to use the schema to decode the messages.
7+
- **Parameters**:
8+
- `topic` (string, required): The name of the Kafka topic to consume messages from.
9+
- `group` (string, optional): The consumer group ID to use. If provided, offsets are tracked and committed; otherwise, a random group is used and offsets are not committed.
10+
- `offset` (string, optional): The offset position to start consuming from. One of:
11+
- 'atstart': Begin from the earliest available message (default)
12+
- 'atend': Begin from the next message after the consumer starts
13+
- 'atcommitted': Begin from the last committed offset (only works with specified 'group')
14+
- `max-messages` (number, optional): Maximum number of messages to consume in this request. Default: 10
15+
- `timeout` (number, optional): Maximum time in seconds to wait for messages. Default: 10

‎docs/tools/kafka_client_produce.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#### kafka-client-produce
2+
3+
Produce messages to a Kafka topic. This tool allows you to send single or multiple messages with various options.
4+
5+
- **kafka_client_produce**
6+
- **Description**: Send messages to a Kafka topic, supporting keys, headers, partitions, batching, and file-based payloads.
7+
- **Parameters**:
8+
- `topic` (string, required): The name of the Kafka topic to produce messages to.
9+
- `key` (string, optional): The key for the message. Used for partition assignment and ordering.
10+
- `value` (string, required if 'messages' is not provided): The value/content of the message to send.
11+
- `headers` (array, optional): Message headers in the format of [{"key": "header-key", "value": "header-value"}].
12+
- `sync` (boolean, optional): Whether to wait for server acknowledgment before returning. Default: true.

‎docs/tools/pulsar_admin_broker_stats.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#### pulsar_admin_broker_stats
2+
3+
Unified tool for retrieving Apache Pulsar broker statistics.
4+
5+
- **monitoring_metrics**
6+
- **get**: Get broker monitoring metrics
7+
8+
- **mbeans**
9+
- **get**: Get JVM MBeans statistics from broker
10+
11+
- **topics**
12+
- **get**: Get statistics for all topics managed by the broker
13+
14+
- **allocator_stats**
15+
- **get**: Get memory allocator statistics
16+
- `allocator_name` (string, required): Name of the allocator
17+
18+
- **load_report**
19+
- **get**: Get broker load report

0 commit comments

Comments
 (0)