Dapr state store clickhouse

[middt]

Description

This PR adds a new state store component for ClickHouse, a column-oriented database management system. The ClickHouse state store component provides the following features:

• Basic CRUD operations for state management
• Support for TTL (Time-To-Live)
• ETag support for optimistic concurrency
• Bulk operations support
• Username/password authentication
• Uses ReplacingMergeTree engine for efficient updates

Key implementation details:

• Uses the official ClickHouse Go driver
• Implements the full state.Store interface
• Handles connection management and cleanup
• Includes comprehensive integration tests
• Follows Dapr's component development guidelines

Implementation Details

The component includes:

1. State store implementation (clickhouse.go)

   • Full CRUD operations
   • Bulk operations
   • TTL support
   • ETag support
   • Authentication support

2. Tests (clickhouse_test.go)

   • Integration tests for CRUD operations
   • Metadata validation tests
   • Authentication tests
   • Cleanup handling

3. Configuration options:

   apiVersion: dapr.io/v1alpha1
   kind: Component
   metadata:
     name: statestore
     namespace: default
   spec:
     type: state.clickhouse
     version: v1
     metadata:
     - name: clickhouseURL
       value: "tcp://localhost:9000"
     - name: databaseName
       value: "dapr_state"
     - name: tableName
       value: "state_store"
     - name: username
       value: "default"
     - name: password
       value: ""

Checklist

• Code compiles correctly
• Created/updated tests
• Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

Testing Done

1. Unit tests for metadata validation
2. Integration tests for:
   • Basic CRUD operations
   • TTL functionality
   • ETag support
   • Bulk operations
   • Authentication
   • Error handling

Tests were run against ClickHouse v23.8 using the official Go driver.

Additional Notes

• The component uses ReplacingMergeTree engine for better update performance
• Includes proper cleanup in tests to avoid test pollution
• Follows ClickHouse best practices for table design
• Implements proper connection management and cleanup

[JoshVanL]

Thanks @middt, implementation generally looks good to me!

Would it be possible to add conformance tests for this too? It seems like we should be able to spin up a click house docker compose.

[middt]

Thank you for the feedback! @JoshVanL I've implemented the conformance tests for the ClickHouse state store in this PR:

1. Created a Docker Compose setup in /tests/certification/state/clickhouse/docker-compose.yml that spins up a ClickHouse server with proper configuration
2. Added a conformance test file in /tests/certification/state/clickhouse/clickhouse_test.go that verifies all state store operations
3. Created configuration files in /tests/config/state/clickhouse/clickhouse.yml for the test environment
4. Added ClickHouse to the state store tests configuration in /tests/config/state/tests.yml

The tests verify all the key functionality including:

• Basic CRUD operations
• ETag support for optimistic concurrency
• Error handling for empty and non-existent keys
• Feature reporting

I've also addressed authentication issues by properly configuring username and password in both the Docker Compose setup and the state store implementation.

All unit tests are now passing, confirming that the implementation works correctly with the ClickHouse server.

[JoshVanL]

Thanks @middt, I think the only thing left is to do a go mod tidy and it should be good to go 🙂

Appreciate the work on this!

[github-actions]

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

[middt]

ping pong

[JoshVanL]

Hey @middt, tests are failing

[middt]

Hey @middt, tests are failing

Hi @JoshVanL,
Fixed! Tests now pass:, Ready for review!