refactor(sharding): enforce explicit rollback commands in queue

- Remove RollbackCommandGenerator and legacy fallback code
- Make rollback SQL parameter required for all queue operations
- Update queue schema and docs to mandate rollback commands
- Add rollback commands for shard and cluster operations
- Improve test coverage with explicit rollback parameters
- Simplify rollback handling by assuming always-on support

Author: donhardman

doc/sharding/01-components.md

@@ -287,17 +287,18 @@ Generates reverse SQL commands for rollback operations.
 
 **Usage Pattern:**
 ```php
-// Automatic generation
-$rollback = RollbackCommandGenerator::generate("CREATE TABLE users");
-// Returns: "DROP TABLE IF EXISTS users"
-
-// Specific generators
-$rollback = RollbackCommandGenerator::forCreateTable("users");
-$rollback = RollbackCommandGenerator::forAlterClusterAdd("c1", "users");
-
-// Safety check
-if (RollbackCommandGenerator::isSafeToRollback($command)) {
-    $rollback = RollbackCommandGenerator::generate($command);
+// Direct rollback command provision
+$forwardSql = "CREATE TABLE users (id bigint, name string)";
+$rollbackSql = "DROP TABLE IF EXISTS users";
+$queue->add($nodeId, $forwardSql, $rollbackSql, $operationGroup);
+
+// Common rollback patterns:
+"CREATE TABLE users" → "DROP TABLE IF EXISTS users"
+"CREATE CLUSTER c1" → "DELETE CLUSTER c1"
+"ALTER CLUSTER c1 ADD t1" → "ALTER CLUSTER c1 DROP t1"
+
+// All operations require explicit rollback commands
+$queue->add($node, $sql, $rollbackSql, $operationGroup);
 }
 ```
 
@@ -357,7 +358,7 @@ $health = $monitor->performHealthCheck();
 if ($health['overall_status'] !== 'healthy') {
     // Automatic recovery
     $recovery = $monitor->performAutoRecovery();
-    
+
     // Or manual intervention based on recommendations
     foreach ($health['recommendations'] as $recommendation) {
         echo $recommendation;
@@ -370,7 +371,7 @@ if ($health['overall_status'] !== 'healthy') {
 ### Rollback Flow
 
 ```
-Table.shard() 
+Table.shard()
     ├── Creates operation_group
     ├── Queue.addWithRollback() [multiple times]
     ├── On failure:

doc/sharding/04-queue-system.md

@@ -13,21 +13,21 @@ The Queue class manages distributed command execution with the following key fea
 - **Node Targeting**: Routes commands to specific cluster nodes
 - **Parallel Execution**: Supports concurrent operations where safe
 - **Synchronization Points**: Ensures critical operations complete before proceeding
-- **Rollback Support**: Stores rollback commands alongside forward commands
+- **Rollback Support**: Stores rollback commands alongside forward commands (REQUIRED)
 - **Operation Groups**: Groups related commands for atomic execution
 - **Automatic Rollback**: Executes rollback sequence on failure
 
-### Enhanced Queue Table Structure
+### Queue Table Structure
 
 ```sql
 CREATE TABLE system.sharding_queue (
     `id` bigint,                    -- Primary key
     `node` string,                   -- Target node
     `query` string,                  -- Forward command
-    `rollback_query` string,         -- Rollback command (NEW)
+    `rollback_query` string,         -- Rollback command (REQUIRED)
     `wait_for_id` bigint,           -- Forward dependency
-    `rollback_wait_for_id` bigint,  -- Rollback dependency (NEW)
-    `operation_group` string,        -- Operation group ID (NEW)
+    `rollback_wait_for_id` bigint,  -- Rollback dependency
+    `operation_group` string,        -- Operation group ID
     `tries` int,                     -- Retry count
     `status` string,                 -- Command status
     `created_at` bigint,            -- Creation timestamp
@@ -136,7 +136,7 @@ When a rollback is triggered:
 ```php
 protected function executeRollbackSequence(array $rollbackCommands): bool {
     $allSuccess = true;
-    
+
     foreach ($rollbackCommands as $command) {
         try {
             $this->client->sendRequest($command['rollback_query']);
@@ -147,7 +147,7 @@ protected function executeRollbackSequence(array $rollbackCommands): bool {
             // Continue with other rollback commands
         }
     }
-    
+
     return $allSuccess;
 }
 ```

doc/sharding/07-error-handling.md

@@ -16,12 +16,12 @@ All related operations are grouped together for atomic execution:
 public function shard(Queue $queue, int $shardCount, int $replicationFactor = 2): Map {
     // Create unique operation group
     $operationGroup = "shard_create_{$this->name}_" . uniqid();
-    
+
     try {
         // All operations in this group
         $queue->addWithRollback($node, $createSql, $rollbackSql, $operationGroup);
         // ... more operations
-        
+
         return $result;
     } catch (\Throwable $t) {
         // Automatic rollback of entire group
@@ -31,12 +31,17 @@ public function shard(Queue $queue, int $shardCount, int $replicationFactor = 2)
 }
 ```
 
-### Rollback Command Generation
+### Rollback Command Handling
 
-The system automatically generates reverse commands for common operations:
+The system requires explicit rollback commands for all operations:
 
 ```php
-// RollbackCommandGenerator examples
+// Rollback commands must be provided when queuing operations
+$forwardSql = "CREATE TABLE users (id bigint, name string)";
+$rollbackSql = "DROP TABLE IF EXISTS users";
+$queue->add($nodeId, $forwardSql, $rollbackSql, $operationGroup);
+
+// Common rollback patterns:
 "CREATE TABLE users" → "DROP TABLE IF EXISTS users"
 "CREATE CLUSTER c1" → "DELETE CLUSTER c1"
 "ALTER CLUSTER c1 ADD t1" → "ALTER CLUSTER c1 DROP t1"

doc/sharding/11-rollback-system.md

@@ -1,16 +1,16 @@
-# Sharding Rollback and Recovery System
+# Sharding Rollback System
 
 ## Overview
 
-The ManticoreSearch Buddy sharding system now includes a comprehensive rollback and recovery mechanism that ensures system consistency and reliability by automatically reversing failed operations and providing tools for health monitoring and resource cleanup.
+The ManticoreSearch Buddy sharding system includes a simplified rollback mechanism that ensures system consistency by storing rollback commands directly when operations are queued. Rollback is always enabled and commands are provided upfront.
 
 ## Key Features
 
-### 1. Automatic Rollback
-- **Operation Groups**: Related commands grouped for atomic execution
-- **Rollback Commands**: Automatic generation of reverse SQL commands
-- **Failure Detection**: Automatic rollback trigger on operation failure
-- **Queue-Based Execution**: Leverages existing queue infrastructure
+### 1. Always-On Rollback
+- **Required Rollback Commands**: Every queued operation must provide its rollback command
+- **Direct Storage**: Rollback commands stored immediately when operation is queued
+- **No Auto-Generation**: Rollback commands are explicitly provided by the caller
+- **Operation Groups**: Related commands grouped for atomic rollback
 
 ### 2. Rebalancing Control
 - **Stop/Pause/Resume**: Full control over rebalancing operations
@@ -32,17 +32,17 @@ The ManticoreSearch Buddy sharding system now includes a comprehensive rollback
 
 ## Architecture
 
-### Enhanced Queue Table Structure
+### Queue Table Structure
 
 ```sql
 CREATE TABLE system.sharding_queue (
     `id` bigint,                    -- Primary key
     `node` string,                   -- Target node
     `query` string,                  -- Forward command
-    `rollback_query` string,         -- Rollback command (NEW)
+    `rollback_query` string,         -- Rollback command (REQUIRED)
     `wait_for_id` bigint,           -- Forward dependency
-    `rollback_wait_for_id` bigint,  -- Rollback dependency (NEW)
-    `operation_group` string,        -- Operation group ID (NEW)
+    `rollback_wait_for_id` bigint,  -- Rollback dependency
+    `operation_group` string,        -- Operation group ID
     `tries` int,                     -- Retry count
     `status` string,                 -- Command status
     `created_at` bigint,            -- Creation timestamp
@@ -58,7 +58,7 @@ Table Operations
     ├── Create operation_group
     ├── Queue.addWithRollback() [multiple commands]
     ├── On Success: Mark complete
-    └── On Failure: 
+    └── On Failure:
         └── Queue.rollbackOperationGroup()
             ├── Get completed commands
             ├── Sort by ID DESC (reverse)
@@ -116,7 +116,7 @@ if ($health['overall_status'] !== 'healthy') {
     foreach ($health['issues'] as $issue) {
         echo "Issue: {$issue['type']} - {$issue['count']} affected";
     }
-    
+
     // Auto-recovery
     $recovery = $monitor->performAutoRecovery();
     echo "Recovered: " . count($recovery['recovered_tables']) . " tables";
@@ -139,9 +139,9 @@ $cleanup->cleanupExpiredQueueItems();
 $cleanup->cleanupStaleStateEntries();
 ```
 
-## Rollback Command Generation
+## Rollback Command Examples
 
-The system automatically generates rollback commands for common operations:
+Common rollback patterns used in the system:
 
 | Forward Command | Rollback Command |
 |----------------|------------------|
@@ -151,13 +151,29 @@ The system automatically generates rollback commands for common operations:
 | `ALTER CLUSTER c1 DROP t1` | `ALTER CLUSTER c1 ADD t1` |
 | `JOIN CLUSTER c1` | `DELETE CLUSTER c1` |
 
-Commands that cannot be safely rolled back (like `DROP TABLE`) return null and require manual intervention.
+All rollback commands must be provided when queuing operations. The system no longer auto-generates rollback commands.
+
+## Usage Examples
+
+### Adding Operations with Rollback
+
+```php
+// Create table with explicit rollback
+$forwardSql = "CREATE TABLE users (id bigint, name string)";
+$rollbackSql = "DROP TABLE IF EXISTS users";
+$queue->add($nodeId, $forwardSql, $rollbackSql, $operationGroup);
+
+// Distributed table creation
+$forwardSql = $this->getCreateShardedTableSQL($shards);
+$rollbackSql = "DROP TABLE IF EXISTS {$this->name}";
+$queue->add($node, $forwardSql, $rollbackSql, $operationGroup);
+```
 
 ## Production Deployment
 
-### Migration
+### Queue Table Setup
 
-For existing systems, run the migration to add rollback support:
+The queue table is automatically created with rollback support:
 
 ```php
 $queue = new Queue($cluster, $client);
@@ -234,4 +250,4 @@ Configure alerts for:
 
 ## Conclusion
 
-The rollback and recovery system transforms the ManticoreSearch Buddy sharding system from a basic distributed system into a production-ready platform with comprehensive error handling, automatic recovery, and resource management capabilities. This ensures high availability, data consistency, and operational reliability in production environments.
+The rollback and recovery system transforms the ManticoreSearch Buddy sharding system from a basic distributed system into a production-ready platform with comprehensive error handling, automatic recovery, and resource management capabilities. This ensures high availability, data consistency, and operational reliability in production environments.

doc/sharding/README.md

@@ -12,7 +12,7 @@ The Manticore Buddy Sharding system provides automatic distribution of data acro
 - **Data Safety**: Ensures no data loss during rebalancing operations
 - **Concurrent Operation Control**: Prevents conflicting rebalancing operations
 - **Queue-Based Processing**: Asynchronous command execution with proper ordering
-- **Automatic Rollback**: Comprehensive rollback system for failed operations
+- **Automatic Rollback**: Simplified rollback system with required rollback commands
 - **Graceful Stop Control**: Ability to stop/pause/resume rebalancing operations
 - **Resource Cleanup**: Automatic cleanup of orphaned resources and failed operations
 - **Health Monitoring**: Built-in health checks and auto-recovery mechanisms