Merge pull request #67 from objectstack-ai/copilot/update-and-remove-objects-fields

Author: hotlong

docs/schema-migration-instructions.md

@@ -0,0 +1,234 @@
+# Schema Migration Instructions - Implementation Summary
+
+## Problem Statement
+
+**如何表达对象和字段的更新与删除指令** (How to express object and field update and delete instructions)
+
+The ObjectQL protocol needed a declarative way to express schema evolution operations - specifically updates and deletions of objects and fields.
+
+## Solution
+
+Added a comprehensive set of TypeScript type definitions to `@objectql/types` that enable declarative expression of schema changes.
+
+### Architecture
+
+Following ObjectQL's "Constitution" principle, all new types were added to `@objectql/types` with:
+- ✅ **Zero dependencies** - Pure TypeScript interfaces
+- ✅ **Universal compatibility** - Works in Node.js, Browser, and Edge
+- ✅ **AI-native** - Declarative, YAML-friendly structure
+- ✅ **Backward compatible** - No breaking changes
+
+### Core Types Added
+
+#### 1. Schema Change Instructions
+
+**Base Type:**
+```typescript
+SchemaChangeType = 'field_update' | 'field_delete' | 'object_update' | 'object_delete'
+```
+
+**Instruction Types:**
+
+1. **`FieldUpdateInstruction`** - Modify field properties
+   - Rename fields
+   - Change field types
+   - Update validation rules
+   - Includes data migration strategies
+
+2. **`FieldDeleteInstruction`** - Remove fields
+   - Deletion strategies (drop, archive, soft)
+   - Archive location specification
+   - Reversibility tracking
+
+3. **`ObjectUpdateInstruction`** - Modify object properties
+   - Rename objects
+   - Update metadata (label, icon, description)
+   - Change datasource assignment
+
+4. **`ObjectDeleteInstruction`** - Remove objects
+   - Deletion strategies (drop, archive, soft)
+   - Cascade strategies (cascade, fail, nullify)
+   - Dependency handling
+
+#### 2. Migration Management
+
+**`MigrationConfig`** - Complete migration definition
+- Version tracking
+- Step dependencies
+- Reversibility flags
+- Tags and metadata
+
+**`MigrationStep`** - Individual migration step
+- Unique identifier
+- Schema change instruction
+- Dependency specification
+- Rollback capability
+
+**`MigrationStatus`** - Execution tracking
+- Status states (pending, running, completed, failed, rolled_back)
+- Progress tracking
+- Error reporting
+
+### Key Features
+
+#### Data Migration Strategies
+```typescript
+type DataMigrationStrategy = 'auto' | 'manual' | 'preserve' | 'clear';
+```
+
+- **auto**: Automatic conversion (safe for renames, simple type changes)
+- **manual**: Requires custom transformation script
+- **preserve**: Keep existing data as-is
+- **clear**: Reset to null/default values
+
+#### Deletion Strategies
+```typescript
+type DeletionStrategy = 'drop' | 'archive' | 'soft';
+```
+
+- **drop**: Permanent removal (irreversible)
+- **archive**: Backup to specified location (reversible)
+- **soft**: Mark as deleted but keep data (reversible)
+
+#### Cascade Strategies (for objects)
+```typescript
+type CascadeStrategy = 'cascade' | 'fail' | 'nullify';
+```
+
+- **cascade**: Delete dependent records
+- **fail**: Prevent deletion if dependencies exist
+- **nullify**: Set foreign key references to null
+
+### Usage Patterns
+
+#### 1. Field Rename (Simple)
+```yaml
+type: field_update
+object_name: users
+field_name: username
+new_field_name: user_name
+changes:
+  label: User Name
+data_migration_strategy: auto
+```
+
+#### 2. Field Type Change (Complex)
+```yaml
+type: field_update
+object_name: products
+field_name: price
+changes:
+  type: currency
+data_migration_strategy: manual
+transform_script: |
+  function transform(oldValue) {
+    return typeof oldValue === 'number' ? oldValue : 0;
+  }
+```
+
+#### 3. Field Deletion with Archive
+```yaml
+type: field_delete
+object_name: users
+field_name: legacy_id
+deletion_strategy: archive
+archive_location: backups/users_legacy_id
+```
+
+#### 4. Object Rename
+```yaml
+type: object_update
+object_name: customers
+new_object_name: accounts
+changes:
+  label: Account
+  icon: standard:account
+```
+
+#### 5. Object Deletion with Cascade
+```yaml
+type: object_delete
+object_name: temp_imports
+deletion_strategy: archive
+archive_location: backups/temp_imports
+cascade_strategy: nullify
+```
+
+### File Naming Convention
+
+Following ObjectQL's metadata standard:
+
+```
+<migration_id>.migration.yml
+```
+
+Examples:
+- `v1.1_rename_username.migration.yml`
+- `v2.0_cleanup_legacy_fields.migration.yml`
+- `v1.3_refactor_crm_objects.migration.yml`
+
+### Documentation
+
+1. **Package README** - Updated with migration types and usage examples
+2. **TypeScript Example** - Comprehensive example with 6 migration patterns
+3. **YAML Examples** - Three complete migration files demonstrating:
+   - Field operations
+   - Object operations
+   - Multi-step migrations
+
+### Benefits
+
+1. **Declarative** - Express "what" not "how"
+2. **Versioned** - Track schema evolution over time
+3. **Reversible** - Built-in rollback support
+4. **Safe** - Multiple safety strategies (archive, soft delete)
+5. **Auditable** - Include reason, author, timestamp
+6. **AI-Friendly** - Perfect for LLM code generation
+7. **Type-Safe** - Full TypeScript support
+
+### Future Implementation (Out of Scope)
+
+This PR provides the **type definitions only**. Future work includes:
+
+1. **Migration Executor** - Implement in `@objectql/core`
+   - Parse migration files
+   - Execute schema changes
+   - Handle data transformations
+   - Track migration state
+
+2. **CLI Commands** - Add to `@objectql/cli`
+   - `objectql migrate up` - Apply migrations
+   - `objectql migrate down` - Rollback migrations
+   - `objectql migrate status` - View migration state
+   - `objectql migrate create` - Generate new migration
+
+3. **Migration State Management**
+   - Track applied migrations
+   - Prevent duplicate execution
+   - Handle dependencies
+   - Store execution history
+
+4. **Driver Integration**
+   - SQL DDL generation (ALTER TABLE, DROP COLUMN, etc.)
+   - MongoDB schema updates
+   - Data migration execution
+   - Backup creation
+
+### Testing
+
+- ✅ Types package builds successfully
+- ✅ No breaking changes to existing APIs
+- ✅ Full monorepo builds without errors
+- ✅ CodeQL security scan passed (0 alerts)
+- ✅ Example files demonstrate proper usage
+
+## Conclusion
+
+This implementation provides a solid foundation for declarative schema evolution in ObjectQL. The types are:
+
+- **Minimal** - Only what's needed for the use case
+- **Extensible** - Easy to add new instruction types
+- **Production-ready** - Safety features built-in
+- **Standards-compliant** - Follows ObjectQL architecture principles
+
+The solution enables both human developers and AI agents to safely express and track schema changes in a consistent, type-safe manner.