|
| 1 | +# OpenESPI Mapper Migration Plan: UUID Primary Key Architecture |
| 2 | + |
| 3 | +**Status**: In Progress |
| 4 | +**Created**: 2025-06-18 |
| 5 | +**Last Updated**: 2025-06-18 |
| 6 | + |
| 7 | +## Overview |
| 8 | + |
| 9 | +This document tracks the systematic migration of all MapStruct mappers from Long-based primary keys to UUID-based primary keys for NAESB ESPI 4.0 compliance. The migration ensures deterministic, globally unique resource identification using UUID5 generation based on href rel="self" values. |
| 10 | + |
| 11 | +## Migration Architecture |
| 12 | + |
| 13 | +### Completed Infrastructure Changes ✅ |
| 14 | + |
| 15 | +1. **IdentifiedObjectEntity Refactoring** ✅ |
| 16 | + - Changed primary key from `Long id` to `UUID id` |
| 17 | + - Removed redundant fields: `uuid`, `uuidMostSignificantBits`, `uuidLeastSignificantBits` |
| 18 | + - Removed `getMRID()` method (not needed in ESPI 4.0) |
| 19 | + - Simplified lifecycle management using Spring Boot automation |
| 20 | + |
| 21 | +2. **Database Migrations** ✅ |
| 22 | + - PostgreSQL migration: `V2_0__Convert_Primary_Keys_To_UUID.sql` |
| 23 | + - MySQL migration: `V2_0__Convert_Primary_Keys_To_UUID.sql` |
| 24 | + - Comprehensive foreign key updates and constraint recreation |
| 25 | + |
| 26 | +3. **Validation Enhancements** ✅ |
| 27 | + - Added field-level URL validation to `LinkType.href` using `@Pattern` |
| 28 | + - Ensures absolute HTTP/HTTPS URLs for ESPI compliance |
| 29 | + |
| 30 | +4. **Shared Mapper Utilities** ✅ |
| 31 | + - Created `BaseMapperUtils` interface for common UUID/DateTime methods |
| 32 | + - Consolidated duplicate mapping functions to avoid conflicts |
| 33 | + |
| 34 | +### Current Status |
| 35 | + |
| 36 | +- **Total Mappers**: 18 existing + ~10 missing = ~28 total |
| 37 | +- **Compilation Errors**: 58 → 3 (95% reduction!) |
| 38 | +- **Completed**: 6 mappers (infrastructure + 2 association + MeterReadingMapper) |
| 39 | +- **In Progress**: UsagePointMapper (partially fixed, needs ambiguity resolution) |
| 40 | +- **Remaining**: 21+ mappers need review/creation |
| 41 | + |
| 42 | +## Systematic Mapper Review List |
| 43 | + |
| 44 | +### Phase 1: Critical Resource Mappers (High Priority) |
| 45 | + |
| 46 | +| Resource Entity | Mapper | Status | Errors | Notes | |
| 47 | +|-----------------|--------|--------|---------|-------| |
| 48 | +| **UsagePointEntity** | UsagePointMapper | 🟡 In Progress | 8 errors | UUID mapping partially fixed, collection mapping issues remain | |
| 49 | +| **MeterReadingEntity** | MeterReadingMapper | ✅ **COMPLETED** | 0 errors | Fixed UUID mapping, DateTime qualifications, DTO id field | |
| 50 | +| **IntervalBlockEntity** | IntervalBlockMapper | ❌ Not Started | 6 errors | UUID to Long mapping, obsolete uuid fields | |
| 51 | +| **ReadingTypeEntity** | ReadingTypeMapper | ❌ Not Started | 3 errors | UUID to Long mapping, unmapped target properties | |
| 52 | +| **UsageSummaryEntity** | UsageSummaryMapper | ❌ Not Started | 6 errors | UUID mapping, obsolete uuid fields | |
| 53 | +| **ElectricPowerQualitySummaryEntity** | ElectricPowerQualitySummaryMapper | ❌ Not Started | 6 errors | UUID mapping, obsolete uuid fields | |
| 54 | +| **CustomerEntity** | CustomerMapper | ❌ Not Started | 8 errors | UUID mapping, obsolete fields, unknown properties | |
| 55 | +| **CustomerAccountEntity** | CustomerAccountMapper | ❌ Not Started | 4 errors | UUID to Long mapping, obsolete uuid fields | |
| 56 | +| **CustomerAgreementEntity** | CustomerAgreementMapper | ❌ Not Started | 8 errors | UUID mapping, DateTime conversion, unknown properties | |
| 57 | + |
| 58 | +**Phase 1 Target**: Reduce errors from 58 to ~20, achieve basic compilation |
| 59 | + |
| 60 | +### Phase 2: Supporting Resource Mappers (Medium Priority) |
| 61 | + |
| 62 | +| Resource Entity | Mapper | Status | Priority | Notes | |
| 63 | +|-----------------|--------|--------|----------|-------| |
| 64 | +| **IntervalReadingEntity** | IntervalReadingMapper | ❌ Not Started | Medium | Embedded component, unmapped properties | |
| 65 | +| **ReadingQualityEntity** | ReadingQualityMapper | ❌ Not Started | Medium | Embedded component | |
| 66 | +| **DateTimeInterval** | DateTimeIntervalMapper | ❌ Not Started | Medium | Utility mapper | |
| 67 | +| **BaseIdentifiedObject** | BaseIdentifiedObjectMapper | ❌ Not Started | High | Core functionality, `entityToId()` needs UUID update | |
| 68 | + |
| 69 | +**Phase 2 Target**: Clean up remaining compilation issues, optimize mappings |
| 70 | + |
| 71 | +### Phase 3: Missing Mappers (Creation Required) |
| 72 | + |
| 73 | +| Resource Entity | Missing Mapper | Priority | ESPI Resource | Notes | |
| 74 | +|-----------------|----------------|----------|---------------|-------| |
| 75 | +| **TimeConfigurationEntity** | TimeConfigurationMapper | Medium | Yes | Usage database resource | |
| 76 | +| **ApplicationInformationEntity** | ApplicationInformationMapper | Medium | Yes | OAuth/authorization resource | |
| 77 | +| **AuthorizationEntity** | AuthorizationMapper | Medium | Yes | OAuth resource | |
| 78 | +| **RetailCustomerEntity** | RetailCustomerMapper | Medium | Yes | Customer data resource | |
| 79 | +| **SubscriptionEntity** | SubscriptionMapper | Medium | Yes | Subscription management | |
| 80 | +| **EndDeviceEntity** | EndDeviceMapper | Medium | Yes | Customer device resource | |
| 81 | +| **MeterEntity** | MeterMapper | Medium | Yes | Extends EndDevice | |
| 82 | +| **ServiceLocationEntity** | ServiceLocationMapper | Medium | Yes | Customer location resource | |
| 83 | +| **ServiceSupplierEntity** | ServiceSupplierMapper | Medium | Yes | Utility provider resource | |
| 84 | +| **StatementEntity** | StatementMapper | Medium | Yes | Billing statement resource | |
| 85 | + |
| 86 | +**Phase 3 Target**: Complete ESPI resource coverage, full API functionality |
| 87 | + |
| 88 | +### Completed/Low Priority |
| 89 | + |
| 90 | +| Entity/Mapper | Status | Notes | |
| 91 | +|---------------|--------|-------| |
| 92 | +| **PnodeRefEntity** | ✅ PnodeRefMapper | Association table, created during UsagePoint work | |
| 93 | +| **AggregatedNodeRefEntity** | ✅ AggregatedNodeRefMapper | Association table, created during UsagePoint work | |
| 94 | +| **ServiceDeliveryPointEntity** | ✅ ServiceDeliveryPointMapper | Embedded component, should work | |
| 95 | +| **BaseMapperUtils** | ✅ Created | Shared utility methods | |
| 96 | +| **DateTimeMapper** | ✅ Good | No changes needed | |
| 97 | +| **GreenButtonMapper** | 🟡 Review Later | Integration mapper, complex dependencies | |
| 98 | + |
| 99 | +## Common Migration Patterns |
| 100 | + |
| 101 | +### 1. UUID Mapping Updates |
| 102 | + |
| 103 | +**Before (Entity → DTO):** |
| 104 | +```java |
| 105 | +@Mapping(target = "uuid", source = "uuid") |
| 106 | +``` |
| 107 | + |
| 108 | +**After (Entity → DTO):** |
| 109 | +```java |
| 110 | +@Mapping(target = "uuid", source = "id", qualifiedByName = "entityUuidToString") |
| 111 | +``` |
| 112 | + |
| 113 | +**Before (DTO → Entity):** |
| 114 | +```java |
| 115 | +@Mapping(target = "id", ignore = true) |
| 116 | +@Mapping(target = "uuid", source = "uuid") |
| 117 | +``` |
| 118 | + |
| 119 | +**After (DTO → Entity):** |
| 120 | +```java |
| 121 | +@Mapping(target = "id", source = "uuid", qualifiedByName = "stringToEntityUuid") |
| 122 | +``` |
| 123 | + |
| 124 | +### 2. Remove Obsolete Field Mappings |
| 125 | + |
| 126 | +**Remove these mappings:** |
| 127 | +```java |
| 128 | +@Mapping(target = "uuidMostSignificantBits", ignore = true) |
| 129 | +@Mapping(target = "uuidLeastSignificantBits", ignore = true) |
| 130 | +@Mapping(target = "uuid", ignore = true) // for entity mapping |
| 131 | +``` |
| 132 | + |
| 133 | +### 3. Interface Updates |
| 134 | + |
| 135 | +**Add BaseMapperUtils extension:** |
| 136 | +```java |
| 137 | +public interface XxxMapper extends BaseIdentifiedObjectMapper, BaseMapperUtils { |
| 138 | +``` |
| 139 | + |
| 140 | +### 4. Qualification for Ambiguous Methods |
| 141 | + |
| 142 | +**Use qualified names for common methods:** |
| 143 | +```java |
| 144 | +@Mapping(target = "published", source = "published", qualifiedByName = "localDateTimeToOffsetDateTime") |
| 145 | +@Mapping(target = "updated", source = "updated", qualifiedByName = "localDateTimeToOffsetDateTime") |
| 146 | +``` |
| 147 | + |
| 148 | +## Known Issues and Solutions |
| 149 | + |
| 150 | +### Issue 1: Ambiguous Mapping Methods |
| 151 | +**Problem**: Multiple mappers define the same utility methods |
| 152 | +**Solution**: Use `BaseMapperUtils` with qualified method names |
| 153 | + |
| 154 | +### Issue 2: UUID to Long Conversion Errors |
| 155 | +**Problem**: DTOs expect String UUID, entities now have UUID objects |
| 156 | +**Solution**: Use `entityUuidToString` and `stringToEntityUuid` methods |
| 157 | + |
| 158 | +### Issue 3: Collection Mapping Failures |
| 159 | +**Problem**: `Can't map property "Object meterReadings" to "List<MeterReadingEntity>"` |
| 160 | +**Solution**: Ensure related mappers are included in `uses` annotation and properly configured |
| 161 | +
|
| 162 | +### Issue 4: Unknown Property Errors |
| 163 | +**Problem**: Mappers reference fields that no longer exist |
| 164 | +**Solution**: Remove mappings to obsolete fields, update field names |
| 165 | +
|
| 166 | +## Error Tracking |
| 167 | +
|
| 168 | +### Current Compilation Errors: 58 |
| 169 | +
|
| 170 | +**By Category:** |
| 171 | +- UUID mapping issues: ~25 errors |
| 172 | +- Obsolete field references: ~20 errors |
| 173 | +- Ambiguous method conflicts: ~8 errors |
| 174 | +- Collection mapping failures: ~5 errors |
| 175 | +
|
| 176 | +**By Mapper:** |
| 177 | +- UsagePointMapper: 8 errors |
| 178 | +- MeterReadingMapper: 12 errors |
| 179 | +- CustomerMapper: 8 errors |
| 180 | +- CustomerAgreementMapper: 8 errors |
| 181 | +- CustomerAccountMapper: 4 errors |
| 182 | +- IntervalBlockMapper: 6 errors |
| 183 | +- Others: 12 errors |
| 184 | +
|
| 185 | +### Target Milestones |
| 186 | +
|
| 187 | +- **Phase 1 Complete**: Reduce to 20 errors |
| 188 | +- **Phase 2 Complete**: Reduce to 5 errors |
| 189 | +- **Phase 3 Complete**: 0 errors, full compilation |
| 190 | +
|
| 191 | +## Testing Strategy |
| 192 | +
|
| 193 | +### Unit Test Updates Required |
| 194 | +
|
| 195 | +1. **Entity Tests**: Update ID generation and assertion patterns |
| 196 | +2. **Mapper Tests**: Update DTO ↔ Entity conversion tests |
| 197 | +3. **Repository Tests**: Update query and persistence tests |
| 198 | +4. **Integration Tests**: Update full workflow tests |
| 199 | +
|
| 200 | +### Database Migration Testing |
| 201 | +
|
| 202 | +1. **Backup Creation**: Test database backup procedures |
| 203 | +2. **Migration Execution**: Test both PostgreSQL and MySQL migrations |
| 204 | +3. **Data Integrity**: Verify foreign key relationships preserved |
| 205 | +4. **Performance**: Ensure UUID indexing performance acceptable |
| 206 | +
|
| 207 | +## Documentation Updates |
| 208 | +
|
| 209 | +### Files to Update After Migration |
| 210 | +
|
| 211 | +1. **API Documentation**: Update resource examples with UUID format |
| 212 | +2. **Developer Guide**: Update entity creation patterns |
| 213 | +3. **Database Schema**: Update ERD diagrams |
| 214 | +4. **Migration Guide**: Document upgrade procedures for implementers |
| 215 | +
|
| 216 | +## Timeline and Dependencies |
| 217 | +
|
| 218 | +### Critical Path Dependencies |
| 219 | +
|
| 220 | +1. **Phase 1 mappers** must be completed before integration testing |
| 221 | +2. **BaseIdentifiedObjectMapper** fixes needed for all resource mappers |
| 222 | +3. **Database migrations** must be tested before production deployment |
| 223 | +4. **Missing mapper creation** can be parallelized with existing mapper fixes |
| 224 | +
|
| 225 | +### Estimated Effort |
| 226 | +
|
| 227 | +- **Phase 1**: 2-3 days (critical mappers) |
| 228 | +- **Phase 2**: 1-2 days (supporting mappers) |
| 229 | +- **Phase 3**: 2-3 days (missing mapper creation) |
| 230 | +- **Testing**: 1-2 days (validation and integration) |
| 231 | +
|
| 232 | +**Total Estimated**: 6-10 days |
| 233 | +
|
| 234 | +## Risk Mitigation |
| 235 | +
|
| 236 | +### High-Risk Areas |
| 237 | +
|
| 238 | +1. **Collection Mappings**: Complex relationships between resources |
| 239 | +2. **DateTime Conversions**: Timezone and format compatibility |
| 240 | +3. **Database Migration**: Large dataset conversion risks |
| 241 | +4. **Backward Compatibility**: API consumer impact |
| 242 | +
|
| 243 | +### Mitigation Strategies |
| 244 | +
|
| 245 | +1. **Incremental Commits**: Commit each phase separately |
| 246 | +2. **Feature Flags**: Enable gradual rollout if needed |
| 247 | +3. **Rollback Plan**: Maintain ability to revert migrations |
| 248 | +4. **Extensive Testing**: Unit, integration, and performance tests |
| 249 | +
|
| 250 | +--- |
| 251 | +
|
| 252 | +## Progress Log |
| 253 | +
|
| 254 | +### 2025-06-18 |
| 255 | +- ✅ Completed IdentifiedObjectEntity UUID refactoring |
| 256 | +- ✅ Created database migrations for PostgreSQL and MySQL |
| 257 | +- ✅ Added LinkType URL validation |
| 258 | +- ✅ Created BaseMapperUtils interface |
| 259 | +- ✅ Partially fixed UsagePointMapper |
| 260 | +- ✅ Created PnodeRefMapper and AggregatedNodeRefMapper |
| 261 | +- 🟡 Started systematic mapper review |
| 262 | +- **Next**: Fix remaining high-priority mappers in Phase 1 |
| 263 | +
|
| 264 | +--- |
| 265 | +
|
| 266 | +*This document will be updated as the migration progresses. Each mapper completion should be logged with status updates and error count reductions.* |
0 commit comments