|
| 1 | +# Design Document |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This design outlines the technical approach for modernizing Dynalite's dependencies, with primary focus on updating the LevelDB ecosystem and replacing Mocha with Node.js built-in test runner. The modernization will be executed in phases to minimize risk and ensure compatibility. |
| 6 | + |
| 7 | +## Architecture |
| 8 | + |
| 9 | +### Current Dependency Architecture |
| 10 | + |
| 11 | +``` |
| 12 | +Dynalite Core |
| 13 | +├── LevelDB Ecosystem (CRITICAL - ALL DEPRECATED ⚠️) |
| 14 | +│ ├── levelup@5.1.1 (DEPRECATED) → level@10.x |
| 15 | +│ ├── leveldown@6.1.1 → classic-level (via level@10.x) |
| 16 | +│ ├── memdown@6.1.1 (DEPRECATED) → memory-level@3.x |
| 17 | +│ └── subleveldown@6.0.1 (DEPRECATED) → abstract-level sublevels |
| 18 | +├── Test Framework (SECONDARY PRIORITY) |
| 19 | +│ ├── mocha@10.2.0 → node:test (built-in) |
| 20 | +│ └── should@13.2.3 → node:assert (built-in) |
| 21 | +├── Built-in Replacements |
| 22 | +│ ├── once@1.4.0 → custom wrapper |
| 23 | +│ └── minimist@1.2.8 → util.parseArgs() |
| 24 | +└── Other Dependencies |
| 25 | + ├── async@3.2.4 → async@3.x (latest) |
| 26 | + ├── big.js@6.2.1 → big.js@6.x (latest) |
| 27 | + └── others → latest versions |
| 28 | +``` |
| 29 | + |
| 30 | +### Target Architecture |
| 31 | + |
| 32 | +``` |
| 33 | +Modernized Dynalite |
| 34 | +├── LevelDB Ecosystem (MODERN ABSTRACT-LEVEL) |
| 35 | +│ ├── level@10.x (replaces levelup + leveldown) |
| 36 | +│ ├── memory-level@3.x (replaces memdown) |
| 37 | +│ ├── abstract-level sublevels (replaces subleveldown) |
| 38 | +│ └── classic-level (native binding via level) |
| 39 | +├── Node.js Built-ins |
| 40 | +│ ├── node:test (replaces mocha) |
| 41 | +│ ├── node:assert (replaces should) |
| 42 | +│ ├── util.parseArgs() (replaces minimist) |
| 43 | +│ └── custom once() wrapper |
| 44 | +└── Updated Dependencies |
| 45 | + └── All other deps at latest stable versions |
| 46 | +``` |
| 47 | + |
| 48 | +## Components and Interfaces |
| 49 | + |
| 50 | +### 1. LevelDB Ecosystem Migration |
| 51 | + |
| 52 | +#### Current Implementation (DEPRECATED PACKAGES) |
| 53 | +```javascript |
| 54 | +// db/index.js - Current (ALL DEPRECATED ⚠️) |
| 55 | +var levelup = require('levelup'), // DEPRECATED |
| 56 | + memdown = require('memdown'), // DEPRECATED |
| 57 | + sub = require('subleveldown') // DEPRECATED |
| 58 | + |
| 59 | +var db = levelup(options.path ? require('leveldown')(options.path) : memdown()) |
| 60 | +``` |
| 61 | + |
| 62 | +#### Target Implementation (MODERN ABSTRACT-LEVEL) |
| 63 | +```javascript |
| 64 | +// db/index.js - Target |
| 65 | +var { Level } = require('level'), // Modern replacement |
| 66 | + { MemoryLevel } = require('memory-level') // Modern replacement |
| 67 | + |
| 68 | +var db = options.path ? |
| 69 | + new Level(options.path) : |
| 70 | + new MemoryLevel() |
| 71 | + |
| 72 | +// Sublevel functionality now built into abstract-level |
| 73 | +function getSubDb(name) { |
| 74 | + return db.sublevel(name, { valueEncoding: 'json' }) |
| 75 | +} |
| 76 | +``` |
| 77 | + |
| 78 | +#### Migration Strategy (DEPRECATED → MODERN) |
| 79 | +- **Phase 1**: Research abstract-level migration path and compatibility |
| 80 | +- **Phase 2**: Replace deprecated packages with modern abstract-level ecosystem |
| 81 | + - levelup@5.1.1 (DEPRECATED) → level@10.x |
| 82 | + - memdown@6.1.1 (DEPRECATED) → memory-level@3.x |
| 83 | + - subleveldown@6.0.1 (DEPRECATED) → built-in sublevel functionality |
| 84 | +- **Phase 3**: Update all database access patterns to use new APIs |
| 85 | +- **Phase 4**: Test data compatibility and performance extensively |
| 86 | + |
| 87 | +### 2. Test Framework Migration |
| 88 | + |
| 89 | +#### Current Test Structure |
| 90 | +```javascript |
| 91 | +// test/listTables.js - Current |
| 92 | +var should = require('should'), |
| 93 | + async = require('async'), |
| 94 | + helpers = require('./helpers') |
| 95 | + |
| 96 | +describe('listTables', function() { |
| 97 | + it('should return empty list', function(done) { |
| 98 | + // test implementation |
| 99 | + }) |
| 100 | +}) |
| 101 | +``` |
| 102 | + |
| 103 | +#### Target Test Structure |
| 104 | +```javascript |
| 105 | +// test/listTables.js - Target |
| 106 | +import { test, describe } from 'node:test' |
| 107 | +import assert from 'node:assert' |
| 108 | +import helpers from './helpers.js' |
| 109 | + |
| 110 | +describe('listTables', () => { |
| 111 | + test('should return empty list', async () => { |
| 112 | + // test implementation with assert |
| 113 | + }) |
| 114 | +}) |
| 115 | +``` |
| 116 | + |
| 117 | +#### Migration Strategy |
| 118 | +- Convert 20 test files from Mocha to Node.js test runner |
| 119 | +- Replace `should` assertions with `node:assert` |
| 120 | +- Maintain existing test helper patterns |
| 121 | +- Update npm scripts for new test runner |
| 122 | + |
| 123 | +### 3. Built-in Replacements |
| 124 | + |
| 125 | +#### Once Module Replacement |
| 126 | +```javascript |
| 127 | +// Current usage |
| 128 | +var once = require('once') |
| 129 | +cb = once(cb) |
| 130 | + |
| 131 | +// Target replacement |
| 132 | +function once(fn) { |
| 133 | + let called = false |
| 134 | + return function(...args) { |
| 135 | + if (called) return |
| 136 | + called = true |
| 137 | + return fn.apply(this, args) |
| 138 | + } |
| 139 | +} |
| 140 | +``` |
| 141 | + |
| 142 | +#### Minimist Replacement |
| 143 | +```javascript |
| 144 | +// cli.js - Current |
| 145 | +var argv = require('minimist')(process.argv.slice(2), { |
| 146 | + alias: { debug: ['d'], verbose: ['v'] } |
| 147 | +}) |
| 148 | + |
| 149 | +// cli.js - Target |
| 150 | +import { parseArgs } from 'node:util' |
| 151 | +const { values: argv } = parseArgs({ |
| 152 | + args: process.argv.slice(2), |
| 153 | + options: { |
| 154 | + debug: { type: 'boolean', short: 'd' }, |
| 155 | + verbose: { type: 'boolean', short: 'v' }, |
| 156 | + help: { type: 'boolean', short: 'h' }, |
| 157 | + port: { type: 'string' }, |
| 158 | + host: { type: 'string' }, |
| 159 | + path: { type: 'string' }, |
| 160 | + ssl: { type: 'boolean' } |
| 161 | + } |
| 162 | +}) |
| 163 | +``` |
| 164 | + |
| 165 | +## Data Models |
| 166 | + |
| 167 | +### LevelDB Data Compatibility |
| 168 | + |
| 169 | +The LevelDB ecosystem update must maintain compatibility with existing data formats: |
| 170 | + |
| 171 | +```javascript |
| 172 | +// Key encoding remains identical |
| 173 | +function createKey(item, table, keySchema) { |
| 174 | + // Existing key creation logic preserved |
| 175 | + return keyStr |
| 176 | +} |
| 177 | + |
| 178 | +// Value encoding remains identical |
| 179 | +function itemSize(item, compress, addMetaSize, rangeKey) { |
| 180 | + // Existing size calculation preserved |
| 181 | + return size |
| 182 | +} |
| 183 | +``` |
| 184 | + |
| 185 | +### Test Data Migration |
| 186 | + |
| 187 | +Test helpers and data structures remain unchanged: |
| 188 | + |
| 189 | +```javascript |
| 190 | +// test/helpers.js - Interface preserved |
| 191 | +exports.testHashTable = 'test_table_name' |
| 192 | +exports.request = request |
| 193 | +exports.opts = opts |
| 194 | +// All existing helper functions maintained |
| 195 | +``` |
| 196 | + |
| 197 | +## Error Handling |
| 198 | + |
| 199 | +### LevelDB Error Compatibility |
| 200 | +- Ensure error types and messages remain consistent |
| 201 | +- Map new LevelDB errors to existing error patterns |
| 202 | +- Maintain existing error handling in actions/ |
| 203 | + |
| 204 | +### Test Framework Error Handling |
| 205 | +- Convert Mocha error patterns to Node.js test patterns |
| 206 | +- Preserve existing assertion error messages |
| 207 | +- Maintain test timeout and async error handling |
| 208 | + |
| 209 | +## Testing Strategy |
| 210 | + |
| 211 | +### Phase 1: LevelDB Ecosystem Testing |
| 212 | +1. **Compatibility Tests**: Verify existing data can be read/written |
| 213 | +2. **Performance Tests**: Ensure no regression in operation speed |
| 214 | +3. **Integration Tests**: Full DynamoDB API operation testing |
| 215 | +4. **Migration Tests**: Test upgrade path from old to new versions |
| 216 | + |
| 217 | +### Phase 2: Test Framework Migration Testing |
| 218 | +1. **Conversion Verification**: Each test file converted and verified |
| 219 | +2. **Coverage Maintenance**: Ensure test coverage remains identical |
| 220 | +3. **CI/CD Integration**: Update GitHub Actions for new test runner |
| 221 | +4. **Helper Function Testing**: Verify all test utilities work |
| 222 | + |
| 223 | +### Phase 3: Built-in Replacement Testing |
| 224 | +1. **CLI Testing**: Verify all command-line options work identically |
| 225 | +2. **Callback Testing**: Ensure once() wrapper functions correctly |
| 226 | +3. **Edge Case Testing**: Test error conditions and unusual inputs |
| 227 | + |
| 228 | +## Implementation Phases |
| 229 | + |
| 230 | +### Phase 1: LevelDB Ecosystem Migration (CRITICAL PRIORITY - DEPRECATED PACKAGES) |
| 231 | +**Duration**: 3-4 days |
| 232 | +**Risk**: High (deprecated packages, API changes, data compatibility) |
| 233 | + |
| 234 | +1. Research abstract-level migration guide and breaking changes |
| 235 | +2. Replace deprecated packages in package.json: |
| 236 | + - Remove: levelup, memdown, subleveldown (all DEPRECATED ⚠️) |
| 237 | + - Add: level@10.x, memory-level@3.x |
| 238 | +3. Rewrite db/index.js for abstract-level API |
| 239 | +4. Update all sublevel usage to use built-in sublevel functionality |
| 240 | +5. Run comprehensive tests to verify data compatibility |
| 241 | +6. Performance benchmarking to ensure no regression |
| 242 | + |
| 243 | +### Phase 2: Test Framework Migration (MEDIUM PRIORITY) |
| 244 | +**Duration**: 3-4 days |
| 245 | +**Risk**: Medium (test coverage) |
| 246 | + |
| 247 | +1. Convert test/helpers.js to work with Node.js test runner |
| 248 | +2. Convert individual test files (20 files) from Mocha to node:test |
| 249 | +3. Replace should assertions with node:assert |
| 250 | +4. Update npm scripts and package.json |
| 251 | +5. Verify all tests pass with identical coverage |
| 252 | + |
| 253 | +### Phase 3: Built-in Replacements (LOW PRIORITY) |
| 254 | +**Duration**: 1-2 days |
| 255 | +**Risk**: Low (simple replacements) |
| 256 | + |
| 257 | +1. Replace once module with custom implementation |
| 258 | +2. Replace minimist with util.parseArgs() |
| 259 | +3. Update Node.js version requirement to >=20 |
| 260 | +4. Test CLI functionality thoroughly |
| 261 | + |
| 262 | +### Phase 4: Remaining Dependencies (LOW PRIORITY) |
| 263 | +**Duration**: 1 day |
| 264 | +**Risk**: Low (version updates) |
| 265 | + |
| 266 | +1. Update all remaining dependencies to latest versions |
| 267 | +2. Update ESLint and development tools |
| 268 | +3. Replace PEG.js with @peggyjs/peggy if needed |
| 269 | +4. Final integration testing |
| 270 | + |
| 271 | +## Risk Mitigation |
| 272 | + |
| 273 | +### LevelDB Data Loss Prevention |
| 274 | +- Create backup/restore utilities for testing |
| 275 | +- Implement rollback strategy for LevelDB changes |
| 276 | +- Test with existing production-like data sets |
| 277 | + |
| 278 | +### Test Coverage Preservation |
| 279 | +- Automated test conversion verification |
| 280 | +- Coverage reporting comparison (before/after) |
| 281 | +- Manual verification of critical test paths |
| 282 | + |
| 283 | +### Performance Regression Prevention |
| 284 | +- Benchmark existing performance before changes |
| 285 | +- Continuous performance monitoring during updates |
| 286 | +- Rollback plan if performance degrades significantly |
| 287 | + |
| 288 | +## Success Criteria |
| 289 | + |
| 290 | +1. **LevelDB Ecosystem**: All dependencies updated, data compatibility maintained, performance preserved |
| 291 | +2. **Test Framework**: All tests converted to Node.js test runner, coverage maintained |
| 292 | +3. **Built-ins**: Successfully replaced once and minimist with Node.js built-ins |
| 293 | +4. **Compatibility**: All existing APIs work identically, no breaking changes |
| 294 | +5. **Security**: No new vulnerabilities introduced, dependency count reduced |
0 commit comments