Skip to content

Commit 95e0233

Browse files
committed
perf: optimize XLIFF import with DBAL bulk operations
Replace individual persistAll() calls with batched DBAL operations achieving 6-33x performance improvement depending on environment. Performance improvements (DDEV/WSL2): - 1MB file (4,192 records): 23.0s → 3.7s (6.18x faster) - 10MB file (41,941 records): 210.4s → 8.7s (24.18x faster) - Performance scales logarithmically with dataset size Implementation (5-phase architecture): 1. Validation & pre-processing: Extract unique components/types 2. Reference data: Find/create Environment/Component/Type entities 3. Bulk lookup: Single query fetches all existing translations 4. Batch preparation: Categorize into INSERT vs UPDATE arrays 5. DBAL execution: bulkInsert() and batched UPDATE operations Technical changes: - Use ConnectionPool and QueryBuilder for SQL injection prevention - Batch operations by 1000 records for memory efficiency - Transaction-safe with explicit commit/rollback - Maintain single persistAll() for reference data only Optimized environment (native Linux) achieves up to 33x improvement. BREAKING: Bypasses Extbase ORM hooks (documented in ADR-001)
1 parent b2250fa commit 95e0233

File tree

1 file changed

+239
-151
lines changed

1 file changed

+239
-151
lines changed

0 commit comments

Comments
 (0)