Refactor concurrency adjustment logic in scheduler

The concurrency adjustment logic in the dynamic flush scheduler has been refactored to improve clarity and maintainability. This change moves the calculation of pressure metrics outside of the conditional blocks to ensure they are always determined prior to decision-making.

- The queue pressure and time since last flush calculations were moved up in the code to be independent of the 'backOff' condition.
- This refactor sets up the groundwork for more reliable concurrency scaling and better performance monitoring capabilities. The overall logic of adjusting concurrency based on system pressure metrics remains unchanged.

This adjustment addresses ongoing issues with the scheduler that were not resolved by previous changes.

Author: ericallam

apps/webapp/app/v3/dynamicFlushScheduler.server.ts

@@ -226,15 +226,15 @@ export class DynamicFlushScheduler<T> {
   private adjustConcurrency(backOff: boolean = false): void {
     const currentConcurrency = this.limiter.concurrency;
     let newConcurrency = currentConcurrency;
+    
+    // Calculate pressure metrics - moved outside the if/else block
+    const queuePressure = this.totalQueuedItems / this.memoryPressureThreshold;
+    const timeSinceLastFlush = Date.now() - this.lastFlushTime;
 
     if (backOff) {
       // Reduce concurrency on failures
       newConcurrency = Math.max(this.minConcurrency, Math.floor(currentConcurrency * 0.75));
     } else {
-      // Calculate pressure metrics
-      const queuePressure = this.totalQueuedItems / this.memoryPressureThreshold;
-      const timeSinceLastFlush = Date.now() - this.lastFlushTime;
-
       if (queuePressure > 0.8 || timeSinceLastFlush > this.FLUSH_INTERVAL * 2) {
         // High pressure - increase concurrency
         newConcurrency = Math.min(this.maxConcurrency, currentConcurrency + 2);