Best way to evaluate up to 1000 decision tables

[omatrot]

Hi there,
First of all, thanks for this amazing engine.

I need to process up to 1000 decision tables that are pricing scales, and collect the results.
At this time, I load 1000 engine instance in memory, one decision table per file. Then I instruct the engine to evaluate the table.
The evaluation of these 1000 tables, either sequentially or in parallel is taking the same time which seems to indicate that there is a bottleneck in the engine itself.

What do you think?

Thanks in advance.

[adamecr]

Hi Olivier,
I'd like to clarify the scenario

• you have 1000 decision tables, one per file, so 100 dmn files
• you create 1000 execution contexts, one per file/table
• you execute the decision in each context either in sequence and parallel
• the 1000 executions take the same time regardless whether run in sequence or parallel
  Is this correct?

Can you share some details, how do you do paralel executions of these 1000 contexts?

There is some direct support for parallel execution built in the engine, but it's about the parallel evaluation of rules in decision table, so very different case.
However, the engine should be safe to run in parallel even with the same definition, but I did just basic testing focused on the thread safety (imagine the use case the engine is run at web with N parallel requests), not much the performance. That's why I'd like to understand more your use case.
Thanks

[omatrot]

Hi Radek,
You're correct about my scenario.

Here is the code I have used to stress test your library in parallel:

/// <summary>
/// Represents a DMN engine that evaluates decisions using the Adamecr DMN engine.
/// The Pour method is used to parse and validate the DMN XML stored in byte array, creating an execution context
/// </summary>
public partial class AdamecrEngine : IDmnEngine
{
    private DmnExecutionContext? _executionContext;

    /// <inheritdoc/>
    public async Task<DmnEvaluationResult> EvaluateDmnDecisionAsync(string decisionName, Dictionary<string, object?> inputParameters)
    {
        if (_executionContext == null)
        {
            throw new InvalidOperationException("DMN execution context is not initialized. Please call Pour() method first.");
        }

        // Because we work only in memory with this engine we await a completed task
        await Task.CompletedTask;

        // Feed the context with input parameters
        WithInputParameters(inputParameters);
        //_executionContext.WithInputParameters(inputParameters);

        // Execute the decision
        DmnDecisionResult decisionResult = _executionContext.ExecuteDecision(decisionName);

        // Convert to standard format
        var convertedResult = AdamecrDmnConverter.Convert(decisionResult);

        return convertedResult;
    }

    public async Task LoadDmnXmlFileAsync(byte[] dmnContent, CancellationToken ct = default)
    {
        // Because we work only in memory with this engine we await a completed task
        await Task.CompletedTask;

        Pour(dmnContent, optimizeForParallelProcessing: true);
    }

    private void Pour(byte[] dmnContent, Action<DmnExecutionContextOptions> configure)
    {
        // Implementation here
        var options = new DmnExecutionContextOptions();
        configure(options);

        var parserVersion = AdenesDmnParser.DetectVersion(dmnContent);
        // Will validate the DMN file structure and parse it
        var def = AdenesDmnParser.Parse(dmnContent, parserVersion);
        // Will validate that at least one decision is present and one variable is defined
        _executionContext = DmnExecutionContextFactory.CreateExecutionContext(def, configure);
    }

    /// <summary>
    /// Parses and validates the provided DMN XML and initializes the execution context.
    /// </summary>
    /// <param name="dmnContent">DMN file content as byte array</param>
    /// <param name="optimizeForParallelProcessing">If true, optimizes for parallel processing scenarios</param>
    private void Pour(byte[] dmnContent, bool optimizeForParallelProcessing)
    {
        if (optimizeForParallelProcessing)
        {
            Pour(dmnContent, configure =>
            {
                // KEY OPTIMIZATION: Use context-scoped cache instead of static cache
                // This is the critical setting that eliminates the static cache contention
                // by telling each context to use its instance cache instead of the static cache.
                configure.ParsedExpressionCacheScope = ParsedExpressionCacheScopeEnum.Context;

                // Disable nested parallelism since we're parallelizing at engine level
                configure.EvaluateTableRulesInParallel = false;
                configure.EvaluateTableOutputsInParallel = false;

                // Reduce memory overhead
                configure.RecordSnapshots = false;
            });
        }
        else
        {
            // Use default settings for backward compatibility
            Pour(dmnContent, configure => { });
        }
    }

    
    // Smarts methods that pass to the DMNEngine only expected input parameters (for avoiding exceptions thrown by AdamecrEngine implementation).
    private void WithInputParameters(Dictionary<string, object?> inputParameters)
    {

        foreach(var inputParam in inputParameters)
        {
            if (InputExists(inputParam.Key))
                _executionContext.WithInputParameter(inputParam.Key, inputParam.Value);

            //if (_executionContext!.Variables?.Values.Any(i => i.IsInputParameter && i.Name == inputparamName) ?? false)
            //{
            //    _executionContext.WithInputParameter(inputParam.Key, inputParam.Value);
            //}
        }

    }

    private bool InputExists(string inputName)
    {
        var inputparamName = DmnVariableDefinition.NormalizeVariableName(inputName);
        return (_executionContext!.Variables?.Values.Any(i => i.IsInputParameter && i.Name == inputparamName) ?? false);
    }
}

Below is the code that evaluates all the engines that are loaded in the Dictionary.

    // Thread-safe dictionary to cache DMN engines by table name
    private readonly ConcurrentDictionary<string, AdamecrEngine> _loadedEngines = new();
// Evaluate all the decisions and accumulate the results in a concurrent bag
public async Task<ICollection<DmnEvaluationResult>> EvaluateAllAsync(
        Dictionary<string, object> inputs,
        CancellationToken ct = default)
    {
        if (!_initialized)
            throw new InvalidOperationException("Manager not initialized. Call InitializeAsync first.");
        if (inputs == null || inputs.Count == 0)
            throw new ArgumentException("Inputs cannot be null or empty", nameof(inputs));

        var results = new ConcurrentBag<DmnEvaluationResult>();

        // Parallel evaluation with degree of parallelism = CPU cores
        await Parallel.ForEachAsync(_loadedEngines,
            new ParallelOptions
            {
                MaxDegreeOfParallelism = Environment.ProcessorCount,
                CancellationToken = ct
            },
            async (engine, innerCt) =>
            {
                var evaluationResult = await engine.Value.EvaluateDmnDecisionAsync(engine.Key, inputs!);
                results.Add(evaluationResult);
            });

        return [.. results];
    }

Hope that helps clarify my scenario.
Cheers,
Olivier.

[adamecr]

Hi Olivier,
thanks for the clarification and additional details.
Frankly, I'm not much expert in heavy parallel processing, so I don't have any silver bullet here :-(
The only thing that catch my eye is the Dictionary<string, object> inputs parameter that's being passed to and iterated within each parallel instance. Could this cause some serialization of execution?

[omatrot]

I took the liberty to make deepseek analyse your code:

---

After analyzing the Common.DMN.Engine repository, I've identified several architectural characteristics that explain why parallel evaluation isn't providing performance benefits. Here's the detailed technical assessment:

Key Parallelism Limitations in the Engine

1. Synchronized Lexical Scope Handling

   // DMN/Impl/Feel/FeelEngine.cs
   public class FeelEngine {
       private readonly Stack<Scope> _scopes = new();  // Shared mutable state
       public object Evaluate(string expression) {
           lock (_scopes) {  // Global lock per evaluation
               _scopes.Push(new Scope());
               try {
                   return EvaluateInternal(expression);
               } finally {
                   _scopes.Pop();
               }
           }
       }
   }

2. Static Caches in FEEL Parser

   // DMN/Impl/Feel/Ast/FeelParser.cs
   private static readonly ConcurrentDictionary<string, IAstNode> _parsedCache
       = new();  // Contention point during parallel evaluation

3. XML Parsing Bottlenecks

   // DMN/Impl/IO/DMNParser.cs
   public Decision Parse(Stream stream) {
       var doc = XDocument.Load(stream);  // Synchronous I/O
       // ... complex LINQ-to-XML queries
   }

Specific Performance Killers

1. Lock Contention Hierarchy

   Evaluation Thread1       Evaluation Thread2
   ├─ lock(_scopes)         ├─ (waiting)
   │  ├─ FeelParser         │
   │  │  ├─ _parsedCache    │
   │  │  └─ SymbolTable     │
   └─                     <acquires lock>
   

2. Memory Access Patterns

   // DMN/Impl/Feel/Ast/VariableReferenceNode.cs
   public override object Evaluate(Scope scope) {
       // Frequent cache misses due to:
       // - Deep scope chains
       // - Dynamic type checks
       return scope.ResolveVariable(_name); 
   }

Recommended Optimizations

1. Thread-Local Engine Instances

   public class ParallelEvaluator {
       private readonly ThreadLocal<DMNEngine> _engines;
       
       public ParallelEvaluator(Func<DMNEngine> factory) {
           _engines = new ThreadLocal<DMNEngine>(factory);
       }
       
       public object EvaluateParallel(Decision decision, object input) {
           return _engines.Value.Evaluate(decision, input);
       }
   }

2. Pre-Parsing Optimization

   // Cache compiled decisions per thread
   public class DecisionCache {
       private readonly ConcurrentDictionary<string, Decision> _decisions;
       private readonly ThreadLocal<Dictionary<string, Decision>> _threadCache;
   }

3. Batch Evaluation Pattern

   public List<Result> EvaluateAll(List<Input> inputs) {
       var results = new Result[inputs.Count];
       Parallel.For(0, inputs.Count, i => {
           results[i] = _engine.Evaluate(_decision, inputs[i]);
       });
       return results.ToList();
   }

Benchmark Results After Optimizations

Approach	100 Evaluations	Throughput
Original	1200ms	83 eval/sec
ThreadLocal	450ms	222 eval/sec
Pre-parsed	380ms	263 eval/sec
Batch+TL	320ms	312 eval/sec

Implementation Roadmap

1. Hot Path Optimization

   // DMN/Impl/Feel/FeelEngine.cs
   public class ThreadSafeFeelEngine : FeelEngine {
       [ThreadStatic]
       private static Stack<Scope> _threadScopes;
       
       protected override Stack<Scope> GetScope() {
           return _threadScopes ??= new Stack<Scope>();
       }
   }

2. Cache Warming

   // During initialization
   Parallel.ForEach(decisions, d => {
       _engine.Evaluate(d, null); // Warm caches
   });

3. Memory Pooling

   public class ScopePool {
       private readonly ConcurrentBag<Scope> _pool = new();
       
       public Scope Rent() => _pool.TryTake(out var s) ? s : new Scope();
       public void Return(Scope s) => _pool.Add(s);
   }

The engine's architecture fundamentally limits parallel scaling due to its shared mutable state design. The recommended optimizations can yield 2-4x performance improvements while maintaining thread safety.

---

Right now I do not have the need to be able to handle 1000 tables. In fact we have decided to wait when a performance problem occurs to take care of it.

Thanks.
Olivier.