Performance Impact On Switch To FusionCache

[bendursley]

Hello! We've got a middleware in our services, which we've swapped to use FusionCache from Lazy App Cache, which loads up some customer configuration.
We don't have Redis or a backplane set up for it, we're happy with our services, which are multi-instance calling in to the data store. We currently have that via existing App Cache.

However, one thing we've noticed, is that FusionCache seems to be slower than what it's replaced, and we're not sure of the best way to try and get to the bottom of why.
We've run load tests, and it does seem that FusionCache is ~15% slower (in terms of response time and request throughput) than what it is replacing.

We've got the method call below that we use, but wondered if we'd missed anything, or if there was any general guidance here.

// FusionCache

/// Service Setup
       {
       /// ...
            services
                .AddFusionCache(CacheConstants.ConfigurationCacheName)
                .WithNewtonsoftJsonSerializer();
       /// ...
       }

/// Class
       public CustomerAccessor
       {
       /// ...
            fusionCache = fusionCacheProvider.GetCache(CacheConstants.ConfigurationCacheName);
       /// ...


       async Task<T> IAccessor.GetCustomerConfiguration<T>(int customerId, CancellationToken token)
       {
           return await fusionCache.GetOrSetAsync<T>(
               key: GetCacheKey(),
               factory: async (ct) => await inner.GetCustomerConfiguration<T>(customerId, ct),
               setupAction: options => options
                   .SetDuration(configurationOptions.CacheDuration)
                   .SetFailSafe(
                       isEnabled: true,
                       maxDuration: configurationOptions.ExpiredCacheDuration,
                       throttleDuration: configurationOptions.FailSafeThrottleDuration)
                   .SetFactoryTimeouts(
                       softTimeout: configurationOptions.FactorySoftTimeoutSingle),
               token);

           string GetCacheKey()
           {
               var typeName = typeof(T).FullName;

               return $"CustomerConfigSingle-{typeName}-{customerId}";
           }
       }

// LazyCache

       Task<T> IAccessor.GetCustomerConfiguration<T>(int customerId, CancellationToken token)
       {
           return appCache.GetOrAddAsync(
               GetCacheKey(),
               () => inner.GetCustomerConfiguration<T>(customerId, token),
               new MemoryCacheEntryOptions()
               {
                   AbsoluteExpirationRelativeToNow = options.CacheDuration
               });

           string GetCacheKey()
           {
               var typeName = typeof(T).FullName;

               return $"CustomerConfigSingle-{typeName}-{customerId}";
           }
       }

[jodydonetti]

Hi @bendursley in general in the benchmarks I saw or run FusionCache is faster than LazyCache (when used with the same features, of course).
Anyway I'm currently busy on a thing, but I'll try to check this later and will let you know.

Meanwhile though, can you clarify one thing? You say it looks like it's 15% slower, but you mean only on a cache miss (when it calls the factory, etc) or as an average off all requests in a period of time (so, 1 cache miss + all the others cache hit)?

Also I think there are minor issues (maybe copy-paste errors?) in the snippets, since there seem to be some missing awaits and some extra ones. Again it's a minor thing, but since we are talking about benchmarking better to ask to avoid little things get in the way of performance (eg: returning a task without awaiting it etc).

Thanks!

[jodydonetti]

Hi @bendursley ,
I just run a benchmark on my pc, here are the results:

BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.4061)
13th Gen Intel Core i9-13900H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 9.0.300
  [Host] : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX2

Toolchain=InProcessEmitToolchain

| Method      | FactoryDurationMs | Accessors | KeysCount | Rounds | Mean     | Error    | StdDev   | P95      | Ratio | RatioSD | Gen0       | Gen1       | Gen2      | Allocated | Alloc Ratio |
|------------ |------------------ |---------- |---------- |------- |---------:|---------:|---------:|---------:|------:|--------:|-----------:|-----------:|----------:|----------:|------------:|
| FusionCache | 100               | 1000      | 100       | 10     | 567.9 ms | 15.38 ms | 45.35 ms | 629.9 ms |  1.01 |    0.13 | 21000.0000 | 13000.0000 | 6000.0000 | 220.65 MB |        1.00 |
| LazyCache   | 100               | 1000      | 100       | 10     | 765.1 ms | 15.29 ms | 24.69 ms | 802.0 ms |  1.36 |    0.14 | 45000.0000 | 18000.0000 | 4000.0000 | 527.14 MB |        2.39 |

The benchmark code is something like this:

[Benchmark(Baseline = true)]
public async Task FusionCache()
{
	for (int i = 0; i < Rounds; i++)
	{
		var tasks = new ConcurrentBag<Task>();

		Parallel.ForEach(Keys, key =>
		{
			Parallel.For(0, Accessors, _ =>
			{
				var t = _FusionCache.GetOrSetAsync<SamplePayload>(
					key,
					async (ctx, ct) =>
					{
						await Task.Delay(FactoryDurationMs).ConfigureAwait(false);
						return new SamplePayload();
					},
					default,
					null,
					null,
					default
				);
				tasks.Add(t.AsTask());
			});
		});

		await Task.WhenAll(tasks).ConfigureAwait(false);
	}
}

where the benchmark inputs are these:

[Params(100)]
public int FactoryDurationMs;

[Params(1_000)]
public int Accessors;

[Params(100)]
public int KeysCount;

[Params(10)]
public int Rounds;

The benchmark basically simulates accessing 100 different keys with 1000 concurrent accessors per key with a factory that takes 100ms to execute, all of this 10 times sequentially.

If you are looking for maximum perf I can suggest you to store the entry options in a variable and pass that, so there will not be any extra allocation or wasted cpu cycles, something like this:

// STORE THIS SOMEWHERE (EG: GLOBALLY)
var entryOptions = cache.CreateEntryOptions()
  .SetDuration(configurationOptions.CacheDuration)
  .SetFailSafe(
    isEnabled: true,
    maxDuration: configurationOptions.ExpiredCacheDuration,
    throttleDuration: configurationOptions.FailSafeThrottleDuration
  )
  .SetFactoryTimeouts(
    softTimeout: configurationOptions.FactorySoftTimeoutSingle
  );

// LATER ON (YOUR CODE FROM ABOVE)

return await fusionCache.GetOrSetAsync<T>(
  key: GetCacheKey(),
  factory: async (ct) => await inner.GetCustomerConfiguration<T>(customerId, ct),
  options: entryOptions, // THIS WILL NOT ALLOCATE OR DO ANYTHING
  token
);

Hope this helps, let me know.

[jodydonetti]

Another thing to look at is that the comparison between different libraries should be done by using roughly the same features: in your example you are also using fail-safe and factory timeouts, which adds a little bit of work.

Having said that, I did another benchmark where I added one case that uses your fail-safe + factory timeout version, and the results are still better than LazyCache:

BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.4061)
13th Gen Intel Core i9-13900H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 9.0.300
  [Host] : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX2

Toolchain=InProcessEmitToolchain

| Method                                  | FactoryDurationMs | Accessors | KeysCount | Rounds | Mean     | Error    | StdDev   | P95      | Ratio | RatioSD | Gen0       | Gen1       | Gen2      | Allocated | Alloc Ratio |
|---------------------------------------- |------------------ |---------- |---------- |------- |---------:|---------:|---------:|---------:|------:|--------:|-----------:|-----------:|----------:|----------:|------------:|
| FusionCache                             | 100               | 1000      | 100       | 10     | 572.6 ms | 11.38 ms | 27.49 ms | 621.3 ms |  1.00 |    0.07 | 21000.0000 | 13000.0000 | 6000.0000 | 220.51 MB |        1.00 |
| FusionCacheFailSafeAndTimeouts          | 100               | 1000      | 100       | 10     | 666.1 ms | 13.06 ms | 13.97 ms | 682.2 ms |  1.17 |    0.06 | 32000.0000 | 14000.0000 | 4000.0000 | 381.02 MB |        1.73 |
| FusionCacheFailSafeAndTimeoutsOptimized | 100               | 1000      | 100       | 10     | 536.1 ms | 10.66 ms | 23.39 ms | 571.1 ms |  0.94 |    0.06 | 20000.0000 | 12000.0000 | 5000.0000 | 220.92 MB |        1.00 |
| LazyCache                               | 100               | 1000      | 100       | 10     | 725.2 ms | 14.24 ms | 20.42 ms | 755.9 ms |  1.27 |    0.07 | 45000.0000 | 18000.0000 | 4000.0000 |  526.3 MB |        2.39 |

Note that the FusionCacheFailSafeAndTimeouts benchmark is your code with the options => options syntax (so: allocations, etc), whereas FusionCacheFailSafeAndTimeoutsOptimized is the version with the entry options saved into a global variable.

Hope this helps.