Using backplane

[efortunov]

Hello. If I have problems connecting redis how will the cache behave in this case?

[jodydonetti]

Hi @efortunov , I'm not sure I understand what you are asking, but I'll try to give you a complete picture of what FusionCache does in case of problems with the distributed cache or the backplane.

All the options mentioned below, that can be used to customize the behavior of FusionCache, are either on the FusionCacheOptions class (valid for an entire cache instance) or on the FusionCacheEntryOptions class (valid for each cache entry/method call).

You can read more about the available options in the dedicated docs.

Distributed Cache (more)

When FusionCache talks with the distributed cache it can get an error, for various reasons: the Redis instance may have a problem, the network connection may have a problem or something else.

Normally these errors are logged (and you can control with which LogLevel via the option DistributedCacheErrorsLogLevel) but they don't make the entire call throw: this is because the distributed cache is considered a "secondary" component. In case you want to throw when there's a distributed cache error you can set the option ReThrowDistributedCacheExceptions to true.

When there are errors, sometimes the root cause may persist for some time and not be just a single occurrence: in this case it can help to leave the distributed cache alone for some time instead of keep bombarding it with requests. To do that you can set the option DistributedCacheCircuitBreakerDuration so that after the first error the distributed cache will be temporarily ignored for that amount of time.

On top of "hard errors", there may also be just slowdowns, meaning that if Redis or the network are temporarily congested, you may actually get a response, but in a veeery long time. For that you can set the the options DistributedCacheSoftTimeout/DistributedCacheHardTimeout and that will make sure that, no matter what, a slow distributed cache will never slow down your code. When one of these soft/hard timeout occurs, the event is also logged with the LogLevel specified via the option DistributedCacheSyntheticTimeoutsLogLevel.

Finally, the distributed cache operations are normally awaited, but this can be avoided by just setting the option AllowBackgroundDistributedCacheOperations to true, so that all the distributed cache operations will be executed in the background.

Backplane (more)

For the backplane you basically have the same scenarios, the same possible problems and almost the same options:

• BackplaneErrorsLogLevel to control the log level for backplane errors
• BackplaneSyntheticTimeoutsLogLevel to control the log level for the backplane synthetic timeouts
• BackplaneCircuitBreakerDuration to control the amount of time to leave the backplane alone, in case of errors
• AllowBackgroundBackplaneOperations to execute the backplane operations in the background. Differently from the distributed cache though, this is enabled by default so you don't have to do anything

One extra feature of the backplane is the auto-recovery: in case the backplane becomes temporarily unavailable, it will automatically put the missed notifications in a local queue, and will try to send them as soon as it will become available again.

Special care has been put into this feature, so that notifications for cache entries that have already become useless are not sent (think a notification for a cache entry with a 1s duration, which should not be sent if meanwhile 10min have been passed). Also, the order in which they are sent is automatically optimized to obtain the best possible coherence for the state of the entire cache.

Auto-recovery is automatically enabled, but it can be turned off via the option EnableBackplaneAutoRecovery, if you prefer. You can also specify how many items at most can be put into the queue (to avoid consuming too much memory) via the option BackplaneAutoRecoveryMaxItems: by default is set to 100, but you can even set it to null to remove any limit.

Fail-Safe (more)

What happens when a blocking error occurs (eg: while executing a factory, calling the distributed cache, etc)?
The fail-safe mechanism is here to help: if you enable it, it can avoid an error by returning a stale copy of the cached value. In most cases this would be better than just throwing an exception.

Conclusions

As you can see FusionCache should have you covered in any problematic situation: in case you are seeing something missing let me know, and we may think about a new feature or option to be added.

Hope this helps.

[bharatkumarmori]

Let us say that Redis server is temporarily become unavailable. Server reboot or something like this.

During this time, if there is any request coming to Fusion Cache, then it will return "null" for those requests.
var value = await _cache.GetOrDefaultAsync(key);

How do we differentiate that value is genuinely not available in Distributed Cache OR the Distributed cache server itself is not available?

[jodydonetti]

Hi @bharatkumarmori

Let us say that Redis server is temporarily become unavailable. Server reboot or something like this.

During this time, if there is any request coming to Fusion Cache, then it will return "null" for those requests. var value = await _cache.GetOrDefaultAsync(key);

By using a read-only method (eg: TryGet/GetOrDefault) you are explicitly saying "don't go to the database, just give me what is in the cache".

So let's say there's nothing in the cache, both L1 (memory) and L2 (distributed), or that there's nothing in L1 and L2 is unavailable: by calling GetOrDefault you'll get back the default value, as the name implies.

How do we differentiate that value is genuinely not available in Distributed Cache OR the Distributed cache server itself is not available?

If instead you call TryGet you'll be able to differentiate between the 2 things: the default value being returned or the value not being in the cache.
Read here for more.

Also, probably not interesting in this case, but still: by setting ReThrowDistributedCacheExceptions to true you can even have a chance to catch the exception, either a generic FusionCacheDistributedCacheException or, by setting ReThrowOriginalExceptions to true, directly the original one.
Read here for more.

Hope this helps.

[bharatkumarmori]

Thanks for the quick reply @jodydonetti.
Yes, TryGet helps to differentiate between 'value being present in Cache' or 'Not available at all'.

Now I am using ReThrowDistributedCacheExceptions to throw and catch the Exceptions. Hope setting this flag to true will not scale down any capabilities of Fusion Cache architecture. :)

[jodydonetti]

Thanks for the quick reply @jodydonetti. Yes, TryGet helps to differentiate between 'value being present in Cache' or 'Not available at all'.

Great!

Now I am using ReThrowDistributedCacheExceptions to throw and catch the Exceptions. Hope setting this flag to true will not scale down any capabilities of Fusion Cache architecture. :)

Not really. Having said that, if you are not planning on actively acting on that you can just avoid re-throwing it and, at most, take a look at the logs or do some observability (traces, metrics, etc) to keep an eye on errors.

Hope this helps.