Can Spring AOT help with @Cacheable annotation?

[vlsi]

As far as I understand, the current implementation goes with a MethodInterceptor which adds runtime overhead for every invoked method.

I wonder if Spring AOT could be used to generate the interceptors at the build time, so the runtime does not pay overhead.

[snicoll]

To answer that question effectively, we'd need to quantify what the overhead is. There is a single interceptor that uses metadata for the target metadata and this metadata is computed the first time the method is invoked and then cached. As such, yes, we could determine the methods to cache upfront but we have to balance the cost of doing so with the benefit.

Have you quantified the runtime overheard or are you generally asking?

[vlsi]

That is a genuine question based on https://kora-projects.github.io/kora-docs/en/documentation/general/ talks and benchmarks.
They have sourcecode gen for things like @Log, @Cache, @Transactional and they somehow claim they are whatever x%% faster than Spring.

Of course, I doubt it is important to have ultra-fast @Transactional (at the end of the day, a single DB call would likely outweigh the overhead of @Transactional aspect), however, it might be important to have ultra-low overhead for @Cachable, @Loggable, @Lazy.
So I wonder if Spring AOT is the right technology to make those kind of specializations.
An alternative option could be going for invokedynamic so the specialization delays till the runtime (just like the recent Java has for String concatenation, switch statements). However, going with indy might defeat GraalVM native images (I guess GraalVM hard-codes indy they support), so it looks like sourcegen approach might be good enough.

---

@Cacheable's not-so-high performance is somehow known in the community, so I am leaning towards that improving @Cachable performance makes sense for Spring Framework:

• https://medium.com/@nikem/is-spring-cache-abstraction-fast-enough-for-you-a6a5ea1542a9
• https://www.javacodegeeks.com/2019/04/more-about-spring-cache-performance.html
• https://nurkiewicz.com/2013/01/cacheable-overhead-in-spring.html
• https://dzone.com/articles/jmh-benchmark-rest-apis (it relates to @Around, however, it applies to AspectJ-based aspects)

Frankly speaking, I have not benchmarked the overhead of @Cacheable, @Lazy, and others with the latest Spring Framework versions. At the same time, I do not have a production instance suffering from bad @Cacheable performance (well, I do not have Spring-based apps in production).

However, I wonder if Spring maintainers consider it important to optimize @Cacheable, so I raise an issue.

[snicoll]

Without any sort of concrete evidence that what you're referring to is adding an overhead, this issue is not actionable.

[vlsi]

@snicoll , I've created a benchmark, and it looks like there's a true overhead for both cglib-based and jdkproxy-based implementations.

See https://github.com/vlsi/aop-benchmarks

The results with Java 21 and Apple M1 are as follows:

    Benchmark                         (implementation)  Mode  Cnt     Score     Error   Units
    CacheableBenchmark                          direct  avgt    5     0,707 ±   0,020   ns/op
    CacheableBenchmark:gc.alloc.rate            direct  avgt    5    ≈ 10⁻⁵              B/op
    CacheableBenchmark                    spring-cglib  avgt    5   190,043 ±   3,304   ns/op
    CacheableBenchmark:gc.alloc.rate      spring-cglib  avgt    5   608,001 ±   0,001    B/op
    CacheableBenchmark                 spring-jdkproxy  avgt    5   176,445 ±  10,604   ns/op
    CacheableBenchmark:gc.alloc.rate   spring-jdkproxy  avgt    5   592,001 ±   0,001    B/op
    CacheableBenchmark                          manual  avgt    5     7,408 ±   0,295   ns/op
    CacheableBenchmark:gc.alloc.rate            manual  avgt    5    ≈ 10⁻⁴              B/op

In other words:

1. Manual caching with ConcurrentHashMap takes 7ns while spring-based @Cacheable takes 200ns which is ~30 times slower
2. Manual caching does not allocate heap while spring-based implementations allocate 500 bytes for every @Cacheable execution even in the case the values are perfectly cached

Would you consider this as a valid concrete evidence for the overhead?

[snicoll]

No. You're comparing apples and oranges. The lookup of the metadata is one fraction of what the cache abstraction is doing and that's what would be the target for AOT. This benchmark replacing everything by manual caching is irrelevant.

[vlsi]

What else am I missing here? I would be glad to add that it to "manual" so it becomes more "apples to apples".