Lettuce plugin causes CPU load to surge

[oliver-zhang]

Describe the bug

We have a Java application that uses the Lettuce component to operate Redis. When we use OpenTelemetry JavaAgent to collect traces, we find that the CPU usage increases significantly. When we remove the Lettuce plugin, it returns to normal.
The screenshot of using jstack to find the thread that consumes the most CPU is as follows

Steps to reproduce

1. start app without javaagent -> cpu usage normal
2. start app with javaagent -> cpu usage increases significantly
3. start app with javaagent but exclude lettuce plugin -> cpu usage normal

Expected behavior

cpu usage normal

Actual behavior

CPU usage increases significantly

Javaagent or library instrumentation version

javaagent: 2.2.0 lettuce: 5.0.4 spring-data-redis: 2.1.5

Environment

JDK: 1.8
OS: centos

Additional context

No response

[steverao]

When we remove the Lettuce plugin, it returns to normal. The screenshot of using jstack to find the thread that consumes the most CPU is as follows

From your description, it can't be proved that the overhead is from Lettuce instrumentation of OTel Java Agent. The screenshot of jstack doesn't contain any OTel packages. You can compare the overheads by disabled and enabled Lettuce instrumentation.

[oliver-zhang]

When we remove the Lettuce plugin, it returns to normal. The screenshot of using jstack to find the thread that consumes the most CPU is as follows

From your description, it can't be proved that the overhead is from Lettuce instrumentation of OTel Java Agent. The screenshot of jstack doesn't contain any OTel packages. You can compare the overheads by disabled and enabled Lettuce instrumentation.

Thans for reply, I did come to this conclusion by disabling and enabling the lettuce plugin package. When we remove the Lettuce plugin, it returns to normal

[trask]

from the screenshot it looks like there's an IndexOutOfBoundsException (creating exceptions is expensive because of walking the stack and creatiung the stack trace)

does this IndexOutOfBoundsException only occur when the lettuce instrumentation is enabled? if so, that could point to a bug in the lettuce instrumentation

[oliver-zhang]

Looking at the lettuce plugin code, I found that the lettuce CommandArgs.toCommandString method is called when taking statment.

[oliver-zhang]

does this IndexOutOfBoundsException only occur when the lettuce instrumentation is enabled? if so, that could point to a bug in the lettuce instrumentation

Yes

[laurit]

@oliver-zhang would it be possible to provide a minimal sample application that reproduces the issue so we could debug what exactly causes it.

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed automatically if there is no response from the author within 7 additional days from this comment.