Integrate GCP client with telemetry #125278
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mhl-b
commented
Mar 20, 2025
...-gcs/src/main/java/org/elasticsearch/repositories/gcs/GoogleCloudStorageOperationsStats.java
Outdated
Show resolved
Hide resolved
mhl-b
commented
Mar 20, 2025
...sitory-gcs/src/main/java/org/elasticsearch/repositories/gcs/GoogleCloudStorageBlobStore.java
Outdated
Show resolved
Hide resolved
mhl-b
commented
Mar 20, 2025
...s/src/main/java/org/elasticsearch/repositories/gcs/GoogleCloudStorageHttpStatsCollector.java
Show resolved
Hide resolved
mhl-b
added
>enhancement
:Distributed Coordination/Snapshot/Restore
|
Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination) |
|
Hi @mhl-b, I've created a changelog YAML for you. |
mhl-b
commented
Mar 21, 2025
| * @param failIfAlreadyExists whether to throw a FileAlreadyExistsException if the given blob already exists | ||
| */ | ||
| void writeBlob(OperationPurpose purpose, String blobName, BytesReference bytes, boolean failIfAlreadyExists) throws IOException { | ||
| void writeFullBlob(OperationPurpose purpose, String blobName, BytesReference bytes, boolean failIfAlreadyExists) throws IOException { |
There was a problem hiding this comment.
Overloaded writeBlob is hard to read, especially when we call one from the inside of another.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR integrates GCP blob store with APM telemetry. REST API stats measures only requests count for
InsertObject,GetObject,ListObjects. But telemetry tracks wider range of operations and their outcomes. Requests and operations are tracked separately. There are 3 counters for request exceptions:exception,throttle, andrangeNotSatisfied. Theexceptionmetric includes all failures including throttles and ranges.Currently, it's not possible to capture number of throttles/exceptions per single operation(a group of requests). So histograms are updated on request basis:
exceptionHistogram,throttleHistogram.Also it's not possible to measure time per request and update
httpRequestTimeInMillisHistogram. So these values will be 0. With future improvements in GCP SDK we might be able to use provided OpenTelemetry to capture missing metrics.