From d0bc85f1b6c8a777f9f0229b55a5cf8c9863f89b Mon Sep 17 00:00:00 2001 From: raulk Date: Mon, 23 Jun 2025 11:35:53 +0100 Subject: [PATCH] `engine_getBlobsV2`: add `partialResponse` flag to enable partial hits Add optional `partialResponse` boolean flag to reinstate `engine_getBlobsV1`'s partial hit behavior, needed for optimization work. Background: - `engine_getBlobsV1` currently achieves a 70-80% hit rate, expected to remain high even as blob volume increases. - EIP-4844 introduced full blob sidecar broadcasts at the CL layer; with PeerDAS, the CL now gossips column sidecars. - Because columns sidecars require all cells to be present, support for partial hits was previously dropped [in this commit][cad419]. Motivation: - The P2P networking team is actively working on backwards-compatible optimizations that can be deployed in-between forks. - For instance, we're prototyping cell-level deltas to facilitate column reconciliation based on local availability. - This would reduce redundant full-column transmissions and reclaim useful bandwith to further scale blobs. Remarks: - We're aware this change might be procedurally late, but its ease of implementation and its ability to unlock an entire class of interim improvements makes it worthwhile to consider (and we only noticed this opportunity in Berlin). - We'll happily submit PRs to implement this in EL and CL clients if needed. Just let us know! - We gated partial responses behind a flag to avoid unnecessary overhead while unused. [cad419]: https://github.com/ethereum/execution-apis/pull/630/commits/cad4194e3fa37359a1be95e4aad2752d69691077 --- src/engine/openrpc/methods/blob.yaml | 7 +++++++ src/engine/osaka.md | 16 +++++++++++----- 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/src/engine/openrpc/methods/blob.yaml b/src/engine/openrpc/methods/blob.yaml index 1dbcea57f..694ac5b95 100644 --- a/src/engine/openrpc/methods/blob.yaml +++ b/src/engine/openrpc/methods/blob.yaml @@ -45,6 +45,11 @@ type: array items: $ref: '#/components/schemas/hash32' + - name: Partial response requested + required: true + schema: + title: partialResponse + type: boolean result: name: List of blobs and corresponding cell proofs schema: @@ -60,6 +65,8 @@ - name: Blob versioned hashes value: - '0x000657f37554c781402a22917dee2f75def7ab966d7b770905398eba3c444014' + - name: Partial response requested + value: true result: name: List of blobs and corresponding cell proofs value: diff --git a/src/engine/osaka.md b/src/engine/osaka.md index e9e059718..2b787319a 100644 --- a/src/engine/osaka.md +++ b/src/engine/osaka.md @@ -96,11 +96,14 @@ Consensus layer clients **MAY** use this method to fetch blobs from the executio * method: `engine_getBlobsV2` * params: 1. `Array of DATA`, 32 Bytes - Array of blob versioned hashes. + 2. `partialResponse` : `BOOLEAN` - Controls whether to return partial hits. * timeout: 1s #### Response -* result: `Array of BlobAndProofV2` - Array of [`BlobAndProofV2`](#BlobAndProofV2) or `null` in case of any missing blobs. +* result: `Array of BlobAndProofV2`: + * if `partialResponse=false`: Array of [`BlobAndProofV2`](#BlobAndProofV2), or `null` in case of ANY missing blobs. + * if `partialResponse=true`: Array of [`BlobAndProofV2`](#BlobAndProofV2), containing `null` for EACH missing blob. * error: code and message set in case an error occurs during processing of the request. #### Specification @@ -108,12 +111,15 @@ Consensus layer clients **MAY** use this method to fetch blobs from the executio Refer to the specification for [`engine_getBlobsV1`](./cancun.md#engine_getblobsv1) with changes of the following: 1. Given an array of blob versioned hashes client software **MUST** respond with an array of `BlobAndProofV2` objects with matching versioned hashes, respecting the order of versioned hashes in the input array. -2. Client software **MUST** return `null` in case of any missing or older version blobs. For instance, +2. When `partialResponse=false`, client software **MUST** return `null` in case of any missing or older version blobs. For instance, 1. if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has data for blobs `A` and `C`, but doesn't have data for `B`, the response **MUST** be `null`. 2. if the request is `[A_versioned_hash_for_blob_with_blob_proof]`, the response **MUST** be `null` as well. -3. Client software **MUST** support request sizes of at least 128 blob versioned hashes. The client **MUST** return `-38004: Too large request` error if the number of requested blobs is too large. -4. Client software **MUST** return `null` if syncing or otherwise unable to serve blob pool data. -5. Callers **MUST** consider that execution layer clients may prune old blobs from their pool, and will respond with `null` if a blob has been pruned. +3. When `partialResponse=true`, client software **MUST** place responses in the order given in the request, using `null` for any missing blobs. For instance, + 1. if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has data for blobs `A` and `C`, but doesn't have data for `B`, the response **MUST** be `[A, null, C]`. + 2. if the request is `[A_versioned_hash_for_blob_with_blob_proof]`, the response **MUST** be `null`. +4. Client software **MUST** support request sizes of at least 128 blob versioned hashes. The client **MUST** return `-38004: Too large request` error if the number of requested blobs is too large. +5. Client software **MUST** return `null` if syncing or otherwise unable to serve blob pool data. +6. Callers **MUST** consider that execution layer clients may prune old blobs from their pool, and will respond with `null` if a blob has been pruned. ### Update the methods of previous forks