Activate improved compression

[Mustang98]

• Activate improved compression for all types of broadcast
• Activate compression with state for block_broadcast and block_full
• Turn on compression for broadcast in public overlay and block_full
• Extend benchmark logs with more details

All types of broadcast and compression configuration for them:

Depends on #1944

[github-actions]

• Block proof parsing for state fetch is incorrect for shard blocks. extract_prev_blocks_from_proof (validator/full-node-serializer.cpp:144-170) unpacks a BlockProof record, but shard broadcasts/data-full carry a proof link, not a block proof (ValidateBroadcast still calls create_proof_link for non-masterchain). As soon as need_state_for_decompression returns true, all shard block broadcasts/data-full take this path (validator/full-node-shard.cpp:822-882; validator/net/download-block-new.cpp:229-265), the parser errors with “invalid block proof in broadcast”, and the message is dropped before decompression. With state-based compression now enabled by default, this would prevent processing shard broadcasts and shard block downloads entirely.

[github-actions]

Findings

• crypto/vm/boc-compression.cpp:716 – When rebuilding a pruned branch from the provided state during “with state” decompression, the code strips the highest significant bit from the pruned branch mask (base_mask = mask & (highest_bit_mask - 1)), then builds the LevelMask and max_level from this reduced mask. The highest significant level is never marked/significant, so the reconstructed pruned branch omits its top-level hash/depth while still advertising the original mask. Finalization can fail or produce an incorrect cell hash whenever a Merkle Update’s left subtree contains a pruned branch with level >0. Use the full mask to build the LevelMask/max_level and iterate over all significant levels.

Notes / Risks

• State-dependent decompression will still fail for any MU left subtree containing pruned branches until the mask handling is fixed.

[github-actions]

• validator-session/candidate-serializer.cpp:44-48 now calls compress_candidate_data without returning a decompressed size, yet still builds a validatorSession_compressedCandidate (V1) with an uninitialized decompressed_size. Receivers will treat the payload as legacy LZ4/V1 and use that bogus size, while the data is encoded with the new improved compression, so decompression will fail. Use the V2 schema (or restore size calculation) and ensure the improved_compression path is selected on decode.

[github-actions]

• validator-session/candidate-serializer.cpp:44-48 – serialize_candidate now calls compress_candidate_data (which produces ImprovedStructureLZ4 data and no longer returns the decompressed size) but still wraps the result in the legacy validatorSession_compressedCandidate message and populates decompressed_size with an uninitialized value. Peers will try to LZ4-decompress using that garbage size and fail to deserialize the broadcast. Use the V2 schema (with validatorSession_compressedCandidateV2) or revert to baseline compression and pass a real decompressed size.

[github-actions]

• validator/full-node-serializer.cpp:146 uses BlockProof to derive prev blocks, but shard block broadcasts/dataFull carry proof links (see is_proof_link usage). When V2 state-based compression is used, shard broadcasts and block_full downloads hit this helper (e.g., full-node-shard.cpp:848, validator/net/download-block-new.cpp:243) and unpack_cell will fail on a proof link, so state is never fetched and the message is dropped/aborted. Need to handle proof links when is_proof_link is true (or otherwise extract prev block IDs) so shard blocks can be decompressed with state.

[github-actions]

I can’t run any shell commands in this environment (every command fails with a landlock “Permission denied”), so I can’t access the git diff. Please paste the PR diff or the specific files/patches you want reviewed, and I’ll give a focused review of those changes.

[github-actions]

I can’t access the repository or run any commands in this environment: every shell command fails with a landlock “Permission denied” error, and there are no MCP resources available to read files. Please provide the diff (or relevant file snippets) for the PR range so I can review the changes.

[github-actions]

I can’t access the repo or run any commands in this environment: every shell invocation fails with error running landlock: Permission denied, so I’m unable to inspect the PR diff or files to perform a review. Please provide the diff or re-run with a working sandbox so I can review the changes.

[github-actions]

I can’t access the repository: every shell command fails with a landlock PermissionDenied in this environment. Please paste the PR diff (or the specific files/patch hunks) so I can review the changes only.

[github-actions]

No issues found.

[github-actions]

• validator/full-node-serializer.cpp:114 + validator/manager.cpp:962: extract_prev_blocks_from_proof drops the after_split flag from unpack_block_prev_blk_try, and wait_state_by_prev_blocks always returns the raw prev block state (or merge). For after-split blocks the MU left subtree is derived from a split state, so using the parent shard state will likely make boc_decompress_improved_structure_lz4 fail (refs mismatch) or reconstruct the wrong tree. Consider propagating after_split and applying split logic similar to ValidatorManagerImpl::wait_prev_block_state in validator/manager.cpp:980.