Skip to content

feat(io): fix ancillary API to avoid UB#737

Open
fantix wants to merge 11 commits intocompio-rs:masterfrom
fantix:refactor-cmsg-3
Open

feat(io): fix ancillary API to avoid UB#737
fantix wants to merge 11 commits intocompio-rs:masterfrom
fantix:refactor-cmsg-3

Conversation

@fantix
Copy link
Contributor

@fantix fantix commented Mar 6, 2026

This PR was initally created to enhance the AncillaryBuilder API to be like:

use compio_io::ancillary::{AncillaryBuf, ancillary_space};

const LEVEL: i32 = 1;
const TYPE: i32 = 2;

// Build a buffer containing two `u32` ancillary messages.
let mut buf = AncillaryBuf::<{ ancillary_space::<u32>() * 2 }>.new();
let mut builder = buf.builder();
builder.try_push(LEVEL, TYPE, 42u32).unwrap();
builder.try_push(LEVEL, TYPE, 43u32).unwrap();
assert!(builder.try_push(LEVEL, TYPE, 44u32).is_none()); // buffer is full
// no builder.finish() as `set_len()` is done per try_push()

This is not a breaking change because:

  1. AncillaryBuilder is never released
  2. CMsgBuilder preserves old deprecated behavior

UPDATE

During review, @George-Miao spotted an API design defect that allows users to trigger undefined behaviors sending/receiving control messages using structs that have paddings between fields.

This PR is then repurposed to fix this issue in a way that:

  1. Added a new trait AncillaryData that is semantically safe to encode/decode any types of value to/from control message data payload;
  2. Added optional bytemuck dependency to automatically implement AncillaryData for basic types;
  3. compio-io now implements AncillaryData for networking types required by compio-quic.

This PR also changed the interface to copy data during encoding/decoding, with a side-effect of no longer requiring data alignment.

Now it looks like:

use compio_io::ancillary::{AncillaryBuf, ancillary_space};

const LEVEL: i32 = 1;
const TYPE: i32 = 2;

// Build a buffer containing two `u32` ancillary messages.
let mut buf = AncillaryBuf::<{ ancillary_space::<u32>() * 2 }>.new();
let mut builder = buf.builder();
builder.push::<u32>(LEVEL, TYPE, &42).unwrap();  // this is now a Result instead of Option
builder.push::<u32>(LEVEL, TYPE, &43).unwrap();
assert!(builder.push(LEVEL, TYPE, &44u32).is_err()); // buffer is full

And AncillaryRef.data() returns Result<T, CodecError> instead of &T.

Refs #730 #734

@fantix fantix mentioned this pull request Mar 6, 2026
4 tasks
@Berrysoft
Copy link
Member

Berrysoft commented Mar 6, 2026

We need to be cautious to allocations here. The previous CMsgBuilder avoid allocations through accepting a maybe-uninit slice. This new builder introduces Box.

Somehow I would like to avoid the box. Maybe make the CMsgIter storing an offset to the buffer instead of a pointer. That way, we can store the whole buffer in the CMsgIter, instead of referencing an outer one.

@fantix fantix force-pushed the refactor-cmsg-3 branch from b96b649 to 034429c Compare March 6, 2026 03:29
@fantix
Copy link
Contributor Author

fantix commented Mar 6, 2026

Good point on allocation! This change was partially to eliminate the now-unnecessary finish() + unsafe set_len(), so I just pushed a fix to store an &mut AncillaryBuf in the builder directly.

@George-Miao George-Miao added package: io Related to compio-io enhancement New feature or request labels Mar 6, 2026
@George-Miao George-Miao changed the title feat(io): new AncillaryBuf::build() API feat(io): new AncillaryBuf::build() API Mar 6, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new AncillaryBuf::builder()-style construction API for ancillary/control messages in compio-io, updates downstream usage (QUIC socket + tests), and preserves the deprecated compio-net::CMsgBuilder API via a compatibility type.

Changes:

  • Refactors AncillaryBuilder to build into an AncillaryBuf<N> and adds AncillaryBuf::builder().
  • Introduces a deprecated-compat CMsgBuilder and updates compio-net’s deprecated alias to point to it.
  • Adds docs/example + a new ancillary_space<T>() helper intended for const-generic sizing; adjusts tests and Cargo test gating.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
compio-quic/src/socket.rs Switches send-side control-message construction to AncillaryBuf::builder() and removes manual finish/set_len.
compio-net/src/lib.rs Updates deprecated CMsgBuilder alias and deprecation note to reflect new API path.
compio-io/tests/ancillary.rs Migrates tests to AncillaryBuf-based builder and uses compat CMsgBuilder for alignment panic test.
compio-io/src/ancillary/windows.rs Makes wsa_cmsg_space pub(crate) for reuse.
compio-io/src/ancillary/mod.rs Implements new builder API, adds compat CMsgBuilder, adds docs/example, and adds ancillary_space<T>().
compio-io/Cargo.toml Removes aligned-array dev-dep and gates the ancillary integration test behind the ancillary feature.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Berrysoft Berrysoft added this to the v0.19 milestone Mar 11, 2026
@fantix fantix changed the title feat(io): new AncillaryBuf::build() API feat(io): fix ancillary API to avoid UB Mar 16, 2026
@Berrysoft Berrysoft requested a review from George-Miao March 16, 2026 14:14
@fantix
Copy link
Contributor Author

fantix commented Mar 16, 2026

@AsakuraMizu I would love to have your review on this change too!

Berrysoft
Berrysoft previously approved these changes Mar 16, 2026
Copy link
Member

@Berrysoft Berrysoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it LGTM currently, but would like @George-Miao to review.

builder.try_push(libc::IPPROTO_IP, libc::IP_TOS, ecn as libc::c_int);
builder
.push(libc::IPPROTO_IP, libc::IP_TOS, &(ecn as libc::c_int))
.expect("cmsg push");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be tolerant of push failures.

Copy link
Contributor Author

@fantix fantix Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, good catch! This is a change-of-behavior indeed.

push() now fails when:

  1. Insufficient capacity in the buffer
  2. encode() fails with custom error (not currently used)

while try_push() was returning None when:

  1. Insufficient capacity in the buffer
  2. Data type is not aligned to cmsg (no longer used)

So, changing expect("cmsg push") to ok() would preserve the old behavior.

However, given that the buffer size was set to a fixed number of 128 bytes, my question is like, is 128 guaranteed to contain the worst-case CMSG_SPACE() sum of all three:

  1. ECN
  2. pktinfo / destination address
  3. GSO

If yes, I think we should keep the panic; otherwise, it would be ideal to compute a proper total size to replace the magic number.

... unless, it was by design that, if any of the 3 control messages go beyond 128 bytes, we silently ignore the issue, skip the rest, and send with whatever cmsg we can?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request package: io Related to compio-io

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants