Skip to content

[WIP]: Do not store tag in uninhabited enum variants, or in the single inhabited variant. #145337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

zachs18
Copy link
Contributor

@zachs18 zachs18 commented Aug 13, 2025

This PR contains 3 or 4 changes which could probably be split up:

First (first commit): Remove TmpLayout from layout_of_enum. 09a3846 from #103693 made LayoutData not be interned in Variants::Multiple::variants anymore, so there's no need for TmpLayout. This should have no noticable effect.

Second (second commit): Do not make space for the enum tag in uninhabited variants in the tagged enum layout calculation. This optimizes enums where the uninhabited variants are the largest, and reduces their size by up to the size of the tag. e.g. enum Foo { A, B(i32, !) } was 8 bytes A: (tag, uninit), B: (tag, i32, !) and is now 4 bytes A: (tag,), B: (i32, !). Additionally, enums with only zero-sized uninhabited variants are special-cased to return an uninhabited layout of the correct size and alignment without a tag at all, as otherwise we get weird edge cases1

Third-ish (seventh commit) (needed for the second and fourth to be sound with offset_of! on uninhabited variants' fields): Keep track of the uninhabited variant layouts, even if we use an untagged or uninhabited layout. This is necessary as otherwise offset_of! will "conjure" a layout for uninhabited variants where all fields are at offset 0, which conflicts with partial initialization2.

Fourth (ninth commit): Add an additional enum layout that does not encode a tag. This is similar to the existing "single-present-variant" optimization, but works even if some of the uninhabited variants are "present" (have non-1-ZST fields). This works by calculating the layout of the inhabited variant as a struct, and padding it to the maximum size and alignment necessary to fit each uninhabited variant.

The layout changes probably need an MCP and I haven't squashed fully, so draft for now.

r? ghost

Footnotes

  1. Specifically, without the special-case, enum Foo { A(Aligned2Never), B(Aligned2Never) } got a size-0 layout with an i16 tag field, which I assume would cause problems.

  2. This isn't an issue before this PR because the layout algorithm only produces Variants::Empty or Variants::Single when all uninhabited variants are "absent", i.e. have only 1-ZST fields

@rustbot rustbot added A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 13, 2025
@rust-log-analyzer

This comment has been minimized.

@zachs18 zachs18 force-pushed the enum-uninhabited-or-single-variant-no-tag branch from 7095fd5 to 76ded17 Compare August 13, 2025 06:41
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants