[UI/UX support for attestations] Interface Audit

[kborchers]

• Audit current attestation displays on PyPI, RubyGems, and npm.
• Audit available documentation for attestation displays.
• Review parallel UX patterns from security-heavy interfaces in other domains (e.g., certificates, SBOMs).

[nlhkabu]

Breaking this up into smaller tickets:

• [UI/UX support for attestations] PyPI Audit #71
• [UI/UX support for attestations] npm Audit #72
• [UI/UX support for attestations] RubyGems Audit #73
• [UI/UX support for attestations] Audit security-heavy UIs #74
• [UI/UX support for attestations] Audit Supported Attestation Types #75
• [UI/UX support for attestations] Audit existing recommendations from OSSF #79

I will use our personas (#70) in these audits.

Re: "Audit available documentation for attestation displays" - @kborchers , @di - had you imagined this to be auditing the PyPI, npm and RubyGems docs, or are you looking for a more general audit of "what documentation exists on the web to help package consumers to understand attestations?"

For context, I have already audited the PyPI docs as part of the PyPI audit, and will do the same for npm and RubyGems. I think there could be some value in looking at wider attestations documentation, but am not clear if this is within the scope of the project.

[kborchers]

@nlhkabu I believe this was originally written to mean just an audit of the 3 package repos but I agree that there could be value in a slightly wider review in case there are others doing it better. I think a high level review of a couple more repos would be in scope but will defer to @di on a final decision there and a recommendation of which other repos to look at.

[nlhkabu]

Summary of Audit Recommendations - Input for UX Brief

Our audit of PyPI, npm, and RubyGems reveals that attestations are currently hard to discover, difficult to understand, and easily confused with weaker security signals.

The following recommendations provide a framework for making attestations clear, consistent, and user-friendly. The goal is to empower users to make better security decisions by translating complex technical data into intuitive visual cues and plain-language explanations.

Note: these recommendations are based on our audit only. Further research, including user interviews, is required to validate and refine them.

Guiding Principles

1. Establish a Clear Visual Hierarchy

To prevent user confusion, attestations must not be presented as just another feature in a list.

Instead, the design must visually communicate the relative importance of attestations vs other security signals. This may mean intentionally de-emphasizing weaker signals (like "Verified URLs" or basic file hashes on PyPI) to prevent user confusion.

2. Frame Attestations as a Trust Signal, Not a Guarantee

From OSSF's implementation guidance for build provenance:

The registry web UI should convey to consumers that a package has provenance, but not that it is automatically trustworthy. Remember, build provenance just provides links back to the source code and build instructions. Package consumers still need to follow those links to assess trustworthiness.

The iconography and text used must be chosen carefully to manage user expectations. The design should convey trust and verification without implying that a package is "100% secure."

3. Provide Recommendations for Adopting New Attestations

Repositories need a clear framework for evaluating which future attestations to support.

Priority should be given to attestations that are:

1. Verifiable and Reputable: Sourced from known, trusted entities (e.g., established vendors, open source foundations).
2. Understandable: The value proposition can be clearly and concisely explained to a non-expert user.
3. Actionable: The information helps a user make a clear decision or understand a specific risk mitigation.

Implementation Recommendations

1. Prioritise Discoverability and Prominence

To improve discoverability, we should explore adding a dedicated "Security" or "Provenance" link to the main navigation on PyPI and npm.

Example on GitHub:

On npm, this could be an additional tab on the top level menu:

On PyPI, this could be an additional menu item in the "Navigation" section in the sidebar:

Note: This is less critical on RubyGems, where the simpler UI already gives attestations greater prominence.

2. Use a Simple, Consistent Badge

Repositories should use a single, primary visual indicator (e.g., a stamp or shield) to signal that a package has at least one attestation. This indicator must be displayed consistently across the package repository to build a clear visual language:

1. As a filter on the search results page (if supported).
2. On the package summary card.
3. At the top of the package detail page.
4. Directly within the attestation section itself.

Examples on PyPI:

3. Differentiate Attestation Types with Thematic Badges

We should consider designing distinct but thematically linked badges for each of the different types of attestations. For example, all attestation badges could share the same shape but use different internal icons:

4. Use Plain-English Explanations and Tooltips

The interface must translate technical concepts into practical, plain-English explanations, e.g:

5. Design Clear Error and Warning States

The UI must include clear error states. When provenance cannot be established (e.g., the source repository is deleted), the UI should display an explicit warning, as already implemented by npm:

6. Mandate an "Escape Hatch" for Experts

Every UI must provide a clear and direct link that allows expert users to access the raw, unabridged attestation data and, ideally, its predicate definition.

See also OSSF's implementation guidance for build provenance for further recommendations.

Documentation recommendations

To be effective, new attestation documentation must be integrated thoughtfully into the repository's existing help content. We recommend repositories improve their documentation by:

1. Developing New Content: Create user-friendly documentation that answers "What is an attestation?" and "Why should I care?" in plain language. This content should be linked directly from the attestation UI.
2. Improving Existing Documentation Architecture: This may include creating a new top-level "Security" or "Trust & Safety" section in their documentation to house all related content.
3. Creating a Holistic Security Overview: Particularly on PyPI, a high-level page is needed to describe all security features (Attestations, Trusted Publishing, etc.), explaining their limitations and how they fit together. This helps users build an accurate mental model of the package repo's security.

Outstanding research questions

These recommendations are based on our audit; however, user research is critical to validate these assumptions. Key outstanding questions to explore include:

1. Terminology and Iconography:

• What visual language (icons, colors, badges) most effectively communicates "attestation" without overstating security and creating a false sense of trust?
• Which term - "attested," "verified," "signed," "with provenance" or another - is most accurate, while being intuitive for end-users?

2. User Perception and Trust:

• How should the absence of an attestation be presented? Should it be a neutral state or an explicit warning? How does this choice impact user trust and maintainer behavior? Note: we want to avoid creating a "badge of shame" for maintainers.
• How should we present information if a package previously had an attestation, but no longer does?
• What is the minimum set of information a user needs to see in a tooltip or summary to feel confident about a package's provenance?

3. Information Hierarchy for Attestation Details:

• What is the optimal information hierarchy for the attestation summary? Which specific fields (e.g., source commit, builder identity, transparency log link) must be surfaced by default to satisfy most user needs, and which can be deferred to the full, raw attestation view?
• What are the most effective plain-language labels and tooltips for each exposed field?