[UI/UX support for attestations] Improving the RubyGems documentation

[nlhkabu]

Improving the RubyGems Documentation - DRAFT (v 0.1)

Goals

The goals of the OSSF attestations project include:

• Helping users to understand attestations, including what they are, how they work and what their limitations are
• Helping users to understand attestations within the context of other security features
• (slightly out of scope - bonus objective) Directing maintainers towards workflows that generate attestations (i.e. by using Trusted Publishing)

For new security documentation to be successful, it should be contained within an overall structure that allows users to find and understand the content they need. On auditing the existing RubyGems documentation, we found that its structure and clarity could be improved. Therefore, we extended our goals for the RubyGems documentation to also include:

• Reorganising the documentation to help users better find the content they are looking for
• Differentiating between content for consumers and producers

Summary of recommendations

We recommend updating the RubyGems documentation to:

1. Improve the structure (information architecture) to help users differentiate between documentation for gem consumers and documentation for gem authors
2. Update the "What is a gem" and "RubyGems basics" content to enhance the documentation introduction for end users (especially beginners)
3. Add a new "gem security" page to explain security concepts (including attestations) in easy to understand terms
4. Update gem publishing and security guides to prioritize Trusted Publishing
5. Add search functionality (e.g "search docs" or "search guides")
6. Enhance the reading experience of each guide / page, by exposing heading hyperlinks and providing a table of contents

The rest of this document addresses points 1-4.
Points 5 & 6 can/should be included in UI recommendations (to be made during the design phase), if determined to be within the project scope.

Recommendations

See https://docs.google.com/document/d/1WN33a4uV-YZs8Xh37_T2m3Nt_aH3T7aY21ZJQtKugh4/edit?usp=sharing

(Content has been moved to GoogleDocs to allow for easier commenting/feedback/collaboration. Once recommendations are finalised, they will be re-published here.)

[nlhkabu]

Note to self: We are missing recommendations on how to improve the guide homepage

[nlhkabu]

I am looking for feedback on the above proposal, both in general (any/all feedback feedback is welcome!) and in response to the following questions:

1. Does the proposed information architecture (separating content for gem consumers from gem authors) seem like a useful and logical approach?

2. Is the security content accurate and complete? For example, in the "User's guide to gem security," have I missed any common practices or threats?

3. Is the tone appropriate for a wide audience (including non-native English speakers), and is the terminology (e.g., "provenance," "dependency confusion") explained well and used correctly?

4. Are there any upcoming features on the RubyGems or Bundler roadmaps that should be considered for this documentation? Specifically, are there any plans for CLI support related to attestations, such as viewing them with the gem command or verifying them with Bundler?

5. Are there any great resources we should link to to help gem consumers understand security? Likewise, are there any great security resources we should point maintainers to?

Thank you in advance!

[nlhkabu]

Points 5 & 6 can/should be included in UI recommendations (to be made during the design phase), if determined to be within the project scope.

@di @kborchers: looking for specific feedback on this point. Would improving the RubyGems documentation UI to better house security/attestations documentation be within or out-with the scope of this project?

[nlhkabu]

I've updated this content based on new information surfaced during a recent user interview:

Verifying source code security

RubyGems.org is a repository for distributing packages, not a service for auditing them. RubyGems does not guarantee the quality or security of the code within any given gem. Assessing the trustworthiness of a gem's source code is the installer's responsibility.

The only definitive way to verify a gem's security is to inspect the source code yourself. When a full code audit is not practical, the signals listed below can help you build confidence and assess the trustworthiness of a new gem:

• Project activity & popularity

  • A high download count suggests wide usage (though is not itself a signal for quality, as many niche or new projects may have fewer downloads)
  • An active commit history and recent releases indicate the project is maintained
  • A large community can mean more eyes on the code, potentially finding issues faster
  • Sponsorship or affiliation with reputable organizations may indicate more resources for securing the codebase

• Security posture

  • The presence of a SECURITY.md file in the repository (or a security policy on the project website) shows the project has a formal process for handling vulnerability reports
  • How maintainers respond to security issues reported in the project's issue tracker is another key indicator of its security posture

• Dependencies

  • Each dependency is another project you are implicitly trusting, which subsequently increases your attack surface
  • A minimal dependency tree, or one composed of other well-known projects, can be a positive signal
  • A gem's dependencies are listed in its .gemspec file. You can explore dependencies using the gem dependency command, or through third-party services like GitHub's dependency graph
  • GitHub's Dependabot can also be used to scan your project for dependencies with known vulnerabilities

• Known vulnerabilities

  • The RubySec database is the central, community-maintained repository of all publicly disclosed security advisories for RubyGems. It can be searched manually, and it serves as the data source for bundler-audit - a tool you can use to scan your project's dependencies for known vulnerabilities
  • Third-party security platforms like Socket or Snyk also provide vulnerability information for gems.

[nlhkabu]

Updated the attestations section to use the term "certificate" instead of "document" and "receipt". Also added the following point to attestations limitations (as surfaced during a recent user interview):

Implicit Trust: When you trust an attestation, you are implicitly trusting the processes and authority of the organisation that issued it

[di]

Updated the attestations section to use the term "certificate" instead of "document" and "receipt".

I would be careful with this as it's overloading the term "certificate" a bit -- in the context of attestations, it MAY mean that the attestation was signed with help from a Certificate Authority (e.g. Sigstore's Fulcio) and resulted in an entry in a Certificate Transparency Log (e.g. Sigstore's Rekor). In this sense, the attestation is not the literal certificate, so this metaphor might be confusing.

[nlhkabu]

Thanks for this feedback @di:

A little confused:

Right now, the attestations that exist across packaging repositories#75 all have a signed rekor log.

Question: Is there a scenario where a packaging repository would/could generate an attestation without creating a signed rekor log?

If I'm understanding you correctly, what you are saying is that there is a distinction between the "attestation" itself and the resulting "certificate"

• "Attestation" is the issuing authority saying something about the origin of the artifact
• "Certificate" is the signed rekor log

Is that correct?

[di]

Question: Is there a scenario where a packaging repository would/could generate an attestation without creating a signed rekor log?

Yes, it's possible to create and sign an attestation without using Sigstore (e.g. using a public/private keypair). For example, SLSA attestations from Cloud Build do not use Sigstore: https://slsa.dev/blog/2022/12/gcb-slsa-verification.

Currently, the repositories that support attestations also have an additional requirement that the signatures in them are Sigstore signatures (because these are verifiable without having to handle key management) but this could change to be more permissive (although I wouldn't recommend it).

Still, I think it would be good to disambiguate between "attestation" and "certificate".

"Attestation" is the issuing authority saying something about the origin of the artifact
"Certificate" is the signed rekor log

I'd say:

"Attestation" is a collection of metadata about some artifact, with a signature.
"Signature" is a cryptographic signature that is made by a public/private keypair (either managed or ephemeral)
"Certificate" is what you get from a certificate authority (like Fulcio), which binds an OIDC identity to an ephemeral key
"Entry" is an entry in a transparency log (like Rekor), which makes the signing event public, verifiable and immutable

[nlhkabu]

Content updated at https://docs.google.com/document/d/1WN33a4uV-YZs8Xh37_T2m3Nt_aH3T7aY21ZJQtKugh4/edit?usp=sharing

[indirect]

Hi there, I'm the RubyGems OSS team lead and I was pointed over here by Marty from Ruby Central and Dustin. This is really great! I've left a few comments, but overall this project is something that we had identified we needed but hadn't figured out how to have budget or staff to handle yet. Thanks so much for helping with this!

[nlhkabu]

Thank you very much @indirect!
I've updated the content based on your feedback - could you please take another quick look and resolve any comments as you see fit?

There are some questions there too - if you can't answer them, could you point me in the direction of someone who can?