Skip to content
268 changes: 144 additions & 124 deletions llvm/docs/DeveloperPolicy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,90 +16,87 @@ to eliminate miscommunication, rework, and confusion that might arise from the
distributed nature of LLVM's development. By stating the policy in clear terms,
we hope each developer can know ahead of time what to expect when making LLVM
contributions. This policy covers all llvm.org subprojects, including Clang,
LLDB, libc++, etc.
LLDB, libc++, MLIR, etc.

This policy is also designed to accomplish the following objectives:
The developer policy supports the following LLVM project objectives:

#. Attract both users and developers to the LLVM project.
#. Attract both users and new contributors to the LLVM project.

#. Make life as simple and easy for contributors as possible.
#. Help people contribute to LLVM by documenting our development practices.

#. Keep the top of tree as stable as possible.
#. Maintain the stability, performance, and quality of the ``main`` branch.

#. Establish awareness of the project's :ref:`copyright, license, and patent
policies <copyright-license-patents>` with contributors to the project.

This policy is aimed at frequent contributors to LLVM. People interested in
contributing one-off patches can do so in an informal way by sending them to the
`llvm-commits mailing list
<http://lists.llvm.org/mailman/listinfo/llvm-commits>`_ and engaging another
developer to see it through the process.
#. Establish the project's :ref:`copyright, license, and patent
policies <copyright-license-patents>` policies.

Developer Policies
==================

This section contains policies that pertain to frequent LLVM developers. We
always welcome `one-off patches`_ from people who do not routinely contribute to
LLVM, but we expect more from frequent contributors to keep the system as
efficient as possible for everyone. Frequent LLVM contributors are expected to
meet the following requirements in order for LLVM to maintain a high standard of
quality.

Stay Informed
-------------

Developers should stay informed by reading the `LLVM Discourse forums`_ and subscribing
to the categories of interest for notifications.

Paying attention to changes being made by others is a good way to see what other people
are interested in and watching the flow of the project as a whole.
Communication Channels
----------------------

Contibutions to the project are made through :ref:`GitHub Pull Requests <github-reviews>`.
You can subscribe to notification for areas of the codebase by joining
LLVM is a large project with many subcomponents, and it has a wide array of
communication channels that you can use to keep track of recent developments,
upcoming projects, new designs, enhancements, and other community business.

First and foremost is the `LLVM Discourse forums`_, which is the successor
to our former mailing lists (llvm-dev@, cfe-dev@, lldb-dev@, etc). This is
probably the most vital and active communication channel to our highly
distributed open source project. It enables long-form asyncronous text
communication, and this is where people tend to propose major changes or
propose new designs in the form of RFCs (Request For Comment), which are
described later. Please be aware that the discourse forums are public and
archived, and that notices of confidentiality or non-disclosure cannot be
respected.

We accept code contributions as :ref:`GitHub Pull Requests <github-reviews>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thing I don't have a strong opinion on, but witnessed: some PRs or Github issue discussions grow very long and sometimes contentious, but much fewer people engage with them. Maybe we want some sort of soft guideline on when to move such a discussion to the forums for broader community involvement.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that's useful information, but I think I'd move that to the specific CodeReview docs, and I don't want to touch that file in this PR. This section is supposed to be an overview of the main communication spaces (Discourse, Discord, GitHub).

Our project is generally too large to subscribe to all github notifications, so
if you want to be notified of pull requests affecting a specific parts of the
code, you can join
one of the `pr-subscribers-* <https://github.com/orgs/llvm/teams?query=pr-subscribers>`_
GitHub teams. This `mapping <https://github.com/llvm/llvm-project/blob/main/.github/new-prs-labeler.yml>`_
indicates which team is associated with a particular paths in the repository.

You can also subscribe to the "commits" mailing list for a subproject you're interested in,
such as `llvm-commits
<http://lists.llvm.org/mailman/listinfo/llvm-commits>`_, `cfe-commits
<http://lists.llvm.org/mailman/listinfo/cfe-commits>`_, or `lldb-commits
<http://lists.llvm.org/mailman/listinfo/lldb-commits>`_.
documents the paths that trigger notifications for each of the listed teams.

Missing features and bugs are tracked through our `GitHub issue tracker <https://github.com/llvm/llvm-project/issues>`_
and assigned labels. We recommend that active developers monitor incoming issues.
You can subscribe for notification for specific components by joining
one of the `issue-subscribers-* <https://github.com/orgs/llvm/teams?query=issue-subscribers>`_
teams.
You may also subscribe to the `llvm-bugs
<http://lists.llvm.org/mailman/listinfo/llvm-bugs>`_ email list to keep track
of bugs and enhancements occurring in the entire project. We really appreciate people
who are proactive at catching incoming bugs in their components and dealing with them
promptly.

Please be aware that all public LLVM mailing lists and discourse forums are public and archived, and
that notices of confidentiality or non-disclosure cannot be respected.
and assigned labels. You can subscribe for notification for specific components
by joining
one of the `issue-subscribers-*
<https://github.com/orgs/llvm/teams?query=issue-subscribers>`_ teams. You may
also subscribe to the `llvm-bugs
<http://lists.llvm.org/mailman/listinfo/llvm-bugs>`_ email list to subscribe to
the firehose of all issue notifications, which some community members use to
perform custom filtering.

Beyond the main formal communication channels, LLVM has a Discord server for
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it sound like working groups are not formal communication channels. Perhaps:

Beyond the Discourse forums, LLVM has...

instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I said "asynchronous written", since I think that's the differentiator.

real-time chat communication, as well as a community calendar with many regular
workgroup video calls and office hours. See :doc:`GettingInvolved` for more
information on other ways to engage with the community.

.. _patch:
.. _one-off patches:

Making and Submitting a Patch
-----------------------------

When making a patch for review, the goal is to make it as easy for the reviewer
to read it as possible. As such, we recommend that you:
Submitting patches for review is straightforward with GitHub. Follow the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Submitting patches for review is straightforward with GitHub.

I'd probably reword this to "Patches are submitted through GitHub" rather than claim it's straightforward for everyone (not everyone is familiar with GitHub or comfortable with git in general).

:ref:`Getting Started Guide <sources>` to check out sources, make a patch, and
then follow the :ref:`GitHub Pull Request <github-reviews>` guide to upload a
pull request.

#. Make your patch against git main, not a branch, and not an old version
of LLVM. This makes it easy to apply the patch. For information on how to
clone from git, please see the :ref:`Getting Started Guide <sources>`.
Here are some tips to enable a successful code review:

#. Similarly, patches should be submitted soon after they are generated. Old
patches may not apply correctly if the underlying code changes between the
time the patch was created and the time it is applied.
* :ref:`Include a test <include a testcase>`. This tends to be one of the first
things a reviewer will ask for and look at to understand what a new patch
does.

#. Once you have created your patch, create a
:ref:`GitHub Pull Request <github-reviews>` for
it (or commit it directly if applicable).
* Identify 2-3 individuals to review the patch. Look through the relevant
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do random GIthub users have permissions to add PR reviewers? If not, we should document the recommended way of attracting attention, e.g., by @-pinging the relevant people in a comment and explicitly stating you don't have permission to add reviewers (for one-off PRs) and/or asking for "triage" set of permissions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do random GIthub users have permissions to add PR reviewers?

In my experience, they do not have the ability to do this themselves unless they're a member of the organization. Though I wonder if that can be relaxed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added guidance to @-ping in the absence of permission changes. It's good enough for the time being.

:ref:`Maintainers` file or browse git blame for likely stakeholders for the
code want to modify.

* Make sure your patch is based on a recent commit from ``main``, rather than a
previous release branch. If you want to make changes to a release branch, land
a change in ``main`` first and ask a release manager to backport it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we encourage developers to backport their patches themselves, when possible?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I should link the instructions for backporting and leave it at that. It looks like the current state is that we have special github issue comments that bots pick up and act on:
https://llvm.org/docs/GitHub.html#backporting-fixes-to-the-release-branches

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you planning on adding that link here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, done, sorry I missed that.


See :doc:`CodeReview` for more info on what to expect.

When submitting patches, please do not add confidentiality or non-disclosure
notices to the patches themselves. These notices conflict with the LLVM
Expand All @@ -115,19 +112,20 @@ GitHub platform about their past contributions. Primarily, our buildbot
infrastructure uses emails to contact contributors about build and test
failures.

Therefore, the LLVM community requires contributors to have a public
email address associated with their GitHub commits, so please ensure that "Keep
my email addresses private" is disabled in your
`account settings <https://github.com/settings/emails>`_.
Therefore, the LLVM community requires contributors to have a public email
address associated with their GitHub commits, so please ensure that "Keep my
email addresses private" is disabled in your `account settings
<https://github.com/settings/emails>`_. There are many free email forwarding
services available if you wish to keep your identity private.

.. _code review:

Code Reviews
------------

LLVM has a code-review policy. Code review is one way to increase the quality of
software. Please see :doc:`CodeReview` for more information on LLVM's code-review
process.
LLVM uses code review, which is a generally accepted software engineering best
practice for maintaining high code quality. Please see :doc:`CodeReview` for
more information on LLVM's code review process.

.. _maintainers:

Expand All @@ -147,17 +145,17 @@ Community members can find active and inactive maintainers for a project in the
Maintainers are volunteering to take on the following shared responsibilities
within an area of a project:

* ensure that commits receive high-quality review, either by the maintainer
or by someone else,
* help to confirm and comment on issues,
* mediate code review disagreements through collaboration with other
maintainers (and other reviewers) to come to a consensus on how best to
proceed with disputed changes,
* actively engage with relevant RFCs,
* aid release managers with backporting and other release-related
activities,
* be a point of contact for contributors who need help (answering questions
on Discord/Discourse or holding office hours).
* ensure that commits receive high-quality review, either by the maintainer
or by someone else,
* help to confirm and comment on issues,
* mediate code review disagreements through collaboration with other
maintainers (and other reviewers) to come to a consensus on how best to
proceed with disputed changes,
* actively engage with relevant RFCs,
* aid release managers with backporting and other release-related
activities,
* be a point of contact for contributors who need help (answering questions
on Discord/Discourse or holding office hours).

Each top-level project in the monorepo will specify one or more
lead maintainers who are responsible for ensuring community needs are
Expand All @@ -173,10 +171,10 @@ project should be discontinued.
All contributors with commit access to the LLVM Project are eligible to be a
maintainer. However, we are looking for people who can commit to:

* engaging in their responsibilities the majority of the days in a month,
* ensuring that they, and the community members they interact with, abide by
the LLVM Community Code of Conduct, and
* performing these duties for at least three months.
* engaging in their responsibilities the majority of the days in a month,
* ensuring that they, and the community members they interact with, abide by the
LLVM Community Code of Conduct, and
* performing these duties for at least three months.

We recognize that priorities shift, job changes happen, burnout is real,
extended vacations are a blessing, and people's lives are generally complex.
Expand Down Expand Up @@ -220,27 +218,34 @@ Test Cases
Developers are required to create test cases for any bugs fixed and any new
features added. Some tips for getting your testcase approved:

* All feature and regression test cases are added to the ``llvm/test``
directory. The appropriate sub-directory should be selected (see the
* All feature and regression test cases are added to the ``test`` subdirectory
of each LLVM subproject, i.e. ``llvm-project/llvm/test`` for LLVM itself. The
appropriate sub-directory should be selected (see the
:doc:`Testing Guide <TestingGuide>` for details).

* Test cases should be written in :doc:`LLVM assembly language <LangRef>`.
* Changes to libraries, such as Support, which are not directly observable
through tool invocations, are often best tested with unit tests. Unit tests
are located under the ``unittests`` subdirectory of each subproject.

* Test cases, especially for regressions, should be reduced as much as possible,
by :doc:`bugpoint <Bugpoint>` or manually. It is unacceptable to place an
entire failing program into ``llvm/test`` as this creates a *time-to-test*
burden on all developers. Please keep them short.
* Test cases are written in the relevant input language of the relevant
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we want to highlight more prominently that, whenever possible, tests should be written against a tool such as optrather than as unit tests and use command-line tools such as FileCheck to verify the output for testing efficiency reasons.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I scrapped this paragraph, reordered it, and mentioned our usual preferred approaches.

component. For LLVM, this is typically the :doc:`LLVM assembly language
<LangRef>`. For Clang, it is often C/C++/ObjC.

* Test cases should be targeted. Large inputs exhibiting bugs should be reduced
with tools like ``llvm-reduce`` before committing them to the suite. It is not
acceptable to place an entire failing program into ``llvm/test`` as this
creates a *time-to-test* burden on all developers. Please keep them short.

* Avoid adding links to resources that are not available to the entire
community, such as links to private bug trackers, internal corporate
documentation, etc. Instead, add sufficient comments to the test to provide
the context behind such links.

Note that llvm/test and clang/test are designed for regression and small feature
tests only. More extensive test cases (e.g., entire applications, benchmarks,
etc) should be added to the ``llvm-test`` test suite. The llvm-test suite is
for coverage (correctness, performance, etc) testing, not feature or regression
testing.
Note that ``llvm/test`` and ``clang/test`` are designed for regression and small
feature tests only. More extensive test cases (e.g., entire applications,
benchmarks, etc) should be added to the ``llvm-test`` test suite. The ``llvm-test``
suite is for integration and application testing (correctness, performance, etc)
testing, not feature or regression testing.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we also want to mention that it's the place to put tests which are incompatible with the license from the main LLVM repo?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, done.


Release Notes
-------------
Expand Down Expand Up @@ -336,12 +341,12 @@ you follow these guidelines to help review, search in logs, email formatting
and so on. These guidelines are very similar to rules used by other open source
projects.

Most importantly, the contents of the message should be carefully written to
convey the rationale of the change (without delving too much in detail). It
also should avoid being vague or overly specific. For example, "bits were not
set right" will leave the reviewer wondering about which bits, and why they
weren't right, while "Correctly set overflow bits in TargetInfo" conveys almost
all there is to the change.
Commit messages should communicate briefly what the change does, and provide
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, the "why" part is the most important in the commit message. We can understand the "what" from the commit itself, although it is useful to summarize the "what" in the message for the reviewer to check whether the code actually does what the author intended.

background, possibly with context, for why the change is being made. Commit
messages should be thoughtfully written and specific, rather than vague. For
example, "bits were not set right" will leave the reviewer wondering about which
bits, and why they weren't right, while "Correctly set overflow bits in
TargetInfo" conveys almost all there is to the change.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also remind that the final commit, potentially squashed from multiple commits in a PR, will have the description taken from the PR description. The latter is set once when the PR is created, defaulting to the commit message if the PR only contains one commit, and does not track the changes to the commit message. As a corollary, developers must manually update the PR description as they iterate on the change...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a paragraph after the following bullets


Below are some guidelines about the format of the message itself:

Expand Down Expand Up @@ -369,8 +374,6 @@ Below are some guidelines about the format of the message itself:
or "[OpenMP] ...". This helps email filters and searches for post-commit
reviews.

* The body, if it exists, should be separated from the title by an empty line.

* The body should be concise, but explanatory, including a complete
reasoning. Unless it is required to understand the change, examples,
code snippets and gory details should be left to bug comments, web
Expand All @@ -382,7 +385,7 @@ Below are some guidelines about the format of the message itself:
* If the commit is a bug fix on top of another recently committed patch, or a
revert or reapply of a patch, include the git commit hash of the prior
related commit. This could be as simple as "Revert commit NNNN because it
caused PR#".
caused issue #".

* If the patch has been reviewed, add a link to its review page, as shown
`here <https://www.llvm.org/docs/Phabricator.html#committing-a-change>`__.
Expand Down Expand Up @@ -512,11 +515,12 @@ someone with commit access commits on your behalf. When doing so, please
provide the name and email address you would like to use in the Author
property of the commit.

For external tracking purposes, committed changes are automatically reflected
on a commits mailing list soon after the commit lands (e.g. llvm-commits_).
Note that these mailing lists are moderated, and it is not unusual for a large
commit to require a moderator to approve the email, so do not be concerned if a
commit does not immediately appear in the archives.
For external tracking purposes, committed changes are automatically reflected on
a commits mailing list soon after the commit lands (e.g.
[email protected]). Note that these mailing lists are moderated, and
it is not unusual for a large commit to require a moderator to approve the
email, so do not be concerned if a commit does not immediately appear in the
archives.

If you have recently been granted commit access, these policies apply:

Expand Down Expand Up @@ -552,26 +556,42 @@ to do so.

.. _discuss the change/gather consensus:

Making a Major Change
---------------------

When a developer begins a major new project with the aim of contributing it back
to LLVM, they should inform the community with a post to the `LLVM Discourse forums`_, to the extent
possible. The reason for this is to:

#. keep the community informed about future changes to LLVM,

#. avoid duplication of effort by preventing multiple parties working on the
same thing and not knowing about it, and
Proposing Major Changes (RFCs)
------------------------------

#. ensure that any technical issues around the proposed work are discussed and
resolved before any significant work is done.
LLVM is a large community with many stakeholders, and before landing any major
change, it is important to discuss the design of a change publicly with the
community. This is done by posting an Request For Comments (RFC) on the `LLVM
Discourse forums`_.

The design of LLVM is carefully controlled to ensure that all the pieces fit
together well and are as consistent as possible. If you plan to make a major
change to the way LLVM works or want to add a major new extension, it is a good
idea to get consensus with the development community before you start working on
it.
idea to get consensus with the development community before you invest
significant effort in an implementation. Prototype implementations, however, can
often be helpful in making design discussions more concrete by demonstrating
what is possible.

TODO: Elaborate on best practices for making a successful RFC.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Random ideas for this from recent experience:

  • make it targeted, don't touch components irrelevant to the task
  • explain how the change improves LLVM for all/most stakeholders rather than your specific use case
  • periodically summarize the current state of the discussion and clearly separate points where consensus seems to emerge from those where further discussion is necessary
  • by default, the burden of proof is on the proposer (though we'd like to avoid situations when somebody keeps asking for more proof as a way of exhausting the proposer)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that list, but I'm sure we can add more here :) I think my goal with this PR was to get a better base for future edits.


After posting a major proposal, it is common to receive lots of conflicting
feedback from different parties, or no feedback at all, leaving authors without
clear next steps. As a community, we are aiming for `"rough consensus"
<https://en.wikipedia.org/wiki/Rough_consensus>`_, similar in spirit to what is
desribed in `IETF RFC7282 <https://datatracker.ietf.org/doc/html/rfc7282>`_.
This requires considering and addressing all of the objections to the RFC, and
confirming that we can all live with the tradeoffs embodied in the proposal.

The LLVM Area Teams (defined in `LP0004
<https://github.com/llvm/llvm-www/blob/main/proposals/LP0004-project-governance.md>`_)
are responsible for facilitating project decision making. In cases were there
isn't obvious agreement, area teams should step in to restate their perceived
consensus. In cases of deeper disagreement, area teams should try to identify
the next steps for the proposal, such as gathering more data, changing the
proposal, or rejecting it outright. They can also act as moderators by
scheduling calls for participants to speak directly to resolve disagreements,
subject to normal :ref:`Code of Conduct <LLVM Community Code of Conduct>`
guidelines.

Once the design of the new feature is finalized, the work itself should be done
as a series of `incremental changes`_, not as a long-term development branch.
Expand Down
Loading