Skip to content

Conversation

@KOTungseth
Copy link
Contributor

@KOTungseth KOTungseth commented Nov 14, 2025

Summary

Introduces a new pull request template for the Elastic Docs repository that includes required GenAI disclosure questions. The template is designed to help internal and external contributors provide clear and accurate documentation updates while ensuring transparency around AI-assisted contributions.

What’s changing

  • Adds a new PR template to .github/PULL_REQUEST_TEMPLATE.md.
  • Incorporates two GenAI disclosure questions drafted by Legal:
    • Whether a GenAI tool was used.
    • Which tool(s) and model(s) were used, if applicable.

Why this change is needed

The Elastic Legal and Open Source teams have requested that all public-facing repositories include standardized questions about generative AI usage in documentation contributions. These questions help us:

  • Maintain transparency regarding AI-assisted content.
  • Support responsible and high-quality use of AI for documentation.
  • Ensure reviewers have the context needed to evaluate contributions.
  • Establish consistency across Elastic open repositories, particularly as external contributors begin using AI more frequently.

Adding this template to the Elastic Docs repo also helps contributors understand our expectations upfront and aligns with the AI & Docs Guidelines we’ve developed internally.

What's needed from reviewers

  • Content accuracy and clarity: Does the wording in the template feel clear, welcoming, and consistent with our documentation standards?
  • Completeness: Are there additional checklist items or guidance we should add for contributors?

Additional question

Should we also include a separate template specifically for external contributors, or is the single unified template sufficient?

@KOTungseth KOTungseth self-assigned this Nov 14, 2025
@KOTungseth KOTungseth requested a review from a team as a code owner November 14, 2025 18:28
@KOTungseth KOTungseth added the supportability ability enable self-service or support of product label Nov 14, 2025
@KOTungseth KOTungseth requested a review from Copilot November 14, 2025 18:42
Copilot finished reviewing on behalf of KOTungseth November 14, 2025 18:43
@bmorelli25
Copy link
Member

Thanks for doing this. The things that are most important to me in a PR description are:

  • Links to related issues
  • Explanation of the change
  • Explanation of the rationale behind the change
  • Exactly what you need from reviewers. Who do you need a review from and what should they check/verify/decide/answer? Are there specific questions you want answered? Sections that should be looked at more deeply than others?

The AI stuff makes sense too.

If we're going to include a checklist, I think it should include testing the changes and/or validating against the UI. And ensuring folks are writing cumulative docs. But it's a bit of a slippery slope from there. Was the code validated? Were the snippets tested? Did you explore other areas where this content could live? Were all instances updated?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a standardized pull request template for the Elastic Docs repository to streamline the contribution process and ensure consistency across submissions.

  • Adds a structured PR template with Summary and Checklist sections to guide contributors
  • Includes a Generative AI disclosure section to track usage of AI tools in contributions
  • Provides a link to contribution guidelines for additional reference

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bmorelli25
Copy link
Member

bmorelli25 commented Nov 14, 2025

Top PR Review Issues - Quick Reference

Quick scan of the most common issues found in PR reviews. Based on analysis of 300 recent PRs with 1,547 review comments (including 585 suggestion blocks).

This comment was generated by Claude. Maybe it's helpful for determining what folks are generally missing or forgetting about in their PRs?


Top Issues by Frequency

  1. Links and References (79 PRs, 294 comments) - Broken internal/external links, incorrect link anchors, wrong relative paths, links pointing to non-existent content, missing links

  2. Content Structure and Organization (70 PRs, 147 comments) - Sections in wrong order, headings that don't accurately describe content, poor paragraph structure, related content not grouped together, tutorial steps not properly structured

  3. Content Improvements (58 PRs, 445 comments) - General content suggestions, additions, removals, rewrites, and improvements suggested by reviewers

  4. Technical Accuracy (53 PRs, 91 comments) - Code examples that don't work, incorrect version numbers or compatibility info, invalid configuration examples, outdated information

  5. Markdown Formatting Issues (49 PRs, 137 comments) - Indentation and whitespace inconsistencies, missing backticks for inline code, improper markdown syntax, missing language tags in code blocks, inconsistent list formatting, incorrect heading hierarchy

  6. Attributes and Variables (46 PRs, 94 comments) - Hardcoded product names instead of attributes ({{es}}, {{kib}}, {{esql}}), incorrect attribute syntax, abbreviations not using available attributes

  7. Context and Completeness (40 PRs, 60 comments) - Missing background information, lack of context, missing links to related documentation, prerequisites not mentioned, unclear explanations

  8. Applies_to Metadata / Cumulative Docs (33 PRs, 58 comments) - Missing or incorrect applies_to tags, incorrect syntax for applies_to badges, not using applies_to when content is version-specific or deployment-specific, not following cumulative docs guidelines, incorrect version ordering in cumulative content

  9. Clarity and Precision (26 PRs, 31 comments) - Unclear or ambiguous language, unnecessary words, technical terms not explained, instructions not actionable

  10. Preview and Testing (24 PRs, 35 comments) - Changes not previewed before submission, code examples not tested, formatting not verified in rendered output

  11. Images and Media (20 PRs, 40 comments) - Missing alt text, incorrect image file paths, file names don't follow conventions, images not optimized, inaccurate captions

  12. Terminology and Consistency (16 PRs, 19 comments) - Inconsistent terminology, capitalization doesn't match style guide, inconsistent use of abbreviations, naming conventions

  13. Spelling and Grammar (10 PRs, 13 comments) - Typos and misspellings, incorrect capitalization, punctuation errors, inconsistent terminology

  14. Snippets and Includes (8 PRs, 13 comments) - Duplicating content instead of using snippets, incorrect snippet paths, referenced snippets don't exist

  15. Frontmatter and Metadata (5 PRs, 5 comments) - Incorrect frontmatter YAML, missing required metadata fields, inaccurate title or description

  16. Quality Checks (2 PRs, 3 comments - but highly recommended) - Acrolinx quality checks not run, Vale rule violations not fixed, linter warnings ignored

@KOTungseth KOTungseth marked this pull request as draft November 14, 2025 22:22
@KOTungseth KOTungseth marked this pull request as ready for review November 19, 2025 22:08
Comment on lines 7 to 9
Describe what your PR changes or improves.
Describe the reason you are making this change.
If your PR fixes an issue, link it here.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make all of the instructions comments. Otherwise people will forget and we'll have super long PR descriptions of mostly instructions.

Suggested change
Describe what your PR changes or improves.
Describe the reason you are making this change.
If your PR fixes an issue, link it here.
<!--
Describe what your PR changes or improves.
Describe the reason you are making this change.
If your PR fixes an issue, link it here.
-->

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also like thinking about: What problem does it solve? Any tradeoffs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Describe the reason you are making this change.

This IMO should come from the issue and shouldn't need repeating.
What is most important here is the implementation detail (what the changes are, and optionally how do these changes solve the issue if not already captured in the issue)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I combined the summary section so it now appears as:

Describe what your PR changes or improves.
If your PR fixes an issue, link it here. If not, describe the reason you are making the change.

Love the idea to turn all the instructions into comments.

Comment on lines +19 to +21
2. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).

Tool(s) and model(s) used:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. I'd comment out the instructions if we can.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).
Tool(s) and model(s) used:
<!--
2. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).
-->
Tool(s) and model(s) used:

Copy link
Contributor

@leemthompo leemthompo Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure if there's much point in specifying what model or tool one used TBH, the point is signaling what information the author isn't confident about because they relied on AI to generate it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to know where the data might have been ingested / processed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, just not sure if a PR template is the right place for this. I think you're getting at the fact that non-public information shouldn't be used with these tools, but it's too late if the PR is already up.

Shouldn't we have rules against using certain tools if that's the case, rather than asking for information after the fact? I'm just trying to get at the real point of this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we aim to avoid is merging code / docs that might have been processed from models we haven't approved — in this case, the PR gate provides a measure of liability / transparency / honesty.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more I think about this, the more I think we should just bake this into our CLA 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll touch base with Legal about including this in our CLA, but in the meantime, this is the verbiage Legal is going forward with to meet our policy standards.


--

## Reviewers and timeline
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The table feels overly complex. I also don't really understand the review timelines still. More useful IMO is the PR author saying things like:

@person for technical accuracy in the X section and @team-docs for content and structure
or
@team please focus extra attention on the new ### Limitations section and the updated screenshots.
or open questions like:
Are there edge cases or user journeys I'm missing? Anything that might confuse readers?

Essentially being more explicit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd vote for de-scoping this PR to do the minimal AI legal compliance thing, and not try to cram general PR hygiene stuff into it TBH

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd vote for de-scoping this PR to do the minimal AI legal compliance thing, and not try to cram general PR hygiene stuff into it TBH

This is exactly what I'm going to do.

@leemthompo
Copy link
Contributor

leemthompo commented Nov 20, 2025

Few thoughts:

  • Can we not just bake this into our CLA?
  • I'm not really sure if there's much point in specifying what model or tool one used, there's AI baked into everything already today and that's trending towards ubiquity
  • I think we need to distinguish between using AI for syntax versus for semantics, if folks are using an LLM to generate hypothetical technical "facts", or generate code examples, this must be flagged for reviewers. On the other hand, I don't think anybody cares whether you used Gemini 3.1 to make a table out of a list.
  • @theletterf already drafted AI guidelines, and I think long term the best approach would be to publish those and have a single checkbox on PRs that says "I have read and confirm compliance with the Elastic AI usage policy", we shouldn't try to capture the policy details in a PR template
  • From hard-won experience, I know how reluctant folks are to fill in PR templates, so keeping additional bureaucracy to a minimum has to be a priority IMO
  • Ultimately the responsibility falls on the reviewer to verify and quality control any PRs, whether they're generated by a human, an LLM, or a combo.

@theletterf
Copy link
Contributor

@theletterf already drafted AI guidelines, and I think long term the best approach would be to publish those and have a single checkbox on PRs that says "I have read and confirm compliance with the Elastic AI usage policy", we shouldn't try to capture the policy details in a PR template

Sadly, we can't still do that, though I'm discussing this internally.

@KOTungseth
Copy link
Contributor Author

Thank you for your feedback, everyone! I agree with Liam's comment to include only the AI disclosure requirements from Legal.

We can add more to this template in the future to meet our team needs.

@KOTungseth KOTungseth merged commit b6e0a3d into elastic:main Nov 20, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

supportability ability enable self-service or support of product

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants