Skip to content

Swap to new structure of documentation#494

Open
jacomago wants to merge 12 commits intoarchiver-appliance:masterfrom
jacomago:refactor/docs-structure
Open

Swap to new structure of documentation#494
jacomago wants to merge 12 commits intoarchiver-appliance:masterfrom
jacomago:refactor/docs-structure

Conversation

@jacomago
Copy link
Collaborator

@jacomago jacomago commented Feb 26, 2026

This PR updates the archiver documentation to the outline discussed at the codathon.

The goal of the PR is to move the content into the new structure, I'm purposefully not rewriting any documentation nor adding new documentation.
It does NOT write new content other than some summary paragraphs at the top of the index pages, and renaming some of the links.
It does add TODOs for where content is missing from the breakdown worked out at the codathon.

You can see the changes live at https://epicsarchiver--494.org.readthedocs.build/en/494/index.html

p.s. On the current state: Currently I think we have a mix of really good documentation (especially the explanations), alot missing (but not clear how much should be in the clients instead) and a load out of sync from the actual code. Hopefully this restructure is a good push to clean it up.

@jacomago jacomago self-assigned this Feb 26, 2026
@MatInGit
Copy link

As of the end of codethon here is the sate of things:

  • Made a best effort first pass at splitting the existing deocumetation into Diátaxis categories. Minimal changes to actual content, fixed some of the linking where obvious.
  • Some documents may need further breaking down or re-writing.
  • Added arrow key navigation.
  • Added uv sphinx build script.

Reference:
Structure Discussion Doc

@jacomago jacomago force-pushed the refactor/docs-structure branch 3 times, most recently from 4f95de2 to 677a5e4 Compare March 6, 2026 07:52
jacomago and others added 7 commits March 6, 2026 09:01
Set up new role-based doc structure (operator, integrator, sysadmin,
developer, contributor) each with tutorials/guides/explanations/references
quadrants. Move old monolithic docs to old_source/. First pass of content
distribution, plus uv build script and JS keybindings.
Split userguide.md into operator and integrator pages. Split installguide.md
into sysadmin guides (deployment, tomcat-setup, mysql-setup, tutorials).
Distribute faq.md content into appropriate explanations. Delete old source
files.
Move deployment guides, scripting guide, storage explanation, overview and
features, system requirements, and env vars from contributor/ to sysadmin/
and developer/. Slim customization.md, extract channel archiver guide,
distribute monitoring and policies to explanations.
The 9-line scripting.md was a redirect with no original content.
Absorb it as a "Scripting for bulk operations" section at the end of
monitoring.md, where the cross-references to developer/guides/scripting
and the mgmt scriptables reference are more discoverable.

Remove the now-deleted file from sysadmin/explanations/index.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
check-pv-status.md was 14 lines covering a natural follow-on step to
submitting a PV. Absorb it as a "Check the status of a PV" section at
the end of integrator/tutorials/submit-pv.md.

Remove the now-deleted file from integrator/guides/index.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jacomago jacomago force-pushed the refactor/docs-structure branch from 677a5e4 to b844c52 Compare March 6, 2026 08:06
jacomago and others added 3 commits March 6, 2026 09:29
Content will be folded into the developer/references/retrieval-api.md page,
which covers both the time-range and point-in-time retrieval endpoints.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ieval API ref

- Rename operator/guides/web-client.md → developer/guides/fetch_data_guide.md;
  the URL anatomy and Python/Bokeh example are developer content
- Replace web-client entry in operator/guides/index.md with [Web client] stub
- Add fetch_data_guide to developer/guides/index.md
- Move retrieval-api.md from operator/references/ to developer/references/
- Replace [Retrieval API] stub in developer/references/index.md with the real page
- Fold save-restore (getDataAtTime) endpoint into retrieval-api.md as a section
- Update all cross-references

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y-file

The file's title is "Create your policies file" and its content is a
procedural how-to (create policies.py, set env vars, set up storage
folders). It belongs in the guides quadrant, not explanations.

- Rename/move: sysadmin/explanations/policies.md
             → sysadmin/guides/create-policy-file.md
- Remove from sysadmin/explanations/index.md.
- Add to top of sysadmin/guides/index.md (before deployment, as it is
  a prerequisite).
- Update cross-reference in customization.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jacomago jacomago force-pushed the refactor/docs-structure branch from b844c52 to f5aa68b Compare March 6, 2026 08:33
jacomago and others added 2 commits March 6, 2026 10:04
- Prefix every placeholder toctree entry with "TODO: " so they are
  clearly marked as unwritten pages (e.g. [Web client] → [TODO: Web client])
- Remove deployment-guides/ sub-directory (deploy-bm, deploy-docker,
  deploy-swarm, deploy-kubernetes, deploy.md were all stubs or duplicated
  content already in deployment.md)
- Replace the Deployment guides <deployment-guides/index> toctree entry
  in sysadmin/guides/index.md with four individual TODO stubs for each
  deployment target

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each index page previously rendered as a bare heading and toctree.
Add one or two sentences describing the scope of that quadrant so
readers landing on the page have context before following a link.
@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 6, 2026

@jacomago jacomago marked this pull request as ready for review March 6, 2026 09:44
@jacomago
Copy link
Collaborator Author

jacomago commented Mar 6, 2026

@minijackson FYI

@minijackson
Copy link
Collaborator

I don't have the time to read the whole thing today, but can I suggest removing the empty sections, for example developer/tutorials/? This is to avoid the frustrating experience of clicking on the section, just to find that there's nothing currently.

Copy link
Contributor

@anderslindho anderslindho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is a good start, but needs some further thinking as well as some cleanup.

On the first:
What you term developer is really on par with operator/user. What you call integrator would maybe be more suited as (control system) developer. Contributing doesn't merit that much space in the toctree; I would personally prefer a - de facto standard - non-published CONTRIBUTING.md in the repo.

On the second:
You have some commits mixing up refactoring (/reorganisation) with adding and/or changing content. You are adding stubs that would be better registered as tasks or issues in an issue tracker, to be added when actually written. And you are keeping empty sections, as pointed out already by Rémi.

@anderslindho
Copy link
Contributor

Thinking more about profiles and personas, I think there's just user (operator plus developer plus integrator), sys admin, and maintainer.

@jacomago
Copy link
Collaborator Author

jacomago commented Mar 9, 2026

Thinking more about profiles and personas, I think there's just user (operator plus developer plus integrator), sys admin, and maintainer.

I disagree that there are only three personas. If I were to add RBAC I would split as I have done.

actions:

  • operator/scientist: only retrieve data
  • integrator: submit pvs, check pvs, pause, resume
  • sysadmin: everything else

Developer is supposed to be someone who is developing a client to the archiver, which I think is quite a different role to the others, but yes could be folded in to the others.

Contributor stuff I could put in CONTRIBUTING.md, but a bit a of a struggle since its so much.


An alternative split is by the wars split:

  • retrieval
  • mgmt
  • etl
  • engine

which somewhat matches the profiles with

  • retrieval (operator)
  • mgmt (sys admin)
  • etl (sys admin)
  • engine (integrator)

On the second: You have some commits mixing up refactoring (/reorganisation) with adding and/or changing content. You are adding stubs that would be better registered as tasks or issues in an issue tracker, to be added when actually written. And you are keeping empty sections, as pointed out already by Rémi.

Commit structure... this kind of change is really difficult to have nice commits...

@anderslindho
Copy link
Contributor

I disagree that there are only three personas. [...]

You are thinking of ESS workflow and roles, and even then this is not true. Consider:

  • Integrators are not called as such at all sites - even at ESS we also have automation engineers and other roles perform the same task
  • The control system developers at ESS only set up archival as a service - they act as a proxy for a system expert, operator, or data scientist/analyst - because of our workflows; they generally don't archive things they themselves care about (but they might: consider IOC heartbeat, etc.)
  • Smaller sites may not have the same role names or responsibility divisions

It is ultimately the (potential) data consumer that decides on archival, which is why I argue for a different split.

Commit structure... this kind of change is really difficult to have nice commits...

I disagree 🤷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants