Detail Design Document

[dcalavrezo-qorix]

I'd like to start a discussion on how we can best structure detail design documentation using Sphinx-Needs.

Maybe we can use as a reference this demo

https://sphinx-needs-demo.readthedocs.io/en/latest/automotive-adas/swe_3_sw_detailed_design.html

[dcalavrezo-qorix]

Ok, so here is how the documentation is generated in the demo (DDD):

• The source code uses special sphinx-style docstring annotations, such as:

"""
.. impl:: Lane Marking Detection Algorithm
🆔 IMPL_001
:links: SWREQ_001, SWARCH_001

Implements the lane marking detection algorithm using camera inputs.
"""

• The documenation files use the Sphinx directive:

.. automodule:: automotive_adas
   :members:
   :undoc-members:
   :show-inheritance:

This tells Sphinx to extract and include the docstrings (with the embedded Sphinx-Needs objects) from the automotive_adas Python module.
All annotated classes and methods, with their implementation-level documentation, get pulled into the generated docs.

• The extension processing

  • During the Sphinx build process, the Sphinx-Needs extension scans for needs objects (like impl) in both .rst files and Python docstrings.

  • It parses these and creates structured, cross-linked design artifacts:

    • Implementation details are automatically linked to requirements (SWREQ_xxx) and architecture (SWARCH_xxx) via the :links: field.
    • Each implementation block appears in the generated documentation, with all metadata and traceability.

[dcalavrezo-qorix]

sphinx-rust does not have a direct equivalent of .. automodule::.

You cannot just point to a Rust crate or module and have all public items and their doc comments imported and rendered with one directive.

Instead, you have to explicitly document each module, struct, enum, or function using the provided directives:

.. rust:mod:: my_module

.. rust:struct:: MyStruct

.. rust:enum:: MyEnum

.. rust:fn:: my_function

These directives will extract doc comments for each item, but you must list them manually.

[dcalavrezo-qorix]

Objective

I've tried out to build a custom Sphinx extension that would automatically extract "needs objects" from Rust source code (specifically, from annotated Rust doc comments) and make them available as structured documentation elements in the generated docs. This is inspired by the workflow used in Python codebases, where docstrings can be parsed for such objects using Sphinx extensions like sphinx-needs and custom directives.

---

Goals

• Enable docs-as-code for Rust: Allow requirements, design elements, and traceability objects to live directly in Rust code and sync them automatically with Sphinx documentation builds.
• Support detailed design ("needs objects"): Especially for static (dd_sta__*) and dynamic (dd_dyn__*) detailed design objects, as required by the project metamodel.
• Integrate with Sphinx workflows: So engineers can reference these objects (via :need:) in .rst documents.

---

Approach

1. Annotation Format

   • Chose a comment annotation style compatible with Rust, using triple slashes and Sphinx-like block structure.
   • Example:

     /// .. dd_sta:: kvs_design
     ///    :id: dd_sta__kvs_design
     ///    :status: valid
     ///    :security: YES
     ///    :safety: ASIL_B
     ///
     ///    Key-Value-Store overall design.
     pub struct DummyForNeedsTest;

2. Extension Design

   • On Sphinx build initialization, recursively scan the Rust source directory for .rs files.
   • Use a regular expression to extract contiguous doc-comment blocks matching the needs annotation pattern.
   • Aggregate these blocks and write them into an autogenerated .rst file (e.g., _rust_needs_autogen.rst).
   • Inject this file into the documentation tree for Sphinx to parse, making the needs objects available for cross-referencing.

3. Usage

   • Add the extension to the Sphinx conf.py’s extensions list.
   • Reference extracted objects in other .rst files via their IDs (e.g., :need:dd_sta__kvs_design``).

---

Key Implementation Details

• Configurable Source Path:
  The Rust source root is set via a Sphinx config value (rust_needs_source_path).
• Regex Extraction:
  Looks for blocks starting with /// .. dd_sta:: or /// .. dd_dyn:: and includes all following contiguous /// lines.
• Output File:
  Writes extracted blocks to _rust_needs_autogen.rst in the documentation source directory.
• Automatic Inclusion:
  Modifies index.rst or other top-level docs to include this file in the table of contents.

---

Troubleshooting and Status

• File Found But Empty:
  If _rust_needs_autogen.rst exists but is empty, the most likely reasons are:
  • Rust source files lack the expected annotation blocks.
  • Regex extraction does not match the actual annotation style.
  • Extension points to the wrong source directory.
• No Needs Objects Available:
  Sphinx will warn about missing needs objects if references (e.g., :need:) are used but the extracted file is empty or missing the relevant object IDs.

---

Next Steps

• Ensure correct annotation format is present in Rust code.
• Confirm the source path in conf.py matches the project structure.
• Test with a minimal Rust file containing an example annotation.
• Improve error reporting/logging in the extension for easier debugging.

---

Example Reference Usage

.. toctree::
   :maxdepth: 1

   ddd

----

See :need:`dd_sta__kvs_design` for the static detailed design extracted from the source code.

---

Links and Related Work

• Sphinx-Needs
• sphinx-needs autodoc integration (Python)
• Rust doc comments

---

[dcalavrezo-qorix]

It turns out the demo extension just replicates an existing sphinx directive

[dcalavrezo-qorix]

🧩Using needextract as an alternative

• Document Rust projects using Sphinx
• Extract structured metadata from Rust code (like requirements/specs)
• Optionally link to source code in GitHub or generated Rust docs
• Understand the difference between Sphinx's Python autodoc and Rust handling

---

✅ Recommended Approach for Rust Projects

1. Use sphinx-needs with .. needextract:: in :file: mode

You can annotate your Rust code with NEEDS: comments:

/// NEEDS:id=REQ-LOGIN
/// NEEDS:title=User login functionality
/// NEEDS:status=approved
/// NEEDS:tags=auth,security
/// NEEDS:source=https://github.com/myorg/mycrate/blob/main/src/auth.rs#L10
pub fn login_user(...) { ... }

Then, in your .rst file:

.. needextract:: ../src/auth.rs
   :file:
   :filter: NEEDS:

This reads the .rs file as plain text, extracts lines with NEEDS: fields, and renders them as need objects in your documentation.

⚠️ This does not parse Rust syntax; it only extracts metadata from comments.

---

2. Enable Custom Fields like source=

In conf.py:

needs_extra_options = ["source"]

Now, sphinx-needs will display a clickable link in the rendered documentation:

🔗 Source: auth.rs#L10

---

3. Compare: sphinx.ext.autodoc (Python) vs. Rust Handling

Feature	Python (autodoc)	Rust (needextract :file:)
Language Support	Python only	Any (Rust, YAML, etc.)
Uses real imports	✅ Yes	❌ No
Interprets type signatures	✅ Yes	❌ No
Extracts docstring	✅ Yes	✅ Yes (via NEEDS fields)
Supports [source] links	✅ Via sphinx.ext.viewcode	❌ Use manual source= field
Native API doc generation	✅	❌ Use cargo doc instead

---

❌ sphinx.ext.viewcode Doesn’t Work for Rust

• Only works with Python modules
• Requires the code to be importable into Python
• Doesn’t recognize or parse .rs files

Use needs_extra_options with source= instead.

---

🛠️ Optional Tools for Advanced Rust Doc Integration

Tool	Purpose	Notes
cargo doc	Native Rust API documentation	Recommended for full API docs
sphinx-needs	Traceability, requirements, metadata	Best for structured documentation
Doxygen + Breathe	Parse Rust code as C-like syntax	Advanced setup with doxyfilter-rust
literalinclude	Embed Rust code snippets	Pure Sphinx feature, syntax-highlighted

---

📁 Example Project Structure

my-docs/
├── docs/
│   ├── index.rst
│   ├── conf.py
├── rust_src/
│   └── auth.rs
└── requirements.txt

auth.rs

/// NEEDS:id=REQ-LOGIN
/// NEEDS:title=User login functionality
/// NEEDS:status=approved
/// NEEDS:source=https://github.com/myorg/mycrate/blob/main/src/auth.rs#L10
pub fn login_user(...) { ... }

index.rst

.. needextract:: ../rust_src/auth.rs
   :file:
   :filter: NEEDS:

conf.py

extensions = ['sphinx_needs']
needs_extra_options = ["source"]

---