Support section-folding comments in Python (similar to R’s # ---- convention)

[kv9898]

Summary

Resolves #2019.

Implements structured comment headers for code organization, inspired by R's convention. Sections like # Title ---- (4+ trailing delimiters: -, #, =) create hierarchical structure based on # count.

Example:

# Level 1 ----
class person:
    name: str

## Level 2 ----
a = 1

def greeting():
    print("hello, world")

### Level 3 -=-=-----

# Another Level 1 ----
b = 2

## Another Level 2 ====
c = 3
d = 4

# Final Level 1 ####

Outline and folding:

Provides:

• Folding ranges: Sections fold hierarchically from their line to the next same/higher level section
• Document symbols: Sections appear in outline tree with proper nesting. AST symbols (variables, functions, classes) are nested as children of their preceding comment section, enabling proper code organization in the outline view.

Implementation

• comment_section.rs: Regex-based parser (^(#+)\s+(.+?)\s+([-#=]{4,})\s*$) extracts sections with level detection
• folding.rs: Integrates sections as FoldingRangeKind::Region with hierarchical folding support
• document_symbols.rs: Sequential processing of statements and sections. AST symbols are added as children of the current section context. Uses path-based navigation for robust parent-child relationships in the symbol tree.
• Safe integer conversions prevent overflow on large files

Test Plan

• 6 unit tests for parsing (levels, delimiters, edge cases)
• 2 integration tests for folding (basic + nested hierarchy)
• 2 integration tests for document symbols (nesting validation, AST children under sections)
• All existing folding/symbol tests pass (no regressions)
• Verified proper nesting: variables, functions, and classes appear as children of their preceding comment section

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[stroxler]

Thanks for the PR!

Assigning to @yangdanny97 because our team chatted about this - we'd like to support the feature which seems useful, but we also want to make sure it doesn't interfere with traditional semantic folding, so we may add decide to a feature flag for this behavior so that it is opt-in

[kv9898]

Thanks for the PR!

Assigning to @yangdanny97 because our team chatted about this - we'd like to support the feature which seems useful, but we also want to make sure it doesn't interfere with traditional semantic folding, so we may add decide to a feature flag for this behavior so that it is opt-in

That would be perfect! Thanks!

[meta-codesync]

@yangdanny97 has imported this pull request. If you are a Meta employee, you can view this in D90510122.