Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/agents/code-quality-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Ensure the project is:

1. **Build**: Zero warnings (TreatWarningsAsErrors=true)
2. **Linting**:
- markdownlint (`.markdownlint.json`)
- markdownlint (`.markdownlint-cli2.jsonc`)
- cspell (`.cspell.json`)
- yamllint (`.yamllint.yaml`)
- dotnet format (`.editorconfig`)
Expand Down
30 changes: 29 additions & 1 deletion .github/agents/repo-consistency-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ The agent reviews the following areas for consistency with the template:

#### Quality Configuration

- **Linting Rules**: `.cspell.json`, `.markdownlint.json`, `.yamllint.yaml`
- **Linting Rules**: `.cspell.json`, `.markdownlint-cli2.jsonc`, `.yamllint.yaml`
- Note: Spelling exceptions will be repository-specific
- **Editor Config**: `.editorconfig` settings (file-scoped namespaces, 4-space indent, UTF-8+BOM, LF endings)
- **Code Style**: C# code style rules and analyzer configuration
Expand Down Expand Up @@ -88,6 +88,34 @@ The agent reviews the following areas for consistency with the template:
3. **Recommend Updates**: Suggest specific files or patterns that should be updated
4. **Respect Customizations**: Recognize valid project-specific customizations

### Tracking Template Evolution

To ensure downstream projects benefit from recent template improvements, review recent pull requests
merged into the template repository:

1. **List Recent PRs**: Retrieve recently merged PRs from `demaconsulting/TemplateDotNetTool`
- Review the last 10-20 PRs to identify template improvements

2. **Identify Propagatable Changes**: For each PR, determine if changes should apply to downstream
projects:
- Focus on structural changes (workflows, agents, configurations) over content-specific changes
- Note changes to `.github/`, linting configurations, project patterns, and documentation
structure

3. **Check Downstream Application**: Verify if identified changes exist in the downstream project:
- Check if similar files/patterns exist in downstream
- Compare file contents between template and downstream project
- Look for similar PR titles or commit messages in downstream repository history

4. **Recommend Missing Updates**: For changes not yet applied, include them in the consistency
review with:
- Description of the template change (reference PR number)
- Explanation of benefits for the downstream project
- Specific files or patterns that need updating

This technique ensures downstream projects don't miss important template improvements and helps
maintain long-term consistency.

### What NOT to Flag

- Project-specific naming (tool names, package IDs, repository URLs)
Expand Down
17 changes: 17 additions & 0 deletions .github/agents/requirements-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,23 @@ Follow the `requirements.yaml` structure:
- Linked to appropriate test(s)
- Enforced via: `dotnet reqstream --requirements requirements.yaml --tests "test-results/**/*.trx" --enforce`

### Test Source Filters

Test links in `requirements.yaml` can include a source filter prefix to restrict which test results count as
evidence. This is critical for platform and framework requirements - **never remove these filters**.

- `windows@TestName` - proves the test passed on a Windows platform
- `ubuntu@TestName` - proves the test passed on a Linux (Ubuntu) platform
- `net8.0@TestName` - proves the test passed under the .NET 8 target framework
- `net9.0@TestName` - proves the test passed under the .NET 9 target framework
- `net10.0@TestName` - proves the test passed under the .NET 10 target framework
- `dotnet8.x@TestName` - proves the self-validation test ran on a machine with .NET 8.x runtime
- `dotnet9.x@TestName` - proves the self-validation test ran on a machine with .NET 9.x runtime
- `dotnet10.x@TestName` - proves the self-validation test ran on a machine with .NET 10.x runtime

Without the source filter, a test result from any platform/framework satisfies the requirement. Removing a
filter invalidates the evidence for platform/framework requirements.

## Defer To

- **Software Developer Agent**: For implementing self-validation tests
Expand Down
3 changes: 3 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ Before submitting this pull request, ensure you have completed the following:

- [ ] Code builds successfully: `dotnet build --configuration Release`
- [ ] All tests pass: `dotnet test --configuration Release`
- [ ] Self-validation tests pass:
`dotnet run --project src/DemaConsulting.SonarMark --configuration Release --framework net10.0`
`--no-build -- --validate`
- [ ] Code produces zero warnings

### Code Quality
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@ jobs:
9.x
10.x

- name: Restore Tools
run: >
dotnet tool restore

- name: Restore Dependencies
run: >
dotnet restore
Expand Down Expand Up @@ -419,7 +423,7 @@ jobs:
run: |
echo "Capturing tool versions..."
dotnet versionmark --capture --job-id "build-docs" -- \
dotnet git node npm pandoc weasyprint sarifmark reqstream buildmark versionmark
dotnet git node npm pandoc weasyprint sarifmark sonarmark reqstream buildmark versionmark
echo "✓ Tool versions captured"

# === GENERATE MARKDOWN REPORTS ===
Expand Down
1 change: 1 addition & 0 deletions .markdownlint-cli2.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"MD003": { "style": "atx" },
"MD007": { "indent": 2 },
"MD013": { "line_length": 120 },
"MD024": { "siblings_only": true },
"MD025": false,
"MD033": false,
"MD041": false
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ SonarQube/SonarCloud analysis results.

## Tech Stack

- C# 12, .NET 8.0/9.0/10.0, MSTest, dotnet CLI, NuGet
- C# (latest), .NET 8.0/9.0/10.0, MSTest, dotnet CLI, NuGet

## Key Files

- **`requirements.yaml`** - All requirements with test linkage (enforced via `dotnet reqstream --enforce`)
- **`.editorconfig`** - Code style (file-scoped namespaces, 4-space indent, UTF-8+BOM, LF endings)
- **`.cspell.json`, `.markdownlint.json`, `.yamllint.yaml`** - Linting configs
- **`.cspell.json`, `.markdownlint-cli2.jsonc`, `.yamllint.yaml`** - Linting configs
- **`.vscode/tasks.json`** - VS Code tasks for build, test, lint, and quality checks

## Requirements (SonarMark-Specific)
Expand Down
47 changes: 30 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,29 +137,41 @@ sonarmark --validate
sonarmark --validate --results validation-results.trx
```

### Self-Validation Tests
## Self Validation

SonarMark includes built-in self-validation tests that verify the tool's functionality without requiring
a live SonarQube/SonarCloud server. These tests use mock data to validate core features and generate test
result files in TRX or JUnit format.
Running self-validation produces a report containing the following information:

The self-validation suite includes the following tests:
```text
# DEMA Consulting SonarMark

| Information | Value |
| :------------------ | :------------------------------------------------- |
| SonarMark Version | <version> |
| Machine Name | <machine-name> |
| OS Version | <os-version> |
| DotNet Runtime | <dotnet-runtime-version> |
| Time Stamp | <timestamp> UTC |

✓ SonarMark_QualityGateRetrieval - Passed
✓ SonarMark_IssuesRetrieval - Passed
✓ SonarMark_HotSpotsRetrieval - Passed
✓ SonarMark_MarkdownReportGeneration - Passed

Total Tests: 4
Passed: 4
Failed: 0
```

| Test Name | Description |
| :-------- | :---------- |
| `SonarMark_QualityGateRetrieval` | Verifies fetching and processing quality gate status |
| `SonarMark_IssuesRetrieval` | Verifies fetching and processing code issues |
| `SonarMark_HotSpotsRetrieval` | Verifies fetching and processing security hot-spots |
| `SonarMark_MarkdownReportGeneration` | Verifies generating markdown reports with quality metrics |
Each test in the report proves:

These tests provide evidence of the tool's functionality and are particularly useful for:
- **`SonarMark_QualityGateRetrieval`** - Verifies fetching and processing quality gate status from SonarQube/SonarCloud.
- **`SonarMark_IssuesRetrieval`** - Verifies fetching and processing code issues with severity classification.
- **`SonarMark_HotSpotsRetrieval`** - Verifies fetching and processing security hot-spots and vulnerabilities.
- **`SonarMark_MarkdownReportGeneration`** - Verifies generating markdown reports with quality metrics and findings.

- Verifying the installation is working correctly
- Running automated tests in CI/CD pipelines without requiring SonarQube access
- Generating test evidence for compliance and traceability requirements
See the [User Guide][link-guide] for more details on the self-validation tests.

For detailed usage instructions, command-line options, and examples, including tool update instructions, see the
[Usage Guide](https://github.com/demaconsulting/SonarMark/blob/main/docs/guide/guide.md).
On validation failure the tool will exit with a non-zero exit code.

## Report Format

Expand Down Expand Up @@ -251,6 +263,7 @@ SonarMark is built with the following open-source projects:

<!-- Link References -->
[link-forks]: https://github.com/demaconsulting/SonarMark/network/members
[link-guide]: https://github.com/demaconsulting/SonarMark/blob/main/docs/guide/guide.md
[link-stars]: https://github.com/demaconsulting/SonarMark/stargazers
[link-contributors]: https://github.com/demaconsulting/SonarMark/graphs/contributors
[link-license]: https://github.com/demaconsulting/SonarMark/blob/main/LICENSE
Expand Down
72 changes: 33 additions & 39 deletions docs/guide/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,70 +437,64 @@ If no security hot-spots are found:
Found no security hot-spots
```

# Running Self-Validation
# Self-Validation

SonarMark includes built-in self-validation tests to verify functionality without requiring access to a real
SonarQube/SonarCloud server. The validation uses mock data to test core features.
Self-validation produces a report demonstrating that SonarMark is functioning
correctly. This is useful in regulated industries where tool validation evidence is required.

## Running Validation

To perform self-validation:

```bash
sonarmark --validate
```

## Validation Tests

The self-validation suite includes the following tests that verify core functionality:

| Test Name | Description |
| :-------- | :---------- |
| `SonarMark_QualityGateRetrieval` | Verifies fetching and processing quality gate status from SonarQube/SonarCloud |
| `SonarMark_IssuesRetrieval` | Verifies fetching and processing code issues with severity classification |
| `SonarMark_HotSpotsRetrieval` | Verifies fetching and processing security hot-spots and vulnerabilities |
| `SonarMark_MarkdownReportGeneration` | Verifies generating markdown reports with quality metrics and findings |
To save validation results to a file:

These tests provide evidence of the tool's functionality and are particularly useful for:
```bash
sonarmark --validate --results results.trx
```

- Verifying the installation is working correctly on different platforms and .NET versions
- Running automated tests in CI/CD pipelines without requiring SonarQube access
- Generating test evidence for compliance and traceability requirements
- Validating tool functionality before deployment
The results file format is determined by the file extension: `.trx` for TRX (MSTest) format,
or `.xml` for JUnit format.

**Note**: The test names with the `SonarMark_` prefix are designed for clear identification in test
result files (TRX/JUnit) when integrating with larger projects or test frameworks.
## Validation Report

## Validation Output
The validation report contains the tool version, machine name, operating system version,
.NET runtime version, timestamp, and test results.

Example output:
Example validation report:

```text
SonarMark version 1.0.0
Copyright (c) DEMA Consulting

# DEMA Consulting SonarMark
## Self-Validation Tests

[PASS] Quality Gate Status Retrieval
[PASS] Issues Retrieval
[PASS] Hot-Spots Retrieval
[PASS] Markdown Report Generation
| Information | Value |
| :------------------ | :------------------------------------------------- |
| SonarMark Version | 1.0.0 |
| Machine Name | BUILD-SERVER |
| OS Version | Ubuntu 22.04.3 LTS |
| DotNet Runtime | .NET 10.0.0 |
| Time Stamp | 2024-01-15 10:30:00 UTC |

✓ SonarMark_QualityGateRetrieval - Passed
✓ SonarMark_IssuesRetrieval - Passed
✓ SonarMark_HotSpotsRetrieval - Passed
✓ SonarMark_MarkdownReportGeneration - Passed

Total Tests: 4
Passed: 4
Failed: 0
```

## Saving Validation Results
## Validation Tests

Save results in TRX or JUnit XML format for integration with test reporting tools:
Each test proves specific functionality works correctly:

```bash
# TRX format (for Azure DevOps, Visual Studio)
sonarmark --validate --results validation-results.trx

# JUnit XML format (for Jenkins, GitLab CI)
sonarmark --validate --results validation-results.xml
```
- **`SonarMark_QualityGateRetrieval`** - Verifies fetching and processing quality gate status from SonarQube/SonarCloud.
- **`SonarMark_IssuesRetrieval`** - Verifies fetching and processing code issues with severity classification.
- **`SonarMark_HotSpotsRetrieval`** - Verifies fetching and processing security hot-spots and vulnerabilities.
- **`SonarMark_MarkdownReportGeneration`** - Verifies generating markdown reports with quality metrics and findings.

# Best Practices

Expand Down
Loading