Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/agents/code-quality-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Ensure the project is:

1. **Build**: Zero warnings (TreatWarningsAsErrors=true)
2. **Linting**:
- markdownlint (`.markdownlint.json`)
- markdownlint (`.markdownlint-cli2.jsonc`)
- cspell (`.cspell.json`)
- yamllint (`.yamllint.yaml`)
- dotnet format (`.editorconfig`)
Expand Down
30 changes: 29 additions & 1 deletion .github/agents/repo-consistency-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ The agent reviews the following areas for consistency with the template:

#### Quality Configuration

- **Linting Rules**: `.cspell.json`, `.markdownlint.json`, `.yamllint.yaml`
- **Linting Rules**: `.cspell.json`, `.markdownlint-cli2.jsonc`, `.yamllint.yaml`
- Note: Spelling exceptions will be repository-specific
- **Editor Config**: `.editorconfig` settings (file-scoped namespaces, 4-space indent, UTF-8+BOM, LF endings)
- **Code Style**: C# code style rules and analyzer configuration
Expand Down Expand Up @@ -88,6 +88,34 @@ The agent reviews the following areas for consistency with the template:
3. **Recommend Updates**: Suggest specific files or patterns that should be updated
4. **Respect Customizations**: Recognize valid project-specific customizations

### Tracking Template Evolution

To ensure downstream projects benefit from recent template improvements, review recent pull requests
merged into the template repository:

1. **List Recent PRs**: Retrieve recently merged PRs from `demaconsulting/TemplateDotNetTool`
- Review the last 10-20 PRs to identify template improvements

2. **Identify Propagatable Changes**: For each PR, determine if changes should apply to downstream
projects:
- Focus on structural changes (workflows, agents, configurations) over content-specific changes
- Note changes to `.github/`, linting configurations, project patterns, and documentation
structure

3. **Check Downstream Application**: Verify if identified changes exist in the downstream project:
- Check if similar files/patterns exist in downstream
- Compare file contents between template and downstream project
- Look for similar PR titles or commit messages in downstream repository history

4. **Recommend Missing Updates**: For changes not yet applied, include them in the consistency
review with:
- Description of the template change (reference PR number)
- Explanation of benefits for the downstream project
- Specific files or patterns that need updating

This technique ensures downstream projects don't miss important template improvements and helps
maintain long-term consistency.

### What NOT to Flag

- Project-specific naming (tool names, package IDs, repository URLs)
Expand Down
17 changes: 17 additions & 0 deletions .github/agents/requirements-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,23 @@ Follow the `requirements.yaml` structure:
- Linked to appropriate test(s)
- Enforced via: `dotnet reqstream --requirements requirements.yaml --tests "test-results/**/*.trx" --enforce`

### Test Source Filters

Test links in `requirements.yaml` can include a source filter prefix to restrict which test results count as
evidence. This is critical for platform and framework requirements - **never remove these filters**.

- `windows@TestName` - proves the test passed on a Windows platform
- `ubuntu@TestName` - proves the test passed on a Linux (Ubuntu) platform
- `net8.0@TestName` - proves the test passed under the .NET 8 target framework
- `net9.0@TestName` - proves the test passed under the .NET 9 target framework
- `net10.0@TestName` - proves the test passed under the .NET 10 target framework
- `dotnet8.x@TestName` - proves the self-validation test ran on a machine with .NET 8.x runtime
- `dotnet9.x@TestName` - proves the self-validation test ran on a machine with .NET 9.x runtime
- `dotnet10.x@TestName` - proves the self-validation test ran on a machine with .NET 10.x runtime

Without the source filter, a test result from any platform/framework satisfies the requirement. Removing a
filter invalidates the evidence for platform/framework requirements.

## Defer To

- **Software Developer Agent**: For implementing self-validation tests
Expand Down
3 changes: 3 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ Before submitting this pull request, ensure you have completed the following:

- [ ] Code builds successfully: `dotnet build --configuration Release`
- [ ] All tests pass: `dotnet test --configuration Release`
- [ ] Self-validation tests pass:
`dotnet run --project src/DemaConsulting.SonarMark --configuration Release --framework net10.0`
`--no-build -- --validate`
- [ ] Code produces zero warnings

### Code Quality
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@ jobs:
9.x
10.x

- name: Restore Tools
run: >
dotnet tool restore

- name: Restore Dependencies
run: >
dotnet restore
Expand Down Expand Up @@ -419,7 +423,7 @@ jobs:
run: |
echo "Capturing tool versions..."
dotnet versionmark --capture --job-id "build-docs" -- \
dotnet git node npm pandoc weasyprint sarifmark reqstream buildmark versionmark
dotnet git node npm pandoc weasyprint sarifmark sonarmark reqstream buildmark versionmark
echo "✓ Tool versions captured"

# === GENERATE MARKDOWN REPORTS ===
Expand Down
1 change: 1 addition & 0 deletions .markdownlint-cli2.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"MD003": { "style": "atx" },
"MD007": { "indent": 2 },
"MD013": { "line_length": 120 },
"MD024": { "siblings_only": true },
"MD025": false,
"MD033": false,
"MD041": false
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ SonarQube/SonarCloud analysis results.

## Tech Stack

- C# 12, .NET 8.0/9.0/10.0, MSTest, dotnet CLI, NuGet
- C# (latest), .NET 8.0/9.0/10.0, MSTest, dotnet CLI, NuGet

## Key Files

- **`requirements.yaml`** - All requirements with test linkage (enforced via `dotnet reqstream --enforce`)
- **`.editorconfig`** - Code style (file-scoped namespaces, 4-space indent, UTF-8+BOM, LF endings)
- **`.cspell.json`, `.markdownlint.json`, `.yamllint.yaml`** - Linting configs
- **`.cspell.json`, `.markdownlint-cli2.jsonc`, `.yamllint.yaml`** - Linting configs
- **`.vscode/tasks.json`** - VS Code tasks for build, test, lint, and quality checks

## Requirements (SonarMark-Specific)
Expand Down
47 changes: 30 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,29 +137,41 @@ sonarmark --validate
sonarmark --validate --results validation-results.trx
```

### Self-Validation Tests
## Self Validation

SonarMark includes built-in self-validation tests that verify the tool's functionality without requiring
a live SonarQube/SonarCloud server. These tests use mock data to validate core features and generate test
result files in TRX or JUnit format.
Running self-validation produces a report containing the following information:

The self-validation suite includes the following tests:
```text
# DEMA Consulting SonarMark

| Information | Value |
| :------------------ | :------------------------------------------------- |
| SonarMark Version | <version> |
| Machine Name | <machine-name> |
| OS Version | <os-version> |
| DotNet Runtime | <dotnet-runtime-version> |
| Time Stamp | <timestamp> UTC |

✓ SonarMark_QualityGateRetrieval - Passed
✓ SonarMark_IssuesRetrieval - Passed
✓ SonarMark_HotSpotsRetrieval - Passed
✓ SonarMark_MarkdownReportGeneration - Passed

Total Tests: 4
Passed: 4
Failed: 0
```

| Test Name | Description |
| :-------- | :---------- |
| `SonarMark_QualityGateRetrieval` | Verifies fetching and processing quality gate status |
| `SonarMark_IssuesRetrieval` | Verifies fetching and processing code issues |
| `SonarMark_HotSpotsRetrieval` | Verifies fetching and processing security hot-spots |
| `SonarMark_MarkdownReportGeneration` | Verifies generating markdown reports with quality metrics |
Each test in the report proves:

These tests provide evidence of the tool's functionality and are particularly useful for:
- **`SonarMark_QualityGateRetrieval`** - Verifies fetching and processing quality gate status from SonarQube/SonarCloud.
- **`SonarMark_IssuesRetrieval`** - Verifies fetching and processing code issues with severity classification.
- **`SonarMark_HotSpotsRetrieval`** - Verifies fetching and processing security hot-spots and vulnerabilities.
- **`SonarMark_MarkdownReportGeneration`** - Verifies generating markdown reports with quality metrics and findings.

- Verifying the installation is working correctly
- Running automated tests in CI/CD pipelines without requiring SonarQube access
- Generating test evidence for compliance and traceability requirements
See the [User Guide][link-guide] for more details on the self-validation tests.

For detailed usage instructions, command-line options, and examples, including tool update instructions, see the
[Usage Guide](https://github.com/demaconsulting/SonarMark/blob/main/docs/guide/guide.md).
On validation failure the tool will exit with a non-zero exit code.

## Report Format

Expand Down Expand Up @@ -251,6 +263,7 @@ SonarMark is built with the following open-source projects:

<!-- Link References -->
[link-forks]: https://github.com/demaconsulting/SonarMark/network/members
[link-guide]: https://github.com/demaconsulting/SonarMark/blob/main/docs/guide/guide.md
[link-stars]: https://github.com/demaconsulting/SonarMark/stargazers
[link-contributors]: https://github.com/demaconsulting/SonarMark/graphs/contributors
[link-license]: https://github.com/demaconsulting/SonarMark/blob/main/LICENSE
Expand Down
72 changes: 33 additions & 39 deletions docs/guide/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,70 +437,64 @@ If no security hot-spots are found:
Found no security hot-spots
```

# Running Self-Validation
# Self-Validation

SonarMark includes built-in self-validation tests to verify functionality without requiring access to a real
SonarQube/SonarCloud server. The validation uses mock data to test core features.
Self-validation produces a report demonstrating that SonarMark is functioning
correctly. This is useful in regulated industries where tool validation evidence is required.

## Running Validation

To perform self-validation:

```bash
sonarmark --validate
```

## Validation Tests

The self-validation suite includes the following tests that verify core functionality:

| Test Name | Description |
| :-------- | :---------- |
| `SonarMark_QualityGateRetrieval` | Verifies fetching and processing quality gate status from SonarQube/SonarCloud |
| `SonarMark_IssuesRetrieval` | Verifies fetching and processing code issues with severity classification |
| `SonarMark_HotSpotsRetrieval` | Verifies fetching and processing security hot-spots and vulnerabilities |
| `SonarMark_MarkdownReportGeneration` | Verifies generating markdown reports with quality metrics and findings |
To save validation results to a file:

These tests provide evidence of the tool's functionality and are particularly useful for:
```bash
sonarmark --validate --results results.trx
```

- Verifying the installation is working correctly on different platforms and .NET versions
- Running automated tests in CI/CD pipelines without requiring SonarQube access
- Generating test evidence for compliance and traceability requirements
- Validating tool functionality before deployment
The results file format is determined by the file extension: `.trx` for TRX (MSTest) format,
or `.xml` for JUnit format.

**Note**: The test names with the `SonarMark_` prefix are designed for clear identification in test
result files (TRX/JUnit) when integrating with larger projects or test frameworks.
## Validation Report

## Validation Output
The validation report contains the tool version, machine name, operating system version,
.NET runtime version, timestamp, and test results.

Example output:
Example validation report:

```text
SonarMark version 1.0.0
Copyright (c) DEMA Consulting

# DEMA Consulting SonarMark
## Self-Validation Tests

[PASS] Quality Gate Status Retrieval
[PASS] Issues Retrieval
[PASS] Hot-Spots Retrieval
[PASS] Markdown Report Generation
| Information | Value |
| :------------------ | :------------------------------------------------- |
| SonarMark Version | 1.0.0 |
| Machine Name | BUILD-SERVER |
| OS Version | Ubuntu 22.04.3 LTS |
| DotNet Runtime | .NET 10.0.0 |
| Time Stamp | 2024-01-15 10:30:00 UTC |

✓ SonarMark_QualityGateRetrieval - Passed
✓ SonarMark_IssuesRetrieval - Passed
✓ SonarMark_HotSpotsRetrieval - Passed
✓ SonarMark_MarkdownReportGeneration - Passed

Total Tests: 4
Passed: 4
Failed: 0
```

## Saving Validation Results
## Validation Tests

Save results in TRX or JUnit XML format for integration with test reporting tools:
Each test proves specific functionality works correctly:

```bash
# TRX format (for Azure DevOps, Visual Studio)
sonarmark --validate --results validation-results.trx

# JUnit XML format (for Jenkins, GitLab CI)
sonarmark --validate --results validation-results.xml
```
- **`SonarMark_QualityGateRetrieval`** - Verifies fetching and processing quality gate status from SonarQube/SonarCloud.
- **`SonarMark_IssuesRetrieval`** - Verifies fetching and processing code issues with severity classification.
- **`SonarMark_HotSpotsRetrieval`** - Verifies fetching and processing security hot-spots and vulnerabilities.
- **`SonarMark_MarkdownReportGeneration`** - Verifies generating markdown reports with quality metrics and findings.

# Best Practices

Expand Down
Loading