News: Roadmap update: HTML report comes next

Author: gasparnagy

_posts/2025-06-09-roadmap-update-html-report.md

@@ -0,0 +1,91 @@
+---
+layout: post
+title: "Roadmap update: HTML report comes next"
+date: 2025-06-09
+categories: News
+tags: Release Roadmap LivingDoc
+image: assets/images/2025/reqnroll_html_report_header.webp
+image_caption: "HTML report generated by Reqnroll Cucumber Messages PR"
+image_caption_style: "color: var(--wp--preset--color--body)"
+author: gasparnagy
+---
+
+We released the last update for our V2 line in April, and then jumped into preparing for the next major release: V3. On this occasion, I would like to share some updates on our roadmap and some good news about the HTML report.
+
+## TL;DR
+
+Native HTML report generation is coming in V3! This will work out of the box, without additional conversion tools, and will be free and open source!
+
+## V2 was about test execution
+
+The V2 versions of Reqnroll focused on stabilizing the Reqnroll codebase and the test execution workflow. The SpecFlow codebase [was forked at version v4-beta](2024-02-08-from-specflow-to-reqnroll-why-and-how.md). That version was never officially released and therefore was only used by a few SpecFlow projects. It contained many changes, especially related to asynchronous execution of tests. This was a significant change affecting many aspects of execution, and because it was only tried by a limited set of users, it was not "battle tested." We could say that Reqnroll V2 is the "proper SpecFlow v4" release.
+
+Another important and outstanding need that we inherited from SpecFlow was to clean up the execution workflow. Probably the most notable element of that was supporting test-level parallelization, but in the background we also had to redesign the entire resource handling, hook execution, and error handling infrastructure.
+
+## V3 is about SpecFlow ecosystem parity
+
+Users have seen the SpecFlow ecosystem as a homogeneous set of frameworks and supporting tools that consisted of the SpecFlow core framework, the Visual Studio and Rider extensions, the SpecFlow plugins, and the SpecFlow+ toolset — particularly the SpecFlow+ LivingDoc Generator. In reality, each of these parts and tools has its own story, and most importantly, the SpecFlow+ toolset was never open source, so we could not fork it as we did with the other assets.
+
+This has been a pain point for many users, because LivingDoc Generator was the go-to tool for generating nice HTML reports about their specifications and execution results. And we could not take that over.
+
+Although there are alternative solutions for generating HTML reports (notably [Allure Report](https://allurereport.org/), one of the most popular open-source reporting tools, which was one of the first to integrate with Reqnroll), we felt that we needed to provide a built-in solution as well.
+
+Our strategy has been to adopt the concept of "Cucumber Messages" from the Cucumber tool family. Cucumber Messages is basically a standardized file format that can capture all execution details of a BDD framework, including step or hook executions and attachments. Our idea was that if we were able to provide a Cucumber Messages report, we would open up visualization possibilities that have already been available to Cucumber tools. We cannot talk about this feature without mentioning [Chris Rudolphi](https://github.com/clrudolphi), who made an incredible effort to bring this strategy to reality. Thank you, Chris!
+
+While V3 will contain many other interesting updates, most importantly, it will include the first version of our native Reqnroll HTML report generation! With this step, we will finally reach parity with the SpecFlow ecosystem.
+
+## Reqnroll HTML Report
+
+While this is still in the beta phase, I would like to share some details about Reqnroll HTML report generation. The beta phase in this case means that the code has been implemented and is working, but we are in the middle of the review process and making some small fixes.
+
+As mentioned above, our strategy was to be able to produce Cucumber Messages so that we could use existing tooling to turn that into HTML. We were aware of the [Cucumber React Components](https://github.com/cucumber/react-components) project, which provides HTML visualization for Cucumber Messages, but in the end, we found another important open-source component, [Cucumber HTML Formatter](https://github.com/cucumber/html-formatter/), which provides an end-to-end wrapper for producing a complete HTML report from Cucumber Messages. Exactly what we needed!
+
+Have you heard of "Cucumber Formatters"? That is a feature and extensibility point in Cucumber tools that can be used to produce reports, outputs, or other results in a specific format. Since Cucumber started as a command-line tool, the concept of formatters has existed there for a long time — basically, when you executed Cucumber, it was a formatter that produced the console output and another formatter that generated the test result JSON file. As SpecFlow/Reqnroll has been executed by the .NET test execution hosts (e.g., `dotnet test` or *Visual Studio Test Explorer*), we had not implemented this particular concept before.
+
+During the implementation of the Cucumber Messages output, we realized that what we were doing was essentially the same as what Cucumber calls "formatters," so we (better to say Chris) reframed our solution and implemented the formatter concept from Cucumber into Reqnroll.
+
+And if we have formatters in Reqnroll, then we can implement the Cucumber Messages output as one formatter and the HTML output as another! That means we will not need any additional tool to convert Cucumber Messages to HTML (as was needed for "SpecFlow+ LivingDoc Generator"), but it will generate the HTML file out of the box! Yes, even if you run the tests from Visual Studio!
+
+It is important to mention that all parts of the HTML generation toolchain (Reqnroll, formatters, Cucumber HTML formatter, Cucumber React Components) are free and open source.
+
+The formatter concept has two important characteristics that make it even more appealing for Reqnroll:
+1. You can register multiple formatters. If you need to have both Cucumber Messages and an HTML report, you can register both!
+2. You can write your own formatter if you need a customized report or output format! This is all integrated into the Reqnroll Plugin infrastructure.
+
+Currently, the Cucumber Messages and HTML formatters will be packaged with Reqnroll, so these can be used out of the box. Later, we might extend the set of formatters (built-in or as plugins), and I'm pretty sure there will be formatters from the community as well.
+
+To use the formatters, you simply need to add a `formatters` setting to your `reqnroll.json` config file (you can also enable it with environment variables). With the current beta, this is how a sample config file looks that produces both HTML and Cucumber Messages reports:
+
+```
+{
+  "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json",
+  "formatters": {
+    "html": {
+      "outputFilePath": "report/reqnroll_report.html"
+    },
+    "messages": {
+      "outputFilePath": "messages/reqnroll_report.ndjson"
+    }
+  }
+}
+```
+
+This is how the report looks for a Calculator example. Pretty!
+
+{% include figure.html url="/assets/images/2025/reqnroll_html_report.webp" alt="HTML report generated by Reqnroll Cucumber Messages PR" description="HTML report generated by Reqnroll Cucumber Messages PR" %}
+
+## V4 is about structural changes
+
+After V3, we will not stop, but continue with further structural changes that will hopefully make everyone's life easier. We have plans to consolidate the [dependency injection support plugins](https://github.com/orgs/reqnroll/discussions/472), the configuration infrastructure, and we also plan to use the Roslyn infrastructure to improve the overall workflow.
+
+For V4, we will also review the range of different targets we support and deprecate those that require high maintenance costs and are not broadly used. The support for adding feature files to VB.NET projects is one candidate for deprecation.
+
+*.NET Framework 4.8* is currently being used by [almost 10% of projects](2025-06-06-monthly-stats-2025-05.md), so we will need to provide support for that, but we still have to decide what is the best approach. To support *.NET Framework*, we need to keep the codebase at *.NET Standard 2.0* compatibility and cannot utilize features provided by *.NET 5* and later frameworks.
+
+## TUnit and xUnit V3 support coming too
+
+There are also two new unit test frameworks we plan to support: TUnit and xUnit V3. As these are additional features and not breaking changes, they can probably be included as updates for V3.
+
+## Your feedback counts
+
+Thank you for reading through this long post. Please share your ideas in the [Reqnroll Discussion Board](https://github.com/orgs/reqnroll/discussions/categories/ideas) or on our [Discord server](https://go.reqnroll.net/discord-invite).