Integration with ReportPortal

[nikhilbhandari]

Use case

Maestro should be able to integrate with ReportPortal as it is standard report aggregation framework used widely. All the integrations available except Maestro. There is manual import or API import available thru Junit plugin, but it will only upload the summary of test result. ReportPortal is rich functionality where you can upload step-by-step execution summary, attach screen-shots, video recordings.

Proposal

https://reportportal.io/features#quality-gates

Anything else?

No response

[linear]

MAE-405

[Fishbowler]

Maestro already has JUnit output, as you mention.

Since it's a CLI tool, intentionally doesn't attempt to integrate with cloud systems. Is this a cloud system? It's unclear from this info.

You mention the JUnit summary - that's a function of the existing reporting, that the JUnit only contains tests. I suspect there are other issues in this tracker you can follow about extending JUnit support to include steps.

[nikhilbhandari]

ReportPortal is a self-hosted solution deployed securely within an organization’s own infrastructure rather than the cloud. It acts as a centralized reporting hub for QA and engineering teams, consolidating results across various testing frameworks (e.g., Playwright for Web; Karate, Cypress, and RestAssured for APIs; and AndroidTest for Mobile).

While developing a custom integration requires an initial engineering effort to ensure detailed reports are published correctly, the return on investment is highly valuable for mature automation practices.

Key Benefits:

• Centralized Triaging: Once tests are integrated into CI/CD pipelines and DeviceFarm solutions, ReportPortal provides a single interface for engineers to quickly troubleshoot failures using rich artifacts like execution steps, logs, screenshots, and video recordings.

• Advanced Analytics: Its powerful dashboards offer deep insights into overall automation health, making it easy to identify flaky tests, track execution times, monitor historical trends, and measure test automation growth.

[Fishbowler]

But the integration is still an HTTP endpoint?
Presently, Maestro intentionally keeps to file-based report outputs only.

[nikhilbhandari]

Yes it will be HTTP endpoint.
Reporting is critical aspect when tests are running on various platforms (Devices, OS, Browsers) to take decision to move to Production or Troubleshoot failures. It is also one of the factors in decision making while selecting the framework. So please keep this in consideration when you plan for new features roadmap. Thank you.

[Fishbowler]

Can you elaborate on the implementation requirements for this?

[Fishbowler]

I've taken a brief look at the docs. Staff are not likely to pick this up as a work item - it doesn't benefit enough users.
A graceful implementation by an open source contributor might be considered.
I still think the best implementation here is an extended JUnit implementation that's useful for ReportPortal, but also other tools too.

[nikhilbhandari]

Thank you for quick response and considering the new feature request.
ReportPortal is one of the options. But I still feel there is need for rich test execution reporting where someone can see steps, screen-shots, errors, logs, recording in a single view.