diff --git a/assets/images/hyperexecute/html-report.webp b/assets/images/hyperexecute/html-report.webp deleted file mode 100644 index 74e2b7c87..000000000 Binary files a/assets/images/hyperexecute/html-report.webp and /dev/null differ diff --git a/assets/images/hyperexecute/knowledge-base/reports/cypress-mochawesome-report.gif b/assets/images/hyperexecute/knowledge-base/reports/cypress-mochawesome-report.gif new file mode 100644 index 000000000..6e48ec73e Binary files /dev/null and b/assets/images/hyperexecute/knowledge-base/reports/cypress-mochawesome-report.gif differ diff --git a/assets/images/hyperexecute/knowledge-base/reports/error-categorization-report.gif b/assets/images/hyperexecute/knowledge-base/reports/error-categorization-report.gif index 2965f7871..6bba78500 100644 Binary files a/assets/images/hyperexecute/knowledge-base/reports/error-categorization-report.gif and b/assets/images/hyperexecute/knowledge-base/reports/error-categorization-report.gif differ diff --git a/assets/images/hyperexecute/knowledge-base/reports/mochawesome-report.png b/assets/images/hyperexecute/knowledge-base/reports/mochawesome-report.png deleted file mode 100644 index 36cc4ece3..000000000 Binary files a/assets/images/hyperexecute/knowledge-base/reports/mochawesome-report.png and /dev/null differ diff --git a/assets/images/hyperexecute/knowledge-base/reports/playwright-html-report.gif b/assets/images/hyperexecute/knowledge-base/reports/playwright-html-report.gif new file mode 100644 index 000000000..c1894930b Binary files /dev/null and b/assets/images/hyperexecute/knowledge-base/reports/playwright-html-report.gif differ diff --git a/docs/allure-reports.md b/docs/allure-reports.md index e457e8caf..f58b29fcb 100644 --- a/docs/allure-reports.md +++ b/docs/allure-reports.md @@ -2,7 +2,7 @@ id: allure-reports title: Allure Reports on HyperExecute hide_title: false -sidebar_label: Allure Reports +sidebar_label: Allure description: Learn how to generate Allure test report for HyperExecute on lambdatest and download the reports from the dashboard keywords: - allure reports diff --git a/docs/cucumber-report.md b/docs/cucumber-report.md index 7f03e22e8..892ec8f53 100644 --- a/docs/cucumber-report.md +++ b/docs/cucumber-report.md @@ -2,7 +2,7 @@ id: cucumber-report title: Cucumber Report on HyperExecute hide_title: true -sidebar_label: Cucumber Report +sidebar_label: Cucumber description: Learn how to generate Cucumber Report on lambdatest and download the reports from the dashboard keywords: - cucumber testing reports diff --git a/docs/cypress-mochaawesome-report.md b/docs/cypress-mochaawesome-report.md index cb2a9a10a..affbb0c0d 100644 --- a/docs/cypress-mochaawesome-report.md +++ b/docs/cypress-mochaawesome-report.md @@ -1,8 +1,8 @@ --- id: cypress-mochaawesome-report -title: Consolidated Mochawesome Report for Cypress on HyperExecute +title: Consolidated Mochawesome Report for Cypress hide_title: false -sidebar_label: Cypress Mochawesome Report +sidebar_label: Cypress Mochawesome description: Discover how to create MochaAwesome test reports for Cypress framework testing on LambdaTest and how to download these reports from the dashboard keywords: - cypress testing @@ -33,46 +33,21 @@ slug: cypress-mochaawesome-report/ },{ "@type": "ListItem", "position": 3, - "name": "Playwright Testing With Playwright Test", + "name": "Cypress Mochawesome Report", "item": "https://www.lambdatest.com/support/docs/cypress-mochaawesome-report/" }] }) }} > -Cypress is an open-source, end-to-end testing framework made especially for contemporary web applications. It enables programmers to create and run tests that mimic user interactions right inside a web browser. +The Mochawesome reporter is a custom Cypress reporter designed to generate standalone HTML reports. These reports are clean, modern, and interactive, making it easier to visualize test results. With features like filtering tests and displaying stack traces for failures, the Mochawesome report is an essential tool in Cypress for clear and concise test insights. -### Built-in Reporters -There several pre-defined or built-in reporters and since Cypress is built on top of Mocha, that means any reporter built for Mocha can be used with Cypress as well. Each reporter may present the test results in a different way, suitable for specific needs or preferences. These built-in reporters offer different levels of detail and visualization options, helping developers to understand the test results and identify any potential issues more effectively. +## Generating a Consolidated HTML Report +Follow these steps to generate a consolidated HTML report using Mochawesome: -:::note info -Following are some of the built-in reporters: -- Spec Reporter -- Dot Matrix Reporter -- Nyan Reporter -- Tap Reporter -- Landing Strip Reporter -- List Reporter -::: - -There are many other built-in reporters as well. By default, Cypress uses the `spec` reporter to output information to `STDOUT` - -### Custom Reporters -Apart from the built-in reporters, Cypress Test also allows developers to create custom reporters i.e. you have the flexibility to define your own format and layout for displaying test results. Custom reporters are beneficial when you have specific reporting requirements or when you want to integrate the test results seamlessly into your existing development workflow. - -You can load custom Mocha reporters through a relative or absolute path which can be specified in your **Cypress Configuration file** or via the **CLI**. - -## Mochawesome Report on HyperExecute -*** - -Mochawesome reporter is a custom reporter that generates a standalone HTML report which helps to help visualize the test runs. It has simple, clean, and modern design. The report has filters to display only the desired tests and renders stack trace for failed tests. The Mochawesome report is one of the most important reports in Cypress. +### Step 1: Update Your Cypress Configuration +In your Cypress configuration file `cypress.config.js`, add the following code to enable Mochawesome as a reporter: -### Steps to Generate HTML Report on HyperExecute - -**Step 1:** Update your Cypress Configuration File by adding the below code - -#### Cypress Config File - -```bash +```javascript title="cypress.config.js" "reporter": "cypress-multi-reporters", "reporterOptions": { "reporterEnabled": [ @@ -87,17 +62,16 @@ Mochawesome reporter is a custom reporter that generates a standalone HTML repor } ``` -> The `overwrite` parameter should be set to **true** and `html` should be set to **false** because the mocha-merge utility which we use to merge reports do not support html files in path while merging json files - -:::tip Note - -The path to report is set to **``"reportDir": "cypress/results"``** and it should not be changed as we use this path for showing command logs on a dashboard. - +:::note +- The `overwrite` parameter should be set to `true` to ensure the report is replaced with the latest run results. +- The `html` option should be set to `false` because we will be merging JSON files later, and the mocha-merge utility does not support HTML files. +- Ensure the `reportDir` path is set to `"cypress/results"` —this path is used to generate logs that will be visible on the dashboard. **Do not change this path.** ::: -**Step 2:** Now configure the report parameters in the **HyperExecute YAML** file. +### Step 2: Configure the HyperExecute YAML File +In your HyperExecute YAML configuration, define the [`report`](https://www.lambdatest.com/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: -```bash +```yaml title="hyperexecute.yaml" report: true partialReports: frameworkName: cypress @@ -105,6 +79,7 @@ partialReports: type: html ``` -**Step 3:** Now Execute your job by triggering the HyperExecute CLI. You can visit the HyperExecute dashboard to download the report after job completion. +### Step 3: Execute Your Tests +Run your Cypress tests on HyperExecute using the CLI. After your job completes, you can visit the HyperExecute dashboard to download and view the consolidated Mochawesome report. -Image \ No newline at end of file +Image \ No newline at end of file diff --git a/docs/error-categorization-report.md b/docs/error-categorization-report.md index b175f699a..934c7b268 100644 --- a/docs/error-categorization-report.md +++ b/docs/error-categorization-report.md @@ -1,6 +1,6 @@ --- id: error-categorization-report -title: Error Categorization Report on HyperExecute +title: Error Categorization Report hide_title: false sidebar_label: Error Categorization description: Streamline error analysis with the Error Categorization Report. Identify patterns in test failures to enhance quality and efficiency in your testing process. @@ -45,7 +45,7 @@ You can generate and download other reports as well, along with the error-report This report is generated when your job includes multiple error categories. To enable the error categorization report, simply pass the [`errorCategorizedReport`](https://www.lambdatest.com/support/docs/deep-dive-into-hyperexecute-yaml/#errorcategorizedreport) flag to your YAML file: -```yaml +```yaml title="hyperexecute.yaml" errorCategorizedReport: enabled: true ``` diff --git a/docs/extent-report.md b/docs/extent-report.md index 9e25c68c9..f4642c217 100644 --- a/docs/extent-report.md +++ b/docs/extent-report.md @@ -2,7 +2,7 @@ id: extent-report title: Extent Report on HyperExecute hide_title: true -sidebar_label: Extent Report +sidebar_label: Extent description: Learn how to generate Extent Report on lambdatest and download the reports from the dashboard keywords: - extent testing reports diff --git a/docs/hyperexecute-job-reports.md b/docs/hyperexecute-job-reports.md index e4102e395..73950f480 100644 --- a/docs/hyperexecute-job-reports.md +++ b/docs/hyperexecute-job-reports.md @@ -36,70 +36,65 @@ slug: hyperexecute-job-reports/ }) }} > -HyperExecute reports are a powerful tool for analyzing the quality of your builds and identifying potential issues. By understanding the different types of reports available and how to use them, you can improve your testing process and deliver high-quality software. +HyperExecute Job Reports are designed to offer deep insights into the performance of your tests. By utilizing these reports, you can identify issues, track progress, and enhance your software testing workflow. Whether you're a developer or QA engineer, understanding and leveraging the various report types is crucial to delivering high-quality software efficiently. -## Types of HyperExecute Reports +## Extent Report +The [Extent Report](/support/docs/extent-report/) is a robust reporting framework commonly used with Java TestNG, and Selenium tests. It provides a feature-rich environment to explore test results, offering: -HyperExecute supports a wide variety of test reports, including: +- Comprehensive test case summaries. +- Visuals like screenshots and videos of test executions. +- Detailed execution logs. +- Graphical data, including charts and graphs to simplify test result analysis. -### Extent Report +## Extent Native +[Extent Native](/support/docs/native-extent-report/) is a more modern reporting solution from the Extent Reports family. Extent Native offers a native reporting experience that is highly interactive and responsive. It supports Java, TestNG, and Selenium tests and includes: -[**Extent Report**](/support/docs/extent-report/) is a popular reporting framework for Java, TestNG, and Selenium tests. It provides a comprehensive set of features for reporting test results, including detailed test case summaries, screenshots and videos of test execution, execution logs, and charts and graphs to analyze test results. +- Drill-down analysis to examine results in depth. +- Custom filtering options for tailored views. +- Real-time updates, accessible on any device for a seamless experience. -### Extent Native +## Native TestNG Reports +The [Native TestNG Reports](/support/docs/hyperexecute-native-testng-report/) provide a standardized summary extracted from raw Extent reports for each Virtual Machine (VM) at the conclusion of a HyperExecute job. These reports deliver a consolidated HTML view, generated by the TestNG framework, ensuring easy access to comprehensive testing results -[**Extent Native**](/support/docs/native-extent-report/) is a newer reporting framework from the Extent Reports team. It provides a native reporting experience for Java, TestNG, and Selenium tests. Extent Native reports are interactive and responsive, and they can be viewed on any device. They also include features such as drill-down analysis, custom filters, and real-time updates. +## Allure Report +[Allure](/support/docs/allure-reports/) is a cross-platform reporting tool compatible with various testing frameworks and languages. It offers a rich feature set for comprehensive analysis, including: -### Allure Report +- Interactive and responsive reporting dashboards. +- Drill-down analysis to explore test outcomes. +- Custom filters and real-time updates. +- CI/CD tool integration for streamlined workflows. -[**Allure**](/support/docs/allure-reports/) is a cross-platform reporting framework that can be used with a variety of testing frameworks and languages. It provides a rich reporting experience with features such as interactive and responsive reports, drill-down analysis, custom filters, real-time updates, and integration with CI/CD tools. +## Cucumber Report +[Cucumber Report](/support/docs/cucumber-report/) focuses on Cucumber test executions. It provides: -### Cucumber Report +- Test step summaries with detailed logs. +- Screenshots and videos for visual verification of results. +- Charts and graphs for analyzing test trends. -[**Cucumber Report**](/support/docs/cucumber-report/) is a reporting framework for Cucumber tests. It provides a detailed overview of the test execution, including test step summaries, screenshots and videos of test execution, execution logs, and charts and graphs to analyze test results. +## Playwright HTML +[Playwright HTML](/support/docs/playwright-html-report/) report is specifically designed for Playwright tests and offers detailed execution summaries, including: -### Playwright HTML +- Test suite, case, and step summaries. +- Visual assets such as screenshots and test execution videos. +- Execution logs along with graphical data for analyzing results. -[**Playwright HTML**](/support/docs/playwright-html-report/) is a reporting framework for Playwright tests. It provides a detailed overview of the test execution, including test suite summaries, test case summaries, test step summaries, screenshots and videos of test execution, execution logs, and charts and graphs to analyze test results. +## Specflow +[Specflow](/support/docs/specflow-report/) is a popular testing framework for Behavior-Driven Development (BDD), supporting languages like C#, Java, and Ruby. Its reports include: -### Specflow +- Feature summaries and scenario breakdowns. +- Step-by-step execution details. +- Screenshots, videos, and execution logs for comprehensive result visualization. -[**Specflow**](/support/docs/specflow-report/) is a testing framework for BDD (Behavior Driven Development). It can be used with a variety of programming languages, including C#, Java, and Ruby. Specflow reports provide a detailed overview of the test execution, including feature summaries, scenario summaries, step summaries, screenshots and videos of test execution, and execution logs. +## Cypress Mochawesome Report +[Cypress Mochawesome](/support/docs/cypress-mochaawesome-report/) is designed for Cypress and Mocha tests, this report offers: -### Cypress Mochawesome Report +- Test suite and case summaries. +- Test step breakdowns with logs. +- Screenshots and videos to validate test executions. -[**Cypress Mochawesome**](/support/docs/cypress-mochaawesome-report/) report is a reporting framework for Mocha tests. It provides a detailed overview of the test execution, including test suite summaries, test case summaries, test step summaries, screenshots and videos of test execution, and execution logs. +## Error Categorization Report +The [Error Categorization Report](/support/docs/error-categorization-report/) report is crafted to help users classify and understand test failures systematically. Key benefits include: -### Error Categorization Report - -The [**Error Categorization Report**](/support/docs/error-categorization-report/) serves as a tailored solution designed to assist users in classifying errors based on failed tests, fostering a systematic and lucid understanding of test failures with shared characteristics. Its primary objective is to streamline the process of pinpointing problematic stages and associated tests, offering users the ability to seamlessly navigate through detailed views and download pertinent information for in-depth analysis. - -### Cypress Report - -Cypress Reports is a reporting framework for Cypress tests. It provides a detailed overview of the test execution, including test suite summaries, test case summaries, test step summaries, screenshots and videos of test execution, execution logs, and charts and graphs to analyze test results. - -### Junit Report - -Junit Report is a reporting framework for Junit tests. It provides a detailed overview of the test execution, including test suite summaries, test case summaries, test step summaries, and execution logs. - -### Specflow Custom - -Specflow Custom reports are custom reports that can be generated using the Specflow reporting API. You can use the Specflow reporting API to generate reports in any format you want, such as HTML, PDF, or XML. - -## Benefits of Using HyperExecute Reports - -There are many benefits to using HyperExecute reports, including: - -- **Identify failed tests quickly and easily:** HyperExecute reports can help you to quickly identify which tests failed and why. This can help you to prioritize your debugging efforts and resolve issues more quickly. - -- **Analyze test results in depth:** HyperExecute reports provide a detailed overview of your test results, including charts and graphs to help you identify trends and patterns. This information can be used to improve your testing process and deliver high-quality software. - -- **Track progress over time:** HyperExecute reports can be used to track your team's progress over time and identify areas where improvement is needed. For example, you can use reports to track the percentage of tests that are passing and the average execution time. - -- **Improve communication and collaboration:** HyperExecute reports can be shared with other members of your team and stakeholders. This can help to improve communication and collaboration, and ensure that everyone is on the same page. - -Tips for Using HyperExecute Reports Effectively - -:::tip -Different HyperExecute reports provide different information. Choose the report that is most relevant to your needs. For example, if you want to quickly identify which tests failed, you would use the Summary Report -::: \ No newline at end of file +- Simplified error classification by grouping failures with shared characteristics. +- Detailed navigation through the problematic test stages. +- Downloadable data for thorough analysis and resolution. \ No newline at end of file diff --git a/docs/hyperexecute-native-testng-report.md b/docs/hyperexecute-native-testng-report.md index b8c523b85..4540e54db 100644 --- a/docs/hyperexecute-native-testng-report.md +++ b/docs/hyperexecute-native-testng-report.md @@ -2,7 +2,7 @@ id: hyperexecute-native-testng-report title: Native TestNG Reports on HyperExecute hide_title: false -sidebar_label: Native TestNG Reports +sidebar_label: Native TestNG description: Learn how to generate Native TestNG test report for HyperExecute on lambdatest and download the reports from the dashboard keywords: - native testng reports diff --git a/docs/native-extent-report.md b/docs/native-extent-report.md index e470c729e..a3b47277a 100644 --- a/docs/native-extent-report.md +++ b/docs/native-extent-report.md @@ -2,7 +2,7 @@ id: native-extent-report title: Native Extent Report on HyperExecute hide_title: true -sidebar_label: Extent Native Report +sidebar_label: Extent Native description: Learn how to generate Native Extent Report on lambdatest and download the reports from the dashboard keywords: - native testing reports diff --git a/docs/playwright-html-report.md b/docs/playwright-html-report.md index fc59c771e..6b9bcf754 100644 --- a/docs/playwright-html-report.md +++ b/docs/playwright-html-report.md @@ -1,8 +1,8 @@ --- id: playwright-html-report -title: Consolidated Playwright HTML Report on HyperExecute -hide_title: true -sidebar_label: Playwright HTML Report +title: Consolidated Playwright HTML Report +hide_title: false +sidebar_label: Playwright HTML description: Learn how to generate html test report for playwright framework testing on lambdatest and download the reports from the dashboard keywords: - playwright testing with Playwright test runner @@ -40,71 +40,34 @@ slug: playwright-html-report/ }) }} > +Playwright offers built-in HTML reports to help visualize the results of your test runs. These reports are detailed, user-friendly, and interactive, allowing you to analyze each test's status, logs, and errors in an organized manner. -# Consolidated Playwright HTML Report on HyperExecute -* * * +## Steps to Generate Consolidated HTML Report +Follow these steps to generate a consolidated Playwright HTML report: -Playwright is a Node.js library that uses a single API to automate Chromium, Firefox, and WebKit. It is designed to enable powerful, reliable, and efficient automated browser testing. Playwright can also automate Microsoft Edge since it is built on the open-source Chromium web framework. +### Step 1: Update Your Playwright Configuration -LambdaTest allows you to run Playwright tests across 40+ real browsers and operating system combinations. This guide will help you on how to generate a HTML Report for Playwright testing on the LambdaTest platform. +In your Playwright configuration file `playwright.config.js`, add the following code to enable the reporter: -## Reporters -*** - -Reporters are components or tools that generate human-readable output to display the results of test executions. They play a crucial role in providing developers, testers, and other stakeholders with valuable insights into the outcomes of automated tests. Reporters take the raw test execution data and format it into a structured and readable presentation. - -### Built-in Reporters -Playwright Test provides several pre-defined or built-in reporters. Each reporter may present the test results in a different way, suitable for specific needs or preferences. These built-in reporters offer different levels of detail and visualization options, helping developers to understand the test results and identify any potential issues more effectively. -> -Following are the Playwright built-in reporters: -- List Reporter -- Line Reporter -- Dot Reporter -- HTML Reporter -- JSON Reporter -- JUnit Reporter -- GitHub Actions annotations - -### Custom Reporters -Apart from the built-in reporters, Playwright Test also allows developers to create custom reporters i.e. you have the flexibility to define your own format and layout for displaying test results. Custom reporters are beneficial when you have specific reporting requirements or when you want to integrate the test results seamlessly into your existing development or CI/CD (Continuous Integration/Continuous Deployment) workflow. - -## HTML Report on HyperExecute -*** - -There are several built-in reporters in **Playwright Test** for various needs, along with an option to create Custom Reporters. - -HTML reporter produces a self-contained folder that contains a report for the test run that can be served as a web page. With Hyperexecute, we support the generation of a single HTML report for the complete job. - -By default, an HTML report is opened automatically if some of the tests failed. However, you can control this behavior via the `open` property in the **Playwright config file**. The possible values for that property are `always`, `never`, and `on-failure` (default). - -> In the case of Hyperexecute, we always have to set it to `never`. - -### Steps to Generate HTML Report on HyperExecute - -**Step 1:** Update your Playwright Configuration File by adding the below code - -#### playwright.config.js File - -```javascript +```javascript title="playwright.config.js" import { devices } from '@playwright/test'; -const config = { - timeout: 5 * 60 * 1000, // Maximum time one test can run for - testDir: './tests', - testMatch: /.*\.spec\.js/, +const config = { + ...// reporter: [['html', { open: 'never' }]], + ...// }; ``` +:::note +- In the case of HyperExecute, we always have to set the value of `open: never`. -:::tip Note - -By default, the report is written into the **playwright-report folder** in the current working directory. The same location can be used in the report parameters in the YAML file. - +- By default, the report is written into the **playwright-report folder** in the current working directory. The same location can be used in the report parameters in the YAML file. ::: -**Step 2:** Now configure the report parameters in the **HyperExecute YAML** file. +### Step 2: Configure the HyperExecute YAML File +In your HyperExecute YAML configuration, define the [`report`](https://www.lambdatest.com/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: -```bash +```yaml title="hyperexecute.yaml" report: true partialReports: frameworkName: playwright @@ -112,6 +75,7 @@ partialReports: type: html ``` -**Step 3:** Now execute your job by triggering the HyperExecute CLI. You can visit the HyperExecute dashboard to download the report after job completion. +### Step 3: Execute Your Tests +Run your Playwright tests on HyperExecute using the CLI. After your job completes, you can visit the HyperExecute dashboard to download and view the consolidated Plywright HTML report. -Image \ No newline at end of file +Image \ No newline at end of file diff --git a/docs/specflow-report.md b/docs/specflow-report.md index 60493f3a3..505d8e375 100644 --- a/docs/specflow-report.md +++ b/docs/specflow-report.md @@ -2,7 +2,7 @@ id: specflow-report title: SpecFlow Report on HyperExecute hide_title: true -sidebar_label: SpecFlow Report +sidebar_label: SpecFlow description: Learn how to generate SpecFlow Report on lambdatest and download the reports from the dashboard keywords: - specflow testing reports