You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When end-to-end testing Grafana plugins, it's recommended to use the [`@grafana/plugin-e2e`](https://www.npmjs.com/package/@grafana/plugin-e2e?activeTab=readme) testing tool. `@grafana/plugin-e2e` extends [`@playwright/test`](https://playwright.dev/) capabilities with relevant fixtures, models, and expect matchers; enabling comprehensive end-to-end testing of Grafana plugins across multiple versions of Grafana. For information on how to get started with Plugin end-to-end testing and Playwright, checkout the [Get started](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/get-started) guide.
3
+
When end-to-end testing Grafana plugins, a best practice is to use the [`@grafana/plugin-e2e`](https://www.npmjs.com/package/@grafana/plugin-e2e?activeTab=readme) testing tool. The `@grafana/plugin-e2e`tool extends [`@playwright/test`](https://playwright.dev/) capabilities with relevant fixtures, models, and expect matchers. Use it to enable comprehensive end-to-end testing of Grafana plugins across multiple versions of Grafana.
4
4
5
-
## Adding end-to-end tests for a core plugin
5
+
> **Note:** To learn more, refer to our documentation on [plugin development](https://grafana.com/developers/plugin-tools/) and [end-to-end plugin testing](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/get-started).
6
6
7
-
Playwright end-to-end tests for plugins should be added to the [`e2e/plugin-e2e`](https://github.com/grafana/grafana/tree/main/e2e/plugin-e2e) directory.
7
+
## Add end-to-end tests for a core plugin
8
8
9
-
1. Add a new directory that has the name as your plugin [`here`](https://github.com/grafana/grafana/tree/main/e2e/plugin-e2e). This is where your plugin tests will be kept.
9
+
You can add Playwright end-to-end tests for plugins to the [`e2e/plugin-e2e`](https://github.com/grafana/grafana/tree/main/e2e/plugin-e2e) directory.
10
10
11
-
2. Playwright uses [projects](https://playwright.dev/docs/test-projects) to logically group tests together. All tests in a project share the same configuration.
12
-
In the [Playwright config file](https://github.com/grafana/grafana/blob/main/playwright.config.ts), add a new project item. Make sure the `name` and the `testDir` sub directory matches the name of the directory that contains your plugin tests.
13
-
Adding `'authenticate'` to the list of dependencies and specifying `'playwright/.auth/admin.json'` as storage state will ensure all tests in your project will start already authenticated as an admin user. If you wish to use a different role for and perhaps test RBAC for some of your tests, please refer to the plugin-e2e [documentation](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/use-authentication).
11
+
1. Add a new directory that has the name as your plugin [`here`](https://github.com/grafana/grafana/tree/main/e2e/plugin-e2e). This is the directory where your plugin tests will be kept.
12
+
13
+
1. Playwright uses [projects](https://playwright.dev/docs/test-projects) to logically group tests together. All tests in a project share the same configuration.
14
+
In the [Playwright config file](https://github.com/grafana/grafana/blob/main/playwright.config.ts), add a new project item. Make sure the `name` and the `testDir` subdirectory match the name of the directory that contains your plugin tests.
15
+
Add `'authenticate'` to the list of dependencies and specify `'playwright/.auth/admin.json'` as the storage state to ensure that all tests in your project will start already authenticated as an admin user. If you want to use a different role for and perhaps test RBAC for some of your tests, refer to our [documentation](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/use-authentication).
14
16
15
17
```ts
16
18
{
@@ -24,14 +26,16 @@ Playwright end-to-end tests for plugins should be added to the [`e2e/plugin-e2e`
24
26
},
25
27
```
26
28
27
-
3. Update the [CODEOWNERS](https://github.com/grafana/grafana/blob/main/.github/CODEOWNERS/#L315) file so that your team is owner of the tests in the directory you added in step 1.
29
+
1. Update the [CODEOWNERS](https://github.com/grafana/grafana/blob/main/.github/CODEOWNERS/#L315) file so that your team is owner of the tests in the directory you added in step 1.
28
30
29
31
## Commands
30
32
31
-
-`yarn e2e:playwright` will run all Playwright tests. Optionally, you can provide the `--project mysql` argument to run tests in a certain project.
33
+
-`yarn e2e:playwright` runs all Playwright tests. Optionally, you can provide the `--project mysql` argument to run tests in a specific project.
34
+
35
+
The `yarn e2e:playwright` script assumes you have Grafana running on `localhost:3000`. You may change this with an environment variable:
32
36
33
-
The script above assumes you have Grafana running on `localhost:3000`. You may change this by providing environment variables.
37
+
`HOST=127.0.0.1 PORT=3001 yarn e2e:playwright`
34
38
35
-
`HOST=127.0.0.1 PORT=3001 yarn e2e:playwright`
39
+
The `yarn e2e:playwright:server` starts a Grafana [development server](https://github.com/grafana/grafana/blob/main/scripts/grafana-server/start-server) on port 3001 and runs the Playwright tests.
36
40
37
-
-`yarn e2e:playwright:server` will start a Grafana [development server](https://github.com/grafana/grafana/blob/main/scripts/grafana-server/start-server) on port 3001 and run the Playwright tests. The development server is provisioned with the [devenv](https://github.com/grafana/grafana/blob/main/contribute/developer-guide.md#add-data-sources) dashboards, data sources and apps.
41
+
-You can provision the development server with the [devenv](https://github.com/grafana/grafana/blob/main/contribute/developer-guide.md#add-data-sources) dashboards, data sources, and apps.
Copy file name to clipboardExpand all lines: contribute/style-guides/e2e.md
+36-35Lines changed: 36 additions & 35 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,22 +1,22 @@
1
-
# End-to-End tests
1
+
# End-to-end tests
2
2
3
3
Grafana Labs uses a minimal [homegrown solution](../../e2e/utils/index.ts) built on top of [Cypress](https://cypress.io) for its end-to-end (E2E) tests.
4
4
5
5
Important notes:
6
6
7
7
- We generally store all element identifiers ([CSS selectors](https://mdn.io/docs/Web/CSS/CSS_Selectors)) within the framework for reuse and maintainability.
8
8
- We generally do not use stubs or mocks as to fully simulate a real user.
9
-
- Cypress' promises [do not behave as you'd expect](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Mixing-Async-and-Sync-code).
10
-
-[Testing core Grafana](e2e-core.md) is different than [testing plugins](e2e-plugins.md) - core Grafana uses Cypress whereas plugins use [Playwright test](https://playwright.dev/).
9
+
- Cypress' promises [don't behave as you might expect](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Mixing-Async-and-Sync-code).
10
+
-[Testing core Grafana](e2e-core.md) is different than [testing plugins](e2e-plugins.md)—core Grafana uses Cypress whereas plugins use [Playwright test](https://playwright.dev/).
11
11
12
12
## Framework structure
13
13
14
-
Inspired by https://martinfowler.com/bliki/PageObject.html
14
+
Our framework structure is inspired by [Martin Fowler's Page Object](https://martinfowler.com/bliki/PageObject.html).
15
15
16
-
-`Selector`: A unique identifier that is used from the E2E framework to retrieve an element from the Browser
17
-
-`Page`: An abstraction for an object that contains one or more `Selectors`with `visit` function to navigate to the page.
18
-
-`Component`: An abstraction for an object that contains one or more `Selectors`but without `visit` function
19
-
-`Flow`: An abstraction that contains a sequence of actions on one or more `Pages` that can be reused and shared between tests
16
+
-**`Selector`**: A unique identifier that is used from the E2E framework to retrieve an element from the browser
17
+
-**`Page`**: An abstraction for an object that contains one or more `Selector` identifiers with the `visit` function to go to the page.
18
+
-**`Component`**: An abstraction for an object that contains one or more `Selector` identifiers but without the`visit` function
19
+
-**`Flow`**: An abstraction that contains a sequence of actions on one or more `Page` abstractions that can be reused and shared between tests
20
20
21
21
## Basic example
22
22
@@ -26,13 +26,15 @@ Let's start with a simple [JSX](https://reactjs.org/docs/introducing-jsx.html) e
We _could_ target the field with a CSS selector like `.gf-form-input.login-form-input` but that would be brittle as style changes occur frequently. Furthermore there is nothing that signals to future developers that this input is part of an E2E test. At Grafana, we use `data-testid` attributes as our preferred way of defining selectors. See [Aria-Labels vs data-testid](#aria-labels-vs-data-testid) for more details.
29
+
It is possible to target the field with a CSS selector like `.gf-form-input.login-form-input`. However, doing so is a brittle solution because style changes occur frequently.
30
+
31
+
Furthermore, there is nothing that signals to future developers that this input is part of an E2E test. At Grafana, we use `data-testid` attributes as our preferred way of defining selectors. See [Aria-Labels vs data-testid](#aria-labels-vs-data-testid) for more details.
The next step is to create a `Page` representation in our E2E framework to glue the test with the real implementation using the `pageFactory` function. For that function we can supply a `url` and `selectors` like in the example below:
37
+
The next step is to create a `Page` representation in our E2E framework. Doing so glues the test with the real implementation using the `pageFactory` function. For that function we can supply a `url` and selector like in the following example:
36
38
37
39
```typescript
38
40
exportconst Login = {
@@ -43,9 +45,9 @@ export const Login = {
43
45
};
44
46
```
45
47
46
-
Note that the selector is prefixed with `data-testid` - this is a signal to the framework to look for the selector in the `data-testid` attribute.
48
+
In this example, the selector is prefixed with `data-testid`. The prefix is a signal to the framework to look for the selector in the `data-testid` attribute.
47
49
48
-
The next step is to add the `Login` page to the `Pages` export within [_\<repo-root>/packages/grafana-e2e-selectors/src/selectors/pages.ts_](../../packages/grafana-e2e-selectors/src/selectors/pages.ts) so that it appears when we type `e2e.pages` in our IDE.
50
+
The next step is to add the `Login` page to the `Pages` export within [_\<repo-root>/packages/grafana-e2e-selectors/src/selectors/pages.ts_](../../packages/grafana-e2e-selectors/src/selectors/pages.ts) so that it appears when we type `e2e.pages` in your IDE.
49
51
50
52
```typescript
51
53
exportconst Pages = {
@@ -56,7 +58,9 @@ export const Pages = {
56
58
};
57
59
```
58
60
59
-
Now that we have a `Page` called `Login` in our `Pages` const we can use that to add a selector in our html like shown below and now this really signals to future developers that it is part of an E2E test.
61
+
Now that we have a page called `Login` in our `Pages` const, use it to add a selector in our HTML as shown in the following example. This page really signals to future developers that it is part of an E2E test.
@@ -66,9 +70,8 @@ import { selectors } from '@grafana/e2e-selectors';
66
70
67
71
The last step in our example is to use our `Login` page as part of a test.
68
72
69
-
- The `url` property is used whenever we call the `visit` function and is equivalent to the Cypress' [`cy.visit()`](https://docs.cypress.io/api/commands/visit.html#Syntax).
70
-
71
-
- Any defined selector can be accessed from the `Login` page by invoking it. This is equivalent to the result of the Cypress function [`cy.get(…)`](https://docs.cypress.io/api/commands/get.html#Syntax).
73
+
- Use the `url` property whenever you call the `visit` function. It is equivalent to the [`cy.visit()`](https://docs.cypress.io/api/commands/visit.html#Syntax) in Cypress.
74
+
- Access any defined selector from the `Login` page by invoking it. This is equivalent to the result of the Cypress function [`cy.get(…)`](https://docs.cypress.io/api/commands/get.html#Syntax).
72
75
73
76
```typescript
74
77
describe('Login test', () => {
@@ -83,7 +86,7 @@ describe('Login test', () => {
83
86
84
87
## Advanced example
85
88
86
-
Let's take a look at an example that uses the same `selector` for multiple items in a list for instance. In this example app we have a list of data sources that we want to click on during an E2E test.
89
+
Let's take a look at an example that uses the same selector for multiple items in a list for instance. In this example app, there's a list of data sources that we want to click on during an E2E test.
87
90
88
91
```jsx
89
92
<ul>
@@ -97,7 +100,7 @@ Let's take a look at an example that uses the same `selector` for multiple items
97
100
</ul>
98
101
```
99
102
100
-
Just as before in the basic example we'll start by creating a page abstraction using the `pageFactory` function:
103
+
Like in the basic example, start by creating a page abstraction using the `pageFactory` function:
You might have noticed that instead of a simple `string` as the `selector`, we're using a `function` that takes a string parameter as an argument and returns a formatted string using the argument.
112
+
You might have noticed that instead of a simple string as the selector, there's a function that takes a string parameter as an argument and returns a formatted string using the argument.
110
113
111
-
Just as before we need to add the `DataSources` page to the exported const `Pages` in `packages/grafana-e2e-selectors/src/selectors/pages.ts`.
114
+
Just as before, you need to add the `DataSources` page to the exported const `Pages` in `packages/grafana-e2e-selectors/src/selectors/pages.ts`.
112
115
113
-
The next step is to use the `dataSources` selector function as in our example below:
116
+
The next step is to use the `dataSources` selector function as in the following example:
114
117
115
118
```jsx
116
119
<ul>
@@ -126,15 +129,15 @@ The next step is to use the `dataSources` selector function as in our example be
126
129
</ul>
127
130
```
128
131
129
-
When this list is rendered with the data sources with names `A`, `B` and `C` ,the resulting HTML would look like:
132
+
When this list is rendered with the data sources with names `A`, `B` and `C` ,the resulting HTML looks like this:
130
133
131
134
```html
132
135
<divclass="card-item-name"data-testid="data-testid Data source list item A">A</div>
133
136
<divclass="card-item-name"data-testid="data-testid Data source list item B">B</div>
134
137
<divclass="card-item-name"data-testid="data-testid Data source list item C">C</div>
135
138
```
136
139
137
-
Now we can write our test. The one thing that differs from the [basic example](#basic-example)above is that we pass in which data source we want to click on as an argument to the selector function:
140
+
Now we can write our test. The one thing that differs from the previous [basic example](#basic-example) is that we pass in which data source we want to click as an argument to the selector function:
Our selectors are set up to work with both aria-labels and data-testid attributes. Aria-labels help assistive technologies such as screenreaders identify interactive elements of a page for our users.
155
+
Our selectors are set up to work with both `aria-label` attributes and `data-testid` attributes. The `aria-label` attributes help assistive technologies such as screen readers identify interactive elements of a page for our users.
153
156
154
-
A good example of a time to use an aria-label might be if you have a button with an X to close:
157
+
A good example of a time to use an aria-label might be if you have a button with an **X** to close:
155
158
156
159
```
157
160
<button aria-label="close">X<button>
158
161
```
159
162
160
-
It might be clear visually that the X closes the modal, but audibly it would not be clear for example.
163
+
It might be clear visually that the **X** closes the modal, but audibly it would not be clear, for example.
161
164
162
165
```
163
166
<button aria-label="close">Close<button>
164
167
```
165
168
166
169
The example might read aloud to a user as "Close, Close" or something similar.
167
170
168
-
However adding aria-labels to elements that are already clearly labeled or not interactive can be confusing and redundant for users with assistive technologies.
171
+
However, adding an aria-label to an element that is already clearly labeled or not interactive can be confusing and redundant for users with assistive technologies.
169
172
170
-
In such cases rather than adding unnecessary aria-labels to components so as to make them selectable for testing, it is preferable to use a data attribute that would not be read aloud with an assistive technology for example:
173
+
In such cases, don't add an unnecessary aria-label to a component so as to make them selectable for testing. Instead, use a data attribute that will not be read aloud with an assistive technology. For example:
0 commit comments