Skip to content

Commit 2a8c17a

Browse files
doboothdobooth
committed
Adding docs folder.
1 parent 7609658 commit 2a8c17a

40 files changed

+12859
-0
lines changed

‎docs/best-practices.md

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
---
2+
mftf-release: 2.3.0
3+
redirect_from: /guides/v2.3/magento-functional-testing-framework/2.3/best-practices.html
4+
---
5+
6+
# Best practices
7+
8+
_This topic was updated due to the {{ page.mftf-release }} MFTF release._
9+
{: style="text-align: right"}
10+
11+
Check out our best practices below to ensure you are getting the absolute most out of the Magento Functional Testing Framework.
12+
13+
## Action group
14+
15+
1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used.
16+
Add additional explanation in comments if needed.
17+
2. Provide default values for the arguments that apply to your most common case scenarios.
18+
19+
## `actionGroups` vs `extends`
20+
21+
Use an action group to wraps a set of actions to reuse them multiple times.
22+
23+
Use an [extension] when a test or action group needs to be repeated with the exception of a few steps.
24+
25+
### When to use `extends`
26+
27+
Use `extends` in your new test or action group when at least one of the following conditions is applicable to your case:
28+
29+
1. You want to keep the original test without any modifications.
30+
2. You want to create a new test that follows the same path as the original test.
31+
3. You want a new action group that behaves similarly to the existing action group, but you do not want to change the functionality of the original action group.
32+
33+
### When to avoid `extends`
34+
35+
Do not use `extends` in the following conditions:
36+
37+
1. You want to change the functionality of the test or action group and do not need to run the original version.
38+
2. You plan to merge the base test or action group.
39+
40+
The following pattern is used when merging with `extends`:
41+
42+
1. The original test is merged.
43+
2. The extended test is created from the merged original test.
44+
3. The extended test is merged.
45+
46+
## Annotation
47+
48+
1. Use [annotations] in a test.
49+
2. Update your annotations correspondingly when updating tests.
50+
51+
## Data entity
52+
53+
1. Keep your testing instance clean.
54+
Remove data after the test if the test required creating any data.
55+
Use a corresponding [`<deleteData>`] test step in your [`<after>`] block when using a [`<createData>`] action in a [`<before>`] block.
56+
2. Make specific data entries under test to be unique.
57+
Enable data uniqueness where data values are required to be unique in a database by test design.
58+
Use `unique=”suffix”` or `unique=”prefix”` to append or prepend a unique value to the [entity] attribute.
59+
This ensures that tests using the entity can be repeated.
60+
3. Do not modify existing data entity fields or merge additional data fields without complete understanding and verifying the usage of existing data in tests.
61+
Create a new data entity for your test if you are not sure.
62+
63+
## Naming conventions
64+
65+
### File names
66+
67+
Name files according to the following patterns to make searching in future more easy:
68+
69+
#### Test file name
70+
71+
Format: {_Admin_ or _Storefront_}{Functionality}_Test.xml_, where Functionality briefly describes the testing functionality.
72+
73+
Example: _StorefrontCreateCustomerTest.xml_.
74+
75+
#### Section file name
76+
77+
Format: {_Admin_ or _Storefront_}{UI Description}_Section.xml_, where UI Description briefly describes the testing UI.
78+
79+
Example: _AdminNavbarSection.xml_.
80+
81+
#### Data file name
82+
83+
Format: {Type}_Data.xml_, where Type represents the entity type.
84+
85+
Example: _ProductData.xml_.
86+
87+
### Object names
88+
89+
Use the _Foo.camelCase_ naming convention, which is similar to _Classes_ and _classProperties_ in PHP.
90+
91+
#### Upper case
92+
93+
Use an upper case first letter for:
94+
95+
- File names. Example: _StorefrontCreateCustomerTest.xml_
96+
- Test name attributes. Example: `<test name="TestAllTheThingsTest">`.
97+
- Data entity names. Example: `<entity name="OutOfStockProduct">`.
98+
- Page name. Example: `<page name="AdminLoginPage">`.
99+
- Section name. Example: `<section name="AdminCategorySidebarActionSection">`.
100+
- Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`.
101+
102+
#### Lower case
103+
104+
Use a lower case first letter for:
105+
106+
- Data keys. Example: `<data key="firstName">`.
107+
- Element names. Examples: `<element name="confirmDeleteButton"/>`.
108+
109+
## Page object
110+
111+
Use [parameterized selectors] for constructing a selector when test specific or runtime generated information is needed.
112+
Do not use them for static elements.
113+
114+
{:style="color:red"}
115+
BAD:
116+
117+
``` xml
118+
<element name="relatedProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='{{productType}}']" parameterized="true"/>
119+
```
120+
121+
{:style="color:green"}
122+
GOOD:
123+
124+
Define these three elements and reference them by name in the tests.
125+
126+
``` xml
127+
<element name="relatedProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='related']"/>
128+
<element name="upSellProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='upsell']"/>
129+
<element name="crossSellProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='crosssell']"/>
130+
```
131+
132+
## Test
133+
134+
1. Use actions such as [`<waitForElementVisible>`], [`<waitForLoadingMaskToDisappear>`], and [`<waitForElement>`] to wait the exact time required for the test step.
135+
Try to avoid using the [`<wait>`] action, because it forces the test to wait for the time you specify. You may not need to wait so long to proceed.
136+
1. Keep your tests short and granular for target testing, easier reviews, and easier merge conflict resolution.
137+
It also helps you to identify the cause of test failure.
138+
1. Use comments to keep tests readable and maintainable:
139+
- Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
140+
It helps to inform the reader of what you are testing and to yield a more descriptive Allure report.
141+
- Explain in comments unclear or tricky test steps.
142+
1. Refer to [sections] instead of writing selectors.
143+
144+
## Test step merging order
145+
146+
When setting a [merging] order for a test step, do not depend on steps from Magento modules that could be disabled by an application.
147+
148+
For example, when you write a test step to create a gift card product, set your test step **after** simple product creation and let the MFTF handle the merge order.
149+
Since the configurable product module could be disabled, this approach is more reliable than setting the test step **before** creating a configurable product.
150+
151+
<!-- Link definitions -->
152+
153+
[`<after>`]: test/actions.html#before-and-after
154+
[`<before>`]: test/actions.html#before-and-after
155+
[`<comment>`]: test/actions.html#comment
156+
[`<createData>`]: test/actions.html#createdata
157+
[`<deleteData>`]: test/actions.html#deletedata
158+
[`<wait>`]: test/actions.html#wait
159+
[`<waitForElement>`]: test/actions.html#waitforelement
160+
[`<waitForElementVisible>`]: test/actions.html#waitforelementvisible
161+
[`<waitForLoadingMaskToDisappear>`]: test/actions.html#waitforloadingmasktodisappear
162+
[Action group]: test/action-groups.html
163+
[annotations]: test/annotations.html
164+
[entity]: data.html
165+
[extension]: extending.html
166+
[merging]: merging.html
167+
[parameterized selectors]: section/parameterized-selectors.html
168+
[sections]: section.html

‎docs/commands/codeception.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
mftf-release: 2.3.0
3+
redirect_from: /guides/v2.3/magento-functional-testing-framework/2.3/commands/codeception.html
4+
---
5+
6+
# CLI commands: vendor/bin/codecept
7+
8+
_This topic was updated due to the {{page.mftf-release}} MFTF release._
9+
{: style="text-align: right"}
10+
11+
{:.bs-callout .bs-callout-warning}
12+
We do not recommend using Codeception commands directly as they can break the MFTF basic workflow.
13+
All the Codeception commands you need are wrapped using the [`mftf` tool][].
14+
15+
To run the Codeception testing framework commands directly, change your directory to the `<Magento root>`.
16+
17+
## Usage examples
18+
19+
Run all the generated tests:
20+
21+
```bash
22+
vendor/bin/codecept run functional
23+
```
24+
25+
Run all tests without the `<group value="skip"/>` [annotation][]:
26+
27+
```bash
28+
vendor/bin/codecept run functional --skip-group skip
29+
```
30+
31+
Run all tests with the `<group value="example"/>` [annotation][] but with no `<group value="skpip"/>`:
32+
33+
```bash
34+
vendor/bin/codecept run functional --group example --skip-group skip
35+
```
36+
37+
## `codecept run`
38+
39+
`codecept run` runs the test suites:
40+
41+
```bash
42+
vendor/bin/codecept run
43+
```
44+
45+
{: .bs-callout .bs-callout-info }
46+
The following documentation corresponds to Codeception 2.3.8.
47+
48+
```bash
49+
Full reference:
50+
51+
Arguments:
52+
suite suite to be tested
53+
test test to be run
54+
55+
Options:
56+
-o, --override=OVERRIDE Override config values (multiple values allowed)
57+
--config (-c) Use custom path for config
58+
--report Show output in compact style
59+
--html Generate html with results (default: "report.html")
60+
--xml Generate JUnit XML Log (default: "report.xml")
61+
--tap Generate Tap Log (default: "report.tap.log")
62+
--json Generate Json Log (default: "report.json")
63+
--colors Use colors in output
64+
--no-colors Force no colors in output (useful to override config file)
65+
--silent Only outputs suite names and final results
66+
--steps Show steps in output
67+
--debug (-d) Show debug and scenario output
68+
--coverage Run with code coverage (default: "coverage.serialized")
69+
--coverage-html Generate CodeCoverage HTML report in path (default: "coverage")
70+
--coverage-xml Generate CodeCoverage XML report in file (default: "coverage.xml")
71+
--coverage-text Generate CodeCoverage text report in file (default: "coverage.txt")
72+
--coverage-phpunit Generate CodeCoverage PHPUnit report in file (default: "coverage-phpunit")
73+
--no-exit Don't finish with exit code
74+
--group (-g) Groups of tests to be executed (multiple values allowed)
75+
--skip (-s) Skip selected suites (multiple values allowed)
76+
--skip-group (-x) Skip selected groups (multiple values allowed)
77+
--env Run tests in selected environments. (multiple values allowed, environments can be merged with ',')
78+
--fail-fast (-f) Stop after first failure
79+
--help (-h) Display this help message.
80+
--quiet (-q) Do not output any message.
81+
--verbose (-v|vv|vvv) Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
82+
--version (-V) Display this application version.
83+
--ansi Force ANSI output.
84+
--no-ansi Disable ANSI output.
85+
--no-interaction (-n) Do not ask any interactive question.
86+
```
87+
88+
<!-- Link definitions -->
89+
90+
[`mftf` tool]: mftf.html
91+
[annotation]: ../test/annotations.html

0 commit comments

Comments
 (0)