You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/commands-and-arguments/arguments.md
+6-2Lines changed: 6 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -58,6 +58,11 @@ You can get the full list of arguments by running `cats -h`. Below is a short de
58
58
-`--ignoreResponseWords` COUNT_LIST a comma separated list of words count in the response that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR
59
59
-`--ignoreResponseLines` LINES_COUNT a comma separated list of lines count in the response that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR
60
60
-`--ignoreResponseRegex` a REGEX that will match against the response that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR
61
+
-`--filterResponseCodes` HTTP_CODES_LIST a comma separated list of HTTP response codes that will be filtered and not included in the final report. You can use response code families as `2xx`, `4xx`, etc.
62
+
-`--filterResponseSize` SIZE_LIST a comma separated list of response sizes that will be filtered and not included in the final report
63
+
-`--filterResponseWords` COUNT_LIST a comma separated list of words count in the response that will be filtered and not included in the final report
64
+
-`--filterResponseLines` LINES_COUNT a comma separated list of lines count in the response that will be filtered and not included in the final report
65
+
-`--filterResponseRegex` a REGEX that will match against the response that will be filtered and not included in the final report
61
66
-`--tests` TESTS_LIST a comma separated list of executed tests in JSON format from the cats-report folder. If you supply the list without the .json extension CATS will search the test in the cats-report folder
62
67
-`--ignoreResponseCodeUndocumentedCheck` If supplied (not value needed) it won't check if the response code received from the service matches the value expected by the fuzzer and will return the test result as SUCCESS instead of WARN
63
68
-`--ignoreResponseBodyCheck` If supplied (no value needed) it won't check if the response body received from the service matches the schema supplied inside the contract and will return the test result as SUCCESS instead of WARN
@@ -91,10 +96,9 @@ You can get the full list of arguments by running `cats -h`. Below is a short de
91
96
-`--fuzzersConfig=FILE` A properties file with Fuzzer configuration that changes default behaviour. Configuration keys are prefixed with the fully qualified Fuzzer name
92
97
-`--mutators=FOLDER` A folder containing custom mutators. Only applicable when using the `cats random` sub-command
93
98
-`--allowInvalidEnumValues` When set to true the `InvalidValuesInEnumsFieldsFuzzer` will expect a 2XX response code instead of 4XX
94
-
-`--[no-]filterXxxFromRequestPayloads` In extremely rare cases when CATS fails to generate anyOf/oneOf combinations some requests may still contain ONE_OF/ANY_OF markers. They are filtered out by default. Setting this to false will send them as requests which will probably fail. It's mostly for debug purposes
95
-
-`--generateXxxCombinationsForResponses` Generate anyOf/oneOf combinations also for response schemas. By default it creates one response payload with all possibilities
96
99
-`--selfReferenceDepth=<selfReferenceDepth>` Max depth for objects having cyclic dependencies
97
100
-`--limitXxxOfCombinations=<limitXxxOfCombinations>` Max number of anyOf/oneOf combinations
101
+
-`--limitFuzzedFields=<numberOfFields>` Max number of fields that will be fuzzed
98
102
-`--[no-]useDefaults` If set to true, it will use the default values when generating examples
99
103
-`--nameReplace` If set to true, it will simply do a replacement between the targetFields names provided and the fuzz values
100
104
-`--stopAfterErrors=<stopAfterErrors>` Number of errors after which the continuous fuzzing will stop running. Errors are defined as conditions matching the given match arguments. Only available in `cats random` sub-command.
Copy file name to clipboardExpand all lines: docs/getting-started/filtering-reports.md
+5Lines changed: 5 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,7 @@ description: How to ignore specific HTTP response
7
7
CATS reports can be huge. Opening a report with 20k tests it's not the best user experience.
8
8
Outside the typical [Slicing Strategies](slicing-strategies), you can also choose to ignore reporting for specific HTTP responses.
9
9
10
+
10
11
## Ignoring Specific HTTP Responses
11
12
By default, CATS will report `warns` and `errors` according to the specific behaviour of each Fuzzer. There are cases though when you might want to focus only on critical bugs.
12
13
You can use the `--ignoreResponseXXX` arguments to supply a list of response codes, response sizes, word counts,
@@ -36,3 +37,7 @@ You can ignore specific response codes using the `--ignoreResponseCodes` argumen
36
37
37
38
## Ignore Response Lines, Words, Size, Regex
38
39
You can ignore specific response lines, words, size or regex using the `--ignoreResponseLines`, `--ignoreResponseWords`, `--ignoreResponseSize`, `--ignoreResponseRegex` arguments. This will make CATS report these cases as `success` instead of `warn`.
40
+
41
+
:::info All --ignoreXXX arguments have a corresponding --filterXXX argument
42
+
All `--ignoreXXX` arguments above have a corresponding `--filterXXX` argument which is equivalent to the `--ignoreXXX` argument AND will also consider `--skipReportingForIgnore` as being true.
Copy file name to clipboardExpand all lines: docs/troubleshooting/index.md
+6-4Lines changed: 6 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ These are some of the common reasons why CATS might appear to be stuck:
6
6
7
7
-**Network issues**: CATS might be waiting for a response from the server. Check the network connection and the server status.
8
8
-**Server issues**: The server might be slow or unresponsive. Check the server logs and the server status.
9
-
-**Many fields inside the request(s)**: If the request has many fields, CATS might take longer to generate the tests, as many fuzzers are running for each field.
9
+
-**Many fields inside the request(s)**: If the request has many fields, CATS might take longer to generate the tests, as many fuzzers are running for each field. You can limit the number of fields to be fuzzed using `--limitFuzzedFields` argument.
10
10
-**Usage of anyOf/oneOf in request schemas**: If the request schema contains `anyOf` or `oneOf`, CATS will generate tests for all possible combinations. This can lead to a large number of tests being generated.
11
11
You can limit the number of combinations by using the `--limitXxxCombinations` argument. For example, `--limitOneOfCombinations=10` will limit the number of combinations for all `oneOf/anyOf` to 10.
12
12
-**Self-reference in the request schema**: If the request schema contains self-references, when CATS generates the request sample it might enter an infinite loop. You can limit the depth of the schema by using the `--selfReferenceDepth` argument. For example, `--selfReferenceDepth=3` will limit the self-reference depth of the schema to 3.
@@ -23,7 +23,7 @@ You can set examples for each property or provide specific formats in the schema
23
23
CATS generates tests based on the request schema and the fuzzers that are enabled. If the request schema is complex or has many fields, CATS might generate a large number of tests.
24
24
You can check the following pages for different strategies: [Slicing Strategies](https://endava.github.io/cats/docs/getting-started/slicing-strategies) and [Filtering Options](https://endava.github.io/cats/docs/getting-started/filtering-reports).
25
25
26
-
## CATS stops saying it cannot generate values for a field
26
+
## CATS stops, saying it cannot generate values for a field
27
27
28
28
CATS generates values based on the request schema using examples or request field names and/or formats. If the request schema is not well-defined, CATS might not be able to generate values for some fields.
29
29
There might be cases when the `pattern` for a specific property is too complicated and all internal generators are not able to create a value that it's matching that pattern.
@@ -35,7 +35,7 @@ This is usually when CATS generates a large number of tests and the report is to
35
35
36
36
## CATS generates same number of tests in 2 consecutive runs with same configuration
37
37
38
-
Unlike typical fuzzers that generate random values, CATS generates tests based on the request schema and the fuzzers that are enabled. If the request schema and the configuration are the same, CATS will generate the same tests.
38
+
Unlike typical fuzzers that rely only on randomness, CATS generates tests based on the request schema and the fuzzers that are enabled. If the request schema and the configuration are the same, CATS will generate the same tests.
39
39
CATS uses a deterministic approach to generate tests based on a set of fuzzers which run in a specific order and specific logic. Randomness is introduced only in the values generated for the fields.
40
40
If you want randomness also in the tests generated, you must use the `cats random` sub-command. Check [Continuous Fuzzing](https://endava.github.io/cats/docs/getting-started/running-cats#continuous-fuzzing-mode) for more details.
41
41
@@ -49,7 +49,7 @@ Assuming the OpenAPI file contains at least one valid path with at least one ope
49
49
## CATS generates tests that are not valid
50
50
51
51
If you think some of the test cases generated are not valid in your scenario you can:
52
-
-**Filter out the invalid test cases**: You can use the `--ignoreXXX` arguments to filter out the test cases that are not valid
52
+
-**Filter out the invalid test cases**: You can use the `--ignoreXXX`or `--filterXXX`arguments to filter out the test cases that are not valid
53
53
-**Filter out specific fuzzers**: You can use the `--skipFuzzer` argument to filter out specific fuzzers that are generating invalid test cases
54
54
-**Filter out specific paths**: You can use the `--skpiPath` argument to filter out specific paths that are generating invalid test cases
55
55
-**Filter out specific http methods**: You can use the `--skipHttpMethod` argument to filter out specific http methods that are generating invalid test cases
@@ -58,6 +58,8 @@ If you think some of the test cases generated are not valid in your scenario you
58
58
-**Filter out specific OpenAPI field types**: You can use the `--skipFieldType` argument to filter out specific OpenAPI field types that are generating invalid test cases
59
59
-**Filter out specific request headers**: You can use the `--skipHeader` argument to filter out specific request headers that are generating invalid test cases
60
60
61
+
If you think there might be a bug in CATS, please open an issue on the [GitHub repository](https://github.com/Endava/cats/issues/new/choose).
62
+
61
63
## CATS generates requests that are not valid
62
64
63
65
In very rare cases CATS might generate requests that are not valid.
0 commit comments