You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
.Tech-Talk given at https://www.opensourceindia.in/osi-speakers-2023/gopala-sarma-akshintala/[Open Source Conf], https://speakerdeck.com/gopalakshintala/revoman-a-template-driven-api-automation-tool-for-jvm[🎴Slide deck]
.Tech-Talk given at https://www.opensourceindia.in/osi-speakers-2023/gopala-sarma-akshintala/[Open Source Conf - 2023], https://speakerdeck.com/gopalakshintala/revoman-a-template-driven-api-automation-tool-for-jvm[🎴Slide deck]
* The majority of JVM SaaS applications are REST-based. But the API automation is done through a Mul-*T*-verse of Integration/Functional tests, E2E tests and Manual tests, each with its own frameworks, tools, and internal utilities, testing almost the same code flow.
79
-
* These custom alien automation frameworks, often built using low-level tools like https://rest-assured.io/[**REST Assured**] which are specific to a service or domain and are rigid to reuse, extend and difficult to maintain.
80
+
* These custom alien automation frameworks, often built using low-level tools like https://rest-assured.io/[**REST Assured**], are specific to a service or domain and are rigid to reuse, extend and difficult to maintain.
80
81
* This automation competes on cognitive complexity and learning curve with the Prod code, and mostly, automation wins.
81
82
* After a point, the API automation may deviate from its purpose of augmenting real end-user interaction and turns into a foot-chain for development.
82
83
83
-
image::cognitive-complexity.png[]
84
+
image::cognitive-complexity.png[align="center"]
84
85
85
86
=== The Solution
86
87
@@ -93,24 +94,25 @@ Leveraging it can mitigate writing a lot of code as we translate those manual st
93
94
94
95
____
95
96
96
-
* How _productive_ would it be, if you can plug your exported Postman collection,
97
+
* How _productive_ would it be, if you can plug your exported Postman collection template,
97
98
that you anyway would have created for your manual testing, and execute them through your JVM tests?
98
99
99
-
* How about a Universal API automation tool promotes low code and low-cognitive-complexity and strikes a balance between flexibility and ease of use?
100
+
* How about a Universal API automation tool that promotes low code and low-cognitive-complexity and strikes a balance between flexibility and ease of use?
100
101
101
102
____
102
103
103
104
== API automation with _ReṼoman_
104
105
105
106
=== Template-Driven Testing
106
107
107
-
* The exported Postman collection JSON file is referred to as Postman templates, as it contains some placeholders/variables in the `+{{variable-key}}+` pattern. You can read more about it https://learning.postman.com/docs/sending-requests/variables/[here]
108
+
* The exported Postman collection JSON file is referred to as a Postman template, as it contains some placeholders/variables in the `+{{variable-key}}+` pattern. You can read more about it https://learning.postman.com/docs/sending-requests/variables/[here]
108
109
* ReṼoman understands these templates and replaces these variables at the runtime, similar to Postman. It supports
<13> Ignore Java cert issues when firing HTTP calls
320
-
<14> More assertions on top of <<#_rundown>>
322
+
<14> Run more assertions on the <<Rundown>>
321
323
endif::[]
322
324
323
325
== Debugging UX
324
326
325
327
[.lead]
326
-
This tool has particular emphasis on Debugging experience.
327
-
328
-
The rundown you get as a return from `revUp` after the execution completes has everything you need to know about what happened during each step execution,
329
-
along with environment snapshot during that step execution. Here is what a debugger view of a Rundown looks like:
328
+
This tool has particular emphasis on Debugging experience. Here is what a debugger view of a <<Rundown>> looks like:
330
329
331
330
image:rundown.png[Rundown of all steps]
332
331
@@ -377,7 +376,8 @@ link:{sourcedir}/com/salesforce/revoman/input/json/JsonPojoUtils.kt[JSON POJO Ut
@@ -389,7 +389,7 @@ The configuration offers methods through which the execution strategy can be con
389
389
* `haltOnAnyFailure` — This defaults to `false`. If set to `true`, the execution fails-fast when it encounters a failure.
390
390
* `haltOnFailureOfTypeExcept` — This accepts pairs of `ExeType` and a `PostTxnStepPick` which are used to check if a Step can be ignored for failure for a specific failure type
391
391
* `runOnlySteps`, `skipSteps` — All these accept a `predicate` of type `ExeStepPick`, which is invoked passing the current `Step` instance to decide whether to execute or skip a step.
392
-
** There are some `ExeStepPick` predicates bundled with ReṼoman under `ExeStepPick.PickUtils` e.g `withName`, `inFolder` etc. You can write a custom predicate of you own too.
392
+
** There are some `ExeStepPick` predicates bundled with ReṼoman under `ExeStepPick.PickUtils` e.g `withName`, `inFolder` etc. You can write a custom predicate of your own too.
393
393
394
394
[#_pre_and_post_hooks]
395
395
=== Pre- and Post-Hooks
@@ -428,9 +428,8 @@ which can be populated and picked-up by subsequent steps.
428
428
For example, you may want some `xyzId` but you don't have a Postman collection to create it.
429
429
But instead, you have a Java utility to generate/create it.
430
430
You can invoke the utility in a pre-hook of a step and set the value in `rundown.mutableEnv`,
431
-
so the subsequent steps can pick up value for `{+{xyzId}+}` variable from the environment.
431
+
so the later steps can pick up value for `+{{xyzId}}+` variable from the environment.
432
432
433
-
[#_response_validations]
434
433
==== Response Validations
435
434
436
435
* Post-Hooks are the best place to validate response right after the step.
@@ -439,13 +438,11 @@ so the subsequent steps can pick up value for `{+{xyzId}+}` variable from the en
439
438
440
439
=== Pre-req and Tests scripts
441
440
442
-
[.lead]
443
-
Pre-req and Tests scripts support makes sure that the Postman collection used for manual testing can be used *as-is* for the automation also, without any resistance to modify or overhead of maintaining separate versions for manual and automation.
444
-
445
-
* This tool can execute JavaScript written under the https://learning.postman.com/docs/writing-scripts/script-references/test-examples/[Pre-req and Tests tabs of Postman].
441
+
* Postman lets you write custom javascript in https://learning.postman.com/docs/writing-scripts/script-references/test-examples/[Pre-req and Tests tabs] that get executed before and after a step respectively. When you export the collection as a template, these scripts also come bundled.
442
+
* ReṼoman can execute this javascript on JVM. This support ensures that the Postman collection used for manual testing can be used *as-is* for the automation also, without any resistance to modify or overhead of maintaining separate versions for manual and automation.
446
443
** Pre-req JS is executed as the first step before Unmarshall request.
447
444
** Tests JS is executed right after receiving an HTTP response.
448
-
* ReṼoman supports using of any `npm` modules inside your Pre-req and Tests javascript. You can install `npm` modules in any folder and supply in the Kick config, the relative path of the parent folder that contains the `node_modules` folder using `nodeModulesRelativePath(...)`. Use those `npm` modules inside your scripts with `require(...)`, for example:
445
+
* ReṼoman supports using `npm` modules inside your Pre-req and Tests javascript. You can install `npm` modules in any folder and supply in the `Kick` config, the relative path of the parent folder that contains the `node_modules` folder using `nodeModulesRelativePath(...)`. Use those `npm` modules inside your scripts with `require(...)`, for example:
pm.environment.set("$quantity", _.random(1, 10)); // lodash doesn't need require
454
+
pm.environment.set("$quantity", _.random(1, 10)); // lodash doesn't need `require`
458
455
----
459
456
460
457
[TIP]
461
458
====
462
-
* `node_modules` adds a lot of files to check in. You may replace them with a single distribution file
459
+
* `node_modules` Adds a lot of files to check in. You may replace them with a single distribution file
463
460
464
461
image::node_modules.png[]
465
462
@@ -479,11 +476,12 @@ Each StepReport also has a `pmEnvSnapshot` to assert if a step has executed as e
479
476
480
477
=== Compose Modular Executions
481
478
482
-
* You don't have to squash all your steps into one mega collection. Instead, you can break them into easy-to-manage modular collections. `revUp` accepts a list of collection paths through `templatePaths`
483
-
* But that doesn't mean you have to execute all these templates in one-go.You can make multiple `revUp` calls.
479
+
* You don't have to squash all your steps into one mega collection. Instead, you can break them into easy-to-manage modular collections. `ReVoman.revUp()` accepts a list of collection paths through `templatePaths()`
480
+
* But that doesn't mean you have to execute all these templates in one-go.You can make multiple `ReVoman.revUp()` calls for different collection.
484
481
* If you wish to compose these executions, you can do so by adding the previous execution's `mutableEnv` to current execution using `dynamicEnvironment` parameter. This also comes in handy when you wish to execute a common step (e.g. `UserSetup`) inside a test setup method and use that environment for all the tests.
485
482
486
483
==== Custom Dynamic variables
484
+
487
485
If the in-built dynamic variables don't fit your need, you can plug your own dynamic variable generator
488
486
to be invoked to generate a value for your custom variable-key in the template at runtime.
489
487
@@ -517,10 +515,10 @@ The above test doesn't just have low code, but also low in Cognitive Complexity
517
515
518
516
== Perf
519
517
520
-
This entire execution of **~70 steps**, which includes **10 async steps**, took a mere *122 sec* on localhost.
518
+
This entire execution of **~75 steps**, which includes **10 async steps**, took a mere *122 sec* on localhost.
521
519
This can be much better on auto-build environments.
522
520
523
-
image:pq-revoman-test-time.png[Localhost Test time on FTest console]
521
+
image:pq-revoman-test-time.png[Localhost Test time on FTest console for ~75 steps]
524
522
525
523
WARNING: ReṼoman internally is very light-weight, and the execution times are proportional
526
524
to how your server responds or your network speed.
0 commit comments