Merge branch 'Azure:main' into main

Author: leniatgh

.github/pull_request_assignment.yml

@@ -114,6 +114,12 @@
     reviewers:
       - ArcturusZhang
 
+- rule:
+    paths:
+      - "specification/monitor/**"
+    reviewers:
+      - ArcturusZhang
+
 - rule:
     paths:
       - "specification/authorization/**"

.gitignore

@@ -110,7 +110,10 @@ warnings.txt
 !.vscode/launch.json
 !.vscode/extensions.json
 
+# API Test outputs
+.apitest
+
 *.js
 *.d.ts
 *.js.map
-*.d.ts.map
+*.d.ts.map

README.md

@@ -176,7 +176,7 @@ Specification files and AutoRest configuration files in one RP folder are better
 ## Next steps
 The next step in the process after a spec is completed is to generate SDKs and API reference documentation. If you're Microsoft employee, go to the [Azure SDK - Internal Wiki](https://aka.ms/jointhesdk) for more information.
 
-External Contributors can read [Getting Started with Azure API Specifications](https://github.com/Azure/azure-rest-api-specs/blob/master/documentation/getting-started-with-azure-api-specifications.md).
+External Contributors can read [Getting Started with OpenAPI Specifications](https://github.com/Azure/azure-rest-api-specs/blob/main/documentation/Getting%20started%20with%20OpenAPI%20specifications.md).
 
 ---
 _This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments._

documentation/api-scenario/how-to/QuickStart.md

@@ -5,24 +5,27 @@
  https://opensource.org/licenses/MIT
 -->
 
-# API test quick start
+# Quick start with API Scenario test
 
 ## Install
 
-`oav` is an open-source powerful tool for swagger validation, example generation, and API testing. GitHub: https://github.com/Azure/oav.
+`oav` is an open-source powerful tool for swagger validation, API Scenario testing, and examples generation. GitHub: https://github.com/Azure/oav.
 
 ```sh
-npm install -g oav@latest
+npm install -g oav
 ```
 
 ### OAV Features
 
-- Very easy to use and run.
-- Support postman collection format. Debug easily.
-- Request response validation. `oav` implement a powerful validation algorithm and help developer to detect service issue in the early phase.
-- Validation result report. After each run API scenario, developer will get a validation report which contains detect issue in api test.
+- Very easy to use and run. It supports running API Scenario with ARM Dogfood/Canary/Production environments, and local environment as well.
+- Support Postman collection format. Debug easily.
+- Traffic schema validation and Azure API guidelines validation. `oav` implements a powerful validation algorithm and help developer to detect service issue in the early phase.
+- Generating high quality Swagger examples from API test traffic.
+- Validation result report. After each run API scenario, developer will get a validation report which contains detected issue in API test.
 - Integrate everywhere. Easily integrate with azure-pipeline, cloud-test.
 
+See `oav run -h` to find all available options.
+
 ## Create AAD app
 
 To run API test, first please prepare an AAD app which is used for provisioning Azure resource. Please grant subscription contributor permission to this AAD app.
@@ -31,7 +34,7 @@ For how to create AAD app, please follow this doc https://docs.microsoft.com/en-
 
 ## Authoring steps
 
-We will write API scenario file for SignalR service as an example.
+We will write API scenario file for AppConfiguration service as an example.
 
 #### 1. Write your first API scenario file
 
@@ -44,68 +47,105 @@ Now write your basic API scenario. For more detail about API scenario file forma
 
 ```yaml
 # yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/documentation/api-scenario/references/v1.2/schema.json
+scope: ResourceGroup
+variables:
+  configStoreName:
+    type: string
+    prefix: configstor
 
+scenarios:
+  - scenario: quickStart
+    description: Quick start with AppConfiguration ConfigurationStores
+    steps:
+      - step: Operations_CheckNameAvailability
+        exampleFile: ../examples/CheckNameAvailable.json
+      - step: ConfigurationStores_Create
+        exampleFile: ../examples/ConfigurationStoresCreate.json
+      - step: ConfigurationStores_Get
+        exampleFile: ../examples/ConfigurationStoresGet.json
+```
+
+or use operation based step if Swagger examples are not ready or you want to create more scenarios without writing Swagger examples.
+
+```yaml
+# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/documentation/api-scenario/references/v1.2/schema.json
 scope: ResourceGroup
+variables:
+  configStoreName:
+    type: string
+    prefix: configstor
+
 scenarios:
   - scenario: quickStart
-    description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
+    description: Quick start with AppConfiguration ConfigurationStores
     steps:
-      - step: SignalR_CreateOrUpdate
-        exampleFile: ../examples/SignalR_CreateOrUpdate.json
-      - step: SignalR_Delete
-        exampleFile: ../examples/SignalR_Delete.json
+      - step: Operations_CheckNameAvailability
+        operationId: Operations_CheckNameAvailability
+        parameters:
+          checkNameAvailabilityParameters:
+            name: $(configStoreName)
+            type: Microsoft.AppConfiguration/configurationStores
+      - step: ConfigurationStores_Create
+        operationId: ConfigurationStores_Create
+        parameters:
+          configStoreCreationParameters:
+            location: $(location)
+            sku:
+              name: Standard
+            tags:
+              myTag: myTagValue
+      - step: ConfigurationStores_Get
+        operationId: ConfigurationStores_Get
 ```
 
-#### 2. create your env file
+#### 2. Create env file
 
 The `env.json` file contains required API scenario variables such as, subscriptionId, AAD applicationId, AAD applicationSecret.
 
 ```json
 {
   "subscriptionId": "<my subscription id>",
-  "location": "westus",
+  "location": "westcentralus",
   "tenantId": "<AAD app tenantId>",
   "client_id": "<my add client_id>",
   "client_secret": "<my aad client_secret>"
 }
 ```
 
-#### 3. Run api test
+#### 3. Run API Scenario test
 
 ```sh
-oav run /home/user/azure-rest-api-specs/specification/signalr/resource-manager/Microsoft.SignalRService/preview/2020-07-01-preview/scenarios/signalR.yaml -e env.json
+oav run ~/workspace/azure-rest-api-specs/specification/appconfiguration/resource-manager/Microsoft.AppConfiguration/stable/2022-05-01/scenarios/quickstart.yaml --tag=package-2022-05-01 -e env.json --verbose
 ```
 
-#### 4. Debug with postman
+#### 4. Debug with Postman
 
-Sometimes the command `oav run` may fail due to non 2xx HTTP status code. Now you need to debug the API scenario with postman.
+Sometimes the command `oav run` may fail due to non 2xx HTTP status code. Now you need to debug the API scenario with Postman.
 
-When run `run`, it automatically generate postman collection and postman env in `generated/<providerNamespace>/<apiScenarioFile>/<runId>/<apiScenario>` folder. Here is the generated file folder structure. The `collection.json` and `env.json` is generated postman collection file and environment file. `202105120922-5c3x5` is current runId. For each run command it will generated unique runId.
+When run `run`, it automatically generate Postman collection and postman env in `.apitest/<apiScenarioFile>/<runId>/<scenario>` folder. Here is the output folder structure. The `collection.json` and `env.json` is generated postman collection file and environment file. `202207221820-cyq4mk` is the runId, generated uniquely for each run command.
 
 ```
-generated
-└── Microsoft.SignalRService
-    └── 2020-07-01-preview
-        └── signalR
-            └── 202105120922-5c3x5
-                ├── signalR_0
-                │   ├── collection.json
-                │   └── env.json
-                |   |__ report.json
-                └── signalR_0.json
+.apitest
+└── quickstart.yaml
+    └── 202207221820-cyq4mk
+        ├── quickStart
+        │   ├── collection.json
+        │   ├── env.json
+        │   └── report.json
+        └── quickStart.json
 ```
 
 Postman is a widely used GUI API testing tool. And you could use Postman import the generated postman collection and env for your local debug.
 
 ![import-postman-collection](./import-postman-collection.png)
 
-After you import postman collection, you will get such requests. Now you could debug API test with postman locally.
+After you import Postman collection, you will get such requests. Now you could debug API test with postman locally.
 
-![postman-collection-signalr](./postman-collection-signalr.PNG)
+![postman-collection-list](./postman-collection-list.PNG)
 
-#### 5. manual update example value
+#### 5. Manual update API Scenario or example
 
-After debug with postman, you need to rewrite back all the updated values and run `oav run <api-scenario-file> -e <env.json>` again. The result should be successful.
+After debug with Postman, you need to rewrite back all the updated values and run `oav run <api-scenario-file> -e <env.json>` again. The result should be successful.
 
 ## Feedback