MQE-1671: document using credentials with vault in MFTF tests

Author: jilu1

docs/configuration.md

@@ -251,6 +251,32 @@ Example:
 BROWSER=firefox
 ```
 
+###CREDENTIAL_VAULT_ADDRESS
+
+Api address for vault server.
+
+Default: http://127.0.0.1:8200
+
+Example:
+
+```conf
+# Default api address for local vault dev server
+CREDENTIAL_VAULT_ADDRESS=http://127.0.0.1:8200
+```
+
+###CREDENTIAL_VAULT_SECRET_BASE_PATH
+
+Vault secret engine base path.
+
+Default: secret
+
+Example:
+
+```conf
+# Default base path for kv secret engine in local vault dev server
+CREDENTIAL_VAULT_SECRET_BASE_PATH=secret
+```
+
 <!-- Link definitions -->
 
 [`MAGENTO_CLI_COMMAND_PATH`]: #magento_cli_command_path

docs/credentials.md

@@ -1,8 +1,11 @@
 # Credentials
 
-When you test functionality that involves external services such as UPS, FedEx, PayPal, or SignifyD, use the MFTF credentials feature to hide sensitive [data][] like integration tokens and API keys.
+When you test functionality that involves external services such as UPS, FedEx, PayPal, or SignifyD, 
+use the MFTF credentials feature to hide sensitive [data][] like integration tokens and API keys.
 
-## Define sensitive data in `.credentials`
+Currently MFTF supports two types of credential storage: **.credentials file** and **HashiCorp vault**.
+
+# Configure File Storage
 
 The MFTF creates a sample file for credentials during [initial setup][]: `magento2/dev/tests/acceptance/.credentials.example`.
 The file contains an example list of keys for fields that can require credentials.
@@ -33,49 +36,119 @@ The command outputs the path if the file is excluded:
 .credentials
 ```
 
-### Define sensitive data
+### Define sensitive data in `.credentials` file
 
-Open the `.credentials` file, uncomment the fields you want to use, and add your values:
+Open the `.credentials` file, for Magento core credentials, uncomment the fields you want to use, and add your values:
 
-```config
+```conf
 ...
 # Credentials for the USPS service
-carriers_usps_userid=test_user
-carriers_usps_password=Lmgxvrq89uPwECeV
+magento/carriers_usps_userid=usps_test_user
+magento/carriers_usps_password=Lmgxvrq89uPwECeV
 
 # Credentials for the DHL service
-#carriers/dhl/id_us=
-#carriers/dhl/password_us=
+#magento/carriers_dhl_id_us=dhl_test_user
+#magento/carriers_dhl_password_us=Mlgxv3dsagVeG
 ....
+```  
+
+Or add new key & value pairs for your own credentials. The keys use the following format:
+
+```conf
+<vendor>/<key_name>=<key_value>
 ```
 
 <div class="bs-callout bs-callout-info" markdown="1">
-The `/` symbol is not supported in a key name.
+The `/` symbol is not supported in a `key_name` other than the one after your vendor or extension name.
 </div>
-
-You are free to use any other keys you like, as they are merely the keys to reference from your tests.
+ 
+Otherwise you are free to use any other `key_name` you like, as they are merely the keys to reference from your tests.
 
 ```conf
 # Credentials for the MyAwesome service
-my_awesome_service_token=rRVSVnh3cbDsVG39oTMz4A
+vendor/my_awesome_service_token=rRVSVnh3cbDsVG39oTMz4A
+```
 
-# Credentials for the USPS service
-carriers_usps_userid=test_user
-carriers_usps_password=Lmgxvrq89uPwECeV
-....
+# Configure Vault Storage
+
+Hashicorp vault secures, stores, and tightly controls access to data in modern computing. 
+It provides advanced data protection for your testing credentials. 
+
+MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine. 
+
+### Install vault CLI
+
+Download and install vault CLI tool if you want to run or develop MFTF tests locally. [Download Vault][Download Vault]
+
+### Authenticate to vault via vault CLI
+
+Authenticate to vault server via vault CLI tool. [Login Vault][Login Vault]
+
+```terminal
+vault login -method -path
 ```
+**Do not** use `-no-store` command option, as MFTF will rely on the persisted token in the token helper 
+(usually the local filesystem) for future api requests.
+
+### Store secrets in vault
+
+MFTF uses `KV Version 2` secret engine for secret storage. 
+More information for working with `KV Version 2` can be found in [Vault KV2][Vault KV2]. 
+
+#### Secrets path and key convention
+
+The path and key for secret data must follow:
+
+```conf
+<SECRETS_BASE_PATH>/mftf/<VENDOR>/<SECRET_KEY>
+```
+
+```conf
+# Secret path and key for carriers_usps_userid
+secret/mftf/magento/carriers_usps_userid
+
+# Secret path and key for carriers_usps_password
+secret/mftf/magento/carriers_usps_password
+```
+
+#### Write secrets to vault
+
+You can use vault CLI or Api to write secret data (credentials, etc) to vault. Here is a CLI example:
+
+```terminal
+vault kv put secret/mftf/magento/carriers_usps_userid carriers_usps_userid=usps_test_user
+vault kv put secret/mftf/magento/carriers_usps_password carriers_usps_password=Lmgxvrq89uPwECeV
+```
+
+### Setup MFTF to use vault
+
+Add vault configuration environment variables [`CREDENTIAL_VAULT_ADDRESS`][] and [`CREDENTIAL_VAULT_SECRET_BASE_PATH`][] 
+from `etc/config/.env.example` in `.env`.
+Set values according to your vault server configuration.  
+
+```conf
+# Default vault dev server
+CREDENTIAL_VAULT_ADDRESS=http://127.0.0.1:8200
+CREDENTIAL_VAULT_SECRET_BASE_PATH=secret
+```
+
+# Configure both File Storage and Vault Storage
+
+It's possible and sometimes useful to setup and use both `.credentials` file and vault for secret storage at the same time. 
+In this case, MFTF tests are able to read secret data at runtime from both storage, and local `.credentials` file will take precedence.
 
 <!-- {% raw %} -->
 
-## Use credentials in a test
+# Use credentials in a test
+
+Credentials can be used in actions: [`fillField`][], [`magentoCLI`][], and [`createData`][].
 
-Access the data defined in the `.credentials` file using the [`fillField`][] action with the `userInput` attribute.
-Define the value as a reference to the corresponding key in the credentials file such as `{{_CREDS.my_data_key}}`:
+Define the value as a reference to the corresponding key in the credentials file or vault such as `{{_CREDS.my_data_key}}`:
 
 - `_CREDS` is an environment constant pointing to the `.credentials` file
 - `my_data_key` is a key in the the `.credentials` file that contains the value to be used in a test step
 
-For example:
+For example, reference secret data in the [`fillField`][] action with the `userInput` attribute.
 
 ```xml
 <fillField stepKey="FillApiToken" selector=".api-token" userInput="{{_CREDS.my_data_key}}" />
@@ -88,13 +161,20 @@ For example:
 The generated tests do not contain credentials values.
 The MFTF dynamically retrieves, encrypts, and decrypts the sensitive data during test execution.
 Decrypted credentials do not appear in the console, error logs, or [test reports][].
-The decrypted values are only available in the `.credentials` file.
+The decrypted values are only available in the `.credentials` file or within vault.
 
 <div class="bs-callout bs-callout-info">
 The MFTF tests delivered with Magento application do not use credentials and do not cover external services, because of sensitivity of the data.</div>
 
 <!-- Link definitions -->
 [`fillField`]: test/actions.md#fillfield
+[`magentoCLI`]: test/actions.md#magentocli
+[`createData`]: test/actions.md#createdata
 [data]: data.md
 [initial setup]: getting-started.md
 [test reports]: reporting.md
+[Download Vault]: https://www.hashicorp.com/products/vault/
+[Login Vault]: https://www.vaultproject.io/docs/commands/login.html
+[Vault KV2]: https://www.vaultproject.io/docs/secrets/kv/kv-v2.html
+[`CREDENTIAL_VAULT_ADDRESS`]: configuration.md#CREDENTIAL_VAULT_ADDRESS
+[`CREDENTIAL_VAULT_SECRET_BASE_PATH`]: configuration.md#CREDENTIAL_VAULT_SECRET_BASE_PATH