Merge pull request #551 from sap-contributions/add-file-based-service-binding-docs

Add file-based service binding information to dev docs

Author: anita-flegg

deploy-apps/environment-variable.html.md.erb

@@ -15,7 +15,7 @@ Environment variables are the means <%= vars.app_runtime_abbr %> uses to communi
 For information about setting your own app-specific environment variables, see the [Environment variable](./manifest.html#env-block) in _Deploying with app manifests_.
 
 <p class="note important">
-Do not use user-provided environment variables for security-sensitive information such as credentials. They might unintentionally show up in cf CLI output and Cloud Controller logs. Use <a href="../services/user-provided.html">user-provided service instances</a> instead. The system-provided environment variable <a href="#VCAP_SERVICES">VCAP_SERVICES</a> is properly redacted for user roles such as Space Supporter and in Cloud Controller log files.</p>
+Do not use user-provided environment variables for security-sensitive information such as credentials. They might unintentionally show up in cf CLI output and Cloud Controller logs. Use <a href="../services/user-provided.html">user-provided service instances</a> instead. Credentials which are provided to the app via service bindings are redacted for user roles such as Space Supporter and in Cloud Controller log files.</p>
 
 <p class="note important">
 The maximum size of an environment variable is 130&nbsp;KB. This limit applies also to <%= vars.app_runtime_abbr %> system environment variables such as <code>VCAP_SERVICES</code> and <code>VCAP_APPLICATION</code>.</p>
@@ -25,10 +25,15 @@ The maximum size of an environment variable is 130&nbsp;KB. This limit applies a
 
 Using the Cloud Foundry Command Line Interface (cf CLI), you can run the `cf env` command to view the <%= vars.app_runtime_abbr %> environment variables for your app. The `cf env` command displays the following environment variables:
 
-* The `VCAP_APPLICATION` and `VCAP_SERVICES` variables provided in the container environment
-
+* The `VCAP_APPLICATION` variable, which contains app metadata
 * The user-provided variables set using the `cf set-env` command
 
+There are 3 possible ways of providing service binding data to apps, see [Credential Delivery Methods](../services/application-binding.html#credential-delivery-methods). Depending on the chosen method there will be one of the following environment variables:
+
+* [VCAP_SERVICES](#VCAP-SERVICES)
+* [SERVICE_BINDING_ROOT](#SERVICE-BINDING-ROOT) (experimental)
+* [VCAP_SERVICES_FILE_PATH](#VCAP-SERVICES-FILE-PATH) (experimental)
+
 For more information about the `cf env` command, see [env](http://cli.cloudfoundry.org/en-US/cf/env.html) in the cf CLI documentation. For more information about the `cf set-env` command, see [set-env](http://cli.cloudfoundry.org/en-US/cf/set-env.html) in the cf CLI documentation.
 
 The following example demonstrates the environment variables `cf env` displays:
@@ -100,12 +105,14 @@ The following table lists the system variables available to your application con
 | `PATH` | x | x | x |
 | `PORT` | x |  |  |
 | `PWD` | x | x | x |
+| `SERVICE_BINDING_ROOT` | x | x | x |
 | `TMPDIR` | x |  | x |
 | `USER` | x | x | x |
 | `VCAP_APP_HOST` | x |  |  |
 | `VCAP_APP_PORT` | x |  |  |
 | `VCAP_APPLICATION` | x | x | x |
 | `VCAP_SERVICES` | x | x | x |
+| `VCAP_SERVICES_FILE_PATH` | x | x | x |
 
 ### <a id='CF-INSTANCE-ADDR'></a> CF_INSTANCE_ADDR
 
@@ -222,6 +229,12 @@ The present working directory where the buildpack that processed the app ran.
 
 For example: `PWD=/home/vcap/app`
 
+### <a id='SERVICE-BINDING-ROOT'></a> SERVICE_BINDING_ROOT (experimental)
+
+The root directory location, which contains the service binding information. This needs to be enabled by an app feature flag. See [Service Binding K8s](../services/application-binding.html#service-binding-k8s) for information on how to enable and how service binding data is stored in this directory.
+
+For example: `SERVICE_BINDING_ROOT=/etc/cf-service-bindings`
+
 ### <a id='TMPDIR'></a> TMPDIR
 
 The directory location where temporary and staging files are stored.
@@ -270,6 +283,8 @@ This environment variable contains the associated attributes for a deployed app.
 
 For bindable services, <%= vars.app_runtime_abbr %> adds connection details to the `VCAP_SERVICES` environment variable when you restart your app, after binding a service instance to your app. For more information about bindable services, see [Services overview](../services/index.html).
 
+This is the default [Credential Delivery Method](../services/application-binding.html#credential-delivery-methods).
+
 <%= vars.app_runtime_abbr %> returns the results as a JSON document that contains an object for each service for which one or more instances are bound to the app. The service object contains a child object for each instance of the service that is bound to the app.
 
 The following table defines the attributes that describe a bound service. The key for each service in the JSON document is the same as the value of the "label" attribute.
@@ -341,6 +356,11 @@ VCAP_SERVICES=
 }
 ~~~
 
+### <a id='VCAP-SERVICES-FILE-PATH'></a> VCAP_SERVICES_FILE_PATH (experimental)
+
+The path to the file containing the service binding information in JSON format. This needs to be enabled by an app feature flag. See [File-based VCAP services](../services/application-binding.html#file-based-vcap-services) for more information.
+
+For example: `VCAP_SERVICES_FILE_PATH=/etc/cf-service-bindings/vcap_services`
 
 ## <a id='evgroups'></a> Environment variable groups
 

deploy-apps/large-app-deploy.html.md.erb

@@ -26,6 +26,8 @@ To deploy large apps to <%= vars.app_runtime_abbr %>, ensure that:
 
 * The size of each environment variable for your app does not exceed 130&nbsp;KB. This includes <%= vars.app_runtime_abbr %> system environment variables such as `VCAP_SERVICES` and `VCAP_APPLICATION`. For more information, see [<%= vars.app_runtime_abbr %> environment variables](environment-variable.html).
 
+* If VCAP_SERVICES does get too large either because of too many service bindings or too much data provided by the services, check out the other two [Credential Delivery Methods](../services/application-binding.html#credential-delivery-methods) which have a limit of 1 MB.
+
 * You push only the files that are necessary for your app. To meet this requirement, push only the directory for your app, and remove unneeded files or use the `.cfignore` file to specify excluded files. For more information about specifying excluded files, see [Ignore unnecessary files when pushing](prepare-to-deploy.html#exclude) in _Considerations for Designing and Running an App in the Cloud_.
 
 * You configure Cloud Foundry Command Line Interface (cf CLI) staging, startup, and timeout settings to override settings in the manifest, as necessary:

deploy-apps/manifest-attributes.html.md.erb

@@ -695,7 +695,7 @@ For example:
 <div class="note important">
   <ul>
     <li>You must name variables with alphanumeric characters and underscores. Non-conforming variable names might cause unpredictable behavior.</li>
-    <li>Do not use user-provided environment variables for security sensitive information such as credentials, because they might unintentionally show up in cf CLI output and Cloud Controller logs. Use <a href="../services/user-provided.html">user-provided service instances</a> instead. The system-provided environment variable <a href="./environment-variable.html#VCAP-SERVICES">VCAP_SERVICES</a> is properly redacted for user roles such as Space Supporter and in Cloud Controller log files.</li>
+    <li>Do not use user-provided environment variables for security sensitive information such as credentials, because they might unintentionally show up in cf CLI output and Cloud Controller logs. Use <a href="../services/user-provided.html">user-provided service instances</a> instead. Credentials which are provided to the app via service bindings are redacted for user roles such as Space Supporter and in Cloud Controller log files.</li>
   </ul>
 </div>
 
@@ -776,8 +776,7 @@ For example:
    - instance_XYZ
 ```
 
-You can bind an app to a service instance by setting the `VCAP_SERVICES` environment variable. For more information, see [Bind a
-service](../services/application-binding.html#bind) in _Delivering Service Credentials to an App_.
+For more information, see [Bind a service](../services/application-binding.html#bind) in _Delivering Service Credentials to an App_.
 
 
 ## <a id='deprecated'></a> Deprecated app manifest features

services/_using-vol-services.html.md.erb

@@ -133,6 +133,8 @@ To access the volume service from your app, you must know which file path to use
 
 You can view the file path in the details of the service binding, which are available from the `VCAP_SERVICES` environment variable. See [VCAP_SERVICES](../deploy-apps/environment-variable.html#view-env).
 
+Check [Credential Delivery Methods](./application-binding.html#credential-delivery-methods) on how to retrieve the volume mount information if you chose to use file-based service bindings.
+
 To access the volume service from your app:
 
 1. View environment variables for your app. Run:

services/application-binding.html.md.erb

@@ -9,7 +9,9 @@ You can bind your apps to service instances for the purpose of generating creden
 
 ## <a id='bind'></a>Bind a service instance
 
-Binding a service instance to your app triggers credentials to be provisioned for the service instance and delivered to the app runtime in the [VCAP_SERVICES](../deploy-apps/environment-variable.html) environment variable. For details about consuming these credentials with your app, see [Using bound service instances](#use).
+Binding a service instance to your app triggers credentials to be provisioned for the service instance and delivered to the app runtime. CloudFoundry offers 3 different methods to deliver those credentials to the app.
+
+For details about the different delivery methods and consumption of these credentials with your app, see [Credential Delivery Methods](#credential-delivery-methods).
 
 Not all services support binding, as some services deliver value to users directly without integration with an app. In many cases, binding credentials are unique to an app, and another app bound to the same service instance receives different credentials, but this depends on the service.
 
@@ -25,6 +27,140 @@ $ cf restart my-app
 <p class="note important">
 You must restart or in some cases re-push your app for changes to be applied to the <a href="../deploy-apps/environment-variable.html">VCAP_SERVICES</a> environment variable and for the app to recognize these changes.</p>
 
+### <a id='credential-delivery-methods'></a> Credential Delivery Methods
+
+After your service instance is created and bound to your app, the credentials together with some metadata are available in the app container and can be consumed by your app.
+There are the following three delivery methods, which are mutually exclusive. The developer can choose which one to use and needs to adjust the consumption in the app accordingly.
+For details about consuming credentials specific to your development framework, see the Service Binding section in the documentation for your framework's [buildpack](../../buildpacks/index.html).
+
+#### <a id='vcap-services'></a> Default VCAP_SERVICES environment variable
+
+By default credentials and additional metadata for all bound service instances are stored in the <a href="../deploy-apps/environment-variable.html">VCAP_SERVICES</a> environment variable.
+
+There are two methods developers can use to have their apps consume binding credentials.
+
+* **Parse the JSON yourself:** See the documentation for [VCAP_SERVICES](../deploy-apps/environment-variable.html#VCAP-SERVICES). Helper libraries are available for some frameworks.
+* **Auto-configuration:** Some buildpacks create a service connection for you by creating additional environment variables, updating config files, or passing system parameters to the JVM.
+
+For checking the binding data manually app developers can use `cf env`. See [View Env](../deploy-apps/environment-variable.html#view-env).
+
+<p class="note important">
+The maximum size of content of the VCAP_SERVICES environment variable is 130 KB. If you want to bind more services to the app or the services you want to bind provide lots of data, you can use one of the other methods.</p>
+
+#### <a id='file-based-vcap-services'></a> File-based VCAP services (experimental)
+
+This delivery method is used when the app feature [file-based-vcap-services](https://v3-apidocs.cloudfoundry.org/index.html#app-features) is enabled.
+The environment variable [VCAP_SERVICES_FILE_PATH](../deploy-apps/environment-variable.html#VCAP-SERVICES-FILE-PATH) will be made available in the app container and contains the path to a file containing the same content as the VCAP_SERVICES environment variable above.
+
+The binding data can be viewed like this:
+
+```
+cf ssh <app_name> -c 'cat $VCAP_SERVICES_FILE_PATH'
+```
+
+<p class="note important">
+The vcap_services file content cannot exceed 1 MB, otherwise an <code>IncompatibleBindings</code> error is raised.</p>
+
+#### <a id='service-binding-k8s'></a> Service Binding K8s (experimental)
+
+This delivery method is used when the app feature [service-binding-k8s](https://v3-apidocs.cloudfoundry.org/index.html#app-features) is enabled.
+The environment variable [SERVICE_BINDING_ROOT](../deploy-apps/environment-variable.html#SERVICE-BINDING-ROOT) will be made available in the app container and contains the path to the root folder of a file structure containing binding information.
+This method is fully compatible with the specification on [servicebinding.io](https://servicebinding.io/), which is used by Kubernetes as well.
+
+The binding data can be viewed like this:
+
+```
+cf ssh <app_name> -c 'find $SERVICE_BINDING_ROOT -type f -exec bash -c "echo {}: \$(cat {})" \;'
+```
+
+<p class="note important">
+The bytesize of all files and their content combined cannot exceed 1 MB, otherwise an <code>IncompatibleBindings</code> error is raised.</p>
+
+Here are some examples outlining the transformation of VCAP_SERVICES to the file structure:
+
+Nested structures inside their value will be serialized as JSON objects. The same applies if the value is a list.
+
+```
+VCAP_SERVICES= {
+  "foo": [
+    {
+      "name": "foo",
+      "credentials": {
+        "simple": "value",
+        "deeply": {
+          "nested": "value"
+        },
+        "list": ["v", "a", "l", "u", "e"]
+      }
+    }
+  ]
+}
+
+Service Binding Files:
+  foo/name: foo
+  foo/simple: value
+  foo/deeply: {"nested":"value"}
+  foo/list: ["v","a","l","u","e"]
+```
+
+Existing credential keys can be overwritten by other attributes, that are [VCAP_SERVICES](../deploy-apps/environment-variable.html#VCAP-SERVICES) attributes plus type and provider. The full list of reserved file names is: binding-guid, binding-name, instance-guid, instance-name, name, label, tags, plan, syslog-drain-url, volume-mounts, type, and provider. Overwriting an existing key does not result in an error. 
+
+```
+VCAP_SERVICES= {
+  "foo": [
+    {
+      "name": "foo",
+      "credentials": {
+        "name": "user",
+        "secret": "password"
+      }
+    }
+  ]
+}
+
+Service Binding Files:
+  foo/name: foo
+  foo/secret: password
+```
+
+Empty lists or null values are omitted, i.e. no file will be created.
+
+```
+VCAP_SERVICES= {
+  "foo": [
+    {
+      "name": "foo",
+      "binding_guid": "45436ca8-0a7c-45e3-9439-ca1b44db7a2b",
+      "syslog_drain_url": null,
+      "volume_mounts": []
+    }
+  ]
+}
+
+Service Binding Files:
+  foo/name: foo
+  foo/binding-guid: 45436ca8-0a7c-45e3-9439-ca1b44db7a2b
+```
+
+Binding names and keys resulting in filenames must match [a-z0-9\-.]{1,253}. Invalid binding names and keys will result in IncompatibleBindings error. VCAP_SERVICES attributes are transformed to comply to this schema, i.e. underscores are replaced by hyphens, so that e.g. VCAP_SERVICES attribute binding_guid becomes file name binding-guid.
+
+```
+VCAP_SERVICES= {
+  "foo": [
+    {
+      "name": "foo",
+      "binding_guid": "45436ca8-0a7c-45e3-9439-ca1b44db7a2b",
+      "instance_guid": "83745ca8-0a7c-45e3-9439-ca1b44db7a2b"
+    }
+  ]
+}
+
+Service Binding Files:
+  foo/name: foo
+  foo/binding-guid: 45436ca8-0a7c-45e3-9439-ca1b44db7a2b
+  foo/instance-guid: 83745ca8-0a7c-45e3-9439-ca1b44db7a2b
+```
+
 ### <a id='arbitrary-params-binding'></a> Arbitrary parameters
 
 Some services support additional configuration parameters with the bind request. These parameters are passed in a valid JSON object containing service-specific configuration parameters, provided either in-line or in a file. For a list of supported configuration parameters, see documentation for the particular service offering.
@@ -90,7 +226,7 @@ OK
 
 The provided name is available in the `name` and `binding_name` properties in the [VCAP_SERVICES](../deploy-apps/environment-variable.html) environment variable:
 
-~~~
+```
 "VCAP_SERVICES": {
   "service-name": [
     {
@@ -100,16 +236,7 @@ The provided name is available in the `name` and `binding_name` properties in th
     }
   ]
 }
-~~~
-
-### <a id='use'></a>Using bound service instances
-
-After your service instance is created and bound to your app, you must configure the app to dynamically fetch the credentials for your service instance. The [VCAP_SERVICES](../deploy-apps/environment-variable.html#VCAP-SERVICES) environment variable contains credentials and additional metadata for all bound service instances. There are two methods developers can use to have their apps consume binding credentials.
-
-* **Parse the JSON yourself:** See the documentation for [VCAP_SERVICES](../deploy-apps/environment-variable.html#VCAP-SERVICES). Helper libraries are available for some frameworks.
-* **Auto-configuration:** Some buildpacks create a service connection for you by creating additional environment variables, updating config files, or passing system parameters to the JVM.
-
-For details about consuming credentials specific to your development framework, see the Service Binding section in the documentation for your framework's [buildpack](../../buildpacks/index.html).
+```
 
 ## <a id='update-credentials'></a>Update service credentials