document extensibility using service.provider

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Author: ndeloof

docs/extension.md

@@ -0,0 +1,102 @@
+# About
+
+Compose application model defines `service` as an abstraction for a computing unit managing (a subset of)
+application needs, which can interact with other service by relying on network(s). Docker Compose is designed 
+to use the Docker Engine ("Moby") API to manage services as containers, but the abstraction _could_ also cover 
+many other runtimes, typically cloud services or services natively provided by host.
+
+Compose extensibility model has been designed to extend the `service` support to runtimes accessible through
+third-party tooling.
+
+# Architecture
+
+Compose extensibility relies on the `provider` attribute to select the actual binary responsible to manage
+resource(s) to actually run a service.
+
+```yaml
+  database:
+    provider:
+      type: awesomecloud
+      options:
+        type: mysql
+        size: 256
+```
+
+`provider.type` tells Compose the binary to run, which can be either:
+- another Docker CLI plugin (typically, `model` to run `docker-model`)
+- an executable in user's `PATH`
+
+To be a valid Compose extension, provider command *MUST* accept subcommand `compose` (which can be hidden)
+with subcommands `up` and `down`.
+
+## Up lifecycle
+
+To execute application `up` lifecycle, Compose executes the provider with dedicated command `compose up` passing 
+project name, service name and options. `provider.options` are translated as command line flags :
+```console
+awesomecloud compose --project-name <NAME> up --type=mysql --size=256 "database"
+```
+
+> __Note:__ `project-name` _should_ be used by provider to tag resources
+> set for project, so that later execution with `down` subcommand releases 
+> all allocated resources set for project.
+
+## Communication with Compose
+
+Provider can interact with Compose using `stdout` as a channel, sending JSON line delimited messages.
+JSON messages MUST include a `type` and a `message` attribute.
+```json
+{ "type": "info", "message": "preparing mysql ..." }
+```
+
+`type` can be either:
+- `info` to report status to the user. Compose will render message as service state in the progress UI
+- `error` to let user know something went wrong with details on the error. Compose will render message as service failure reason.
+- `setenv` for plugin to tell compose how dependent services can access the created resource. See next section for details
+
+```mermaid
+sequenceDiagram
+    Shell->>Compose: docker compose up
+    Compose->>Provider: compose up --project-name=xx --foo=bar "database"
+    Provider--)Compose: json { "info": "pulling 25%" }
+    Compose-)Shell: pulling 25%
+    Provider--)Compose: json { "info": "pulling 50%" }
+    Compose-)Shell: pulling 50%
+    Provider--)Compose: json { "info": "pulling 75%" }
+    Compose-)Shell: pulling 75%
+    Provider--)Compose: json { "setenv": "URL=http://cloud.com/abcd:1234" }
+    Compose-)Compose: set DATABASE_URL
+    Provider-)Compose: EOF (command complete) exit 0
+    Compose-)Shell: service started
+```
+
+## Connection to a service managed by provider
+
+A service in the Compose application can declare dependency on a service managed by an external provider: 
+
+```yaml
+services:
+  app:
+    image: myapp 
+    depends_on:
+       - database
+
+  database:
+    provider:
+      type: awesomecloud
+```
+
+As provider command sends a `setenv` JSON message, the configured variable is injected into dependent service,
+prefixed by service name. In this illustration example, as `awesomecloud compose up` sends message:
+```json
+{"type": "setenv", "message": "URL=https://awesomecloud.com/db:1234"}
+```
+`app` service which depends on service managed by provider will get `DATABASE_URL` environment variable injected.
+
+> __Note:__  `compose up` provider command _MUST_ be idempotent. If resource is already running, command _MUST_ set
+> the same environment variables for dependent services to be configured.
+
+## Down lifecycle
+
+`down` lifecycle is equivalent to `up` with the `<provider> compose --project-name <NAME> down <SERVICE>` command.
+Provider is responsible to release all resources associated with the service. 

pkg/compose/plugins.go

@@ -59,7 +59,7 @@ func (s *composeService) runPlugin(ctx context.Context, project *types.Project,
 		return err
 	}
 
-	cmd := s.setupPluginCommand(ctx, project, provider, plugin.Path, command)
+	cmd := s.setupPluginCommand(ctx, project, service, plugin.Path, command)
 
 	eg := errgroup.Group{}
 	stdout, err := cmd.StdoutPipe()
@@ -130,11 +130,14 @@ func (s *composeService) getPluginBinaryPath(providerType string) (*manager.Plug
 	return manager.GetPlugin(providerType, s.dockerCli, &cobra.Command{})
 }
 
-func (s *composeService) setupPluginCommand(ctx context.Context, project *types.Project, provider types.ServiceProviderConfig, path, command string) *exec.Cmd {
+func (s *composeService) setupPluginCommand(ctx context.Context, project *types.Project, service types.ServiceConfig, path, command string) *exec.Cmd {
+	provider := *service.Provider
+
 	args := []string{"compose", "--project-name", project.Name, command}
 	for k, v := range provider.Options {
 		args = append(args, fmt.Sprintf("--%s=%s", k, v))
 	}
+	args = append(args, service.Name)
 
 	cmd := exec.CommandContext(ctx, path, args...)
 	// Remove DOCKER_CLI_PLUGIN... variable so plugin can detect it run standalone