Merge pull request #186 from peopledoc/env-write-185

Author: Joachim Jablon

README.rst

@@ -65,7 +65,7 @@ Here are a few things you might do with ``vault-cli``:
     ohsosecret
 
     $ # Load a secret into the environment variables:
-    $ vault-cli env --path mysecret -- env | grep MYSECRET
+    $ vault-cli env --envvar mysecret -- env | grep MYSECRET
     MYSECRET_MYKEY=ohsosecret
 
     $ # Load an ssh key into your ssh-agent:

docs/howto/environment.rst

@@ -16,7 +16,7 @@ works:
     $ vault-cli set test/my_secret value=qwerty
     Done
 
-    $ vault-cli env --path test/my_secret -- bash -c 'echo $MY_SECRET_VALUE'
+    $ vault-cli env --envvar test/my_secret -- bash -c 'echo $MY_SECRET_VALUE'
     qwerty
 
 Environment variable naming
@@ -68,29 +68,29 @@ Let's consider the vault contains only the following secret:
 This table maps input to output. Note that there will always be a single environment
 variable and its value will always be ``mysecret``.
 
-+-------------+-----------------------+---------------------------+
-| ``--path``  | ``--omit-single-key`` | environment variable name |
-+-------------+-----------------------+---------------------------+
-| ``a``       | False                 | ``A_B_C``                 |
-+-------------+-----------------------+---------------------------+
-| ``a``       | True                  | ``A_B``                   |
-+-------------+-----------------------+---------------------------+
-| ``a=D``     | False                 | ``D_B_C``                 |
-+-------------+-----------------------+---------------------------+
-| ``a=D``     | True                  | ``D_B``                   |
-+-------------+-----------------------+---------------------------+
-| ``a/b``     | False                 | ``B_C``                   |
-+-------------+-----------------------+---------------------------+
-| ``a/b``     | True                  | ``B``                     |
-+-------------+-----------------------+---------------------------+
-| ``a/b=D``   | False                 | ``D_C``                   |
-+-------------+-----------------------+---------------------------+
-| ``a/b=D``   | True                  | ``D``                     |
-+-------------+-----------------------+---------------------------+
-| ``a/b:c``   | True or False         | ``C``                     |
-+-------------+-----------------------+---------------------------+
-| ``a/b:c=D`` | True or False         | ``D``                     |
-+-------------+-----------------------+---------------------------+
++---------------+-----------------------+---------------------------+
+| ``--envvar``  | ``--omit-single-key`` | environment variable name |
++---------------+-----------------------+---------------------------+
+| ``a``         | False                 | ``A_B_C``                 |
++---------------+-----------------------+---------------------------+
+| ``a``         | True                  | ``A_B``                   |
++---------------+-----------------------+---------------------------+
+| ``a=D``       | False                 | ``D_B_C``                 |
++---------------+-----------------------+---------------------------+
+| ``a=D``       | True                  | ``D_B``                   |
++---------------+-----------------------+---------------------------+
+| ``a/b``       | False                 | ``B_C``                   |
++---------------+-----------------------+---------------------------+
+| ``a/b``       | True                  | ``B``                     |
++---------------+-----------------------+---------------------------+
+| ``a/b=D``     | False                 | ``D_C``                   |
++---------------+-----------------------+---------------------------+
+| ``a/b=D``     | True                  | ``D``                     |
++---------------+-----------------------+---------------------------+
+| ``a/b:c``     | True or False         | ``C``                     |
++---------------+-----------------------+---------------------------+
+| ``a/b:c=D``   | True or False         | ``D``                     |
++---------------+-----------------------+---------------------------+
 
 Recommended setup
 -----------------
@@ -112,7 +112,7 @@ Your call would look like:
 
 .. code:: console
 
-    $ vault-cli env --omit-single-key --path myapp -- myapp
+    $ vault-cli env --omit-single-key --envvar myapp -- myapp
 
 Ignoring errors
 ---------------
@@ -123,7 +123,7 @@ even if it will be missing some secrets.
 
 .. code:: console
 
-    $ vault-cli env --path myapp --force -- myapp
+    $ vault-cli env --envvar myapp --force -- myapp
 
 .. warning::
 

docs/howto/read.rst

@@ -74,6 +74,11 @@ can write the secret to a specific file:
     with ways to write on ephemeral storage, and check your umask__ and the permissions
     of the created file. See :ref:`SystemD` for safe integration strategies.
 
+.. note::
+
+    ``vault-cli env`` also lets you to write secrets to a file just before launching
+    an arbitrary command.
+
 .. __: https://en.wikipedia.org/wiki/Umask
 
 

docs/howto/ssh.rst

@@ -38,5 +38,5 @@ If you need to have both ssh access and secrets as environment variables (see
 
     $ # If your key is not passphrase-protected
     $ vault-cli ssh --key path/to/ssh_private_key:value \
-      -- vault-cli env --path myapp \
+      -- vault-cli env --envvar myapp \
       -- myapp_that_needs_secrets_and_ssh

docs/howto/systemd.rst

@@ -58,7 +58,7 @@ launch the program through ``vault-cli env``. Let’s launch it as a one-off:
 
 .. code:: console
 
-   $ vault-cli env --path mysecret:value -- myprogram
+   $ vault-cli env --envvar mysecret:value -- myprogram
 
 This will make a variable named ``VALUE`` available to ``myprogram``.
 See the :ref:`vault-cli env <vault-env>` dedicated page for more details on how you can
@@ -85,7 +85,7 @@ We’ll create an override file that will change ExecStart to wrap it in
    # opens a new file for edition
    [Service]
    ExecStart=
-   ExecStart=vault-cli env --path mysecret:value=MYVAR -- myprogram --options
+   ExecStart=vault-cli env --envvar mysecret:value=MYVAR -- myprogram --options
 
 The empty ``ExecStart=`` tells SystemD to ignore the previous command to
 launch and only launch the following one.
@@ -97,8 +97,8 @@ Save and quit the file. Load you new configuration file with:
    $ sudo systemctl daemon-reload
    $ sudo systemctl restart myprogram.service
 
-Writing a single secret to a file before start
-----------------------------------------------
+Writing secrets to files on the filesystem before start
+-------------------------------------------------------
 
 In some cases, you will need to have a file in the filesystem that
 contains directly the secret. This is often the case with private keys.
@@ -110,16 +110,16 @@ be written on disk.
 
 .. __: https://en.wikipedia.org/wiki/RAM_drive
 
-In this case, we’ll also create a service override file, but this time,
-we will be adding a command that launches before our main command:
+In this case, we’ll also create a service override file. We'll add a wrapper
+arount our program like before.
 
 .. code:: console
 
    $ sudo systemctl edit myprogram.service
    # opens a new file for edition
    [Service]
    TemporaryFileSystem=/private
-   ExecStartPre=vault-cli get mysecret --output=/private/path/to/secret/file
+   ExecStart=vault-cli env --file mysecret:key=/private/path/to/secret/file -- myprogram --options
 
 Save and quit the file. Load your new configuration file with:
 
@@ -131,7 +131,7 @@ Save and quit the file. Load your new configuration file with:
 You will need to configure ``myprogram`` to look for your
 secret file at ``/private/path/to/secret/file``.
 
-If you need several files, you can repeat the ``ExecStartPre`` line as
+If you need several files, you can add more ``--file`` flags, as
 many times as needed.
 
 .. note::
@@ -143,6 +143,15 @@ many times as needed.
 Bake secrets into a complex configuration file
 ----------------------------------------------
 
+.. warning::
+
+   It's been reported__ that this approach doesn't work as intended. It's left
+   for inspiration, but as of today, ``ExecStartPre`` cannot write to the
+   private filesystem created by ``TemporaryFileSystem`` in  way that ``ExecStart``
+   can later read. Please refer to the ticket for workarounds.
+
+   .. __: https://github.com/peopledoc/vault-cli/issues/185
+
 In some cases, the program you want to launch doesn’t accept
 configuration through environment but only through configuration files.
 You could be tempted to use the method above, but the configuration file

docs/howto/upgrade.rst

@@ -37,10 +37,10 @@ The following list shows how to update your commands:
    (old) vault get path/to/creds
    (new) vault get path/to/creds value
 
-   (old) vault env --path path/to/creds=FOO -- env  # FOO=xxx
-   (new) vault env --path path/to/creds=FOO -- env  # FOO_VALUE=xxx
-   (new) vault env --path path/to/creds:value=FOO -- env  # FOO=xxx
-   (new) vault env --omit-single_key --path path/to/creds=FOO -- env  # FOO=xxx
+   (old) vault env --envvar path/to/creds=FOO -- env  # FOO=xxx
+   (new) vault env --envvar path/to/creds=FOO -- env  # FOO_VALUE=xxx
+   (new) vault env --envvar path/to/creds:value=FOO -- env  # FOO=xxx
+   (new) vault env --omit-single_key --envvar path/to/creds=FOO -- env  # FOO=xxx
 
 The default output of ``vault get-all`` has also changed and is now flat
 by default (this behavior is controlled with the ``--flat/--no-flat``

docs/quickstart.rst

@@ -208,7 +208,7 @@ Let's try it. First we'll launch the command ``env``, which prints the environme
 
 .. code:: console
 
-    $ vault-cli env --path demo -- env | tail -1
+    $ vault-cli env --envvar demo -- env | tail -1
     DEMO_BLAKE2_SECRET_KEY=du9dibieNg3lei0teidal9
 
 As you can see, the secrets (or, here, the secret) under the path ``demo`` have been
@@ -238,7 +238,7 @@ Ok, now for the real thing:
 
 .. code:: console
 
-    $ vault-cli env --path demo -- ./docs/quickstart_demo.py yay
+    $ vault-cli env --envvar demo -- ./docs/quickstart_demo.py yay
     341c93333a9df726c57671891d6bbea1
 
 **Yay!**

tests/unit/test_cli.py

@@ -414,9 +414,9 @@ def test_env_filter_key(cli_runner, vault_with_token, mocker):
         cli.cli,
         [
             "env",
-            "--path",
+            "--envvar",
             "foo/baz:user=MYNAME",
-            "--path",
+            "--envvar",
             "foo/baz:password",
             "--",
             "echo",
@@ -444,6 +444,40 @@ def test_env_omit_single_key(cli_runner, vault_with_token, mocker):
     assert kwargs["environment"]["FOO_BAZ"] == "yo"
 
 
+def test_env_file(cli_runner, vault_with_token, mocker, tmp_path):
+    mocker.patch("vault_cli.environment.exec_command")
+
+    path = tmp_path / "foo"
+    vault_with_token.db = {"foo/bar": {"value": "yay"}}
+    cli_runner.invoke(
+        cli.cli, ["env", "--file", f"foo/bar:value={path}", "--", "echo", "yay"]
+    )
+    assert path.read_text() == "yay\n"
+
+
+def test_env_file_format_error(cli_runner, vault_with_token, mocker, tmp_path):
+    mocker.patch("vault_cli.environment.exec_command")
+
+    vault_with_token.db = {"foo/bar": {"value": "yay"}}
+    result = cli_runner.invoke(
+        cli.cli, ["env", "--file", "foo/bar", "--", "echo", "yay"]
+    )
+    assert result.exit_code != 0
+    assert "expects both a vault path and a filesystem path" in result.output
+
+
+def test_env_file_yaml(cli_runner, vault_with_token, mocker, tmp_path):
+    mocker.patch("vault_cli.environment.exec_command")
+
+    path = tmp_path / "foo"
+    vault_with_token.db = {"foo/bar": {"value": "yay"}}
+    cli_runner.invoke(
+        cli.cli,
+        ["env", "--file", f"foo/bar={path}", "--", "echo", "yay"],
+    )
+    assert path.read_text() == "---\nvalue: yay\n"
+
+
 def test_main(environ, mocker):
     mock_cli = mocker.patch("vault_cli.cli.cli")
 

vault_cli/cli.py

@@ -465,11 +465,27 @@ def delete(client_obj: client.VaultClientBase, name: str, key: Optional[str]) ->
     formerly deprecated. See command help for details.
     """,
 )
+@click.option(
+    "--file",
+    multiple=True,
+    help="""
+    Write a secret from this path into a file on the filesystem. Expected format is
+    path/in/vault[:key]=/path/in/filesystem . This option is meant to be used when
+    your command can only read its inputs from a file and not from the environment (e.g.
+    secret keys, ...). It's highly recommended to only use the option when provided with
+    a secure private temporary filesystem. Writing to a physical disk should be avoided
+    when possible. If the secret a collection (object or array), it's dumped in YAML
+    format.
+    """,
+)
 @click.option(
     "-o",
     "--omit-single-key/--no-omit-single-key",
     default=False,
-    help="When the secret has only one key, don't use that key to build the name of the environment variable",
+    help="""
+    When the secret has only one key, don't use that key to build the name of the
+    environment variable. This option doesn't affect --file.
+    """,
 )
 @click.option(
     "-f",
@@ -483,6 +499,7 @@ def delete(client_obj: client.VaultClientBase, name: str, key: Optional[str]) ->
 def env(
     client_obj: client.VaultClientBase,
     envvar: Sequence[str],
+    file: Sequence[str],
     omit_single_key: bool,
     force: bool,
     command: Sequence[str],
@@ -501,18 +518,19 @@ def env(
     By default the name is build upon the relative path to the parent of the given path (in parameter) and the name of the keys in the value mapping.
     Let's say that we have stored the mapping `{'username': 'me', 'password': 'xxx'}` at path `a/b/c`
 
-    Using `--path a/b` will inject the following environment variables: B_C_USERNAME and B_C_PASSWORD
-    Using `--path a/b/c` will inject the following environment variables: C_USERNAME and C_PASSWORD
-    Using `--path a/b/c:username` will only inject `USERNAME=me` in the environment.
+    Using `--envvar a/b` will inject the following environment variables: B_C_USERNAME and B_C_PASSWORD
+    Using `--envvar a/b/c` will inject the following environment variables: C_USERNAME and C_PASSWORD
+    Using `--envvar a/b/c:username` will only inject `USERNAME=me` in the environment.
 
     You can customize the variable names generation by appending `=SOME_PREFIX` to the path.
     In this case the part corresponding to the base path is replace by your prefix.
 
-    Using `--path a/b=FOO` will inject the following environment variables: FOO_C_USERNAME and FOO_C_PASSWORD
-    Using `--path a/b/c=FOO` will inject the following environment variables: FOO_USERNAME and FOO_PASSWORD
-    Using `--path a/b/c:username=FOO` will inject `FOO=me` in the environment.
+    Using `--envvar a/b=FOO` will inject the following environment variables: FOO_C_USERNAME and FOO_C_PASSWORD
+    Using `--envvar a/b/c=FOO` will inject the following environment variables: FOO_USERNAME and FOO_PASSWORD
+    Using `--envvar a/b/c:username=FOO` will inject `FOO=me` in the environment.
     """
     envvars = list(envvar) or []
+    files = list(file) or []
 
     env_secrets = {}
 
@@ -534,6 +552,22 @@ def env(
 
         env_secrets.update(env_updates)
 
+    for file in files:
+        path, key, filesystem_path = get_env_parts(file)
+        if not (path and filesystem_path):
+            raise click.BadOptionUsage(
+                "file", "--file expects both a vault path and a filesystem path"
+            )
+        secret_obj = client_obj.get_secret(path=path, key=key or None)
+
+        if not isinstance(secret_obj, (list, dict)):
+            secret = str(secret_obj).strip() + "\n"
+        else:
+            secret = yaml.safe_dump(
+                secret_obj, default_flow_style=False, explicit_start=True
+            )
+        pathlib.Path(filesystem_path).write_text(secret)
+
     if bool(client_obj.errors) and not force:
         raise click.ClickException("There was an error while reading the secrets.")
     environment.exec_command(command=command, environment=env_secrets)