Add new local_command data source for running and retrieving data from local executables

[austinvalle]

Related Issue

Closes #63

There are a number of external provider issues that this new data source would address, so I'll just link a few:

• Feature Request - Allow list/array in query and result terraform-provider-external#2
• Feature Request: Allow Pre-formatted Strings in Query Argument terraform-provider-external#17
• Feature Request: Allow arrays from external datasource result terraform-provider-external#56
• Switch Output to DynamicPseudoType terraform-provider-external#76

Description

This PR introduces a new local_command data source, which achieves a similar goal to the existing external data source in a more generic fashion, allowing practitioners to encode/decode the data going to and from a local executable.

// A toy example using the JSON utility `jq` to process Terraform data
// https://jqlang.org/
data "local_command" "filter_fruit" {
  command   = "jq"
  stdin     = jsonencode([{ name = "apple" }, { name = "lemon" }, { name = "apricot" }])
  arguments = [".[:2] | [.[].name]"] # Grab the first two fruit names from the list
}

output "fruit_tf" {
  value = jsondecode(data.local_command.filter_fruit.stdout)
}

# Outputs:
#
# fruit_tf = [
#   "apple",
#   "lemon",
# ]

This is preferable to returning a dynamic type / pre-decoding data returned from a command, as providers ATM only really have a clear implementation of decoding JSON data, while this approach allows existing decoding definitions to be used, for example:

• https://developer.hashicorp.com/terraform/language/functions/jsondecode
• https://developer.hashicorp.com/terraform/language/functions/yamldecode
• https://developer.hashicorp.com/terraform/language/functions/csvdecode

The acceptance tests at /internal/provider/data_source_local_command_test.go have an example of each of these functions.

In addition, alternative decoding implementations could be introduced via provider defined functions.

Rollback Plan

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

No

[austinvalle]

Well the tests aren't running on this PR since it's pointed at a non-main branch 😅 , so I ran them locally with the latest 1.14.0-rc1 for the time being:

Test logs

 $ make testacc 
TF_ACC=1 go test -v -cover -timeout 120m ./...
        github.com/terraform-providers/terraform-provider-local         coverage: 0.0% of statements
=== RUN   TestFilePermissionValueValidateAttribute
=== PAUSE TestFilePermissionValueValidateAttribute
=== CONT  TestFilePermissionValueValidateAttribute
=== RUN   TestFilePermissionValueValidateAttribute/00700
=== RUN   TestFilePermissionValueValidateAttribute/-1
=== RUN   TestFilePermissionValueValidateAttribute/0777
=== RUN   TestFilePermissionValueValidateAttribute/0644
=== RUN   TestFilePermissionValueValidateAttribute/9999
=== RUN   TestFilePermissionValueValidateAttribute/7
--- PASS: TestFilePermissionValueValidateAttribute (0.00s)
    --- PASS: TestFilePermissionValueValidateAttribute/00700 (0.00s)
    --- PASS: TestFilePermissionValueValidateAttribute/-1 (0.00s)
    --- PASS: TestFilePermissionValueValidateAttribute/0777 (0.00s)
    --- PASS: TestFilePermissionValueValidateAttribute/0644 (0.00s)
    --- PASS: TestFilePermissionValueValidateAttribute/9999 (0.00s)
    --- PASS: TestFilePermissionValueValidateAttribute/7 (0.00s)
PASS
coverage: 27.8% of statements
ok      github.com/terraform-providers/terraform-provider-local/internal/localtypes     0.530s  coverage: 27.8% of statements
=== RUN   TestLocalCommandAction_bash
--- PASS: TestLocalCommandAction_bash (0.64s)
=== RUN   TestLocalCommandAction_bash_stdin
--- PASS: TestLocalCommandAction_bash_stdin (0.47s)
=== RUN   TestLocalCommandAction_bash_all
--- PASS: TestLocalCommandAction_bash_all (0.55s)
=== RUN   TestLocalCommandAction_bash_null_args
--- PASS: TestLocalCommandAction_bash_null_args (0.47s)
=== RUN   TestLocalCommandAction_absolute_path_bash
--- PASS: TestLocalCommandAction_absolute_path_bash (0.49s)
=== RUN   TestLocalCommandAction_not_found
--- PASS: TestLocalCommandAction_not_found (0.17s)
=== RUN   TestLocalCommandAction_stderr
--- PASS: TestLocalCommandAction_stderr (4.83s)
=== RUN   TestLocalCommandDataSource_stdout_json
--- PASS: TestLocalCommandDataSource_stdout_json (0.80s)
=== RUN   TestLocalCommandDataSource_stdout_csv
--- PASS: TestLocalCommandDataSource_stdout_csv (0.52s)
=== RUN   TestLocalCommandDataSource_stdout_yaml
--- PASS: TestLocalCommandDataSource_stdout_yaml (0.97s)
=== RUN   TestLocalCommandDataSource_stdout_no_format_null_args
--- PASS: TestLocalCommandDataSource_stdout_no_format_null_args (0.58s)
=== RUN   TestLocalCommandDataSource_stderr_zero_exit_code
--- PASS: TestLocalCommandDataSource_stderr_zero_exit_code (0.57s)
=== RUN   TestLocalCommandDataSource_stdout_invalid_string
--- PASS: TestLocalCommandDataSource_stdout_invalid_string (0.52s)
=== RUN   TestLocalCommandDataSource_non_zero_exit_code_error
--- PASS: TestLocalCommandDataSource_non_zero_exit_code_error (0.31s)
=== RUN   TestLocalCommandDataSource_allow_non_zero_exit_code
--- PASS: TestLocalCommandDataSource_allow_non_zero_exit_code (0.49s)
=== RUN   TestLocalCommandDataSource_absolute_path_with_working_directory
--- PASS: TestLocalCommandDataSource_absolute_path_with_working_directory (0.51s)
=== RUN   TestLocalCommandDataSource_not_found
--- PASS: TestLocalCommandDataSource_not_found (0.19s)
=== RUN   TestLocalFileDataSource
--- PASS: TestLocalFileDataSource (0.41s)
=== RUN   TestLocalFileSensitiveDataSource
--- PASS: TestLocalFileSensitiveDataSource (0.41s)
=== RUN   TestLocalFileSensitiveDataSourceCheckSensitiveAttributes
--- PASS: TestLocalFileSensitiveDataSourceCheckSensitiveAttributes (0.00s)
=== RUN   TestDirectoryExists_basic
--- PASS: TestDirectoryExists_basic (0.40s)
=== RUN   TestDirectoryExists_invalid_file
--- PASS: TestDirectoryExists_invalid_file (0.16s)
=== RUN   TestDirectoryExists_invalid_symlink
--- PASS: TestDirectoryExists_invalid_symlink (0.15s)
=== RUN   TestLocalFile_Basic
--- PASS: TestLocalFile_Basic (1.32s)
=== RUN   TestLocalFile_Source
--- PASS: TestLocalFile_Source (0.46s)
=== RUN   TestLocalFile_Permissions
--- PASS: TestLocalFile_Permissions (0.36s)
=== RUN   TestLocalFile_Validators
--- PASS: TestLocalFile_Validators (0.20s)
=== RUN   TestLocalFile_Upgrade
--- PASS: TestLocalFile_Upgrade (5.89s)
=== RUN   TestLocalFile_Source_Upgrade
--- PASS: TestLocalFile_Source_Upgrade (2.19s)
=== RUN   TestLocalFile_Permissions_Upgrade
--- PASS: TestLocalFile_Permissions_Upgrade (1.85s)
=== RUN   TestLocalSensitiveFile_Basic
--- PASS: TestLocalSensitiveFile_Basic (1.10s)
=== RUN   TestLocalSensitiveFile_source
--- PASS: TestLocalSensitiveFile_source (0.52s)
=== RUN   TestLocalSensitiveFile_Permissions
--- PASS: TestLocalSensitiveFile_Permissions (0.42s)
=== RUN   TestLocalSensitiveFile_Validators
--- PASS: TestLocalSensitiveFile_Validators (0.20s)
=== RUN   TestLocalSensitiveFile_Upgrade
--- PASS: TestLocalSensitiveFile_Upgrade (4.10s)
=== RUN   TestLocalSensitiveFile_Source_Upgrade
--- PASS: TestLocalSensitiveFile_Source_Upgrade (2.42s)
=== RUN   TestLocalSensitiveFile_Permissions_Upgrade
--- PASS: TestLocalSensitiveFile_Permissions_Upgrade (1.83s)
PASS
coverage: 79.9% of statements
ok      github.com/terraform-providers/terraform-provider-local/internal/provider       38.198s coverage: 79.9% of statements

[stephybun]

I think this looks pretty good, just have one question/thought left in-line.

It's also a shame we can't link to a full list of available encoding/decoding functions in the docs.

[busser]

This looks great! Are there plans for an ephemeral version of this? I would be more than happy to contribute.

A local_command ephemeral resource would close hashicorp/terraform-provider-external#437 and make hashicorp/terraform-provider-external#442 redundant.

[stephybun]

LGTM 💯

[SBGoods]

LGTM!

The base branch was changed.

[stephybun]

LGTM 🧇