[WIP] Integrate ejson loading prototype

[dpetrick]

Idea by Burke here.

Todos

• See code TODOs added (I'll pull them out here when I find the time).
• Tests, I only did manual testing on the go so far.

Shadowlang

Given a made up ejson example:

{
  "_public_key": "a551...",
  "db": {
    "host": "EJ[1:db...",
    "port": 3306,
    "credentials": {
      "user": "EJ[1:db..."
      "pass": "EJ[1:db..."
    }
  },
  "_env": "prod",
  "deployment_tags": [  "EJ[1:db...", "EJ[1:db..." ]
}

The following shadowenv commands are possible:

• Load entire file: (env/ejson "test.ejson") (or (env/ejson "test.ejson" ()))

$ shadowenv diff
+ DB_CREDENTIALS_USER=1234username
+ DEPLOYMENT_TAGS_0=eu-west-1
+ DB_CREDENTIALS_PASS=1234password
+ _ENV=prod
+ DB_PORT=3306
+ DEPLOYMENT_TAGS_1=eu-central-1
+ DB_HOST=127.0.0.1

• Load specific key: (env/ejson "test.ejson" "db") (note that this works with indexing into arrays, see this)

$ shadowenv diff
+ DB_HOST=127.0.0.1
+ DB_PORT=3306
+ DB_CREDENTIALS_USER=1234username
+ DB_CREDENTIALS_PASS=1234password

• Load several paths from a file: (env/ejson "test.ejson" '("db" "deployment_tags"))

$ shadowenv diff
+ DB_HOST=127.0.0.1
+ DB_PORT=3306
+ DB_CREDENTIALS_USER=1234username
+ DB_CREDENTIALS_PASS=1234password
+ DEPLOYMENT_TAGS_1=eu-central-1
+ DEPLOYMENT_TAGS_0=eu-west-1

Reacting to changes

Shadowenv needs to re-run if a referenced ejson file changes. The implemented approach, which probably has holes or may not be a great solution, is:

• During a shadowenv source evaluation, a list of referenced ejson files is tracked. These are written to .shadowenv.d/.esjon-files file as an ordered list of normalized file paths.
• The shadowenv hash is influenced by the last modified time of all referenced ejson paths from this file.
• This means that as soon as a file is touched, shadowenv reevaluates.
• After each run, the file is automatically updated to contain only actually used files.

Plain example:

• No ejson tracking file present, no env/ejson in any program.
• Add env/ejson to some program.
• Shadowenv runs and evaluates because the hash is different since we added a line to a program file.
• Touch the ejson file, which changes the modified time: Shadowenv runs again.

[burke]

I still think this is a cool idea but with some distance I'm now thinking it's silly to merge unless/until we have a concrete usecase for it. We might in the next half year, we'll see