What can we do with compiler output (aka targets)

[MarkMcCulloh]

At the moment, wing has two targets:

• sim
  • wing console synthesis engine. final output is a wx file
  • default polycon factory resolves to actual simulations of each resource intended to be run in a node process
• tf-aws
  • cdktf synthesis engine, final output is a cdktf.out dir intended to be used by the cdktf cli (or just tf I guess as well)
  • default polycon factory resolves to a cdktf AWS construct tree

Maybe "target" is doing too much heavy lifting here.

Few questions for discussion:

• When we add other cloud options, would there be a tf-gcp target? Other than the polycon factory, what changes?
• What if your wing program only needs to change files (like projen)? What target should you use? cdktf can be used to manage file state, but is that the best option
• What would a tf-local target look like? Imagine a polycon factory that has these resources, managed with cdktf. Maybe this could be used to describe a composable build system:
  • Bucket = directory
  • Function = executable file, inflight wing somehow encapsulated
  • Queue = idk, a file with a queue in it lol
• Should there be separate "targets" that act as more specific options, analogous to typical compilers? e.g. target, architecture, platform, etc. What would they mean?
• Should the synthesizer not be part of the target and instead inferred automatically somehow? a la https://github.com/3p3r/unicdk (awesome btw @3p3r)

[Chriscbr]

Lots of thought provoking questions... goes without saying but I think a lot of this is unexplored territory 😄

When we add other cloud options, would there be a tf-gcp target? Other than the polycon factory, what changes?

I'm hoping to try this out soon and report back. :) I think you're right, it's mostly just the polycon factory that changes. I think the root of the user's app will also need to be automatically configured with an Google Provider instead of an AWS provider -- see in the SDK code here:

wing/libs/wingsdk/src/tf-aws/app.ts

Line 67 in 572499b

new AwsProvider(this, "AwsProvider", {});

But besides that, the "App" class isn't doing much heavy lifting, and it could probably be the same for all terraform-based targets.

Should the synthesizer not be part of the target and instead inferred automatically somehow?

Two thoughts:

1. Even though you could say the synthesizer is part of the target, I think it's more accurate to say we're coupling the deployment engine to the targets. In my mind, CDKTF is just an implementation detail -- in the sense that if some other awesome JavaScript library came out that let us generate Terraform manifests programmatically, we could theoretically change the tf-aws polycon implementations to use that instead of CDKTF, while maintaining the same exact public API and nobody noticing the difference.
2. I think the reason we decided to couple the deployment engines to the targets (i.e. tf-aws instead of aws) is so in the future, if someone really wants to create a cfn-aws target or a pulumi-aws target or even if someone wants to create some new deployment engine / format, then it would be possible to create polycons based on that. It's just a foot in the door for now. Now, you might be saying, "that seems inefficient, there's probably a lot of duplicate logic between tf-aws, cfn-aws, and pulumi-aws"! And I suppose if we get that far, we could totally consolidate things into an intermediate layer that maps between cloud resource definitions and how to synthesize them into representations for particular deployment engines. But that might be a long time away...

[eladb]

And maybe initially this policy is a .js file that simply implements a polycons factory?

[3p3r]

This is what I was thinking for validation as well. Instead of having a new format, I suggest we adopt Open Policy Agent's format and compile those policies to web assembly so we can execute them in browsers. An example policy would look like:

package play

import future.keywords.if

# This is what a Wing policy document may look like
# You can learn more here:
# https://play.openpolicyagent.org/
# https://www.openpolicyagent.org/docs/latest/policy-language/#what-is-rego

default encryption := false

true if cloud.Bucket.encryption == encryption 

During validation we can provide a context to the evaluator (e.g app's tree data structure, polycon options, factory selection, etc.)

[eladb]

I love the idea of building on top of an existing standard. But OPA is for defining validations while I am looking for something that will actually apply a policy.

Maybe there's a way to make OPA be "proactive"?

[3p3r]

OPA does both. You would use Rego (OPA’s policy language) to define a policy and use OPA’s engine to apply it to various contexts. OPA calls it evaluation. Workflow is:

1. we get .rego policies from user
2. we compile those to .wasm policies
3. user runs “wing compile —policy sample.wasm app.w”
4. during compilation, we use a runtime like https://github.com/open-policy-agent/npm-opa-wasm to execute/evaluate the policy to any data

Data can be anything. So we can have user write policies for anything. Step 1 and 2 is optional if user already is setup with OPA toolchain.

[eladb]

Sounds like a very interesting candidate for our "infrastructure as policy" component. Would be nice to do a little prototype...