Consider a special prefix for build-time environment variables

[RobvH]

Goals

1. End the confusion about environment variable load time
2. Empower the next build to error if a non-build ENV var is being used in a build-only location
3. Bring the Next perspective on environment variables in line with a 45 year old programming convention

Non-Goals

No response

Background

Environment variables were introduced in Version 7 of Unix in 1979. At that time, a growing goal in programming technologies was portability of programs and code. Many Unix conventions and packages we still use today were born with this purpose. Rather than compiling local configuration values into a program, they could take command line parameters. However, this can quickly grow unwieldy and may include many values not known nor important to the user's intent (what's the hostname I'm running on, what version of the OS is this, should I output color codes, am I relaying my display to a remote x-server, etc). Environment variables were created to let the program pick up configuration at runtime; making it both portable while keeping command line arguments focused on the user's purpose.

Throughout the 90's and 2000's, we made heavy use of this convention to configure things like Apache web servers, MySQL databases, etc. Environment variable were all about runtime. While true, compilers themselves make use of environment variables; they were pretty much always for configuring their own work: compilation, not statically building app configuration into the program they were compiling.

Around 2014-2015, dotenv become very popular and web apps from diverse stacks ranging from Laravel to NodeJS started making common use of .env files, as ways for developers to easily "hoist" values into the env of their application. ECS and Lambdas make heavy use of the unix environment vars, with the former easily hoisting Parameter Store secrets to a container app with very little config via the environment. By 2020, even DotNet had pretty much joined the env team. The theme was universal: build environment agnostic apps - leverage the unix environment at runtime to know what's what.

With the rise of static-site builders, something happened. A really good idea, to pre-build as much of the html of an app as possible, seems to have, misunderstood the concept of environment variables and subverted their purpose. NextJS apps built in CI pipelines started inlining the values of environment vars into their sites; making the build environment the runtime environment. Environment variables to set build settings would be fine, but here the env was being subverted. This was well intentioned 🫶 but very problematic. The use of env vars was, for the first time in their history, completely turned around and the code worked opposite accepted convention both in NodeJS but also every other programming stack since ADA. Of course, some folks had been shipping .env files in their artifacts, but there were plenty of "please stop doing that today" articles. 😄

Much as been done to improve the situation 🤗. For a while now, the NextJS community has realized the problem and the framework now supplies real environment variables in several locations. (I would strongly argue that the term "runtime environment variables" is entirely redundant ☮️ ). However, confusion still ensues, and one of the learning curves and a large source of bugs in NextJS apps derives from the fact that environment variables not only often carry unexpected values, but potentially leak secrets into the compiled artifact.

To mitigate this, a new approach is suggested.

Proposal

When environment variables are intended to be hoisted to the client, we have a wonderfully clear convention - NEXT_PUBLIC_{name}. This enables any developer to clearly state their intentions and (hopefully) easily avoid leaking secrets.

Similarly - it is suggested that env vars intended to be turned "static" at build time, be prefixed NEXT_STATIC_{name} or NEXT_BUILD_{name}. This serves two valuable purposes:

1. A developer setting these will easily know they are intended for build time.
2. The compiler and eslint can look for other patterns and throw compilation/lint errors when a non-STATIC env var is being used in a location that doesn't support "runtime" environment variables.

It is believed that this has the potential to mitigate one of the most painful learning curves in the otherwise developer friendly Next ecosystem. It's recognized that this would require migration - so a whole major version could progressively introduce the change as linter warnings, then errors ahead of a major version change.

Why not prefix runtime environment vars?
Good question! Mainly because, that is already what they are intended to be. It's inherent to their very purpose for existing and is the way they are used universally now. Just as we wouldn't expect to find them in our web clients -- justifying the special prefix for hoisting there -- we wouldn't expect to find their values baked into our build artifacts -- also justifying the special prefix for that.

❤️ ❤️