K8s docs

.github/workflows/ci.yml

@@ -6,6 +6,7 @@ on:
     branches:
       - "main"
       - "preview"
+      - "k8s-docs"
   pull_request:
 
 permissions:

docs/k8s/config.md

@@ -0,0 +1,205 @@
+---
+title: "Configuration"
+description: "Configuring Imageless Kubernetes"
+---
+
+# Configuration
+
+## Authentication
+
+Imageless Kubernetes allows you to run Flox environments in place of or on top of container images.
+Flox environments are accessed centrally via [FloxHub][floxhub] and managed using the Flox CLI.
+
+In the [introduction][intro] we discussed how annotations are used to tell Imageless Kubernetes which Flox environment to run.
+However, we assumed that the referenced environment was publicly available without authentication.
+If you plan to use private environments, you will have to authenticate Imageless Kubernetes using your FloxHub user credentials.
+
+To do so, you need to first login to FloxHub using the Flox CLI using [`flox auth login`][flox_auth], if you haven't done so already.
+You then create a new Kubernetes secret:
+
+```bash
+flox auth token \
+  | kubectl create secret generic floxhub-token \
+    --from-file=floxhub-token=/dev/stdin
+```
+
+!!! note "Token expiry"
+    Tokens generated with `flox auth token` are associated with your user account and will expire 1 month from when they were issued. For a more robust alternative see [Machine Access Tokens for Organizations](../concepts/organizations.md#machine-access-tokens).
+
+!!! note "Flox CLI version"
+    The user creating the token via `flox auth token` will need at least version 1.7.6 of the Flox CLI.
+
+Finally, you add a secret volume to your pod specification and mount it to `/var/run/secrets/flox.dev` inside your container.
+
+A sample specification is shown below:
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: flox-containerd-demo
+  annotations:
+    flox.dev/environment: "flox/echoip"
+spec:
+  runtimeClassName: flox
+
+  # Required for auth: a secret volume referencing the secret created with
+  # `$ kubectl create secret`
+  volumes:
+    - name: secret-volume
+      secret:
+        secretName: floxhub-token
+
+  containers:
+    - name: empty
+      image: flox/empty:1.0.0
+      command: ["echoip"]
+
+      # Required for auth: mount the secret into a known place where Imageless Kubernetes can read it.
+      volumeMounts:
+        - name: secret-volume
+          mountPath: "/var/run/secrets/flox.dev"
+          readOnly: true
+```
+
+## Telemetry
+
+Since Imageless Kubernetes uses the Flox CLI to perform certain operations such as activating your environment, Imageless Kubernetes will report the same telemetry by default that the Flox CLI reports.
+This includes information such as:
+
+- Which subcommands were run
+- Which shell was used for the activation (Bash, Zsh, etc)
+- Whether the environment was remote (e.g. stored on FloxHub) or not
+- etc
+
+We also use Sentry for error reporting.
+This information helps us focus feature development and maintenance on the areas that deliver the most value for our users.
+
+However, we understand that some users either don't want any information collected or work in an environment that does not allow this kind of information to be collected.
+For this reason we offer the ability to disable telemetry.
+
+### Disabling telemetry
+
+When using the Flox CLI directly you can set `FLOX_DISABLE_METRICS=1` in your environment.
+With Imageless Kubernetes, you can set an annotation on the pod specification to accomplish the same goal.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: flox-containerd-demo
+  annotations:
+    flox.dev/environment: "flox/echoip"
+    # Disable telemetry
+    flox.dev/disable-metrics: "true"
+spec:
+  runtimeClassName: flox
+  containers:
+    - name: empty
+      image: flox/empty:1.0.0
+      command: ["echoip"]
+```
+
+## Activation mode
+
+By default, Imageless Kubernetes pods start in `run` mode. `run` mode is intended only to run packages installed in the Flox environment, but not provide any of the installed development dependencies.
+
+The `flox.dev/activate-mode` annotation can be used to configure the mode, which may be useful for applications such as running CI jobs in Flox-enabled pods.
+
+See the [`options.activate.mode`](../man/manifest.toml.md#options) option in the manifest for more details on modes.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: quotes-app
+  annotations:
+    flox.dev/environment: "flox/quotes-app"
+    # Activate in dev mode
+    flox.dev/activate-mode: "dev"
+```
+
+## Generations
+
+A specific [generation][generations] of an environment on FloxHub can be specified as part of the `flox.dev/environment` annotation.
+This can be useful to pin a specific version of an environment to allow for intentional or staged upgrades.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: quotes-app
+  annotations:
+    # Pin to generation 2 of the environment
+    flox.dev/environment: "flox/quotes-app:2"
+```
+
+## Mutability
+
+By default, Imageless Kubernetes pods are immutable, such that `flox install` commands are not possible and `/nix` is mounted read-only.
+
+To enable mutability (e.g. for debugging), the `flox.dev/nix-mutable` annotation can be used.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: quotes-app
+  annotations:
+    flox.dev/environment: "flox/quotes-app"
+    # Enable /nix mutability
+    flox.dev/nix-mutable: "true"
+```
+
+## Mixed Flox/non-Flox pods
+
+Imageless Kubernetes allows mixing Flox and non-Flox-based containers in the same pod, supporting the use of conventional init or sidecar containers combined with Flox-based workloads.
+This is accomplished through the use of two annotations: `flox.dev/skip-containers` and `flox.dev/skip-containers-exec`.
+
+`flox.dev/skip-containers` accepts a comma-separated list of containers that will _not_ be modified by the Flox runtime, and will be run as if they were started with the default runtime (e.g. `runc`). This option is best used for sidecars like `vault-agent` or `istio` that should run completely unmodified.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: quotes-app
+  annotations:
+    vault.hashicorp.com/agent-inject: "true"
+    vault.hashicorp.com/role: "myapp-role"
+    flox.dev/environment: "flox/quotes-app"
+    # Keep these containers unmodified
+    flox.dev/skip-containers: "vault-agent,vault-agent-init"
+  spec:
+    containers:
+...
+    - name: vault-agent
+      image: hashicorp/vault:latest
+      command: ["/bin/sh", "-ec"]
+      args:
+      - |
+        vault agent -config=/vault/configs/agent.hcl
+      env:
+      - name: VAULT_ADDR
+        value: "http://vault.vault.svc.cluster.local:8200"
+      volumeMounts:
+      - name: vault-secrets
+        mountPath: /vault/secrets
+
+    - name: quotes-app
+      image: flox/empty:1.0.0
+      command: ["quotes-app-go"]
+      volumeMounts:
+      - name: vault-secrets
+        mountPath: /vault/secrets
+        readOnly: true
+...
+```
+
+`flox.dev/skip-containers-exec` also accepts a comma separated list of containers, but containers specified in this annotation _will_ contain the Flox environment specified in `flox.dev/environment`.
+
+The difference from `skip-containers` is that while `skip-containers-exec` containers will have their main process run from the Flox environment, commands run via `kubectl exec` or equivalent will be run outside of it. This option is best used when you want the container's main workload to run in the Flox environment, but need exec commands (for debugging, health checks, or auxiliary tasks) to run in the base container environment without Flox.
+
+[intro]: ./intro.md
+[floxhub]: ../concepts/floxhub.md
+[flox_auth]: ../man/flox-auth.md
+[generations]: ../concepts/generations.md

docs/k8s/examples/gitlab-ci.md

@@ -0,0 +1,69 @@
+---
+title: "GitLab CI"
+description: "Demo running GitLab CI with Imageless Kubernetes"
+---
+
+You can use Imageless Kubernetes in many applications where you would rely on a conventional image-based workflow.
+
+This example shows how you can use Imageless Kubernetes with GitLab CI, running jobs inside of the same Flox environment you use for development.
+
+## GitLab runner configuration
+
+To configure your GitLab runner to run Imageless Kubernetes pods, add this section to the runner's `config.toml`:
+
+```toml
+[[runners]]
+  [runners.kubernetes]
+    namespace = {% raw %}"{{ default .Release.Namespace .Values.runners.jobNamespace }}"{% endraw %}
+    runtime_class_name = "flox"
+    image = "flox/empty:1.0.0"
+    pod_annotations_overwrite_allowed = '^flox\.dev.*'
+    [runners.kubernetes.pod_annotations]
+      "flox.dev/skip-containers" = "init-permissions,helper"
+      "flox.dev/skip-containers-exec" = "build"
+      "flox.dev/activate-mode" = "dev" # optional
+```
+
+!!! note "Note"
+    These options can also be passed as part of the job definition in `.gitlab-ci.yml`, see the [GitLab documentation][gitlab-k8s-docs] for more details.
+
+These settings will start all GitLab job pods using the Flox runtime, with an empty container image, and allow setting additional annotations (e.g. `flox.dev/environment`) on a per-job basis.
+
+The `flox.dev/skip-containers` and `flox.dev/skip-containers-exec` annotations are necessary to allow GitLab's init containers to get the code and job script into build container for execution.
+
+`flox.dev/activate-mode` is set to make build dependencies available to the job script.
+
+See the [configuration][config] page for more details on annotations.
+
+## GitLab job configuration
+
+Once you've configured the runner, you will need to supply each job with the desired `flox.dev/environment` annotation, which can be done in `.gitlab-ci.yml`:
+
+```yaml
+stages:
+  - hello
+
+hello-job:
+  stage: hello
+  variables:
+    KUBERNETES_POD_ANNOTATIONS_1: "flox.dev/environment=flox/hello"
+  script:
+    - hello
+```
+
+where the value of the annotation is the name of an environment from [FloxHub][floxhub].
+
+!!! note "Note"
+    The `flox.dev/environment` annotation is *not* optional -- pods will fail to start if it is not supplied.
+
+## Conclusion
+
+Now, any job you target to this runner will be executed with Imageless Kubernetes.
+
+If you utilize the same Flox environment used for development, you will be able to seamlessly test with the exact same packages, regardless of what system or architecture is used.
+
+Since the CI environment doesn’t rely on a container image, updates are instant: run `flox push`, and the job will pick up the changes automatically.
+
+[gitlab-k8s-docs]: https://docs.gitlab.com/runner/executors/kubernetes/
+[config]: ../config.md
+[floxhub]: ../../concepts/floxhub.md

docs/k8s/examples/kind-demo.md

@@ -0,0 +1,115 @@
+---
+title: "Web server with Redis"
+description: "Demo running a simple web server backed by Redis on kind"
+---
+
+With Imageless Kubernetes, you can run a container just like in any other
+Kubernetes deployment, but you don't have to build a container image.
+
+To demonstrate this, we'll run a simple web server backed by Redis using two
+Flox environments.
+
+## Running the example
+
+The entirety of this example can be run locally with the following commands:
+
+```bash
+git clone https://github.com/flox/flox-kind-demo.git
+cd flox-kind-demo
+flox activate
+just up
+```
+
+This starts a local Kubernetes cluster using `kind` and deploys a web server
+backed by Redis.
+
+To fetch something from the web server, run:
+
+```bash
+curl localhost:3000/quotes/0
+```
+
+## Quotes app environment
+
+The example runs a deployment of [`flox/quotes-app`](https://hub.flox.dev/flox/quotes-app), which is just like any other
+Kubernetes deployment, but with a few key differences.
+Here's a snippet from the deployment manifest:
+
+```yaml
+metadata:
+  labels:
+    app: quotes
+  annotations:
+    flox.dev/environment: "flox/quotes-app"
+spec:
+  runtimeClassName: flox
+  containers:
+    - name: quotes
+      image: flox/empty:1.0.0
+      command: ["quotes-app-go", "-r", "redis:6379"]
+```
+
+The full deployment manifest can be found in the
+[flox-kind-demo repo](https://github.com/flox/flox-kind-demo/blob/main/quotes.yaml).
+
+There are two key lines needed to run the container using a Flox environment
+instead of a container image:
+
+- Specifying `runtimeClassName: flox` runs the container using the Flox
+  containerd shim.
+- The annotation `flox.dev/environment: "flox/quotes-app"` specifies the Flox
+  environment to use to bootstrap the container filesystem instead of a container image
+
+Just as with any container, you can specify a startup command, which is
+`["quotes-app-go", "-r", "redis:6379"]` in this case.
+
+When the container starts, the [`flox/quotes-app`](https://hub.flox.dev/flox/quotes-app) Flox environment is pulled from FloxHub and bind mounted into the container.
+This environment contains the [`flox/quotes-app-go`](https://hub.flox.dev/packages/flox/quotes-app-go) package, which is a simple web server published to FloxHub.
+When the container starts, the environment is activated, and then `quotes-app-go` is run inside the activated environment.
+
+### Redis environment
+
+The `quotes-app-go` server uses a Redis instance running in a second deployment.
+Just like the first pod, rather than specifying a container image, the Redis
+deployment runs the environment `flox/redis` which is pulled from
+[hub.flox.dev/flox/redis](https://hub.flox.dev/flox/redis).
+
+Here's the relevant snippet from the Redis deployment manifest:
+
+```yaml
+metadata:
+  labels:
+    app: redis
+  annotations:
+    flox.dev/environment: "flox/redis"
+spec:
+  runtimeClassName: flox
+  containers:
+    - name: redis
+      image: flox/empty:1.0.0
+      command: ["redis-server", "--daemonize", "no", "--dir", "/data", "--bind", "0.0.0.0", "--protected-mode", "no" ]
+      volumeMounts:
+        - name: redis-data
+          mountPath: /data
+```
+
+The full deployment manifest can be found in the
+[flox-kind-demo repo](https://github.com/flox/flox-kind-demo/blob/main/redis.yaml).
+
+## Updating the deployment
+
+Because the environment is hosted on FloxHub, there's no need to rebuild a
+container image to update the deployment.
+After a change to `quotes-app-go`, updating the deployment would require running
+a `flox publish` for `quotes-app-go` and a `flox upgrade -r flox/quotes-app`.
+After that, restarting a pod will pull the latest generation of the environment.
+This allows deploying software with the reproducibility of a container, but
+without the overhead of having to rebuild an entire container image when iterating.
+
+## Cleaning up
+
+To tear down the local kind cluster, run:
+
+```bash
+just down
+```