chore: cleanup instructions for building image

Author: darora

.github/workflows/nix-build.yml

@@ -5,7 +5,6 @@ on:
     branches:
       - develop
       - release/*
-      - da/qemu*
   pull_request:
   workflow_dispatch:
 

.github/workflows/qemu-image-build.yml

@@ -5,7 +5,6 @@ on:
     branches:
       - develop
       - release/*
-      - da/merging-qemu
     paths:
       - '.github/workflows/qemu-image-build.yml'
       - 'qemu-arm64-nix.pkr.hcl'
@@ -62,28 +61,20 @@ jobs:
           sudo chown runner /dev/kvm
           sudo chmod 666 /dev/kvm
 
-      # - name: Install dependencies
-      #   run: |
-      #     sudo apt-get update
-      #     sudo apt-get install -y qemu-efi-aarch64 cloud-image-utils qemu-system-arm qemu-utils
-      #     cd && curl -L "https://releases.hashicorp.com/packer/1.7.8/packer_1.7.8_linux_$(dpkg --print-architecture).zip" -o packer.zip && unzip packer.zip  && rm -f packer.zip  && sudo mv packer /usr/local/bin/
-
       - name: Set PostgreSQL version environment variable
         run: echo "POSTGRES_MAJOR_VERSION=${{ matrix.postgres_version }}" >> $GITHUB_ENV
 
       - name: Generate common-nix.vars.pkr.hcl
         run: |
-          curl -L https://github.com/mikefarah/yq/releases/download/v4.45.1/yq_linux_arm64 -o yq
-          chmod +x yq
+          curl -L https://github.com/mikefarah/yq/releases/download/v4.45.1/yq_linux_arm64 -o yq && chmod +x yq
           PG_VERSION=$(./yq '.postgres_release["postgres'${{ matrix.postgres_version }}'"]' ansible/vars.yml)
           PG_VERSION=$(echo $PG_VERSION | tr -d '"')  # Remove any surrounding quotes
           echo 'postgres-version = "'$PG_VERSION'"' > common-nix.vars.pkr.hcl
           echo 'postgres-major-version = "'$POSTGRES_MAJOR_VERSION'"' >> common-nix.vars.pkr.hcl
-          # echo 'postgresql_major = "'$POSTGRES_MAJOR_VERSION'"' >> common-nix.vars.pkr.hcl
           # Ensure there's a newline at the end of the file
           echo "" >> common-nix.vars.pkr.hcl
 
-
+      # TODO (darora): not quite sure why I'm having to uninstall and re-install these deps, but the build fails w/o this
       - name: Install dependencies
         run: |
           sudo apt-get update
@@ -95,11 +86,7 @@ jobs:
           make init
           GIT_SHA=${{github.sha}}
           export PACKER_LOG=1
-          packer build -var "git_sha=$(UPSTREAM_NIX_GIT_SHA)" -var-file="common-nix.vars.pkr.hcl" qemu-arm64-nix.pkr.hcl
-
-      - name: Install dependencies
-        run: |
-          ls -lh output-cloudimg/
+          packer build -var "git_sha=${GIT_SHA}" -var-file="common-nix.vars.pkr.hcl" qemu-arm64-nix.pkr.hcl
 
       - name: Grab release version
         id: process_release_version
@@ -122,21 +109,21 @@ jobs:
           role-to-assume: ${{ secrets.DEV_AWS_ROLE }}
           aws-region: "us-east-1"
 
-      - uses: docker/setup-buildx-action@v3
-
-      - name: Login to ECR
-        uses: docker/login-action@v2
+      - name: Login to Amazon ECR Public
+        id: login-ecr-public
+        uses: aws-actions/amazon-ecr-login@v2
         with:
-          registry: public.ecr.aws
-
-      - id: build
-        uses: docker/build-push-action@v5
-        with:
-          file: Dockerfile-kubernetes
-          push: true
-          tags: public.ecr.aws/w9p6e7k7/supabase/postgres-v3:${{ steps.process_release_version.outputs.version }}
-          platforms: linux/arm64
-          context: .
+          registry-type: public
+
+      - name: Build, tag, and push docker image to Amazon ECR Public
+        env:
+          REGISTRY: public.ecr.aws/w9p6e7k7
+          REGISTRY_ALIAS: supabase
+          REPOSITORY: postgres-v3
+          IMAGE_TAG: ${{ steps.process_release_version.outputs.version }}
+        run: |
+          docker build -f Dockerfile-kubernetes -t $REGISTRY/$REGISTRY_ALIAS/$REPOSITORY:$IMAGE_TAG .
+          docker push $REGISTRY/$REGISTRY_ALIAS/$REPOSITORY:$IMAGE_TAG
 
       # - name: Upload software manifest to s3 staging
       #   run: |

ansible/tasks/setup-postgres.yml

@@ -273,7 +273,7 @@
   shell: |
     mkdir -p /run/postgresql
     chown -R postgres:postgres /run/postgresql
-  when: stage2_nix and qemu_mode
+  when: stage2_nix and qemu_mode is defined
 
 - name: Restart Postgres Database without Systemd
   become: yes

qemu_artifact.md

@@ -1,68 +1,9 @@
 # QEMU artifact
 
-We build a container image that contains a QEMU qcow2 disk image. This container image can be use with KubeVirt's [containerDisk](https://kubevirt.io/user-guide/storage/disks_and_volumes/#containerdisk) functionality to boot up VMs off the qcow2 image.
-
-Container images are a convenient mechanism to ship the disk image to the nodes where they're needed.
+We build a container image that contains a QEMU qcow2 disk image. Container images are a convenient mechanism to ship the disk image to the nodes where they're needed.
 
 Given the size of the image, the first VM using it on a node might take a while to come up, while the image is being pulled down. The image can be pre-fetched to avoid this; we might also switch to other deployment mechanisms in the future.
 
-# Building QEMU artifact
-
-## Creating a bare-metal instance
-
-We launch an Ubuntu 22 bare-metal instance; we're using the `c6g.metal` instance type in this case, but any ARM instance type is sufficient for our purposes. In the example below the region used is: `ap-south-1`.
-
-```bash
-# create a security group for your instance
-aws ec2 create-security-group --group-name "launch-wizard-1" --description "launch-wizard-1 created 2024-11-26T00:32:56.039Z" --vpc-id "insert-vpc-id"
-
-# using the generated security group ID (insert-sg-group), ensure that it allows for SSH access
-aws ec2 authorize-security-group-ingress --group-id "insert-sg-group" --ip-permissions '{"IpProtocol":"tcp","FromPort":22,"ToPort":22,"IpRanges":[{"CidrIp":"0.0.0.0/0"}]}'
-
-# spin up your instance with the generated security group ID (insert-sg-group)
-aws ec2 run-instances \
---image-id "ami-0a87daabd88e93b1f" \
---instance-type "c6g.metal" \
---key-name "INSERT_KEY_PAIR_NAME" \ # create a key pair, or use other mechanism of getting on to the box
---block-device-mappings '{"DeviceName":"/dev/sda1","Ebs":{"Encrypted":false,"DeleteOnTermination":true,"Iops":3000,"SnapshotId":"snap-0fe84a34403e3da8b","VolumeSize":200,"VolumeType":"gp3","Throughput":125}}' \
---network-interfaces '{"AssociatePublicIpAddress":true,"DeviceIndex":0,"Groups":["insert-sg-group"]}' \
---tag-specifications '{"ResourceType":"instance","Tags":[{"Key":"Name","Value":"qemu-pg-image"}]}' \
---metadata-options '{"HttpEndpoint":"enabled","HttpPutResponseHopLimit":2,"HttpTokens":"required"}' \
---private-dns-name-options '{"HostnameType":"ip-name","EnableResourceNameDnsARecord":true,"EnableResourceNameDnsAAAARecord":false}' \
---count "1"
-
-```
-## Install deps
-
-On the instance, install the dependencies we require for producing QEMU artifacts. Assuming you are the root user:
-
-```bash
-apt-get update
-apt-get install -y qemu-system qemu-system-arm qemu-utils qemu-efi-aarch64 libvirt-clients libvirt-daemon libqcow-utils software-properties-common git make libnbd-bin nbdkit fuse2fs cloud-image-utils awscli
-usermod -aG kvm ubuntu
-curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add -
-apt-add-repository "deb [arch=arm64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
-apt-get update && apt-get install packer=1.11.2-1
-apt-get install -y docker.io
-```
-
-Some dev deps that might be useful:
-
-```bash
-apt-get install -y emacs ripgrep vim-tiny byobu
-```
-
-## Clone repo and build
-
-Logout/login first to pick up new group memberships!
-
-``` bash
-git clone https://github.com/supabase/postgres.git
-cd postgres
-git checkout da/qemu-rebasing # choose appropriate branch here
-make init container-disk-image
-```
-
 ### Build process
 
 The current AMI process involves a few steps:
@@ -87,21 +28,13 @@ b. packer build (`qemu-arm64-nix.pkr.hcl`)
 
 ## Publish image for later use
 
-Following `make init container-disk-image`, the generated image should be found in: `/path/to/postgres/output-cloudimg`. For portability the image is also bundled up as a docker image with the name: `supabase-postgres-test` . Publish the built docker image to a registry of your choosing, and use the published image with KubeVirt.
-
-# Iterating on the QEMU artifact
-
-For a tighter iteration loop on the Postgres artifact, the recommended workflow is to do so on an Ubuntu bare-metal node that's part of the EKS cluster that you're deploying to.
+Following `make init alpine-image`, the generated VM image should be bundled as a container image with the name: `supabase-postgres-test` . Publish the built docker image to a registry of your choosing, and use the published image with e.g. KubeVirt.
 
-- Instead of running `make init container-disk-image`, use `make init host-disk` instead to build the raw image file on disk. (`/path/to/postgres/disk/focal-raw.img`)
-- Update the VM spec to use `hostDisk` instead of `containerDisk`
-    - Note that only one VM can use an image at a time, so you can't create multiple VMs backed by the same host disk.
-- Enable the `HostDisk` feature flag for KubeVirt
-- Deploy the VM to the node
+## Iterating on image
 
-Additionally, to iterate on the container image part of things, you can build the image on the bare-metal node (`eks-node-container-disk-image` target), rather than needing to publish it to ECR or similar registry. However, this part can take a while, so iterating using host disks remains the fastest dev loop.
+For faster iteration, it's more convenient to build the image on an ubuntu bare-metal node that's part of the EKS cluster you're using. Build the image in the `k8s.io` namespace in order for it to be available for immediate use on that node.
 
-## Dependencies note
+### Dependencies note
 
 Installing `docker.io` on an EKS node might interfere with the k8s setup of the node. You can instead install `nerdctl` and `buildkit`: