Progress docker-ifying 11

Author: hjwp

chapter_11_ansible.asciidoc

@@ -189,9 +189,8 @@ At my registrar, the control screens looked a bit like <<registrar-control-scree
 
 [[registrar-control-screens]]
 .Domain setup
-image::images/twp2_0902.png["Registrar control screens for two domains"]
+image::images/gandi_add_dns_a_record.png["Registrar control screen for adding a DNS record"]
 
-//TODO: adjust illustration to show "superlists" not "book-example"
 
 ((("A-Records")))
 In the DNS system, pointing a domain at a specific IP address is called an "A-Record".
@@ -263,42 +262,20 @@ relationship with.
 
   tasks:
 
-    - name: Install docker  <1>
+    - name: Install docker  #<1>
+      ansible.builtin.apt:  #<2>
+        name: docker  #<3>
+        state: latest
+        update_cache: true
       become: true
-      block:
-      - name: Add Docker GPG apt Key
-        ansible.builtin.apt_key:  <2>
-          url: https://download.docker.com/linux/ubuntu/gpg
-          state: present
-
-      - name: Add Docker Repository
-        ansible.builtin.apt_repository:
-          repo: deb https://download.docker.com/linux/ubuntu jammy stable
-          state: present
-
-      - name: Update apt and install docker-ce
-        ansible.builtin.apt:
-          name: docker-ce
-          state: latest
-          update_cache: true
-
-      - name: Install Docker Module for Python
-        ansible.builtin.pip:
-          name: docker
-
-      - name: Add our user to Docker allowed users
-        ansible.builtin.user:
-          name: elspeth
-          groups: docker
-          append: true
 
     - name: Run test container
       community.docker.docker_container:
         name: testcontainer
         state: started
         image: busybox
         command: echo hello world
-        become: true
+      become: true
 ----
 ====
 
@@ -322,56 +299,55 @@ https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.h
 
 
 [subs="specialcharacters,quotes"]
-
 ----
-$ *ansible-playbook --user=elspeth -i 192.168.56.10, infra/ansible-provision.yaml -vv*
+$ *ansible-playbook --user=elspeth -i staging.ottg.co.uk, infra/ansible-provision.yaml -vv*
+ansible-playbook [core 2.16.3]
+  config file = None
+  [...]
+No config file found; using defaults
+Skipping callback 'default', as we already have a stdout callback.
+Skipping callback 'minimal', as we already have a stdout callback.
+Skipping callback 'oneline', as we already have a stdout callback.
+
 PLAYBOOK: ansible-provision.yaml **********************************************
 1 plays in infra/ansible-provision.yaml
 
 PLAY [all] ********************************************************************
 
 TASK [Gathering Facts] ********************************************************
-task path: ...goat-book/infra/ansible-provision.yaml:2
-ok: [192.168.56.10]
-
-TASK [Add Docker GPG apt Key] *************************************************
-task path: ...goat-book/infra/ansible-provision.yaml:9
-changed: [192.168.56.10] => {"after": ["8D81803C0EBFCD88", "7EA0A9C3F273FCD8", "D94AA3F0EFE21092", "871920D1991BC93C"], "before": ["D94AA3F0EFE21092", "871920D1991BC93C"], "changed": true, "fp": "8D81803C0EBFCD88", "id": "8D81803C0EBFCD88", "key_id": "8D81803C0EBFCD88", "short_id": "0EBFCD88"}
-
-TASK [Add Docker Repository] **************************************************
-task path: ...goat-book/infra/ansible-provision.yaml:14
-changed: [192.168.56.10] => {"changed": true, "repo": "deb https://download.docker.com/linux/ubuntu jammy stable", "sources_added": ["/etc/apt/sources.list.d/download_docker_com_linux_ubuntu.list"], "sources_removed": [], "state": "present"}
-
-TASK [Update apt and install docker-ce] ***************************************
-task path: ...goat-book/infra/ansible-provision.yaml:19
-changed: [192.168.56.10] => {"cache_update_time": [...]
-changed: [192.168.56.10] => {"cache_update_time": 1706583891, "cache_updated":
-true, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading
-package lists...\nBuilding dependency tree...\nReading state
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:2
+ok: [staging.ottg.co.uk]
+PLAYBOOK: ansible-provision.yaml **********************************************
+1 plays in infra/ansible-provision.yaml
+
+TASK [Install docker] *********************************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:6
+ok: [staging.ottg.co.uk] => {"cache_update_time": 1708981325, "cache_updated": true, "changed": false}
+
+
+TASK [Install docker] *************************************************************************************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:6
+changed: [staging.ottg.co.uk] => {"cache_update_time": [...]
+"cache_updated": true, "changed": true, "stderr": "", "stderr_lines": [],
+"stdout": "Reading package lists...\nBuilding dependency tree...\nReading [...]
 information...\nThe following additional packages will be installed:\n
-containerd.io docker-buildx-plugin docker-ce-cli docker-ce-rootless-extras\n
-[...]
-TASK [Add our user to Docker allowed users] ***********************************
-changed: [192.168.56.10] => {"append": true, "changed": true, "comment": "",
-"group": 1001, "groups": "docker", "home": "/home/elspeth", "move_home": false,
-"name": "elspeth", "shell": "/bin/bash", "state": "present", "uid": 1001}
+wmdocker\nThe following NEW packages will be installed:\n  docker wmdocker\n0
+
 TASK [Run test container] *****************************************************
-task path: ...goat-book/infra/ansible-provision.yaml:31
-changed: [192.168.56.10] => {"changed": true, "container": {"AppArmorProfile":
-"docker-default", "Args": ["hello", "world"], "Config": {"AttachStderr": true,
-"AttachStdin": false, "AttachStdout": true, "Cmd": ["echo", "hello", "world"],
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:13
+changed: [staging.ottg.co.uk] => {"changed": true, "container":
+{"AppArmorProfile": "docker-default", "Args": ["hello", "world"], "Config":
 [...]
 
-PLAY RECAP ***********************************************************
-192.168.56.10              : ok=6    changed=6    unreachable=0    failed=0
+PLAY RECAP ********************************************************************
+staging.ottg.co.uk         : ok=3    changed=2    unreachable=0    failed=0
 skipped=0    rescued=0    ignored=0
 ----
 
-TODO: stop using local ip
 
 
 ////
-
+old error message when trying to use elspeth user to run docker.
 this goes wrong because groups don't work immediately:
 
 TASK [Run test container] *****************************************************
@@ -382,30 +358,41 @@ PermissionError(13, 'Permission denied'))"}
 waiting a few minutes fixes it
 
 for now i'll just put become:true
-
-
 ////
 
 
 === SSHing Into the Server and Viewing Container Logs
 
-Now ssh into the server, check it worked
+Time to get into some good old-fashioned sysadmin!
+Let's SSH in to our server and see if we can see any evidence that our container has run.
+
+We use `docker ps -a` to view all containers, including old/stopped ones,
+and we can use `docker logs` to view the output from one of them:
 
-TODO: forget podman, just use docker.
 
 [role="server-commands"]
 [subs="specialcharacters,quotes"]
 ----
-elspeth@server:$ *podman ps -a*
-elspeth@server:$ *podman logs testcontainer*
-hello, world
+$ *ssh [email protected]*
+Welcome to Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-67-generic x86_64)
+ [...]
+
+elspeth@server$ *docker ps -a*
+CONTAINER ID   IMAGE     COMMAND              CREATED      STATUS
+PORTS     NAMES
+3a2e600fbe77   busybox   "echo hello world"   2 days ago   Exited (0) 10
+minutes ago             testcontainer
+
+elspeth@server:$ *docker logs testcontainer*
+hello world
 ----
 
 TIP: Look out for that `elspeth@server`
     in the command-line listings in this chapter.
     It indicates commands that must be run on the server,
     as opposed to commands you run on your own PC.
 
+
 SSHing in to check things worked is a key server debugging skill!
 It's something we want to practice on our staging server,
 because ideally we'll want to avoid doing it on production machines.
@@ -419,12 +406,11 @@ and organisations will often run private ones,
 hosted by cloud providers like AWS.
 
 So your process of getting an image onto a server is usually
-* push the image from your machine to the registry
-* pull the image from the registry onto the server.
+* Push the image from your machine to the registry
+* Pull the image from the registry onto the server.
   Usually this step is implicit,
-    in that you just specifying the image name
-    in the format registry-url/image-name:tag,
-    and then `docker run` takes care of pulling down the image for you.
+  in that you just specifying the image name in the format registry-url/image-name:tag,
+  and then `docker run` takes care of pulling down the image for you.
 
 But I don't want to ask you to create a DockerHub account,
 or implicitly endorse any particular provider,
@@ -443,43 +429,132 @@ In ansible config, it looks like this:
 - hosts: all
 
   tasks:
-    - name: Install podman
+    - name: Install docker
       ansible.builtin.apt:
-        name: podman
-        update_cache: yes
+        name: docker
+        state: latest
       become: true
 
-    - name: Export container image locally
-      containers.podman.podman_save:
-        image: superlists
-        dest: /tmp/superlists-img.oci
-        format: oci-archive
-        force: true
+    - name: Export container image locally  #<1>
+      community.docker.docker_image:
+        name: superlists
+        archive_path: /tmp/superlists-img.tar
+        source: local
       delegate_to: 127.0.0.1
 
-    - name: Upload image to server
+    - name: Upload image to server  #<2>
       ansible.builtin.copy:
-        src: /tmp/superlists-img.oci
-        dest: /tmp/superlists-img.oci
+        src: /tmp/superlists-img.tar
+        dest: /tmp/superlists-img.tar
 
-    - name: Import container image on server
-      containers.podman.podman_load:
-        input: /tmp/superlists-img.oci
+    - name: Import container image on server  #<3>
+      community.docker.docker_image:
+        name: superlists
+        load_path: /tmp/superlists-img.tar
+        source: load
+        state: present
+      become: true
 
     - name: Run container
-      containers.podman.podman_container:
+      community.docker.docker_container:
         name: superlists
         image: superlists
         state: started
         recreate: true
 ----
 ====
 
+<1> We export the docker image to a `.tar` file by using the `docker_image` module
+  with the `archive_path` set to temp file, and setting the `delegate_to` attribute
+  to say we're running that command on our local machine rather than the server.
 
-Let's see if that worked!
+<2> We then use the `copy` module to upload the tarfile to the server
 
+<3> And we use `docker_image` again but this time with `load_path` and `source: load`
+  to import the image back on the server
+
+[subs="specialcharacters,quotes"]
+----
+$ *ansible-playbook --user=elspeth -i staging.ottg.co.uk, infra/ansible-provision.yaml -vv*
+[...]
+
+PLAYBOOK: ansible-provision.yaml **********************************************
+1 plays in infra/ansible-provision.yaml
+
+PLAY [all] ********************************************************************
+
+TASK [Gathering Facts] ********************************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:2
+ok: [staging.ottg.co.uk]
+
+TASK [Install docker] *********************************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:5
+ok: [staging.ottg.co.uk] => {"cache_update_time": 1708982855, "cache_updated": false, "changed": false}
+
+TASK [Export container image locally] *****************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:11
+changed: [staging.ottg.co.uk -> 127.0.0.1] => {"actions": ["Archived image
+superlists:latest to /tmp/superlists-img.tar, overwriting archive with image
+11ff3b83873f0fea93f8ed01bb4bf8b3a02afa15637ce45d71eca1fe98beab34 named
+superlists:latest"], "changed": true, "image": {"Architecture": "amd64",
+[...]
+
+TASK [Upload image to server] *************************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:18
+changed: [staging.ottg.co.uk] => {"changed": true, "checksum":
+"313602fc0c056c9255eec52e38283522745b612c", "dest": "/tmp/superlists-img.tar",
+[...]
+
+TASK [Import container image on server] ***************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:23
+changed: [staging.ottg.co.uk] => {"actions": ["Loaded image superlists:latest
+from /tmp/superlists-img.tar"], "changed": true, "image": {"Architecture":
+"amd64", "Author": "", "Comment": "buildkit.dockerfile.v0", "Config":
+[...]
+
+TASK [Run container] **********************************************************
+task path: ...goat-book/superlists/infra/ansible-provision.yaml:32
+changed: [staging.ottg.co.uk] => {"changed": true, "container":
+{"AppArmorProfile": "docker-default", "Args": ["--bind", ":8888",
+"superlists.wsgi:application"], "Config": {"AttachStderr": true, "AttachStdin":
+false, "AttachStdout": true, "Cmd": ["gunicorn", "--bind", ":8888",
+"superlists.wsgi:application"], "Domainname": "", "Entrypoint": null, "Env":
+[...]
+----
+
+TODO: sort out macos M1/arch issues, `docker build --platform linux/amd64` etc.
+
+Looks ok! Let's see if that worked?
+
+[subs="specialcharacters,quotes"]
+----
+$ *ssh [email protected]*
+Welcome to Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-67-generic x86_64)
+ [...]
+
+elspeth@server$ *docker ps -a*
+CONTAINER ID   IMAGE     COMMAND              CREATED      STATUS
+PORTS     NAMES
+3a2e600fbe77   busybox   "echo hello world"   2 days ago   Exited (0) 10
+minutes ago             testcontainer
+
+elspeth@server:$ *docker logs testcontainer*
+[2024-02-26 22:19:15 +0000] [1] [INFO] Starting gunicorn 21.2.0
+[2024-02-26 22:19:15 +0000] [1] [INFO] Listening at: http://0.0.0.0:8888 (1)
+[2024-02-26 22:19:15 +0000] [1] [INFO] Using worker: sync
+[...]
+  File "/src/superlists/settings.py", line 22, in <module>
+    SECRET_KEY = os.environ["DJANGO_SECRET_KEY"]
+                 ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
+  File "<frozen os>", line 685, in __getitem__
+KeyError: 'DJANGO_SECRET_KEY'
+[2024-02-26 22:19:15 +0000] [7] [INFO] Worker exiting (pid: 7)
+[2024-02-26 22:19:15 +0000] [1] [ERROR] Worker (pid:7) exited with code 3
+[2024-02-26 22:19:15 +0000] [1] [ERROR] Shutting down: Master
+[2024-02-26 22:19:15 +0000] [1] [ERROR] Reason: Worker failed to boot.
+----
 
-Nope.  TODO: debug, ssh into the server, show missing env vars.
+Ah woops, we need to set those environment variables on the server too.
 
 
 === Using an env File to Store Our Environment Variables