Docker build fails on Apple Silicon Mac

[zhaoyig]

Running make system_<system_name> on my M1 Macbook fails with following:

docker build -t effecthandlers/effect-handlers:ocaml systems/ocaml
[+] Building 1.7s (12/20)                                                                       docker:desktop-linux
 => [internal] load .dockerignore                                                                               0.0s
 => => transferring context: 2B                                                                                 0.0s
 => [internal] load build definition from Dockerfile                                                            0.0s
 => => transferring dockerfile: 1.33kB                                                                          0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                 1.4s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                   0.0s
 => [ 1/16] FROM docker.io/library/ubuntu:22.04@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e49  0.0s
 => CACHED [ 2/16] RUN apt-get update &&     apt-get -y upgrade &&     apt-get install -y     sudo              0.0s
 => CACHED [ 3/16] RUN useradd -ms /bin/bash ubuntu &&     usermod -aG sudo ubuntu                              0.0s
 => CACHED [ 4/16] RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers                                      0.0s
 => CACHED [ 5/16] WORKDIR /home/ubuntu                                                                         0.0s
 => CACHED [ 6/16] RUN sudo apt install -y wget                                                                 0.0s
 => CACHED [ 7/16] RUN wget https://github.com/sharkdp/hyperfine/releases/download/v1.15.0/hyperfine_1.15.0_am  0.0s
 => ERROR [ 8/16] RUN sudo dpkg -i hyperfine_1.15.0_amd64.deb                                                   0.2s
------
 > [ 8/16] RUN sudo dpkg -i hyperfine_1.15.0_amd64.deb:
0.194 dpkg: error processing archive hyperfine_1.15.0_amd64.deb (--install):
0.194  package architecture (amd64) does not match system (arm64)
0.200 Errors were encountered while processing:
0.200  hyperfine_1.15.0_amd64.deb
------
Dockerfile:25
--------------------
  23 |     RUN sudo apt install -y wget
  24 |     RUN wget https://github.com/sharkdp/hyperfine/releases/download/v1.15.0/hyperfine_1.15.0_amd64.deb
  25 | >>> RUN sudo dpkg -i hyperfine_1.15.0_amd64.deb
  26 |
  27 |     # Install Multicore OCaml
--------------------
ERROR: failed to solve: process "/bin/sh -c sudo dpkg -i hyperfine_1.15.0_amd64.deb" did not complete successfully: exit code: 1
make: *** [system_ocaml] Error 1

Changing the amd64 to arm64 in following lines in Dockerfiles solves the problem.

RUN wget https://github.com/sharkdp/hyperfine/releases/download/v1.15.0/hyperfine_1.15.0_amd64.deb
RUN sudo dpkg -i hyperfine_1.15.0_amd64.deb

[jasigal]

Hi @zhaoyig, thanks for the issue! I don't have an ARM machine to test on, can you please try the branch 53-docker-build-fails-on-apple-silicon-mac and let me know if it works? I added --platform linux/amd64 to all Docker commands, which is supposed to fix your issue.

If it does work and you have time, can you run some benchmarks and let me know if my fix impacts performance compared to yours? If so, I'll try to figure out how to have ARM builds of the Docker containers.

[zhaoyig]

@jasigal Thanks for the fix, but this branch still fails on ocaml and hia. Other systems (koka, eff, effekt) works fine, I will test the performance of those systems later.

Error message for ocaml:
(BTW, I believe the reason for this error is unrelated to this repository because when i run opam switch create -y 4.12.0+domains+effects directly on my machine it gives some C compilation error and it seems to be caused by M1 chips.)

ERROR [14/16] RUN opam switch create -y 4.12.0+domains+effects                                                                                   358.4s 
------                                                                                                                                                      
 > [14/16] RUN opam switch create -y 4.12.0+domains+effects:                                                                                                
1.699                                                                                                                                                       
1.699 <><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>                                                                      
1.699 Switch invariant: ["ocaml-variants" {= "4.12.0+domains+effects"}]                                                                                     
36.38                                                                                                                                                       
36.38 <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
38.07 -> installed base-bigarray.base
38.07 -> installed base-threads.base
38.09 -> installed base-unix.base
38.09 -> installed ocaml-option-nnp.1
113.7 -> retrieved ocaml-variants.4.12.0+domains+effects  (git+https://github.com/ocaml-multicore/ocaml-multicore.git#4.12+domains+effects)
352.3 [ERROR] The compilation of ocaml-variants.4.12.0+domains+effects failed at "make -j7".
352.4 
352.4 #=== ERROR while compiling ocaml-variants.4.12.0+domains+effects ==============#
352.4 # context              2.1.5 | linux/x86_64 |  | https://opam.ocaml.org#9b339ea0
352.4 # path                 ~/.opam/4.12.0+domains+effects/.opam-switch/build/ocaml-variants.4.12.0+domains+effects
352.4 # command              /usr/bin/make -j7
352.4 # exit-code            2
352.4 # env-file             ~/.opam/log/ocaml-variants-8-c9117f.env
352.4 # output-file          ~/.opam/log/ocaml-variants-8-c9117f.out
352.4 ### output ###
352.4 # [...]
352.4 # echo '#!/home/ubuntu/.opam/4.12.0+domains+effects/bin/ocamlrund' > target_camlheaderd
352.4 # echo '#!/home/ubuntu/.opam/4.12.0+domains+effects/bin/ocamlruni' > camlheaderi
352.4 # echo '#!/home/ubuntu/.opam/4.12.0+domains+effects/bin/ocamlruni' > target_camlheaderi
352.4 # ../boot/ocamlrun ../boot/ocamlc -use-prims ../runtime/primitives -strict-sequence -absname -w +a-4-9-41-42-44-45-48 -g -warn-error A -bin-annot -nostdlib -principal -safe-string -strict-formats  -nopervasives -c camlinternalAtomic.ml
352.4 # ../boot/ocamlrun ../boot/ocamlc -use-prims ../runtime/primitives -strict-sequence -absname -w +a-4-9-41-42-44-45-48 -g -warn-error A -bin-annot -nostdlib -principal -safe-string -strict-formats  -nopervasives -c camlinternalFormatBasics.ml
352.4 # ../boot/ocamlrun ../boot/ocamlc -use-prims ../runtime/primitives -strict-sequence -absname -w +a-4-9-41-42-44-45-48 -g -warn-error A -bin-annot -nostdlib -principal -safe-string -strict-formats  -nopervasives -no-alias-deps -w -49  -pp "$AWK -f ./expand_module_aliases.awk" -c stdlib.mli
352.4 # make[2]: *** [Makefile:207: camlinternalAtomic.cmo] Killed
352.4 # make[2]: *** Waiting for unfinished jobs....
352.4 # make[2]: Leaving directory '/home/ubuntu/.opam/4.12.0+domains+effects/.opam-switch/build/ocaml-variants.4.12.0+domains+effects/stdlib'
352.4 # make[1]: *** [Makefile:144: coldstart] Error 2
352.4 # make[1]: Leaving directory '/home/ubuntu/.opam/4.12.0+domains+effects/.opam-switch/build/ocaml-variants.4.12.0+domains+effects'
352.4 # make: *** [Makefile:275: world.opt] Error 2
352.4 
353.0 
353.0 
353.0 <><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
353.0 +- The following actions failed
353.0 | - build ocaml-variants 4.12.0+domains+effects
353.0 +- 
353.0 +- The following changes have been performed (the rest was aborted)
353.0 | - install base-bigarray    base
353.0 | - install base-threads     base
353.0 | - install base-unix        base
353.0 | - install ocaml-option-nnp 1
353.0 +- 
353.1 # Run eval $(opam env --switch=4.12.0+domains+effects) to update the current shell environment
353.1 Switch initialisation failed: clean up? ('n' will leave the switch partially installed) [Y/n] y
------
Dockerfile:35
--------------------
  33 |     RUN echo "test -r /home/ubuntu/.opam/opam-init/init.sh && . /home/ubuntu/.opam/opam-init/init.sh > /dev/null 2> /dev/null || true" >> /home/ubuntu/.profile
  34 |     
  35 | >>> RUN opam switch create -y 4.12.0+domains+effects
  36 |     RUN eval $(opam env)
  37 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c opam switch create -y 4.12.0+domains+effects" did not complete successfully: exit code: 31
make: *** [system_ocaml] Error 1

Also, for HIA the error message is:

#18 [14/18] RUN ghcup install ghc 8.6.5
#18 10.33 [ Warn  ] New GHCup version available: 0.1.19.0. To upgrade, run 'ghcup upgrade'
#18 10.34 [ Info  ] downloading: https://downloads.haskell.org/~ghc/8.6.5/ghc-8.6.5-x86_64-fedora27-linux.tar.xz
#18 10.45   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
#18 10.45                                  Dload  Upload   Total   Spent    Left  Speed
100  180M  100  180M    0     0  19.9M      0  0:00:09  0:00:09 --:--:-- 20.7M
#18 19.54 [ Info  ] verifying digest of: ghc-8.6.5-x86_64-fedora27-linux.tar.xz
#18 21.69 [ Info  ] Unpacking: ghc-8.6.5-x86_64-fedora27-linux.tar.xz to /tmp/ghcup-4a55bf7d324eb814
#18 72.81 [ Info  ] Installing GHC (this may take a while)
#18 110.5 Killed
#18 ERROR: process "/bin/sh -c ghcup install ghc 8.6.5" did not complete successfully: exit code: 137
------
 > [14/18] RUN ghcup install ghc 8.6.5:
10.33 [ Warn  ] New GHCup version available: 0.1.19.0. To upgrade, run 'ghcup upgrade'
10.34 [ Info  ] downloading: https://downloads.haskell.org/~ghc/8.6.5/ghc-8.6.5-x86_64-fedora27-linux.tar.xz
100  180M  100  180M    0     0  19.9M      0  0:00:09  0:00:09 --:--:-- 20.7M
19.54 [ Info  ] verifying digest of: ghc-8.6.5-x86_64-fedora27-linux.tar.xz
21.69 [ Info  ] Unpacking: ghc-8.6.5-x86_64-fedora27-linux.tar.xz to /tmp/ghcup-4a55bf7d324eb814
72.81 [ Info  ] Installing GHC (this may take a while)
110.5 Killed
------
Dockerfile:34
--------------------
  32 |     RUN sudo chmod +x /home/ubuntu/bin/ghcup
  33 |     ENV PATH /home/ubuntu/bin:/home/ubuntu/.ghcup/bin:$PATH
  34 | >>> RUN ghcup install ghc 8.6.5
  35 |     RUN ghcup set ghc 8.6.5
  36 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c ghcup install ghc 8.6.5" did not complete successfully: exit code: 137
make: *** [system_hia] Error 1

No clue why GHC can't be installed.

[jasigal]

Thanks for the update. I may be able to borrow an ARM Mac in the future. In the meantime, I'll try and see if I can understand Docker and the Dockerfiles well enough to make ARM versions.

[jasigal]

I haven't been able to get an ARM Mac, but I do have access to a Raspberry Pi 4. Hopefully it will run into the same issues so I can reproduce them :)