Upgrade cosmopolitan 4.0.2 by dinosaure · Pull Request #52 · dinosaure/esperanto

dinosaure · 2025-06-18T19:48:04Z

And OCaml 5.3.0

…patch'

dinosaure · 2025-06-18T20:54:21Z

HI @shym, I actually try to reproduce the support of Cosmopolitan with OCaml 5.3.0 and took what you did about ocaml-solo5. Actually, I get a weird issue which is: patches are not really applied into the ocaml distribution (they are mainly skipped) and the OCaml compiler can not be compiled with Cosmopolitan.

This is the output:

# if test -d "patches/$VERSION" ; then \
#   git apply -v --unsafe-paths --directory=ocaml "patches/$VERSION"/*; \
# fi
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/aclocal.m4'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/Makefile'.
# Skipped patch 'ocaml/Makefile.build_config.in'.
# Skipped patch 'ocaml/aclocal.m4'.
# Skipped patch 'ocaml/build-aux/ax_prog_cc_for_build.m4'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/Makefile'.
# Skipped patch 'ocaml/Makefile.config.in'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/Makefile'.
# Skipped patch 'ocaml/Makefile.cross'.
# Skipped patch 'ocaml/Makefile.cross'.
# Skipped patch 'ocaml/Makefile.cross'.
# Skipped patch 'ocaml/runtime/debugger.c'.
# Skipped patch 'ocaml/runtime/amd64.S'.
# Skipped patch 'ocaml/runtime/arm64.S'.
# Skipped patch 'ocaml/runtime/power.S'.
# Skipped patch 'ocaml/runtime/riscv.S'.
# Skipped patch 'ocaml/runtime/s390x.S'.
# Skipped patch 'ocaml/aclocal.m4'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/runtime/caml/m.h.in'.
# Skipped patch 'ocaml/utils/config.common.ml.in'.
# Skipped patch 'ocaml/utils/config.fixed.ml'.
# Skipped patch 'ocaml/utils/config.generated.ml.in'.
# Skipped patch 'ocaml/utils/config.mli'.
# Skipped patch 'ocaml/asmcomp/amd64/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/arm64/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/emitaux.ml'.
# Skipped patch 'ocaml/asmcomp/emitaux.mli'.
# Skipped patch 'ocaml/asmcomp/power/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/riscv/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/s390x/emit.mlp'.
# Skipped patch 'ocaml/runtime/amd64.S'.
# Skipped patch 'ocaml/runtime/arm64.S'.
# Skipped patch 'ocaml/runtime/caml/asm.h'.
# Skipped patch 'ocaml/runtime/power.S'.
# Skipped patch 'ocaml/runtime/riscv.S'.
# Skipped patch 'ocaml/runtime/s390x.S'.
# Skipped patch 'ocaml/testsuite/tools/asmgen_arm.S'.
# Skipped patch 'ocaml/testsuite/tools/asmgen_arm64.S'.
# Skipped patch 'ocaml/testsuite/tools/asmgen_power.S'.
# Skipped patch 'ocaml/testsuite/tools/asmgen_s390x.S'.
# Skipped patch 'ocaml/aclocal.m4'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/runtime/caml/m.h.in'.
# Skipped patch 'ocaml/utils/config.common.ml.in'.
# Skipped patch 'ocaml/utils/config.fixed.ml'.
# Skipped patch 'ocaml/utils/config.generated.ml.in'.
# Skipped patch 'ocaml/utils/config.mli'.
# Skipped patch 'ocaml/asmcomp/amd64/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/arm64/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/emitaux.ml'.
# Skipped patch 'ocaml/asmcomp/emitaux.mli'.
# Skipped patch 'ocaml/asmcomp/power/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/riscv/emit.mlp'.
# Skipped patch 'ocaml/asmcomp/s390x/emit.mlp'.
# Skipped patch 'ocaml/runtime/amd64.S'.
# Skipped patch 'ocaml/runtime/arm64.S'.
# Skipped patch 'ocaml/runtime/caml/asm.h'.
# Skipped patch 'ocaml/runtime/power.S'.
# Skipped patch 'ocaml/runtime/riscv.S'.
# Skipped patch 'ocaml/runtime/s390x.S'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/configure'.
# Skipped patch 'ocaml/configure.ac'.
# Skipped patch 'ocaml/runtime/caml/domain.h'.
# Checking patch _opam/.opam-switch/build/aarch64-esperanto.dev/caml/ocaml/runtime/globroots.c...
# Applied patch _opam/.opam-switch/build/aarch64-esperanto.dev/caml/ocaml/runtime/globroots.c cleanly.
# Skipped patch 'ocaml/runtime/signals.c'.
# Skipped patch 'ocaml/runtime/sys.c'.
# Skipped patch 'ocaml/otherlibs/unix/termios.c'.
# Skipped patch 'ocaml/otherlibs/unix/access.c'.
# Skipped patch 'ocaml/otherlibs/unix/open_unix.c'.
# Skipped patch 'ocaml/otherlibs/unix/socket_unix.c'.
# Skipped patch 'ocaml/otherlibs/unix/sockopt_unix.c'.
# Skipped patch 'ocaml/otherlibs/unix/unixsupport_unix.c'.
# Skipped patch 'ocaml/otherlibs/unix/wait.c'.
# Skipped patch 'ocaml/otherlibs/unix/socketaddr.c'.
# Skipped patch 'ocaml/asmcomp/arm64/proc.ml'.
# PATH="$PWD/_build/build-toolchain:$PATH" ; \
# cd ocaml && \
#   ./configure \
# 	--target="aarch64-esperanto-ocaml" \
# 	--prefix="/home/runner/work/esperanto/esperanto/_opam/lib/aarch64-esperanto" \
# 	--disable-shared \
# 	--disable-systhreads \
# 	--disable-instrumented-runtime \
# 	--disable-debug-runtime \
# 	--disable-ocamltest \
# 	--disable-ocamldoc \
# 	--without-zstd \
# 	
# configure: Configuring OCaml version 5.3.0
# checking build system type... x86_64-pc-linux-gnu
# checking host system type... x86_64-pc-linux-gnu
# checking target system type... Invalid configuration 'aarch64-esperanto-ocaml': OS 'ocaml' not recognized
# configure: error: /bin/bash ./build-aux/config.sub aarch64-esperanto-ocaml failed

I don't really understand why we can not apply patches. I tried to use --unsafe-paths which is advertised by git apply if we would like to apply patches regardless a Git repository. Do you have any ideas why it fails? It will be superb to, at least, unlock Cosmopolitan for OCaml 5.3.

shym · 2025-06-21T19:18:22Z

Hi @dinosaure,
I finally understood. When running in a git repository, git apply does refer to the repository root to interpret the patches. So here we go, with a variant that should work both in a git clone and in an archive to publish it on the opam repository, I think.

diff --git i/caml/Makefile w/caml/Makefile
index 97dc862..c3dba6a 100644
--- i/caml/Makefile
+++ w/caml/Makefile
@@ -68,7 +68,7 @@ ocaml:
        cp -r "$$(ocamlfind query ocaml-src)" $@
        VERSION="$$(head -n1 ocaml/VERSION)" ; \
        if test -d "patches/$$VERSION" ; then \
-         git apply -v --unsafe-paths --directory=$@ "patches/$$VERSION"/*; \
+         cd .. && git apply --directory=caml/$@ "caml/patches/$$VERSION"/*; \
        fi

 ocaml/Makefile.config: $(LIBS) $(TOOLCHAIN_FOR_BUILD) | ocaml

(I admit that I stopped at getting that rule to go through, I didn’t try to build the whole project)

dinosaure · 2025-06-24T10:55:12Z

Hmmhmm, it seems that it does not work on the CI again 😕

shym · 2025-07-05T09:14:09Z

No idea came to mind on my end either.
I think I would try to apply patches one by one to see if this provides any clue:

	  cd .. && \
	  for p in "caml/patches/$$VERSION"/*; do \
	    (set -x; git apply --unsafe-path -v --directory=caml/$@ "$$p"; echo $$?) ; \
	  done ; \

and maybe run a check that the content of caml/ocaml is indeed what we hope if that first test doesn’t bring any hint.

dinosaure · 2025-07-11T11:05:27Z

@shym You might be interested in the result (and we should perhaps think carefully about this with regard to ocaml-solo5), but it seems that git apply does some black magic with the .git/index file implicitly. During my research, it is often mentioned that git apply attempts to correlate patches with what is in the repository. In this case, the project is not a Git repository, and therefore git apply cannot correlate the patches with what should be present.

The solution is to initialise a Git repository (with git init - see fb19ccd) so that there is a .git/index file that can track the files and folders. I also think that the fact that ocaml is in a subfolder called caml (unlike ocaml-solo5) plays a role in how git apply interprets the patches (since caml can be interpreted as an "empty folder" by Git and therefore not be recognized).

In short... this raises a lot of questions about the use of git apply and whether we should continue to use it. It might be worth considering using patch instead of git apply.

dinosaure · 2025-09-03T12:48:52Z

Hello @jart, it seems we are reaching the stage where we can produce an OCaml compiler (5.3.0) that uses Cosmopolitan. However, it seems that all syscalls end with ESPIPE.

You can test the project with opam:

$ opam switch create 5.3.0
$ git clone https://github.com/dinosaure/esperanto.git
$ git checkout upgrade-cosmopolitan-4.0.2
$ cd esperanto
$ opam pin add -y .
$ cd example/
$ dune build
$ ./link.sh
$ ./cat.com --strace
^D

I'm not sure I understand why all syscalls emit ESPIPE (perhaps another registry issue that OCaml and Cosmopolitan use?). I would appreciate some help with this part (I am also available via private message if you would like to discuss it). Thanks!

dinosaure · 2025-12-29T13:17:20Z

My last two commits fix the initial issue about our basic program cat.com:

it seems that cosmopolitan is more stricter (POSIX) and lseek fails on TTY file-descriptors (according to POSIX) but OCam tries to lseek regardless it's a file or a TTY. A patch clear errno and ignore the error.
it seems that we have a deadlock. I don't know precisely the assumption that the OCaml runtime have about mutex but if we initialize mutexes with RECURSIVE, it solves the issue. I think the initial problem is a bit more complex than that but let's move on.

dinosaure · 2026-02-17T19:28:56Z

Ok, I think that the last patch is a bug on cosmopolitan. I need to do some reproducible examples but it also seems that the deadlock part was not fundamentally changed. I will merge this PR. I will also try to move on a 5.4 and delete (as we did for ocaml-solo5) the use of git apply.

dinosaure and others added 16 commits June 18, 2025 15:21

Upgrade to Cosmopolitan 4.0.2

f60aac6

Release tarball for v0.0.6-3-gf60aac6-dirty

b4dd75a

Fix the digest file of the Cosmopolitan distribution

3a1455d

Upgrade Esperanto with OCaml 5.3

c96d04b

Lint OPAM files

503f38a

Be able to create the _build directory for when we build OCaml

47b53a0

Remove the verbose mode when we do git apply

cba58f1

Fix the Makefile to be able to install our toolchain

a16ffd2

Update the CI with the new version of esperanto

2c0b2fd

Allow LFS when we clone our GitHub repository

0ea1139

Add opam-installer as a dependency of esperanto

effb6ee

Verbose when we apply patches on OCaml

a0fec7d

Use --unsafe-paths when we apply patch as git-apply is a 'better GNU …

2fffbb5

…patch'

Upgrade checkout action to v4

a7f2c77

Force to use the upstream version of OCaml 5.3.0

ad84e1e

Update how we fix the execution of APE program

5502bcb

dinosaure added 6 commits June 23, 2025 11:05

Fix git apply to be to able to compile OCaml (cc @shym)

726c1f2

Delete --unsafe-path when we invoke git-apply

1a9cc60

Fix how we handle errno codes

45269ec

Fix how we handle signal values

647627b

Use the cache when we would like to checkout the code into our CI

bd50b5b

Re-introduce --unsafe-path when we would like to apply patches on OCaml

a1f9a40

dinosaure added 3 commits July 11, 2025 12:06

Apply Git patches one by one and see the result

2075933

Add undocumented --no-index option when we apply patch to OCaml

df8b797

Initiate a git repo to be able to use git apply

fb19ccd

dinosaure mentioned this pull request Jul 11, 2025

Use patch instead of git apply mirage/ocaml-solo5#151

Closed

dinosaure mentioned this pull request Jul 24, 2025

Support ocaml 5.0.0 #31

Open

dinosaure mentioned this pull request Sep 19, 2025

Show up for logs how we apply patches when we install ocaml-solo5 via opam mirage/ocaml-solo5#153

Closed

dinosaure added 2 commits December 29, 2025 13:26

Ignore and clean errno when we lseek on stdin

d4909a8

Set mutex to be recursive by default

dc6c9bb

dinosaure merged commit 4deab38 into main Feb 17, 2026
2 checks passed

dinosaure deleted the upgrade-cosmopolitan-4.0.2 branch February 17, 2026 19:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade cosmopolitan 4.0.2#52

Upgrade cosmopolitan 4.0.2#52
dinosaure merged 27 commits intomainfrom
upgrade-cosmopolitan-4.0.2

dinosaure commented Jun 18, 2025

Uh oh!

dinosaure commented Jun 18, 2025

Uh oh!

shym commented Jun 21, 2025 •

edited

Loading

Uh oh!

dinosaure commented Jun 24, 2025

Uh oh!

shym commented Jul 5, 2025

Uh oh!

dinosaure commented Jul 11, 2025

Uh oh!

dinosaure commented Sep 3, 2025

Uh oh!

dinosaure commented Dec 29, 2025

Uh oh!

dinosaure commented Feb 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dinosaure commented Jun 18, 2025

Uh oh!

dinosaure commented Jun 18, 2025

Uh oh!

shym commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dinosaure commented Jun 24, 2025

Uh oh!

shym commented Jul 5, 2025

Uh oh!

dinosaure commented Jul 11, 2025

Uh oh!

dinosaure commented Sep 3, 2025

Uh oh!

dinosaure commented Dec 29, 2025

Uh oh!

dinosaure commented Feb 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shym commented Jun 21, 2025 •

edited

Loading