Rules for ctypes

Ctypes is the recommended way to write C stubs for OCaml, Jbuilder have builtin support for it. (related to #131).

[g2p]

I'd be interested in support for ctypes.foreign.
Listing non-stub C sources in c_names doesn't appear to do the trick.
Is there some linker magic I'm missing?

(Edit: yes I was. I needed -rdynamic in c_library_flags, and an external declaration to make sure the C function doesn't get eliminated as dead code)

[XVilka]

I tried to use the rules like:

(jbuild_version 1)
(library
  ((name mylib)
  (public_name mylib)
  (libaries (core ctypes ctypes.foreign))
  (c_library_flags (:standard -lz -lm -llzma -pthread))
  (library_flags (-cclib -Wl,-E))
  (self_build_stubs_archive (libmylib))
))
(rule
((targets (libmylib_stubs.a dllmylib_stubs.so))
  (deps ((files_recursively_in mylib)))
  (action (bash "
  ${MAKE} -s -C mylib clean
  $(MAKE} -s -C mylib libmylib.a
  $(MAKE} -s -C mylib libmylib.so
  cp mylib/libmylib.a libmylib_stubs.a
  cp mylib/libmylib.so dllmylib_stubs.so
  ${MAKE} -s -C mylib clean
"))))

But then when I use this library in any kind of application, it links but doesn't link the symbols from this "libmylib.{a,so}", raising the error when running:

Uncaught exception:
   Dl.DL_error("_build/default/myapp.exe: undefined symbol: "myfunction1")

To be honest, I have no idea why this is happens and why jbuilder can't handle this case.

Try (self_... (mylib)). This is what https://github.com/janestreet/re2/blob/master/src/re2_c/jbuild does and it works

[aantron]

I have the same problem as @XVilka.

The re2 workaround didn't work for me, and I didn't expect it to. re2 has a separate build script for library archive files. In my case, Jbuilder was already correctly building the C library. Adding (self...) just made Jbuilder report that it failed to find rules for building a file that it otherwise successfully built (libfoo_stubs.a). I also confirmed with nm and dlopen/dlsym that the stubs archive being built by Jbuilder does indeed export the symbol that could not be found.

Everything seems to work if I add a dummy external that depends from OCaml on a symbol from the stubs library, however.

It looks like Jbuilder+Ctypes is broken.

Also, to try to find more examples, I tried listing opam packages that depend on both ctypes-foreign and jbuilder, but found none.

[XVilka]

Just for the record, following jbuild file worked for me:

(jbuild_version 1)
(library
  ((name newlib)
   (public_name newlib)
   (wrapped false)
   (libraries (core ctypes ctypes.foreign))
   (c_library_flags (:standard -Wl,--no-as-needed -lz -lm -llzma -pthread))
   (library_flags ((-cclib -Wl,-E) (-cclib -Wl,--no-as-needed) (lib/libmylib_stubs.a)))
   (self_build_stubs_archive (libmylib))
   ))
(rule
((targets (libmylib_stubs.a dllmylib_stubs.so))
	(deps ((files_recursively_in mylib)))
	(action (bash "
	${MAKE} -s -C mylib clean
	${MAKE} -s -C mylib libmylib.a
	${MAKE} -s -C mylib libmylib.so
	cp mylib/liblibmylib.a libmylib_stubs.a
	cp mylib/libmylib.so dllmylib_stubs.so
	${MAKE} -s -C mylib clean
	"))))

It's possible that there is an issue with ctypes-foreign. It works fine with static stub generation though, async_ssl does this for instance.

[rgrinberg]

@diml given your recent change of approach to supporting external tools (implement a satisfying approach inside dune, generalize to plugins later). Do you think we can have good support for ctypes out of the box as well?

Absolutely

[paracetamolo]

Any news on the support in dune for ctypes?

Nothing has been done so far, but we are planning to look at this with @yallop

[vinczedani]

Any news on the support in dune for ctypes?

[yallop]

I think progress on ctypes support may be blocked by my lack of time at the moment. I can't really look at this directly at the moment, but I'm happy to answer questions, or provide feedback, if someone else wants to make a start.

Perhaps the dune/ctypes rules used in @avsm's yaml bindings would be a useful starting ponit.

[rgrinberg]

I'll also recommend the async_ssl bindings.

@vinczedani do you have any particular pain points when using ctypes/dune? We rely on user input to make improvements. AFAIK, ctypes + jbuilder work fine if you're using the stub generation, it's just a little boilerplate heavy.

@aantron would you happen to have a reproducible test case for us?

[aantron]

@rgrinberg Not at this point, it's out of my cache and I won't have time to recreate it soon.

[XVilka]

Just met the problem with manual workarounds - there is no easy way to compile ctypes-generated bindings code statically - C stubs still dynamically load libc, and binary needs libffi.

See also:

• Static linking ? yallop/ocaml-ctypes#41
• Static linking under OSX yallop/ocaml-ctypes#96