meson2hermetic: the world's premiere build system converter #15462

gurchetansingh · 2026-01-12T22:09:24Z

This API converts meson.build into their corresponding hermetic representation (Android Blueprint, Bazel and maybe Buck2 in the future).

This is motivated by the need to integrate Mesa3D into AOSP and Fuchsia trees.

The question "how to build and update Mesa3D drivers for Android?" [1] in particular has led to several methods over the years, none of which used Android's native build system (Soong). This has been an obstacle to adoption of open-source drivers, which everyone knows are more secure, maintainable and faster than closed alternatives.

By integrating into Mesa3D's native build system (Meson), "meson convert" brings shocking and jaw-dropping clarity to the question.

Technically speaking, the tool works via a series of TOML files. Python 3.11 has tomlib in the standard library, and mconvert.py uses a conditional import strategy to prevent issues on older Python versions.

These TOML files specify:

the Meson project that is being converted
where to find the dependencies in a hermetic tree
which compilers that a hermetic tree supports

These TOML files are used to run the Meson interpreter multiple times. For example, the set of C/C++ flags may be different if the compiler targets x86_64 or ARM64. Data from each of of the meson interpreter runs is collated to reconstruct the full set of Soong/Bazel rules.

Although the initial implementation is focused with converting to a existing hermetic build system, this introduces infrastructure that could be useful if Meson itself takes a look at remote-executed, hermetic builds. For example, Meson can download prebuilts from NixPkgs.

[1] https://gitlab.freedesktop.org/mesa/mesa/-/issues/13776

eli-schwartz · 2026-01-12T22:21:25Z

As relevant context, this tool was previously suggested by @gurchetansingh at #14134

/cc @dcbaker based on past discussions.

eli-schwartz · 2026-01-12T22:36:20Z

I haven't yet had an opportunity to take a close look at this. I see that it adds two new commands, one being the command to actually generate the hermetic buildsystem e.g. bazel, and another "check-toolchain" command that appears to be similar in spirit to the existing env2mfile command.

I'd like to hear a high-level explanation for what this tool does, to help better understand it. (And in particular, what are the challenges that make using machine files directly, not sufficient for your purposes? It looks like maybe the main differentiator is a lack of support for specifying the results of cc.has_* checks, but if that's the case then I'd like to implement that for machine files anyway...)

gurchetansingh · 2026-01-12T23:21:06Z

For reference, you can run the tool given the directions at:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39279

Right now, only mesa3d is supported.

I'd like to hear a high-level explanation for what this tool does, to help better understand it. (And in particular, what are the challenges that make using machine files directly, not sufficient for your purposes? It looks like maybe the main differentiator is a lack of support for specifying the results of cc.has_* checks, but if that's the case then I'd like to implement that for machine files anyway...)

One thing I like about .toml is the ability to use comments and have multiple tool-chains (I needed 6 tool-chains in the end to generate the right hermetic rules) in the same file, since IIUC is missing for the .ini files. Essentially, it seems to be the machine-files are just pared down versions of toml, which of course makes sense pre-Python 3.11. But with Python 3.11 on the horizon, using toml was very convenient and feature rich.

dcbaker · 2026-01-13T17:59:25Z

I will make a point to look at this in the near future.

As a bit of an aside: I'm stilling working on the ability to have meson definitions for LLVM, but I need to get more changes into Meson's dependency representation to be able to model that.

dcbaker · 2026-01-13T18:00:45Z

One thing I saw implementation wise from a quick glance, the new convert package has not been added to run_mypy.py, so we're not actually getting type checking. I'd really like to see that turned on for new code.

bonzini · 2026-01-14T15:40:30Z

@gurchetansingh please add stubs to docs/markdown/Commands.md so that CI passes.

To build the documentation you need to install hotdoc, then you can use

          cd docs
          ../meson.py setup _build
          ninja -C _build
          ninja -C _build test

gurchetansingh · 2026-01-15T18:29:37Z

As a bit of an aside: I'm stilling working on the ability to have meson definitions for LLVM, but I need to get more changes into Meson's dependency representation to be able to model that.

Nice. LLVM to hermetic is challenging as well like Mesa. Plus, I think many devs would enjoy an improvement over CMake.

convert package has not been added to run_mypy.py

Added: convert_interpreter.py is failing the CI though due to the current type checking config. That's understandable -- I added questionable code to pass run_mypy.py. BUT, when I remove questionable code, the script fails:

  File "meson/mesonbuild/convert/convertmain.py", line 26, in <module>
    from mesonbuild.convert.convert_interpreter import ConvertInterpreter
  File "meson/mesonbuild/convert/convert_interpreter.py", line 18, in <module>
    from mesonbuild.interpreter import kwargs
  File "meson/mesonbuild/interpreter/kwargs.py", line 15, in <module>
    from ..dependencies.base import Dependency, DependencyMethods, IncludeType
ImportError: cannot import name 'IncludeType' from 'mesonbuild.dependencies.base' (meson/mesonbuild/dependencies/bas

But should IncludeType only be required with T.TYPE_CHECKING? Could be related to recent refactors.

dcbaker · 2026-01-15T19:11:12Z

@gurchetansingh: interpreter/kwargs.py should only be imported inside a T.TYPE_CHECKING block. Or, at least it currently has that assumption.

dcbaker

I've started looking purely at the implementation details of this, but I haven't looked too much at the design yet. It's a lot of code and this is giving me a chance to try to get familiar with it.

I haven't looked at the soong implementation yet, and I'm about half way through the instance code of the third patch. I still need to get to reference/ and to top level convert/ file and mconvert.py

One thing I'd really like to have is some more comments, particularly docstrings to help explain how all of the code fits together, and what each thing is for, there's a lot of code here and it would really help me to review it (and for us to maintain it).

mesonbuild/mchecktoolchain.py

dcbaker · 2026-01-15T19:18:41Z

mesonbuild/mchecktoolchain.py

+    parser.add_argument("-o", "--output", default=None, help="Output file name.")
+
+
+def run(options: argparse.Namespace) -> int:


We do have some uses of Protocol in these front end functions to help with accurate typing vs using Namespace, I'm not sure how practical that is in this case.

left as is since most other commands use namespace

mesonbuild/check_toolchain/android.py

dcbaker · 2026-01-15T19:23:18Z

mesonbuild/convert/abstract/abstract_toolchain.py

+class AbstractCompiler(Compiler):
+    def __init__(self, conf: T.Dict[T.Any, T.Any], *args: T.Any, **kwargs: T.Any):
+        self.conf = conf
+        super().__init__(*args, **kwargs)


I'm not crazy about the heavy use of Any here, but I need to think more about whether there's another good option.

dcbaker · 2026-01-15T19:26:28Z

mesonbuild/convert/abstract/abstract_toolchain.py

+    def find_library(
+        self,
+        libname: str,
+        extra_dirs: T.List[str],
+        libtype: LibType = LibType.PREFER_SHARED,
+        lib_prefix_warning: bool = True,
+        ignore_system_dirs: bool = False,
+    ) -> T.Optional[T.List[str]]:


Please do not use the black style especially this thing it does, it's a really weird style for python and we don't use it elsewhere, the ) -> T.Optional[...]: should not be dedented tot he same level as def

one of:

def find_library(self, libname: str, ...): ... def find_library(self, libname: str, ...): ... def find_library( self, libname: str, ...): ...

Would be consistent with our style elsewhere.

I added a simple .style.yapf based on pep8 . It dedents to the same level as the function definition much less than ruff (the prior tool I used). It does in a few select cases, but I think those cases are reasonable. LMK otherwise, and I can skip the linters and hand-code to match the style you want.

mesonbuild/convert/instance/convert_instance_custom_target.py

This adds the following API: meson check-toolchain [ARGUMENTS] This API checks the the specified toolchains and outputs a TOML file that contains an abstract reprenstation of it's proprties. An example TOML output is: [[toolchain]] name = "android_arm64" [toolchain.host_machine] cpu_family = "aarch64" cpu = "aarch64" system = "android" endian = "little" [toolchain.c] compiler_id = "clang" linker_id = "ld.lld" version = "21.0.0" [toolchain.c.links.fails] "GCC 64bit atomics" = true [toolchain.c.check_header.fails] "pthread_np.h" = true [toolchain.c.has_header_symbol.fails] "sys/mkdev.h" = { major = true, minor = true, makedev = true } "errno.h" = { program_invocation_name = true } [toolchain.c.has_function.fails] qsort_s = true secure_getenv = true The has_function(..) or check_header(..) logs failures, since if one logged the successes, the resultant file would be very long and hard to read. The checks that the toolchain currently performs are modeled on Mesa3D's meson.build file, but can be be a superset of checks needed by the users of the tool in the future. The first user of the tool is meson convert, which is added in a subsequent patch. There is a fastpath for generating outputs given an Android NDK, and subsequent paths may be added later.

The convert tool will use it to generate build targets. It is possible for there to be a dedicated convert "backend" which passes in context like vslite_ctx, but that would require: - Made the core changes more invasive - Split the convert code across more directories, which may be logically harder to follow

This API converts meson.build into their corresponding hermetic representation (Android Blueprint, Bazel and maybe Buck2 in the future). This is motivated by the need to integrate Mesa3D into AOSP and Fuchsia trees. The question "how to build and update Mesa3D drivers for Android?" [1] in particular has led to several methods over the years, none of which used Android's native build system (Soong). This has been an obstacle to adoption of open-source drivers, which everyone knows are more secure, maintainable and faster than closed alternatives. By integrating into Mesa3D's native build system (Meson), "meson convert" brings shocking and jaw-dropping clarity to the question. Technically speaking, the tool works by via series of TOML files. Python 3.11 has tomlib in the standard library, and mconvert.py uses a conditional import strategy to prevent issues on older Python versions. These TOML files specify: * the Meson project that is being converted * where to find the dependencies in a hermetic tree * which compilers that a hermetic tree supports These TOML files are used to run the Meson interpreter multiple times. For example, the set of C/C++ flags may be different if the compiler targets x86_64 or ARM64. Data from each of meson intrepreter runs is collated to reconstruct the full set of Soong/Bazel rules. Although the initial implementation is focused with converting to a existing hermetic build system, this introduces infrastructure that could be useful if Meson itself takes a look at remote-executed, hermetic builds. For example, Meson can download prebuilts from NixPkgs. [1] https://gitlab.freedesktop.org/mesa/mesa/-/issues/13776 Co-developed-by: Craig Stout <[email protected]> Co-developed-by: Brandon Nguyen <[email protected]>

- Add to mesonmain.py - Add to run_mypy.py - Add Commands.md

gurchetansingh · 2026-01-16T20:56:02Z

One thing I'd really like to have is some more comments, particularly docstrings to help explain how all of the code fits together, and what each thing is for, there's a lot of code here and it would really help me to review it (and for us to maintain it).

Added more docstring comment. I would recommend reviewers start in convertmain.py.

gurchetansingh requested a review from jpakkane as a code owner January 12, 2026 22:09

gurchetansingh force-pushed the meson2hermetic branch from c9ebe87 to d3dff63 Compare January 12, 2026 22:13

eli-schwartz added enhancement android labels Jan 12, 2026

gurchetansingh force-pushed the meson2hermetic branch from d3dff63 to f8081d8 Compare January 15, 2026 17:49

dcbaker requested changes Jan 15, 2026

View reviewed changes

gurchetansingh and others added 4 commits January 16, 2026 11:55

meson: core changes to enable new APIs

a2f97c3

- Add to mesonmain.py - Add to run_mypy.py - Add Commands.md

gurchetansingh force-pushed the meson2hermetic branch from f8081d8 to a2f97c3 Compare January 16, 2026 20:11

		parser.add_argument("-o", "--output", default=None, help="Output file name.")


		def run(options: argparse.Namespace) -> int:

Uh oh!

meson2hermetic: the world's premiere build system converter #15462

Are you sure you want to change the base?

meson2hermetic: the world's premiere build system converter #15462

Uh oh!

Conversation

gurchetansingh commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eli-schwartz commented Jan 12, 2026

Uh oh!

eli-schwartz commented Jan 12, 2026

Uh oh!

gurchetansingh commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcbaker commented Jan 13, 2026

Uh oh!

dcbaker commented Jan 13, 2026

Uh oh!

bonzini commented Jan 14, 2026

Uh oh!

gurchetansingh commented Jan 15, 2026

Uh oh!

dcbaker commented Jan 15, 2026

Uh oh!

dcbaker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dcbaker Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

gurchetansingh Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dcbaker Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

dcbaker Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

gurchetansingh Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gurchetansingh commented Jan 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gurchetansingh commented Jan 12, 2026 •

edited

Loading

gurchetansingh commented Jan 12, 2026 •

edited

Loading