experimental: add cargo-optee #245
Conversation
|
new commit:
|
DemesneGH
force-pushed
the
experimental-cargo-build
branch
from
89b136d to
84a9673
Compare
| } | ||
|
|
||
| #[derive(Debug, Subcommand)] | ||
| enum Command { |
There was a problem hiding this comment.
Do we have a high level design about the cargo-optee?
What is the convention | requirement between different roles?
- cargo-optee and underlining build system
- cargo-optee and TA developer
Please refer to the following link as a reference.
https://github.com/automata-network/automata-sgx-sdk?tab=readme-ov-file#getting-started
There was a problem hiding this comment.
Updated in the new commit and cargo-optee/README.md
There was a problem hiding this comment.
Thanks for the effort for showing the current design.
cargo-optee build ta \
--ta_dev_kit_dir <PATH> \
[--path <PATH>] \
[--arch aarch64|arm] \
[--std] \
[--signing_key <PATH>] \
[--uuid_path <PATH>] \
[--debug]However, I don’t think this should be the ideal interface for TA developers.
I suggest we align and compare the design with existing cargo-based workflows such as cargo and cargo-sgx to ensure consistency and ease of use.
Project-related configurations (for both TA and CA) should remain in Cargo.toml.
- For example,
cargo-trustzoneshould automatically infer whether to build with std based on the std feature declared inCargo.toml, as you have proposed using thestdfeature for the std build in previous PR. - For other project-specific parameters, we can follow the convention used by
cargo-sgx, which leverages[package.metadata]as the bridge between developers and the custom cargo toolchain.
Toolchain configurations & Zero-argument usability
By default, running cargo-trustzone build should work without requiring additional parameters.
Since project-specific configurations are defined in Cargo.toml, the toolchain should have clear default discovery rules for common paths (e.g., how ta_dev_kit_dir is located, instead of a required parameter for each invocation of the build cli).
For example, the standard cargo toolchain automatically discovers the Rust sysroot and standard library paths by resolving from rustc --print sysroot, without needing users to specify them manually. Its behavior might also be influenced by environment variables such as RUSTUP_HOME and CARGO_HOME, which allow users to override toolchain locations in a standardized way.
Similarly, cargo-trustzone should support automatic discovery of the default ta_dev_kit_dir (e.g., from an environment variable, configuration file, or standard installation path) to provide a smooth and familiar developer experience consistent with the normal cargo workflow.
Learn from Existing Design
Lastly, as I have suggested multiple times, please take a deeper look at the cargo-sgx tool and compare our current design against it. Many of the challenges we are encountering have already been addressed there — such as providing a developer-friendly interface, automatic build support for customized std environments, and seamless integration of loadable trusted components with untrusted applications. There may be additional insights and techniques we can learn from those design patterns. I strongly recommend gaining a solid understanding of what is already available before reinventing the wheel.
There was a problem hiding this comment.
I've reviewed the cargo-sgx code snippet — although it reads metadata, it still accepts parameters via command line with default values. I assume you're suggesting we should move some params to [package.metadata] in Cargo.toml instead of passing them as command-line arguments?
I actually mentioned the idea in the initial message of this PR, but there are a few considerations. Let's go through each:
arch / std
We should allow building the same app for different arch and modes.
If these values are hardcoded in Cargo.toml, we lose the flexibility. For example, in CI:
cargo-trustzoneshould automatically infer whether to build withstdbased on the feature declared inCargo.toml.
But what should we write in the common examples’ Cargo.toml?
If we set "no-std", how would we build it in std mode for CI? by modifying Cargo.toml each time? I don't have the answer.
Therefore, I believe arch and std should remain as command-line parameters, each with default values.
ta_dev_kit_dir
Seems this might suitable for Cargo.toml, but it’s different for arch — for example:
optee_os/out/arm-plat-vexpress/export-ta_arm64
optee_os/out/arm-plat-vexpress/export-ta_arm32
If we put it in Cargo.toml, we’d have to edit that file each time we change the build architecture.
Maybe try this?
[metadata]
ta_dev_kit.64 = "xxxx_export-ta_arm64"
ta_dev_kit.32 = "xxxx_export-ta_arm32"
use the different entry decided by cmd param arch.
uuid_path (or uuid)
Instead of storing a uuid_path, it may make more sense to store a uuid string directly in the metadata. Since uuid.txt is both used by TA build tools and read by the proto (and imported by the CA when invoking the TA). We could try to write into the Cargo.toml, but to which Cargo.toml? Maybe we should have a workspace?
I’m still thinking about it.
signing_key
It makes sense to write into the Cargo.toml.
and seamless integration of loadable trusted components with untrusted applications
Does it mean the plugin if in optee? I don't really get the point.
Similarly,
cargo-trustzoneshould support automatic discovery of the defaultta_dev_kit_dir(e.g., from an environment variable, configuration file, or standard installation path) to provide a smooth and familiar developer experience consistent with the normal cargo workflow.
Consider the scenario:
- OP-TEE OS developers export the ta_dev_kit, and send it to the TA developer
- TA developer receives it and build the TA
For this tool, I want it less dependent with the environment, for production use cases or experienced developers, the TA developers typically only need a toolchain setup and already know the location of their ta_dev_kit; with those information they can build the TA directly.
Additionally, I plan to provide a helper script for the dev/emulation env for easy use on qemuv8, together with preparing the OP-TEE libraries.
Please feel free to share your suggestions, thanks!
There was a problem hiding this comment.
Before we move further, could you clarify what kind of user experience you envision for TA developers using the cargo-optee toolchain?
From my perspective, the cargo-optee build command should provide a seamless interface aligned with the existing cargo build workflow, either through transparent parameter passing or through complementary extensions. The key principle is to stay consistent with Cargo’s CLI conventions and user expectations.
It’s perfectly fine for the internal implementation to have a different interface, but here I’m mainly referring to the user-facing design.
Here I just expand one point for illustration.
Q: “But what should we write in the common examples’ Cargo.toml? If we set no-std, how would we build it in std mode for CI? By modifying Cargo.toml each time? I don't have the answer.”
My view: when designing the interface, we should ask — how do Rust developers normally address this problem (e.g., switching between no_std and std, or specifying architectures)? We should adopt the same mechanism.
For std / no_std switching, Rust developers use Cargo features declared in Cargo.toml, together with --no-default-features or --features xxx or nothing for cargo build. This convention can also be applied for our customized tooling.
The underlying build logic can follow Cargo’s native feature resolution model (via cargo_metadata), for example:
cargo (optee) build
cargo (optee) build --no-default-features
cargo (optee) build --no-default-features --features no_std // or std
The key concept here is transparent delegation to Cargo, cargo-optee does not need to reimplement or reinterpret feature logic, or use a reinvented --std parameter, but instead query and respect Cargo’s native feature resolution. This ensures consistent developer experience and predictable build behavior.
A similar comparison and concept mapping approach to existing Cargo mechanisms can be applied to other aspects as well, such as toolchain conventions and project conventions, to achieve a more consistent and intuitive CLI experience.
There was a problem hiding this comment.
I think cargo-optee provides the convenience for building TAs, not mandatory to keep consistency with the original cargo cmd parameters.
For example, the cargo-optee cmd:
cargo-optee build ta \
--ta_dev_kit_dir /opt/optee/export-ta_arm64 \
--path ./ta \
--arch aarch64
will be converted to:
TA_DEV_KIT_DIR=/opt/optee/export-ta_arm64 \
RUSTFLAGS="-C panic=abort" \
cargo build --target aarch64-unknown-linux-gnu --release \
--config target.aarch64-unknown-linux-gnu.linker="aarch64-linux-gnu-gcc"
for build.
--arch is not consistent with cargo's cmdline but it will configure the target for developer's convenience. Do you mean we should use the --target in cargo-optee?
If so, why not directly use cargo command, we don't need a wrapper.
feature std is similar. It's not only converted to the cargo build --features std, but also decide the target and whether we use xargo. I can follow the cargo's cmd format, like cargo-optee build ta --features std, but it doesn't mean the same as in cargo.
From my perspective, the cargo-optee build command should provide a seamless interface aligned with the existing cargo build workflow,
I've checked other tools cargo-sgx cargo-near which also defines their cmd lines different with original cargo. Any reasons on why we should follow seamless interface principle?
There was a problem hiding this comment.
cargo_metadata retrieves meta info based on cargo.toml and command-line options in a programatic way, which does not mean convert to cargo build --features std. It is designated to used from within a cargo-* executable. https://crates.io/crates/cargo_metadata
Here is a more concrete example to illustrate the difference.
cargo-optee build ta --stdmeans no matter what default feature developer defines in the Cargo.toml, the customized toolchain will build in using std library (new convention is introduced by cargo-optee).cargo optee build(with other cargo compatible flags) means the the customized toolchain will respect what feature the developer enables in the Cargo.toml (std/no-std) and automatically handles internally (select xargo to build stdlib or not) based on the active feature.
Any reasons on why we should follow seamless interface principle?
There’s no strict requirement to maintain full consistency with Cargo’s original command-line parameters. However, I am just wondering if our goal should be to make OP-TEE development feel like a natural extension of the standard Cargo experience — a design I believe is achievable for our current use cases. To realize this, we can learn from existing customized toolchains, e.g. both tools you mentioned also leverage cargo_metadata to retrieve project setup information.
I will end this conversation here, as I believe I’ve explained my points clearly and don’t want to spend more time on further debate.
|
Summary of the new commit: Users & ScenariosFor developers (dev/emulation environments): For CI:
I've tested the
We can remove all Makefiles from the SDK now. Changes
Notes
Next steps
|
|
Refer to the OP-TEE issue: OP-TEE/build#849 |
|
The new commit:
The CI build script ( How to use this tool:
example log: At this stage, we’ll focus on user-facing functionality and bug fixes. Please feel free to try out the tool and share your feedback, thanks! |
DemesneGH
force-pushed
the
experimental-cargo-build
branch
3 times, most recently
from
de40647 to
fa5661b
Compare
- enable build ca - add sub-command: ta and ca - revise param "arch" from string to enum - fix building error for 32bit TAs, related to optee-utee-build - fix license
DemesneGH
force-pushed
the
experimental-cargo-build
branch
from
fa5661b to
383ef85
Compare
DemesneGH
force-pushed
the
experimental-cargo-build
branch
from
383ef85 to
b48f1cc
Compare
|
Updates: Todos:
Pending: -- |
|
@asir66 Could you help to try out the tool and share your feedback? Thanks! |
It's my pleasure. I'll test it and share my feedback. |
cargo-optee Testing Report and FindingsBelow is a consolidated report of issues, observations, and suggestions identified during testing of 1. Issue:
|
| pub struct BuildConfig { | ||
| pub arch: Arch, | ||
| pub debug: bool, | ||
| pub std: bool, |
There was a problem hiding this comment.
There is an issue here: when --std is not provided, std defaults to false, so cmd_std becomes Some(false) instead of None.
|
@asir66 Thanks for your detailed and careful testing!
The
The For other recommendations:
3 and 4: LGTM.
|
Hi @asir66 thanks for the effort. Please take my previous comment (#245 (comment) compare the current A few highlights from my earlier comment for reference:
Thanks in advance. |
It’s my pleasure — I’d be glad to work on that.
LGTM! I think this approach is great — it allows users to transition smoothly from the old Makefile workflow to the new tool. |
I’ll do my best to review these areas and put together a comprehensive list of the differences from standard cargo conventions. |
@asir66 Let's use |
|
Closing this draft PR and continuing the work in #263 |
4 participants
As discussed in #244, I'm trying to remove Makefiles and provide a Cargo-based tool for building OP-TEE apps.
This is an initial version of the
cargo-opteetool which enables building TAs inno-stdandstd:build the cargo-optee tool:
building no-std TA:
building std TA:
The goal is to make TA building simple: once the environment has the Rust and GCC toolchains installed, developers can build a TA directly using this tool.
It does not rely on global environment variables, only reads parameters explicitly passed. This design makes it more flexible across different development environments and provides clearer control over variable usage. The only mandatory parameter is
TA_DEV_KIT_DIR; all other parameters have default values.However,
TA_DEV_KIT_DIRis a low-level configuration detail. For our dev/emulation environments, we can provide a helper script in the future, to set up all required OP-TEE libraries and passTA_DEV_KIT_DIRto the tool.There are still several areas that can be further refined:
Xargo.tomlfiles from the TA source code.stdfeature is enabled.proto/. The host can read it from the source code, andcargo-opteecan use it when building the TA. For now, it remains as the originaluuid.txt.Cargo.toml(similar to what [cargo-sgx](https://github.com/automata-network/automata-sgx-sdk) does with[package.metadata.sgx]).I’ll move this forward. Please feel free to try out the tool and share any feedback about this, thanks!