feat(apollo_starknet_os_program): add feature for exposing cairo file source map

[dorimedini-starkware]

No description provided.

[reviewable-StarkWare]

This change is 

[github-actions]

Artifacts upload workflows:

• Started at 2025-04-29T13:38:45Z
• Started at 2025-04-29T13:55:13Z
• Started at 2025-04-30T10:57:01Z
• Started at 2025-04-30T11:00:00Z
• Started at 2025-04-30T15:23:34Z
• Started at 2025-04-30T15:27:12Z
• Started at 2025-04-30T17:49:31Z
• Started at 2025-05-01T09:44:48Z
• Started at 2025-05-02T14:17:44Z
• Started at 2025-05-05T10:59:47Z
• Started at 2025-05-05T12:50:44Z
• Started at 2025-05-05T13:08:52Z
• Started at 2025-05-06T12:26:16Z
• Started at 2025-05-06T14:51:42Z
• Started at 2025-05-08T09:22:20Z
• Started at 2025-05-08T09:51:48Z
• Started at 2025-05-08T16:12:18Z
• Started at 2025-05-08T17:34:47Z
• Started at 2025-05-11T09:49:06Z
• Started at 2025-05-11T09:49:06Z

[dorimedini-starkware]

• fix(ci): properly install python deps, in CI and docker #5902 : 2 dependent PRs (#5873 , #6266 )
• feat(apollo_infra_utils): better cairo0 compiler checks #5921
• chore(apollo_infra_utils): explicit instructions in verify_cairo0_compiler_deps #5872
• chore(apollo_infra_utils,blockifier_test_utils): move verify_cairo0_compiler_deps to infra_utils #5871
• feat(starknet_committer_and_os_cli): add command to dump OS source files #6265
• feat(apollo_starknet_os_program): add feature for exposing cairo file source map #6259 👈 (View in Graphite)
• feat(apollo_starknet_os_program): copy OS cairo code #5870
• feat(apollo_starknet_os_program): add apollo_starknet_os_program crate #5869
• feat(ci): upgrade cairo-lang to 0.14.0a1 #6353
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[amosStarkware]

Reviewed 1 of 3 files at r2, 7 of 7 files at r3.
Reviewable status: all files reviewed (commit messages unreviewed), 10 unresolved discussions (waiting on @TzahiTaub)

---

-- commits line 1 at r3:
note there are two commits here - not sure if Graphite will merge well like this

Code quote:

ew commits in r2 on 08/05/2025 at 12:50, probably rebased from r1:

---

crates/apollo_starknet_os_program/Cargo.toml line 17 at r3 (raw file):

[dependencies]
serde_json.workspace = true

you sure we need this?

Code quote:

[dependencies]
serde_json.workspace = true

---

crates/apollo_starknet_os_program/build/main.rs line 3 at r3 (raw file):

#[cfg(feature = "dump_source_files")]
use std::path::PathBuf;

why not call this file build.rs?

---

crates/apollo_starknet_os_program/build/main.rs line 8 at r3 (raw file):

/// Build script for the `apollo_starknet_os_program` crate.
/// Recompiles the OS program if the source files change.

This will happen in a future PR?
consider adding a TODO (if it's annoying then NVM)

Code quote:

/// Build script for the `apollo_starknet_os_program` crate.
/// Recompiles the OS program if the source files change.

---

.gitignore line 15 at r3 (raw file):

*.egg-info
/build

shouldn't you add crates/apollo_starknet_os_program/build?
I'm not familiar with gitignore patterns, so I may be wrong

Code quote:

/build

---

.gitignore line 34 at r3 (raw file):

# Native blockifier artifacts.
/crates/native_blockifier/build

I don't see this dir?

Code quote:

# Native blockifier artifacts.
/crates/native_blockifier/build

---

crates/apollo_starknet_os_program/build/dump_source.rs line 8 at r3 (raw file):

/// Utility function to recursively find all cairo files.
fn get_cairo_file_map_recursive(entry: DirEntry) -> HashMap<String, String> {

do we expect non-cairo files in the OS's directory (crates/apollo_starknet_os_program/src/cairo/starkware/starknet)?
if not you can pass - there's probably a std library function to copy all files in a directory.

Code quote:

fn get_cairo_file_map_recursive(entry: DirEntry) -> HashMap<String, String> {

---

crates/apollo_starknet_os_program/build/dump_source.rs line 22 at r3 (raw file):

                path.to_str().unwrap().to_string(),
                std::fs::read_to_string(path).unwrap(),
            )))

why use an iterator here?

Code quote:

            HashMap::from_iter(std::iter::once((
                path.to_str().unwrap().to_string(),
                std::fs::read_to_string(path).unwrap(),
            )))

---

crates/apollo_starknet_os_program/build/dump_source.rs line 32 at r3 (raw file):

/// the map as JSON to the specified location.
pub fn dump_source_files(dump_to: PathBuf) {
    println!("cargo::warning=Dumping OS source files...");

why warning?

Code quote:

println!("cargo::warning=Dumping OS source files...");

---

crates/apollo_starknet_os_program/build/dump_source.rs line 45 at r3 (raw file):

                .strip_prefix(base_path_string)
                .unwrap()
                .strip_prefix("/cairo/")

why not add this to base_path?

Code quote:

 .strip_prefix("/cairo/")

[dorimedini-starkware]

Reviewable status: 1 of 8 files reviewed, 10 unresolved discussions (waiting on @amosStarkware and @TzahiTaub)

---

-- commits line 1 at r3:

Previously, amosStarkware wrote…

note there are two commits here - not sure if Graphite will merge well like this

WDYM? why not? the merge queue merge method squashes; graphite should handle this just fine, no?

---

.gitignore line 15 at r3 (raw file):

Previously, amosStarkware wrote…

shouldn't you add crates/apollo_starknet_os_program/build?
I'm not familiar with gitignore patterns, so I may be wrong

specifically, the crates/apollo_starknet_os_program/build directory is source code, not build artifacts.
the changes in this file are specifically here to not ignore that directory; without the changes, the build/main.rs and build/dump_source.rs files are ignored by git

---

.gitignore line 34 at r3 (raw file):

Previously, amosStarkware wrote…

I don't see this dir?

after I changed build -> /build, my git status showed me this; I may not be the only one, and it is definitely not something we want to commit. I see no harm in adding it to gitignore

---

crates/apollo_starknet_os_program/Cargo.toml line 17 at r3 (raw file):

Previously, amosStarkware wrote…

you sure we need this?

technically, in this PR, we only need it if the dump_source_files feature is active (see lib.rs); but I'd rather keep it as is, in subsequent PRs we will always need it (for the OS program deserialization itself)

---

crates/apollo_starknet_os_program/build/dump_source.rs line 8 at r3 (raw file):

Previously, amosStarkware wrote…

do we expect non-cairo files in the OS's directory (crates/apollo_starknet_os_program/src/cairo/starkware/starknet)?
if not you can pass - there's probably a std library function to copy all files in a directory.

we may decide to move the constants_template.txt file inside here, so I'd rather keep this search cairo-only. will also help with forward-compatibility, + I like being explicit in what I want to copy

---

crates/apollo_starknet_os_program/build/dump_source.rs line 22 at r3 (raw file):

Previously, amosStarkware wrote…

why use an iterator here?

Done.

---

crates/apollo_starknet_os_program/build/dump_source.rs line 32 at r3 (raw file):

Previously, amosStarkware wrote…

why warning?

this is how you print output in build scripts. see here. regular println! doesn't appear in the console

---

crates/apollo_starknet_os_program/build/dump_source.rs line 45 at r3 (raw file):

Previously, amosStarkware wrote…

why not add this to base_path?

Done. I still want to remove the leading / though, to get relative paths

---

crates/apollo_starknet_os_program/build/main.rs line 3 at r3 (raw file):

Previously, amosStarkware wrote…

why not call this file build.rs?

you either use build.rs, if you have a single build file, or build/main.rs if you have several build files (by convention). note this line in the Cargo.toml:

build = "build/main.rs"

---

crates/apollo_starknet_os_program/build/main.rs line 8 at r3 (raw file):

Previously, amosStarkware wrote…

This will happen in a future PR?
consider adding a TODO (if it's annoying then NVM)

a bit annoying to rebase; the reason this is here is because the entire "check file consistency" feature was added after I already had most of the stack in place (splitting build.rs into modules in build/* was very annoying, this slipped through)
will be implemented in later PRs (specifically, here)

[github-actions]

Benchmark movements:
tree_computation_flow performance improved 😺
tree_computation_flow time: [34.497 ms 34.516 ms 34.536 ms]
change: [-4.8007% -3.0672% -1.5233%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild

[amosStarkware]

Reviewed 7 of 7 files at r5, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @dorimedini-starkware and @TzahiTaub)

---

-- commits line 1 at r3:

Previously, dorimedini-starkware wrote…

WDYM? why not? the merge queue merge method squashes; graphite should handle this just fine, no?

maybe, you know Graphite better. I thought Graphit wants one commit per branch

---

.gitignore line 15 at r3 (raw file):

Previously, dorimedini-starkware wrote…

specifically, the crates/apollo_starknet_os_program/build directory is source code, not build artifacts.
the changes in this file are specifically here to not ignore that directory; without the changes, the build/main.rs and build/dump_source.rs files are ignored by git

so any crate that has a build directory, which was ignored before, won't be ignored now?
are you sure there aren't many such directories?
it seems better to just rename apollo_starknet_os_program/build

---

crates/apollo_starknet_os_program/build/dump_source.rs line 45 at r3 (raw file):

Previously, dorimedini-starkware wrote…

Done. I still want to remove the leading / though, to get relative paths

you can just do join("src/cairo/");, no? non blocking

[dorimedini-starkware]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @amosStarkware and @TzahiTaub)

---

-- commits line 1 at r3:

Previously, amosStarkware wrote…

maybe, you know Graphite better. I thought Graphit wants one commit per branch

apparently not :)

---

.gitignore line 15 at r3 (raw file):

Previously, amosStarkware wrote…

so any crate that has a build directory, which was ignored before, won't be ignored now?
are you sure there aren't many such directories?
it seems better to just rename apollo_starknet_os_program/build

my git status is clean, and I think it's standard to name this dir build, since in the case where it's a single module it's called build.rs (in which case no special setting in the Cargo.toml is required).
rust uses the target terminology for output artifacts

---

crates/apollo_starknet_os_program/build/dump_source.rs line 45 at r3 (raw file):

Previously, amosStarkware wrote…

you can just do join("src/cairo/");, no? non blocking

sure sure if PathBuf::from("a/").to_str() is "a/" or "a"

[amosStarkware]

Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @TzahiTaub)