Add `SCCACHE_BASEDIRS` support #2521

Felixoid · 2025-12-18T21:08:45Z

This is an attempt to fix #35.

The ENV variable SCCACHE_BASEDIRS and configuration parameter basedirs are added.

As well as new tests to validate the behavior.

codecov-commenter · 2025-12-18T21:29:44Z

Codecov Report

❌ Patch coverage is 97.06110% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.00%. Comparing base (d38e9a3) to head (8bc4ca5).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/cache/cache.rs	74.02%	20 Missing ⚠️
src/util.rs	96.13%	10 Missing ⚠️
src/compiler/preprocessor_cache.rs	97.47%	3 Missing ⚠️
src/server.rs	78.57%	3 Missing ⚠️
tests/oauth.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2521      +/-   ##
==========================================
+ Coverage   71.07%   72.00%   +0.93%     
==========================================
  Files          65       65              
  Lines       35737    36894    +1157     
==========================================
+ Hits        25400    26566    +1166     
+ Misses      10337    10328       -9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

src/config.rs

src/util.rs

Felixoid · 2025-12-23T09:24:45Z

I got an idea, that basedirs should be added to the stats command as well

AJIOB · 2025-12-24T09:22:45Z

I got an idea, that basedirs should be added to the stats command as well

I think we can also provide the stat about base dir usage translation:

Number of base dir applied requests
Number of base dir skipped requests

In this case we can see, do we need to provide more/better base dirs or not

docs/Configuration.md

src/cache/cache.rs

src/config.rs

src/util.rs

Felixoid · 2025-12-24T14:58:45Z

Number of base dir applied requests

Number of base dir skipped requests

It's a tricky one. After taking a look, the number of successful substitutions is relatively easy to implement, although the counter should be threaded to the ServerStats.

But how to count the number of skipped directories? What is it? If a base_directory didn't match any of the output at all?

AJIOB · 2025-12-24T19:22:26Z

Number of base dir applied requests

Number of base dir skipped requests

It's a tricky one. After taking a look, the number of successful substitutions is relatively easy to implement, although the counter should be threaded to the ServerStats.

But how to count the number of skipped directories? What is it? If a base_directory didn't match any of the output at all?

No, just to check what number of cache requests has been converted using any of base dirs and what number was kept as an absolute one

AJIOB · 2025-12-24T19:23:42Z

About paths: we can try to normalize paths via https://docs.rs/normpath/latest/normpath/ or something similar instead of doing one more implementation.

Felixoid · 2025-12-29T12:52:06Z

About paths: we can try to normalize paths via https://docs.rs/normpath/latest/normpath/ or something similar instead of doing one more implementation.

It looks like https://doc.rust-lang.org/stable/std/path/fn.absolute.html is what I should use for both cases. It normalizes slashes, but keeps the cases as is.

Although see my comment regarding the preprocessor_output. normalize_path is used not only to needles from basedirs, but to a haystack as well. Both https://doc.rust-lang.org/stable/std/path/fn.absolute.html and https://docs.rs/normpath/latest/normpath/trait.PathExt.html#tymethod.normalize_virtually work on Path-like objects. They don't fit the purpose of the plaintext normalization, as far as I can tell.

Felixoid · 2025-12-30T00:33:50Z

No, just to check what number of cache requests has been converted using any of base dirs and what number was kept as an absolute one

I am afraid it causes changes in too many places, including the signatures of hash_key and other significant components. I want to cut this task from the PR, if you don't mind; it's already quite huge.

src/config.rs

src/util.rs

docs/Configuration.md

README.md

docs/Configuration.md

src/cache/readonly.rs

src/compiler/c.rs

src/util.rs

mathstuf · 2025-12-30T16:01:39Z

src/util.rs

+    for basedir_path in basedirs.iter() {
+        let basedir_str = basedir_path.to_string_lossy();
+        let basedir = basedir_str
+            .trim_end_matches(|c| c == '/' || c == '\\')


\ should not be trimmed on non-Windows.

#2521 (comment)

so, stripping \|/ on windows, / on other OSes, and then adding the trailing /

I'll use https://docs.rs/typed-path/latest/typed_path/ here.

Then, the trailing / will be added explicitly and completely stripped from the preprocessor_output instead of being replaced by ..

The last question is whether ./ should be treated as a special case. Because ./test.c and test.c won't match

In 87151ba, I implemented this logic. The basedir is stripped completely to match /dir/test.c and test.c in the cache.

The ./test.c is still not covered by the code.

src/util.rs

Co-authored-by: Alex Overchenko <aleksandr9809@gmail.com>

Co-authored-by: Ben Boeckel <mathstuf@users.noreply.github.com>

Felixoid · 2026-01-21T09:42:38Z

Another rebase:

The COS backend was added
Added integration tests for basedirs for each already tested backend

- Use autotools - Add __FILE__ macros - Use includes to get complex command line

Felixoid · 2026-01-22T20:33:27Z

I am pretty sure I didn't touch anything related to the TimeMacroFinder::find_time_macros in this PR

upd: yes, it's the same in another PR

Felixoid mentioned this pull request Dec 18, 2025

Implement an equivalent to CCACHE_BASEDIR #35

Open

Felixoid force-pushed the add-basedir-configuration branch from bfba6ec to 9eb3241 Compare December 18, 2025 23:45

Felixoid commented Dec 22, 2025

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

Felixoid commented Dec 22, 2025

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

Felixoid force-pushed the add-basedir-configuration branch from e818064 to d2e6edd Compare December 22, 2025 10:47

AJIOB reviewed Dec 22, 2025

View reviewed changes

src/util.rs Outdated Show resolved Hide resolved

Felixoid force-pushed the add-basedir-configuration branch from ca0a0db to 6f47f36 Compare December 23, 2025 13:57

Felixoid changed the title ~~Add SCCACHE_BASEDIR support~~ Add SCCACHE_BASEDIRS support Dec 23, 2025

Felixoid force-pushed the add-basedir-configuration branch from b5a7d22 to cf0b871 Compare December 23, 2025 23:03

whisperity reviewed Dec 24, 2025

View reviewed changes

Felixoid force-pushed the add-basedir-configuration branch 2 times, most recently from a97218c to bccc8f4 Compare December 29, 2025 23:13

Felixoid requested review from AJIOB, mathstuf and whisperity December 30, 2025 00:36

Felixoid force-pushed the add-basedir-configuration branch 2 times, most recently from 0887cea to e42a1f6 Compare December 30, 2025 12:12

AJIOB reviewed Dec 30, 2025

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

AJIOB approved these changes Dec 30, 2025

View reviewed changes

src/util.rs Outdated Show resolved Hide resolved

src/util.rs Outdated Show resolved Hide resolved

src/util.rs Outdated Show resolved Hide resolved

src/util.rs Outdated Show resolved Hide resolved

mathstuf suggested changes Dec 30, 2025

View reviewed changes

Felixoid force-pushed the add-basedir-configuration branch 2 times, most recently from 6529696 to fae67f8 Compare December 30, 2025 21:50

Felixoid and others added 20 commits January 21, 2026 10:02

Address review point for directories separator

5be4d03

Simplify strip_basedirs a bit more

9034c80

Final cleanup of the code for leftovers

9ff31bf

Review: less reallocations

aa4d6f1

Apply suggestions from code review

f078a57

Co-authored-by: Alex Overchenko <aleksandr9809@gmail.com>

Move normalization to the config building, use bytes for basedirs

91bbce7

Return unnormalized output, fix tests

f94565a

Empty SCCACHE_BASEDIRS overrides, but unset does not

c76e858

Apply suggestions from code review

65703c5

Co-authored-by: Ben Boeckel <mathstuf@users.noreply.github.com>

Do not trim spaces from paths, safer env overwrite in tests

dd3b7d8

Add tests that check uniq basedir per case

428c961

Return Cow from strip_basedirs

777b3ef

Use typed_path::Utf8TypedPathBuf to normalize directories

a9e59c7

Strip basedir completely, normalize once, add integration tests

a0f0740

Use ascii + utf in normalize_win_path

c6c2e1c

Another review round

81d6535

Avoid config conflicts while set_env manipulations

f3d4576

Update README.md

11ccdb6

Co-authored-by: Ben Boeckel <mathstuf@users.noreply.github.com>

Deduplicate basedirs in Config

fa346df

Refactor hash_key to HashKeyParams::compute

b32d59c

Felixoid force-pushed the add-basedir-configuration branch from 61c6747 to 861d150 Compare January 21, 2026 09:38

Add integration tests for basdirs

ebbd0fb

Felixoid force-pushed the add-basedir-configuration branch from 861d150 to ebbd0fb Compare January 21, 2026 09:44

Felixoid added 2 commits January 22, 2026 21:01

Improve basedirs integration tests

a383d35

- Use autotools - Add __FILE__ macros - Use includes to get complex command line

Sort profiles and services in Makefile

f8e0ecb

Felixoid force-pushed the add-basedir-configuration branch from 2d2be61 to f8e0ecb Compare January 22, 2026 20:01

Check CACHE_MISSES did not change between runs for autotools

8bc4ca5

AJIOB mentioned this pull request Jan 23, 2026

Distributed compilation support for asm & preprocessor outputs #2557

Open

Add SCCACHE_BASEDIRS support #2521

Are you sure you want to change the base?

Add SCCACHE_BASEDIRS support #2521

Conversation

Felixoid commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Felixoid commented Dec 23, 2025

Uh oh!

AJIOB commented Dec 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Felixoid commented Dec 24, 2025

Uh oh!

AJIOB commented Dec 24, 2025

Uh oh!

AJIOB commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Felixoid commented Dec 29, 2025

Uh oh!

Felixoid commented Dec 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mathstuf Dec 30, 2025

Choose a reason for hiding this comment

Uh oh!

Felixoid Dec 30, 2025

Choose a reason for hiding this comment

Uh oh!

Felixoid Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Felixoid Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Felixoid commented Jan 21, 2026

Uh oh!

Felixoid commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Add `SCCACHE_BASEDIRS` support #2521

Add `SCCACHE_BASEDIRS` support #2521

Felixoid commented Dec 18, 2025 •

edited

Loading

codecov-commenter commented Dec 18, 2025 •

edited

Loading

AJIOB commented Dec 24, 2025 •

edited

Loading

Felixoid commented Jan 22, 2026 •

edited

Loading