sccache

[lorenzleutgeb]

I would like to use crane together with sccache in hopes of speeding up builds. I have had some initial success, but got stuck.

Set up sccache

(This is rough, I know, but good enough for an experiment.)

1. Add extra-sandbox-paths=/tmp/sccache.sock? to nix.conf.
2. Install sccache.
3. Run SCCACHE_SERVER_UDS=/tmp/sccache.sock sccache --start-server.
4. Run chmod a+rwx /tmp/sccache.sock (to avoid thinking about permissions for now).
5. In another terminal window, run SCCACHE_SERVER_UDS=/tmp/sccache.sock watch sccache -s (to see if any of the stats change as you build).
6. (After all experiments, to clean up: Run SCCACHE_SERVER_UDS=/tmp/sccache.sock sccache --stop-server and rm /tmp/sccache.sock.)

Adjust the Flake

Setting environment variables for the build (you know how lib and pkgs are defined...):

{
  RUSTFLAGS = "--emit link";
  RUSTC_WRAPPER = lib.getExe pkgs.sccache;
  SCCACHE_SERVER_UDS = "/tmp/sccache.sock";
  SCCACHE_LOG = "debug";
}

Build

Run nix build as usual. Confirm that the sccache client process running inside the sandbox can use /tmp/sccache.sock to escape Nix's sandbox and talk to the sccache server process outside of the sandbox, by just watching the statistics change (via watch, see above) during build.

Stuck

Compilation seems to work fine, I get lots of green lines of compiled crates.
However, I run into issues, when it comes to linking and/or compiling C code, for example compiling SQLite:

Details

warning: build failed, waiting for other jobs to finish...
warning: sqlite3-src@0.5.1: [2025-10-21T22:59:39Z DEBUG sccache::config] Attempting to read config file at "/homeless-shelter/.config/sccache/config"
warning: sqlite3-src@0.5.1: [2025-10-21T22:59:39Z DEBUG sccache::config] Couldn't open config file: failed to open file `/homeless-shelter/.config/sccache/config`
warning: sqlite3-src@0.5.1: [2025-10-21T22:59:39Z DEBUG sccache::commands] Server sent CompileStarted
warning: sqlite3-src@0.5.1: source/sqlite3.c: In function 'jsonParseValue':
warning: sqlite3-src@0.5.1: source/sqlite3.c:201197:7: warning: this statement may fall through [-Wimplicit-fallthrough=]
warning: sqlite3-src@0.5.1: 201197 |     if( strncmp(z+i,"null",4)==0 && !sqlite3Isalnum(z[i+4]) ){
warning: sqlite3-src@0.5.1:        |       ^
warning: sqlite3-src@0.5.1: source/sqlite3.c:201203:3: note: here
warning: sqlite3-src@0.5.1: 201203 |   default: {
warning: sqlite3-src@0.5.1:        |   ^~~~~~~
warning: sqlite3-src@0.5.1: In function 'sqlite3Strlen30',
warning: sqlite3-src@0.5.1:     inlined from 'sqlite3ColumnSetColl' at source/sqlite3.c:118921:10:
warning: sqlite3-src@0.5.1: source/sqlite3.c:33946:28: warning: 'strlen' reading 1 or more bytes from a region of size 0 [-Wstringop-overread]
warning: sqlite3-src@0.5.1: 33946 |   return 0x3fffffff & (int)strlen(z);
warning: sqlite3-src@0.5.1:       |                            ^~~~~~~~~
warning: sqlite3-src@0.5.1: In function 'sqlite3ColumnSetColl':
warning: sqlite3-src@0.5.1: cc1: note: source object is likely at address zero
warning: sqlite3-src@0.5.1: Assembler messages:
warning: sqlite3-src@0.5.1: Fatal error: can't create /build/source/target/release/build/sqlite3-src-555766ef64aadaa2/out/beafe50bf357af0b-sqlite3.o: No such file or directory
error: failed to run custom build command for `sqlite3-src v0.5.1`
Caused by:

process didn't exit successfully: /build/source/target/release/build/sqlite3-src-4d54194c254821ea/build-script-build (exit status: 1)

--- stdout

OUT_DIR = Some(/build/source/target/release/build/sqlite3-src-555766ef64aadaa2/out)

OPT_LEVEL = Some(3)

TARGET = Some(x86_64-unknown-linux-gnu)

HOST = Some(x86_64-unknown-linux-gnu)

cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu

CC_x86_64-unknown-linux-gnu = None

cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu

CC_x86_64_unknown_linux_gnu = None

cargo:rerun-if-env-changed=HOST_CC

HOST_CC = None

cargo:rerun-if-env-changed=CC

CC = Some(gcc)

cargo:rerun-if-env-changed=CC_KNOWN_WRAPPER_CUSTOM

CC_KNOWN_WRAPPER_CUSTOM = None

RUSTC_WRAPPER = Some(/nix/store/wz945a0cbmhk0mc0v6zh7lnkb67xcna8-sccache-0.10.0/bin/sccache)

cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT

cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS

CRATE_CC_NO_DEFAULTS = None

DEBUG = Some(false)

CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)

cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu

CFLAGS_x86_64-unknown-linux-gnu = None

cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu

CFLAGS_x86_64_unknown_linux_gnu = None

cargo:rerun-if-env-changed=HOST_CFLAGS

HOST_CFLAGS = None

cargo:rerun-if-env-changed=CFLAGS

CFLAGS = None

CARGO_ENCODED_RUSTFLAGS = Some(--emitlink)

cargo:warning=[2025-10-21T22:59:39Z DEBUG sccache::config] Attempting to read config file at "/homeless-shelter/.config/sccache/config"

cargo:warning=[2025-10-21T22:59:39Z DEBUG sccache::config] Couldn't open config file: failed to open file /homeless-shelter/.config/sccache/config

cargo:warning=[2025-10-21T22:59:39Z DEBUG sccache::commands] Server sent CompileStarted

cargo:warning=source/sqlite3.c: In function 'jsonParseValue':

cargo:warning=source/sqlite3.c:201197:7: warning: this statement may fall through [-Wimplicit-fallthrough=]

cargo:warning=201197 |     if( strncmp(z+i,"null",4)==0 && !sqlite3Isalnum(z[i+4]) ){

cargo:warning=       |       ^

cargo:warning=source/sqlite3.c:201203:3: note: here

cargo:warning=201203 |   default: {

cargo:warning=       |   ^~~~~~~

cargo:warning=In function 'sqlite3Strlen30',

cargo:warning=    inlined from 'sqlite3ColumnSetColl' at source/sqlite3.c:118921:10:

cargo:warning=source/sqlite3.c:33946:28: warning: 'strlen' reading 1 or more bytes from a region of size 0 [-Wstringop-overread]

cargo:warning=33946 |   return 0x3fffffff & (int)strlen(z);

cargo:warning=      |                            ^~~~~~~~~

cargo:warning=In function 'sqlite3ColumnSetColl':

cargo:warning=cc1: note: source object is likely at address zero

cargo:warning=Assembler messages:

cargo:warning=Fatal error: can't create /build/source/target/release/build/sqlite3-src-555766ef64aadaa2/out/beafe50bf357af0b-sqlite3.o: No such file or directory
--- stderr
error occurred: Command LC_ALL="C" "/nix/store/wz945a0cbmhk0mc0v6zh7lnkb67xcna8-sccache-0.10.0/bin/sccache" "gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-Wall" "-Wextra" "-o" "/build/source/target/release/build/sqlite3-src-555766ef64aadaa2/out/beafe50bf357af0b-sqlite3.o" "-c" "source/sqlite3.c" with args gcc did not execute successfully (status code exit status: 1).

Of course I understand that this is an exotic usage of crane, and I am in no way trying to "blame" crane here. But I just don't know the build process well enough, and probably people around here are much more knowledgeable in tweaking Rust builds within Nix. I do wonder whether you would be interested in supporting such use-case, and whether you have any ideas for me on how to proceed? Maybe sccache is "only" missing some configuration here... Thanks for your consideration.