False Sharing Lab

The counters were independent. The cache line was not.

This deterministic C++20 simulator shows how logically independent per-thread counters can still contend at the cache-coherence layer. The same seeded update cohort is replayed through four counter layouts:

• Packed atomics place adjacent counters on the same modeled cache line.
• Cache-line padding gives each exact counter a separate line.
• Sampled thread shards keep hot updates local and publish periodically.
• Batched aggregation merges local deltas into one shared counter.

With the default 16-worker workload, the packed layout completes about 34% of attempted updates. Padding restores modeled capacity and reduces ownership transfers by roughly four orders of magnitude, at the cost of 8× the counter footprint. Sampling and batching reveal two more useful trades: freshness versus hot-path cost, and read fan-in versus merge traffic.

Technology

• C++20 analytical model and dependency-free POSIX socket server
• Vanilla HTML, CSS, JavaScript, and Canvas interface
• Native assertion tests plus live HTTP integration tests
• GCC 15 Linux CI and a multi-stage container
• No application package dependencies

Run it

Requires a C++20 compiler, Node.js, curl, jq, Make, and a POSIX environment.

make run

Open http://localhost:8080. To choose another port:

PORT=9090 make run

The server binds to loopback by default. Set HOST=0.0.0.0 when running directly in a container or when you intentionally need a non-loopback bind.

The simulation is also available as JSON from the CLI and HTTP API:

./build/false-sharing-lab --json | jq

curl 'http://localhost:8080/api/simulate?worker_threads=32&updates_per_second_per_thread=8000000&cache_line_bytes=64&atomic_update_ns=4&coherence_transfer_ns=90&snapshot_interval_ms=100&publish_interval_ms=50&merge_batch_size=512&run_seconds=60&seed=7219'

Verify it

make check
make docker-check

make check rebuilds with warnings treated as errors, runs model and normalization assertions, verifies deterministic JSON, checks the browser script, starts the server on an ephemeral port, and tests assets, input clamping, health, API shape, and HTTP errors. make docker-check builds and exercises the live container.

What the layouts reveal

Layout	Hot writes	Snapshot	Primary tradeoff
Packed atomics	Exact, adjacent shared lines	Exact	Small footprint, high ownership traffic
Cache-line padded	Exact, isolated lines	Exact	Predictable writes, larger footprint
Sampled shards	Thread-local	Bounded by publish interval	Low hot-path cost, stale observations
Batched aggregate	Thread-local plus shared flush	One aggregate read	Low read fan-in, batch-dependent merges

Padding is marked as the exact default for this lab, not as a universal recommendation. A real system should choose using measured workload behavior, freshness requirements, memory and TLB cost, snapshot frequency, core topology, and the bottleneck exposed after the first fix.

Telemetry model

Field	Why it matters
thread.id	Identifies the logical counter writer
cpu.core_id	Connects writers to coherence topology
counter.layout	Names the counter policy in use
counter.cache_line_id	Groups unrelated fields sharing a physical coherence unit
cache.coherence_transfers	Estimates ownership movement between writers
perf.remote_hitm	Highlights loads served by a modified line on another socket
cpu.stalled_cycles	Connects line movement to lost CPU capacity
counter.update_p99_ns	Makes the hot-path latency visible
counter.publish_p99_ns	Separates local updates from publication or flush cost
counter.snapshot_staleness_ms	Makes aggregation freshness explicit
counter.read_fan_in	Shows the work required to produce one snapshot
counter.memory_bytes	Keeps the padding cost in the same decision

Start with cache lines ranked by coherence events, remote HITM, stalled cycles, and write rate. Resolve each line to its structure and field offsets, then compare the writers by thread, core, and operation. Verify a candidate layout against throughput, tail latency, freshness, memory, and the next bottleneck.

Semantics behind the model

Linux kernel documentation describes false sharing as coherence activity on a shared cache line when concurrent accesses include at least one write. It also documents perf c2c and pahole as tools for finding hot lines and resolving their offsets. Its mitigation guidance explicitly calls out the performance versus space tradeoff of isolating hot data and the option of per-CPU data with periodic global synchronization.

C++ exposes implementation-defined constructive and destructive hardware interference sizes in <new>. This lab accepts an explicit modeled cache-line size so the mechanism and footprint tradeoff remain visible instead of silently assuming one host value.

• Linux kernel false-sharing documentation
• C++ working draft: hardware interference size

Model equations

The packed layout estimates line capacity as:

packed capacity / second =
  packed cache lines × 1 second
  ÷ (atomic update cost + ownership transfer cost)

For packed counters, the modeled transfer rate approaches one transfer per completed update as the number of writers per line increases. Padded and sampled layouts model only snapshot or publication traffic. Batched aggregation amortizes one shared-line merge over the configured number of local updates. Seeded ±4% capacity variance makes the timeline legible while keeping identical inputs byte-for-byte deterministic.

Model boundaries

This is an explanatory analytical simulator, not a microbenchmark, profiler, or prediction for a particular processor. It does not execute the configured number of atomics and does not read hardware performance counters. Values named coherenceTransfers, estimatedRemoteHitm, stalledCpuMs, and latency fields are modeled estimates.

Real behavior depends on cache hierarchy, coherence protocol, CPU topology, NUMA placement, scheduling and affinity, compiler output, atomic ordering, prefetching, snapshot readers, surrounding data, write distribution, and other contenders. A production investigation should validate the hypothesis with hardware counters and layout tools on representative hardware.

Built with C++ and telemetry.sh in mind. MIT licensed.