Skip to content

feat: ARM64 runtime guards (SMT, CPU info, seccomp, UFFD) #2259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tomassrnka wants to merge 6 commits into
base: main
Choose a base branch
from
Open

feat: ARM64 runtime guards (SMT, CPU info, seccomp, UFFD) #2259

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
da3757b
feat: ARM64 runtime guards for SMT, CPU info, seccomp, and UFFD
Mar 29, 2026
064b1a6
fix: move archARM64 const to package level, document CPU compat limits
Mar 30, 2026
32ed031
fix: remove --no-seccomp on ARM64, verified unnecessary
Mar 30, 2026
97e9af2
Merge branch 'main' into feat/arm64-runtime-guards
tomassrnka Mar 31, 2026
22e145a
Merge branch 'main' into feat/arm64-runtime-guards
tomassrnka Apr 1, 2026
2bb018d
Merge branch 'main' into feat/arm64-runtime-guards
tomassrnka Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion packages/orchestrator/cmd/smoketest/smoke_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
Expand Down Expand Up @@ -285,7 +286,7 @@ func findOrBuildEnvd(t *testing.T) string {

cmd := exec.CommandContext(t.Context(), "go", "build", "-o", binPath, ".") //nolint:gosec // trusted input
cmd.Dir = envdDir
cmd.Env = append(os.Environ(), "CGO_ENABLED=0", "GOOS=linux", "GOARCH=amd64")
cmd.Env = append(os.Environ(), "CGO_ENABLED=0", "GOOS=linux", "GOARCH="+runtime.GOARCH)
out, err := cmd.CombinedOutput()
if err != nil {
t.Skipf("failed to build envd: %v\n%s", err, out)
Expand Down
12 changes: 11 additions & 1 deletion packages/orchestrator/pkg/sandbox/fc/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package fc
import (
"context"
"fmt"
"runtime"

"github.com/bits-and-blooms/bitset"
"github.com/firecracker-microvm/firecracker-go-sdk"
Expand All @@ -18,6 +19,8 @@ import (
"github.com/e2b-dev/infra/packages/shared/pkg/utils"
)

const archARM64 = "arm64"

type apiClient struct {
client *client.Firecracker
}
Expand Down Expand Up @@ -326,7 +329,14 @@ func (c *apiClient) setMachineConfig(
memoryMB int64,
hugePages bool,
) error {
smt := true
// SMT (Simultaneous Multi-Threading / Hyper-Threading) must be disabled on
// ARM64 because ARM processors use a different core topology (big.LITTLE,
// efficiency/performance cores) rather than hardware threads per core.
// Firecracker validates this against the host CPU and rejects SMT=true on ARM.
// See: https://github.com/firecracker-microvm/firecracker/blob/main/docs/cpu_templates/cpu-features.md
// We use runtime.GOARCH (not TARGET_ARCH) because the orchestrator binary
// always runs on the same architecture as Firecracker.
smt := runtime.GOARCH != archARM64
trackDirtyPages := false
machineConfig := &models.MachineConfiguration{
VcpuCount: &vCPUCount,
Expand Down
12 changes: 11 additions & 1 deletion packages/orchestrator/pkg/sandbox/uffd/testutils/page_mmap.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package testutils

import (
"errors"
"fmt"
"math"
"syscall"
Expand All @@ -20,7 +21,16 @@ func NewPageMmap(t *testing.T, size, pagesize uint64) ([]byte, uintptr, error) {
}

if pagesize == header.HugepageSize {
return newMmap(t, size, header.HugepageSize, unix.MAP_HUGETLB|unix.MAP_HUGE_2MB)
b, addr, err := newMmap(t, size, header.HugepageSize, unix.MAP_HUGETLB|unix.MAP_HUGE_2MB)
// Hugepage allocation can fail with ENOMEM on CI runners that don't
// have enough (or any) hugepages pre-allocated in /proc/sys/vm/nr_hugepages.
// Skip gracefully rather than failing the test.
if err != nil && errors.Is(err, syscall.ENOMEM) {
pages := int(math.Ceil(float64(size) / float64(header.HugepageSize)))
t.Skipf("skipping: hugepage mmap failed (need %d hugepages): %v", pages, err)
}

return b, addr, err
}

return nil, 0, fmt.Errorf("unsupported page size: %d", pagesize)
Expand Down
24 changes: 21 additions & 3 deletions packages/orchestrator/pkg/service/machineinfo/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,31 @@ func Detect() (MachineInfo, error) {
}

if len(info) > 0 {
if info[0].Family == "" || info[0].Model == "" {
family := info[0].Family
model := info[0].Model

// On ARM64, gopsutil doesn't populate Family/Model from /proc/cpuinfo.
// Provide fallback values so callers don't get an error.
// NOTE: Using a generic "arm64" family treats all ARM64 CPUs as compatible.
// This works for same-host snapshot restore but cross-host restore between
// different ARM CPU implementations (e.g. Graviton2 vs Graviton3) may fail.
// For finer granularity, consider using MIDR_EL1 register values.
if runtime.GOARCH == "arm64" {
if family == "" {
family = "arm64"
}
if model == "" {
Comment on lines +34 to +38
Copy link
Copy Markdown

@claude claude bot Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The ARM64 CPU family fallback at line 32 sets family = "arm64" (an architecture name), but the PR description explicitly states the intended default is "8" (the numeric ARMv8 family identifier). This is semantically wrong and inconsistent with all x86 nodes, which report numeric families like "6"; fix by changing the fallback to family = "8".

Extended reasoning...

What the bug is

In packages/orchestrator/pkg/service/machineinfo/main.go (lines 30-34), the ARM64 fallback for CPU family is set to the string "arm64" rather than the numeric string "8". The PR description explicitly states: "we provide sensible defaults (family "8", model "0")". The code correctly sets model = "0" but incorrectly uses family = "arm64" instead of family = "8".

How it manifests

On any ARM64 node where gopsutil cannot populate Family from /proc/cpuinfo (the stated motivation for this fallback), the MachineInfo.Family field will be set to the architecture name "arm64" rather than the ARMv8 numeric family identifier "8". This value propagates to the cpu_family database column and gRPC messages sent to the orchestrator.

Why existing code does not prevent it

There is no validation that Family must be numeric. The guard if family == "" || model == "" only checks for empty strings — it passes through any non-empty value. So "arm64" is stored without error.

Impact

CPU family from /proc/cpuinfo is conventionally a numeric string (e.g., "6" for x86 Haswell, "8" for ARMv8). The CpuFamily field is used in template build records, snapshot creation, and orchestrator pool/node matching. Any downstream code that parses CpuFamily as a number (for Firecracker CPU template selection or similar) will fail on ARM64 nodes reporting "arm64". It also breaks semantic consistency: all x86 nodes report numeric families, while ARM64 nodes would report an architecture name.

How to fix

Change line 32 from family = "arm64" to family = "8". This matches the PR description and the conventional ARMv8 numeric family identifier.

Step-by-step proof

  1. ARM64 orchestrator starts; gopsutil reads /proc/cpuinfo but finds no CPU family field (ARM64 format does not include it).
  2. info[0].Family is "".
  3. runtime.GOARCH == "arm64" is true, so the fallback block is entered.
  4. family == "" is true, so family = "arm64" is assigned (bug — should be "8").
  5. MachineInfo{Family: "arm64", ...} is returned to callers.
  6. The value "arm64" is stored in cpu_family in the database and sent via gRPC.
  7. Any code attempting to parse cpu_family as an integer receives an error; any comparison with numeric families from x86 nodes is semantically inconsistent.

Copy link
Copy Markdown
Member Author

@tomassrnka tomassrnka Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code uses family = "arm64" which is correct — it's a label for CPU family grouping consistent with runtime.GOARCH semantics, not the ARMv8 numeric family identifier. The PR description mentioning "8" was a mistake in the bot's interpretation.

Copy link
Copy Markdown
Member

@jakubno jakubno Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we make sure that 2 different ARM CPUs are compatible?

Copy link
Copy Markdown
Member Author

@tomassrnka tomassrnka Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment documenting this limitation. Using generic "arm64" family works for same-host snapshots. Cross-host restore between different ARM CPUs (e.g. Graviton2 vs Graviton3) may need MIDR_EL1 register values — noted as a follow-up.

Copy link
Copy Markdown
Member Author

@tomassrnka tomassrnka Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment in the code documenting this limitation — generic arm64 family works for same-host, cross-host may need MIDR_EL1.

Copy link
Copy Markdown
Member

@jakubno jakubno Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw what's in ModelName? Could using that for Model work for now as a workaround?

model = "0"
}
}

if family == "" || model == "" {
return MachineInfo{}, fmt.Errorf("unable to detect CPU platform from CPU info: %+v", info[0])
}

return MachineInfo{
Family: info[0].Family,
Model: info[0].Model,
Family: family,
Model: model,
ModelName: info[0].ModelName,
Flags: info[0].Flags,
Arch: runtime.GOARCH,
Expand Down
Loading