Skip to content

feat(C4): add unsafe deserialization prohibition for model artifacts (4.5.10) #635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RicoKomenda wants to merge 1 commit into
base: main
Choose a base branch
from
+12 −10
Open

feat(C4): add unsafe deserialization prohibition for model artifacts (4.5.10) #635

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions 1.0/en/0x10-C04-Infrastructure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ AI infrastructure must be hardened against privilege escalation, supply chain ta
Prevent container escapes and privilege escalation through OS-level isolation primitives.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.1.1** | **Verify that** all AI workloads run with minimal permissions needed on the operating system, by e.g. dropping unnecessary Linux capabilities in case of a container. | 1 |
| **4.1.2** | **Verify that** workloads are protected by technologies limiting exploitation such as sandboxing, seccomp profiles, AppArmor, SELinux or similar, and that the configuration is appropriate. | 1 |
| **4.1.3** | **Verify that** workloads run with a read-only root filesystem, and that any writable mounts are explicitly defined and hardened with restrictive options that prevent execution and privilege escalation (e.g., noexec, nosuid, nodev). | 2 |
Expand All @@ -25,7 +25,7 @@ Prevent container escapes and privilege escalation through OS-level isolation pr
Ensure cryptographic integrity and supply chain security through reproducible builds and signed artifacts.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.2.1** | **Verify that** builds are completely automated with no manual steps that could introduce untracked changes. | 1 |
| **4.2.2** | **Verify that** build artifacts are cryptographically signed with build-origin metadata (source repository, build pipeline, commit hash) that can be independently verified. | 2 |
| **4.2.3** | **Verify that** build artifact signatures and build-origin metadata are validated at deployment admission, and unverified artifacts are rejected. | 2 |
Expand All @@ -39,7 +39,7 @@ Ensure cryptographic integrity and supply chain security through reproducible bu
Implement zero-trust networking with default-deny policies and encrypted communications.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.3.1** | **Verify that** network policies enforce default-deny ingress and egress, with only required services explicitly allowed. | 1 |
| **4.3.2** | **Verify that** AI workloads across environments (development, testing, production) run in isolated network segments with no direct internet access and no cross-environment network connectivity. | 1 |
| **4.3.3** | **Verify that** administrative and remote access protocols and access to cloud metadata services are restricted and require strong authentication. | 1 |
Expand All @@ -54,7 +54,7 @@ Implement zero-trust networking with default-deny policies and encrypted communi
Protect secrets and cryptographic keys with secure storage, automated rotation, and strong access controls.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.4.1** | **Verify that** secrets are stored in a dedicated secrets management system with encryption at rest and isolated from application workloads. | 1 |
| **4.4.2** | **Verify that** access to production secrets requires strong authentication. | 1 |
| **4.4.3** | **Verify that** secrets are deployed to applications at runtime through a dedicated secrets management system. Secrets must never be embedded in source code, configuration files, build artifacts, container images, or environment variables. | 1 |
Expand All @@ -68,16 +68,17 @@ Protect secrets and cryptographic keys with secure storage, automated rotation,
Isolate untrusted AI models in secure sandboxes and protect sensitive AI workloads using trusted execution environments (TEEs) and confidential computing technologies.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| **4.5.1** | **Verify that** external or untrusted AI models execute in isolated sandboxes.| 1 |
| **4.5.2** | **Verify that** sandboxed workloads have no outbound network connectivity by default, with any required access explicitly defined.| 1 |
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.5.1** | **Verify that** external or untrusted AI models execute in isolated sandboxes. | 1 |
| **4.5.2** | **Verify that** sandboxed workloads have no outbound network connectivity by default, with any required access explicitly defined. | 1 |
| **4.5.3** | **Verify that** workload attestation is performed before model loading, ensuring cryptographic proof that the execution environment has not been tampered with. | 2 |
| **4.5.4** | **Verify that** confidential workloads execute within a trusted execution environment (TEE) that provides hardware-enforced isolation, memory encryption, and integrity protection. | 3 |
| **4.5.5** | **Verify that** confidential inference services prevent model extraction through encrypted computation with sealed model weights and protected execution. | 3 |
| **4.5.6** | **Verify that** TEE orchestration manages the full lifecycle of trusted execution environments, including provisioning, suspension, and termination. | 3 |
| **4.5.7** | **Verify that** secure multi-party computation (SMPC) enables collaborative AI training without exposing individual datasets or model parameters. | 3 |
| **4.5.8** | **Verify that** TEE orchestration performs remote attestation before each workload placement to confirm the execution environment has not been tampered with. | 3 |
| **4.5.9** | **Verify that** communication channels between TEE orchestration components and execution environments are encrypted and mutually authenticated. | 3 |
| **4.5.10** | **Verify that** model artifact loading prohibits or restricts serialization formats that permit arbitrary code execution at deserialization time (e.g., Python pickle with unrestricted globals), and that model files are scanned using format-aware tools before loading in any environment. | 1 |

---

Expand All @@ -86,7 +87,7 @@ Isolate untrusted AI models in secure sandboxes and protect sensitive AI workloa
Prevent resource exhaustion attacks and ensure fair resource allocation through quotas and monitoring. Maintain infrastructure resilience through secure backups, tested recovery procedures, and disaster recovery capabilities.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.6.1** | **Verify that** workload resource consumption is limited through quotas and limits (e.g., CPU, memory, GPU) to mitigate denial-of-service attacks. | 1 |
| **4.6.2** | **Verify that** resource exhaustion triggers automated protections (e.g., rate limiting or workload isolation) once defined CPU, memory, or request thresholds are exceeded. | 2 |
| **4.6.3** | **Verify that** backup systems run in isolated networks with separate credentials that are not shared with production workloads. | 2 |
Expand All @@ -99,7 +100,7 @@ Prevent resource exhaustion attacks and ensure fair resource allocation through
Secure AI-specific hardware components including GPUs, TPUs, and specialized AI accelerators.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.7.1** | **Verify that** before workload execution, AI accelerator integrity is validated using hardware-based attestation mechanisms (e.g., TPM, DRTM, or equivalent). | 2 |
| **4.7.2** | **Verify that** accelerator (GPU) memory is isolated between workloads through partitioning mechanisms with memory sanitization between jobs. | 2 |
| **4.7.3** | **Verify that** AI accelerator firmware is version-pinned, signed, and attested at boot; unsigned or debug firmware is blocked. | 2 |
Expand All @@ -116,7 +117,7 @@ Secure AI-specific hardware components including GPUs, TPUs, and specialized AI
Secure distributed AI deployments including edge computing, federated learning, and multi-site architectures.

| # | Description | Level |
|:--------:|--------------------------------------------------------------------------------------------|:---:|
| :--------: | ------------------------------------------------------------------------------------------ | :---: |
| **4.8.1** | **Verify that** edge AI devices authenticate to central infrastructure using mutual authentication with certificate validation (e.g., mutual TLS). | 1 |
| **4.8.2** | **Verify that** models deployed to edge or mobile devices are cryptographically signed during packaging, and that the on-device runtime validates these signatures or checksums before loading or inference; unverified or altered models must be rejected. | 1 |
| **4.8.3** | **Verify that** edge devices implement secure boot with verified signatures and rollback protection to prevent firmware downgrade attacks. | 2 |
Expand Down
1 change: 1 addition & 0 deletions 1.0/en/0x93-Appendix-D_AI_Security_Controls_Inventory.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,7 @@ Verify origin and authenticity, scan dependencies, and enforce integrity of mode
| Expired and unmaintained dependency detection | 6.3.3 |
| Approved source and internal registry enforcement | 6.4.1 |
| Malicious layer and trojan trigger scanning | 6.1.2 |
| Unsafe deserialization format prohibition and format-aware scanning at load time | 4.5.10 |
| External dataset poisoning assessment (fingerprinting, outlier detection) | 6.5.1 |
| Copyright and PII detection in external datasets | 6.5.2 |
| Dataset origin and lineage documentation | 6.5.3 |
Expand Down
Loading