datadog example

.github/workflows/pdd-secrets-dispatch.yml

@@ -0,0 +1,126 @@
+name: PDD Secrets Dispatch
+
+on:
+  repository_dispatch:
+    types: [pdd-secrets]
+
+jobs:
+  provide-secrets:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Install Dependencies
+        run: pip install cryptography requests
+
+      - name: Encrypt and Send Secrets
+        shell: python
+        env:
+          SECRETS_CONTEXT: ${{ toJSON(secrets) }}
+          PAYLOAD_CONTEXT: ${{ toJSON(github.event.client_payload) }}
+          WORKER_PUBLIC_KEY: |
+            -----BEGIN PUBLIC KEY-----
+            MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwLc2Vn2VErKtvPa27Iek
+            ePSZuWdagaiA9wfce4+4GR3aDNCsv24btyttNapgqO1pkB7mVATp4WJCD7F3CDMU
+            ddDsi07PReXJrdMp+/IC0eGblHPPpJyZOh/5ZoIo2oJPOHjGWwEQO75JgT8jtvUh
+            L2o7lYuFS1BwMQpdHifpxDWdn8yXo6SvV7k0UISEWKn5sa4gcalOhYiEjFUNt4pt
+            a7/HJQQu6K3AJKoUT8eTsftV5o1SHP0C8wB85hOehdzEM5uSpCb6aZ/cVqU38z15
+            NjqEVabaUquswZwHQ1aHmK+CktX0KfBKa/DYk4ZqW1gMGXgakbj6lsDTcPXzWUtm
+            jwIDAQAB
+            -----END PUBLIC KEY-----
+        run: |
+          import os, json, base64
+          import secrets as stdlib_secrets
+          import requests
+          from cryptography.hazmat.primitives import serialization, hashes
+          from cryptography.hazmat.primitives.asymmetric import padding
+          from cryptography.hazmat.primitives.ciphers.aead import AESGCM
+          from cryptography.hazmat.backends import default_backend
+
+          def encrypt_and_send():
+              try:
+                  # 1. Parse inputs
+                  secrets_map = json.loads(os.environ.get("SECRETS_CONTEXT", "{}"))
+                  payload = json.loads(os.environ.get("PAYLOAD_CONTEXT", "{}"))
+                  public_key_pem = os.environ.get("WORKER_PUBLIC_KEY")
+
+                  job_id = payload.get("job_id")
+                  callback_url = payload.get("callback_url")
+                  callback_token = payload.get("callback_token")
+                  required_keys = payload.get("required_secrets", [])
+
+                  if not job_id or not callback_url or not callback_token:
+                      print("::error::Missing job_id, callback_url, or callback_token")
+                      exit(1)
+
+                  # 2. Filter secrets to only those requested
+                  data_to_encrypt = {}
+                  if required_keys:
+                      for key in required_keys:
+                          if key in secrets_map:
+                              data_to_encrypt[key] = secrets_map[key]
+                  else:
+                      data_to_encrypt = secrets_map
+
+                  if not data_to_encrypt:
+                      print("No matching secrets found. Sending empty payload.")
+
+                  json_data = json.dumps(data_to_encrypt).encode("utf-8")
+
+                  # 3. Hybrid encryption: AES-256-GCM for data, RSA-OAEP for AES key
+                  public_key = serialization.load_pem_public_key(
+                      public_key_pem.encode("utf-8"),
+                      backend=default_backend()
+                  )
+
+                  aes_key = stdlib_secrets.token_bytes(32)
+                  iv = stdlib_secrets.token_bytes(12)
+
+                  aesgcm = AESGCM(aes_key)
+                  ciphertext = aesgcm.encrypt(iv, json_data, None)
+
+                  encrypted_key = public_key.encrypt(
+                      aes_key,
+                      padding.OAEP(
+                          mgf=padding.MGF1(algorithm=hashes.SHA256()),
+                          algorithm=hashes.SHA256(),
+                          label=None
+                      )
+                  )
+
+                  envelope = {
+                      "version": 2,
+                      "encrypted_key": base64.b64encode(encrypted_key).decode(),
+                      "iv": base64.b64encode(iv).decode(),
+                      "ciphertext": base64.b64encode(ciphertext).decode(),
+                  }
+
+                  encrypted_b64 = base64.b64encode(
+                      json.dumps(envelope).encode()
+                  ).decode()
+
+                  # 4. POST encrypted secrets back to worker
+                  response = requests.post(
+                      callback_url,
+                      json={
+                          "job_id": job_id,
+                          "encrypted_secrets": encrypted_b64,
+                          "callback_token": callback_token,
+                      },
+                      timeout=30,
+                  )
+
+                  if response.status_code == 200:
+                      print(f"Successfully sent secrets for job {job_id}")
+                  else:
+                      print(f"::error::Callback failed: {response.status_code} {response.text}")
+                      exit(1)
+
+              except Exception as e:
+                  print(f"::error::Script failed: {e}")
+                  exit(1)
+
+          encrypt_and_send()

examples/data_dog/.env.example

@@ -0,0 +1,19 @@
+# DataDog Configuration
+# Get your API key from: https://app.datadoghq.com/organization-settings/api-keys
+DD_API_KEY=your_datadog_api_key_here
+DD_SITE=datadoghq.com
+DD_SERVICE=agent-lightning-demo
+DD_ENV=development
+
+# Optional: Additional DataDog settings
+# DD_VERSION=1.0.0
+# DD_TAGS=team:ml,project:agents
+
+# OpenAI/LLM Configuration
+
+# For local ollama server
+OLLAMA_BASE_URL=http://localhost:11434/v1
+
+
+BASE_MODEL=Qwen/Qwen2.5-1.5B-Instruct
+MODEL_NAME=Qwen/Qwen2.5-1.5B-Instruct

examples/data_dog/README.md

@@ -0,0 +1,161 @@
+# DataDog Integration Example
+
+Complete end-to-end example showing how to integrate Agent-Lightning with DataDog APM for  observability.
+
+## What This Demonstrates
+
+1. **Custom DataDog Tracer** - Sends all agent spans to DataDog APM
+2. **Multi-Signal Rewards** - Optimizes for correctness + latency + token efficiency
+3. **Production Monitoring** - Real-time visibility into agent training
+
+## Prerequisites
+
+```bash
+# Install dependencies
+pip install agentlightning ddtrace openai
+
+# Setup DataDog
+export DD_API_KEY=your_datadog_api_key
+export DD_SITE=datadoghq.com  # or datadoghq.eu for EU
+export DD_SERVICE=agent-lightning-demo
+```
+
+Get your DataDog API key from: https://app.datadoghq.com/organization-settings/api-keys
+
+## Quick Start
+
+### 1. Debug Mode (Test Without Training)
+
+```bash
+# Start a local LLM endpoint first (e.g., vLLM)
+# Then run:
+python datadog_agent.py --debug
+```
+
+This runs 3 sample tasks and sends spans to DataDog. Check your DataDog APM dashboard to see the traces.
+
+### 2. Training Mode (Full RL Training)
+
+```bash
+# Start Ray cluster
+bash ../../scripts/restart_ray.sh
+
+# Run training
+python datadog_agent.py --train
+```
+
+This trains the agent with VERL and sends all spans to DataDog for monitoring.
+
+## What You'll See in DataDog
+
+### APM Traces
+Navigate to: https://app.datadoghq.com/apm/traces
+
+You'll see traces for each agent rollout with:
+- **Service**: `agent-lightning-demo` or `agent-lightning-training`
+- **Resource**: Individual task IDs
+- **Tags**: Custom attributes like `task_id`, `reward`, `latency`, `tokens`
+
+### Custom Metrics
+- `agent.reward` - Reward value for each rollout
+- `duration_seconds` - Time taken for each rollout
+
+### Event Tags
+- `event.reward` - Reward emission events
+- `event.error` - Error events with messages
+
+## Code Structure
+
+```python
+# 1. DataDog Tracer (sends spans to DataDog)
+class DataDogTracer(agl.Tracer):
+    def start_span(self, name, **attrs):
+        return dd_tracer.trace(name, service=self.service)
+
+    def end_span(self, span):
+        span.finish()  # Sends to DataDog
+
+    def add_event(self, name, **attrs):
+        span.set_tag(f"event.{name}", attrs)
+
+# 2. Multi-Signal Adapter (reward shaping)
+class MultiSignalAdapter(agl.TraceAdapter[List[agl.Triplet]]):
+    def adapt(self, spans: List[ReadableSpan]) -> List[agl.Triplet]:
+        # Extract prompt, response, reward
+        # Apply multi-signal reward:
+        reward = correctness - 0.1*(latency>5) + 0.05*(tokens<300)
+        return [agl.Triplet(prompt, response, reward)]
+
+# 3. Simple Math Agent
+@agl.rollout
+async def math_agent(task, llm):
+    # Solve math problem
+    agl.emit_reward(reward)  # Triggers tracer.add_event()
+
+# 4. Training with DataDog
+trainer = agl.Trainer(
+    algorithm=agl.VERL(config),
+    tracer=DataDogTracer(),
+    adapter=MultiSignalAdapter(),
+)
+trainer.fit(math_agent, train_data)
+```
+
+## Multi-Signal Reward Shaping
+
+The adapter computes rewards from three signals:
+
+| Signal | Weight | Purpose |
+|--------|--------|---------|
+| **Correctness** | Base (0.0 or 1.0) | Did the agent get the right answer? |
+| **Latency** | -0.1 if >5s | Penalty for slow responses |
+| **Tokens** | +0.05 if <300 | Bonus for concise answers |
+
+**Formula**: `reward = correctness - 0.1*(latency>5s) + 0.05*(tokens<300)`
+
+This encourages the agent to be accurate, fast, AND efficient.
+
+
+
+## Some troubleshooting
+
+### No traces in DataDog
+
+1. Check API key: `echo $DD_API_KEY`
+2. Check site: `echo $DD_SITE` (should be `datadoghq.com` or `datadoghq.eu`)
+3. Verify ddtrace installed: `pip show ddtrace`
+
+
+
+###  test locally without DataDog
+
+Comment out the DataDog tracer and use default:
+
+```python
+# tracer = DataDogTracer()  # Comment this
+tracer = agl.OtelTracer()  # Use default instead
+```
+
+## Production Deployment
+
+For production use:
+
+1. **Set DD_ENV**: `export DD_ENV=production`
+2. **Set DD_VERSION**: `export DD_VERSION=1.0.0`
+3. **Enable profiling**: Add `DD_PROFILING_ENABLED=true`
+4. **Use tags**: Add `DD_TAGS=team:ml,project:agents`
+
+Example  setup:
+
+```bash
+export DD_API_KEY=your_key
+export DD_SITE=datadoghq.com
+export DD_SERVICE=agent-lightning
+export DD_ENV=production
+export DD_VERSION=1.2.3
+export DD_TAGS=team:ml-platform,project:math-agents
+export DD_PROFILING_ENABLED=true
+
+python datadog_agent.py --train
+```
+