EvalOps

All

44 repositories

cerebro
Public
Entity Intelligence Engine — security, business, and operational posture management
Go
•
Apache License 2.0
•0•0•15•1•Updated Mar 21, 2026Mar 21, 2026
diffscope
Public
A composable code review engine for automated diff analysis
Rust
•
Apache License 2.0
•2•12•44•2•Updated Mar 21, 2026Mar 21, 2026
keep
Public
PoC zero-trust access stack with Google SSO, Envoy, OPA, and device attestation
Go
•
Other
•0•0•0•6•Updated Mar 19, 2026Mar 19, 2026
fabric
Public
Agent Fabric - Reimagining Slack for AI agents
TypeScript
•0•0•0•0•Updated Mar 18, 2026Mar 18, 2026
verdict
Public
Inference-time scaling for LLMs-as-a-judge.
Jupyter Notebook
•
MIT License
•25•0•0•2•Updated Mar 15, 2026Mar 15, 2026
grimoire
Public
The AI coding agent built for the terminal.
TypeScript
•
MIT License
•13k•0•0•2•Updated Mar 15, 2026Mar 15, 2026
maestro
Public
Lightweight agent orchestration hooks for Claude Code and OpenAI Codex. Keep your coding agents working until the job is done.
Python
•
MIT License
•0•0•0•0•Updated Mar 14, 2026Mar 14, 2026
asb
Public
Agents-first secret broker control plane in Go
Go
•0•0•23•0•Updated Mar 12, 2026Mar 12, 2026
ensemble-tap
Public
Ingest SaaS webhooks, polls, and CDC into NATS JetStream + ClickHouse — gives Ensemble continuous awareness of customer business systems
go golang open-source
go golang open-source webhooks clickhouse nats ensemble cdc event-streaming cloudevents
Go
•0•0•0•0•Updated Mar 9, 2026Mar 9, 2026
open-associate-skills
Public
Skills bundle for simulating a top‑decile VC associate: market mapping, sourcing, diligence, memos, and portfolio ops.
skills salesforce diligence
skills salesforce diligence sourcing venture-capital vc agent-skills market-mapping investment-memo portfolio-ops
Python
•0•0•0•0•Updated Jan 29, 2026Jan 29, 2026
max
Public
Max - AI Agent Dashboard built with Next.js, TypeScript, and Tailwind CSS
TypeScript
•0•1•0•0•Updated Dec 27, 2025Dec 27, 2025
dspy-micro-agent
Public
Minimal agent runtime built with DSPy modules and a thin Python loop. Includes CLI, FastAPI server, and eval harness with OpenAI/Ollama support.
python agent cli
python agent cli ai openai fastapi dspy llm ollama agent-runtime
Python
•6•71•0•0•Updated Dec 22, 2025Dec 22, 2025
petri-tribunal
Public
Multi-critic alignment auditing: Petri + LLM Tribunal integration for robust AI safety evaluation
Python
•0•1•0•0•Updated Dec 16, 2025Dec 16, 2025
tokio-hotel
Public
Ephemeral, cancellable task rooms with scoped resources for Tokio runtimes
rust async concurrency
rust async concurrency tracing tokio cancellation resource-management
Rust
•
MIT License
•0•0•0•0•Updated Dec 6, 2025Dec 6, 2025
nimbus
Public
Self-hosted CI infrastructure optimized for AI evaluation workloads. Run evals on bare metal with Firecracker isolation, built-in observability, and zero cloud …
ci smith github-actions
ci smith github-actions firecracker
Python
•0•7•0•1•Updated Nov 10, 2025Nov 10, 2025
cursor-triplane
Public
Cursor-style tri-plane RL stack: Firecracker microVM envs, Ray inference, FP8 MoE trainer
Python
•0•1•0•0•Updated Nov 8, 2025Nov 8, 2025
agent-pm
Public
Agent PM: OpenAI Agents-powered product management orchestrator with automated PRDs, tickets, and comms.
product-management fastapi openai-agents
product-management fastapi openai-agents evalops agent-pm
Python
•
MIT License
•2•5•0•0•Updated Oct 21, 2025Oct 21, 2025
provenance
Public
Agent provenance & risk analytics API: FastAPI service with Semgrep detection, Redis, analytics dashboards
redis security analytics
redis security analytics fastapi streamlit semgrep
Python
•0•0•0•0•Updated Oct 17, 2025Oct 17, 2025
cognitive-dissonance-dspy
Public
A multi-agent LLM system for detecting and resolving cognitive dissonance.
Python
•22•276•0•1•Updated Oct 14, 2025Oct 14, 2025
meta-secdev-agent
Public
Static analysis meta agent for security and tenant isolation workflows
Python
•0•0•0•0•Updated Oct 13, 2025Oct 13, 2025
github-ona
Public
Organizational Network Analysis tool for GitHub repositories - analyze collaboration patterns through PR and review data
Python
•0•0•0•0•Updated Oct 11, 2025Oct 11, 2025
env-drift-detector
Public
Detect environment variable drift across monorepo services using OpenAI Codex SDK - catches naming inconsistencies, missing vars, and undocumented secrets
devops monorepo environment-variables
devops monorepo environment-variables configuration-management openai developer-tools codex drift-detection
TypeScript
•0•1•0•0•Updated Oct 8, 2025Oct 8, 2025
tenant-isolation-guard
Public
Detect missing tenant isolation in multi-tenant FastAPI apps using Codex SDK - prevents cross-tenant data leaks by catching missing workspace filters
security sqlalchemy multi-tenant
security sqlalchemy multi-tenant typescript saas openai codex fastapi tenant-isolation
TypeScript
•0•0•0•0•Updated Oct 8, 2025Oct 8, 2025
tinker-eval-loop
Public
Evaluation-driven LoRA fine-tuning loop using Tinker API - automatically improve models through iterative training and evaluation cycles
machine-learning evaluation lora
machine-learning evaluation lora tinker fine-tuning llm evalops inspect-ai
Python
•0•0•0•0•Updated Oct 2, 2025Oct 2, 2025
stack
Public
A cohesive Transformers stack built on PyTorch with uv for reproducible dependency management
Python
•0•0•0•0•Updated Sep 30, 2025Sep 30, 2025
mocktopus
Public
🐙 Multi-armed mocks for LLM apps - Drop-in replacement for OpenAI/Anthropic APIs for deterministic testing
python testing continuous-integration
python testing continuous-integration mock-server evaluation pytest openai developer-tools testing-tools deterministic
Python
•
Other
•0•6•0•0•Updated Sep 24, 2025Sep 24, 2025
metaethical_breach_dspy
Public
DSPy implementation for detecting metaethical breaches in AI systems through systematic evaluation of moral reasoning patterns
benchmark machine-learning evaluation
benchmark machine-learning evaluation ai-safety ethics ai-alignment dspy moral-reasoning
Python
•
MIT License
•0•0•0•0•Updated Sep 23, 2025Sep 23, 2025
dspy-lean-prover-hint-clipping
Public
DSPy + Lean (mock) iterative prover with hint clipping; sweeps on clipping vs KL, noise, sparsity; scalable dataset generator; curated training; frozen tools.
experiment evaluation program-synthesis
experiment evaluation program-synthesis dataset rl lean clipping variance-reduction ppo tool-use
Python
•0•2•0•0•Updated Sep 19, 2025Sep 19, 2025
EvalATS
Public
Modern, AI-powered Applicant Tracking System with evaluation focus
TypeScript
•3•4•0•0•Updated Sep 18, 2025Sep 18, 2025
override-cascade-dspy
Public
DSPy framework for detecting and preventing safety override cascades in LLM systems. Research-grade implementation for studying when completion urgency override…
python machine-learning ai-safety
python machine-learning ai-safety cognitive-architecture research-framework dspy llm-evaluation llm-safety safety-research override-cascade
Python
•
MIT License
•0•5•0•0•Updated Sep 14, 2025Sep 14, 2025