Introduction

Orion Proxy is a high performance and memory safe implementation of popular Envoy Proxy. Orion Proxy is implemented in Rust using high-quality open source components.

Key features

Memory Safety

Rust programming language allows Orion Proxy implementation to avoid a whole set of bugs related to memory management and data races making Orion Proxy a very robust and secure application.

Performance

Orion Proxy offers 2x-4x better throughput and latency than Envoy Proxy. Refer to Performance to see more performance figures and more details on how Orion Proxy was tested.

HTTP Benchmark Results
Requests per Second	Latency (microseconds)
HTTPS Benchmark Results
Requests per Second	Latency (microseconds)

Use Cases and Compatibility

Orion Proxy configuration is generated from Envoy's xDS protobuf definitions. Orion Proxy aims to be a drop in replacement for Envoy for the most common or popular use-cases.

Kubernetes Gateway - Orion Proxy can be used as Kubernetes Gateway API and Orion Proxy is passing basic conformance tests. See Kubvernor documentation on how to run conformance tests with Orion Proxy. Kubernetes Gateway for Inference Flows - Orion Proxy can also be used to route Inference Flows and it is passing Gateway API Inference Extension conformance tests. See Kubvernor documentation on how to run inference extension conformance tests with Orion Proxy.

Architecture

Orion Proxy is designed as a high-performance L7 proxy compatible with Envoy's xDS API while delivering superior performance through Rust's zero-cost abstractions and memory safety guarantees.

Orion Proxy has been built on a share-nothing principle. In the default configuration, Orion Proxy tries to create and pin one instance of Tokio Runtime per CPU/Thread, so the spawned tasks and actions are always executed in the local context minimizing cross CPU communication. Other modes are available through configuration settings.

Core Components

• xDS Client: Subscribes to and processes Envoy xDS APIs (LDS, RDS, CDS, EDS) for dynamic configuration updates
• Configuration Manager: Manages runtime configuration, converts Envoy protobuf definitions to Orion's internal representation
• Listener Manager: Handles incoming connections, applies listener filters (TLV, Proxy Protocol, TLS Inspector)
• Router (L7): HTTP routing, header manipulation, retries, timeouts, and traffic shifting
• Cluster Manager: Manages upstream clusters, implements load balancing algorithms (round-robin, least-request, random)
• Transport Layer: High-performance async I/O using Tokio, supports HTTP/1.1, HTTP/2, and raw TCP
• TLS Engine: Powered by rustls for memory-safe TLS/mTLS, client certificate validation
• Observability: Prometheus metrics export, OpenTelemetry tracing integration
• Admin Interface: HTTP API for runtime configuration inspection, metrics, and health checks

Key Design Principles

• Zero-Copy I/O: Minimizes memory allocations and copies through Rust's ownership system and Bytes buffers
• Async Runtime: Built on Tokio for efficient handling of thousands of concurrent connections
• Memory Safety: Eliminates entire classes of bugs (use-after-free, data races) through Rust's type system
• Envoy Compatibility: Direct protobuf compatibility with Envoy xDS APIs for seamless integration with Istio and other control planes

Building

git clone https://github.com/kmesh-net/orion
cd orion
git submodule init
git submodule update --force
cargo build

Kubernetes Integration

A great use case for Orion Proxy is to use Orion Proxy as Gateway API Service in Kubernetes environments.

Orion Proxy can be used as Kubernetes Gateway API and Orion Proxy is passing basic conformance tests. See Kubvernor documentation on how to run conformance tests with Orion Proxy.

Running

cargo run --bin orion -- --config orion/conf/orion-runtime.yaml

Docker

Build and run with Docker:

# Build
docker build -t orion-proxy -f docker/Dockerfile .

# Run
docker run -p 8000:8000 --name orion-proxy orion-proxy

# Verify service
curl -v http://localhost:8000/direct-response # Should return HTTP 200 with "meow! 🐱"

For detailed Docker configuration options, see docker/README.md.

Orion Configuration

Orion has two levels of configuration options.

Runtime configuration allows controlling how Orion uses the CPUs of the operating system. Orion exposes tuning parameters allowing fine-tuning how Orion's worker threads are pinned to CPUs/Threads.

Bootstrap configuration is the same as for Envoy bootstrap configuration.

See example config file for more details.

Testing with Backend Servers

For testing load balancing with real backend servers:

# Start two nginx containers
docker run -d -p 4001:80 --name backend1 nginx:alpine
docker run -d -p 4002:80 --name backend2 nginx:alpine

# Start Orion Proxy (uses host networking to access localhost:4001/4002)
docker run -d --network host --name orion-proxy orion-proxy

# Test load balancing
curl http://localhost:8000/  # Proxies to nginx backends!

# Cleanup
docker rm -f backend1 backend2 orion-proxy

Examples and Demos

TLV Listener Filter Demo

Orion includes a comprehensive TLV (Type-Length-Value) listener filter demo compatible with the Kmesh project for service mesh integration. This demo provides end-to-end testing of the TLV filter functionality.

To test the TLV filter:

cd examples/tlv-filter-demo
./test_tlv_config.sh

This demo will:

• Start Orion with TLV filter configuration matching Kmesh format
• Load the TLV listener filter using TypedStruct configuration
• Send actual TLV packets to test the filter functionality
• Extract and verify original destination information from TLV data
• Show debug logs confirming successful TLV processing
• Verify compatibility with Kmesh TLV configuration format

For detailed information, see examples/tlv-filter-demo/README.md.

License

Orion Proxy is licensed under the Apache License, Version 2.0.