|
1 | | -# Imgproxy to Tigris Proxy |
| 1 | +# ImgProxy cache |
2 | 2 |
|
3 | | - |
| 3 | +A transparent caching reverse proxy for [imgproxy](https://imgproxy.net/) that automatically uploads processed images to S3-compatible storage (Tigris, AWS S3, MinIO, etc.). |
4 | 4 |
|
5 | | -## Features |
6 | | -- Dual-process architecture (Go proxy + imgproxy) |
7 | | -- Non-blocking S3 writes with best-effort cleanup |
8 | | -- Namespaced environment configuration |
9 | | -- Structured error logging |
| 5 | +## What It Does |
10 | 6 |
|
11 | | -## Security |
12 | | -- Automatic CVE scanning |
13 | | -- Distroless base image |
14 | | -- Signed containers |
| 7 | +This application acts as a transparent layer in front of imgproxy: |
15 | 8 |
|
16 | | -## Environment Variables |
17 | | -| Prefix | Purpose | |
18 | | -|--------|---------| |
19 | | -| `PROXY_*` | Go proxy configuration | |
20 | | -| `IMGPROXY_*` | Native imgproxy settings | |
| 9 | +1. **Receives** image processing requests |
| 10 | +2. **Proxies** them to imgproxy for processing |
| 11 | +3. **Returns** the processed image to the client immediately |
| 12 | +4. **Uploads** the processed image to Tigris or S3 asynchronously for future use |
| 13 | + |
| 14 | +The upload happens in the background, so client responses are not delayed. This creates a "cache-on-write" pattern where every successfully processed image is automatically stored in the target bucket. |
| 15 | + |
| 16 | +## Architecture |
| 17 | + |
| 18 | +``` |
| 19 | +Client Request |
| 20 | + ↓ |
| 21 | +[imgproxy-cache :8080] ← This application |
| 22 | + ↓ |
| 23 | +Response → Client (immediate) |
| 24 | + ↓ |
| 25 | +S3 Upload (background) |
| 26 | +``` |
| 27 | + |
| 28 | +The proxy: |
| 29 | +- Buffers the complete response in memory |
| 30 | +- Sends it immediately to the client |
| 31 | +- Spawns a goroutine to upload to S3 |
| 32 | +- Logs upload success/failure without blocking |
| 33 | + |
| 34 | +## Use Cases |
| 35 | + |
| 36 | +- **Persistent Cache**: Ensure processed images are stored durably, even if imgproxy's local cache is cleared |
| 37 | +- **Multi-Region**: Process images once, store in Tigris or S3, serve from multiple regions |
| 38 | +- **Cost Optimization**: Reduce repeated processing of the same images |
| 39 | + |
| 40 | +## Installation |
| 41 | + |
| 42 | +### Using Docker (Recommended) |
| 43 | + |
| 44 | +The Docker image includes both imgproxy (v3.30) and the Go proxy in a single container: |
21 | 45 |
|
22 | | -## Deployment |
23 | 46 | ```bash |
24 | | -docker build -t imgproxy-tigris . |
| 47 | +docker build -t imgproxy-cache . |
| 48 | +docker run -p 8080:8080 \ |
| 49 | + -e S3_BUCKET="your-bucket" \ |
| 50 | + -e AWS_ACCESS_KEY_ID="your-key" \ |
| 51 | + -e AWS_SECRET_ACCESS_KEY="your-secret" \ |
| 52 | + imgproxy-cache |
| 53 | +``` |
| 54 | + |
| 55 | +When the container starts: |
| 56 | +1. imgproxy starts on `127.0.0.1:8081` (internal) |
| 57 | +2. The Go proxy starts on `:8080` (exposed) |
| 58 | +3. Both processes run under supervision - if either exits, the container stops |
25 | 59 |
|
| 60 | +You can pass imgproxy-specific configuration via environment variables prefixed with `IMGPROXY_`: |
| 61 | + |
| 62 | +```bash |
26 | 63 | docker run -p 8080:8080 \ |
27 | | - -e PROXY_S3_ENDPOINT="your.tigris.endpoint" \ |
28 | | - -e PROXY_S3_BUCKET="your-bucket" \ |
29 | | - -e PROXY_S3_REGION="auto" \ |
| 64 | + -e S3_BUCKET="your-bucket" \ |
| 65 | + -e AWS_ACCESS_KEY_ID="your-key" \ |
| 66 | + -e AWS_SECRET_ACCESS_KEY="your-secret" \ |
30 | 67 | -e IMGPROXY_MAX_SRC_RESOLUTION=16384 \ |
31 | | - imgproxy-tigris |
| 68 | + -e IMGPROXY_QUALITY=90 \ |
| 69 | + imgproxy-cache |
32 | 70 | ``` |
33 | 71 |
|
34 | | -## Local Development |
| 72 | +See [imgproxy documentation](https://docs.imgproxy.net/configuration) for all available options. |
| 73 | + |
| 74 | +### From Source |
| 75 | + |
35 | 76 | ```bash |
36 | | -docker-compose -f docker-compose.local.yml up --build |
37 | | -aws --endpoint-url=http://localhost:4566 s3 mb s3://test-bucket |
| 77 | +go build -o imgproxy-cache . |
| 78 | +./imgproxy-cache |
38 | 79 | ``` |
39 | 80 |
|
40 | | -## Operational Notes |
41 | | -- S3 writes happen in parallel with client streaming |
42 | | -- Partial uploads are automatically cleaned up |
43 | | -- Process crashes are handled by supervisord |
| 81 | +**Note**: When running from source, you must have imgproxy running separately on `localhost:8081`. The application will wait up to 30 seconds for imgproxy to become healthy before starting. |
| 82 | + |
| 83 | +## Configuration |
| 84 | + |
| 85 | +### Environment Variables |
| 86 | + |
| 87 | +| Variable | Required | Default | Description | |
| 88 | +|----------|----------|---------|-------------| |
| 89 | +| `S3_BUCKET` | **Yes** | - | S3 bucket name where images will be stored | |
| 90 | +| `S3_FOLDER` | No | `""` | Prefix/folder path within the bucket | |
| 91 | +| `S3_ENDPOINT` | No | `https://fly.storage.tigris.dev` | S3-compatible endpoint URL | |
| 92 | +| `IMGPROXY_BIND` | No | `:8080` | Address and port for the proxy to bind to | |
| 93 | +| `HEALTH_CHECK_TIMEOUT_IN_SEC` | No | `30` | Seconds to wait for imgproxy to become healthy | |
| 94 | + |
| 95 | +### AWS Credentials |
| 96 | + |
| 97 | +The application uses the AWS SDK v2, which automatically loads credentials from: |
| 98 | +- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) |
| 99 | +- Shared credentials file (`~/.aws/credentials`) |
| 100 | +- IAM roles (when running on EC2/ECS/Lambda) |
| 101 | +- Web identity tokens (when running on EKS) |
| 102 | + |
| 103 | +See [AWS SDK documentation](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/) for full details. |
| 104 | + |
| 105 | +## How Caching Works |
| 106 | + |
| 107 | +### Key Generation |
| 108 | + |
| 109 | +S3 keys are generated by MD5 hashing the imgproxy URL path: |
| 110 | + |
| 111 | +``` |
| 112 | +Request: /resize:fill:300:300/plain/https://example.com/image.jpg |
| 113 | +S3 Key: a3f8c9d2e1b4f7a6c8d9e2f1b3a4c5d6 |
| 114 | +``` |
| 115 | + |
| 116 | +This ensures: |
| 117 | +- **Consistent**: Same URL always maps to the same S3 key |
| 118 | +- **Compact**: Keys are fixed-length 32 characters |
| 119 | +- **Safe**: No special characters or path traversal issues |
| 120 | + |
| 121 | +### Upload Behavior |
| 122 | + |
| 123 | +- **Only successful responses** (HTTP 200) are uploaded |
| 124 | +- **Uploads are asynchronous** - client doesn't wait for S3 confirmation |
| 125 | +- **Failed uploads are logged** but don't affect the client response |
| 126 | +- **No deduplication** - same request will re-upload (consider implementing checks) |
| 127 | + |
| 128 | +### Storage Structure |
| 129 | + |
| 130 | +``` |
| 131 | +s3://your-bucket/ |
| 132 | + └── processed/ (if S3_FOLDER is set) |
| 133 | + ├── a3f8c9d2e1b4f7a6... (image 1) |
| 134 | + ├── b2e7d8c1f0a9e3b7... (image 2) |
| 135 | + └── c9f1a2b3e4d5c6a7... (image 3) |
| 136 | +``` |
| 137 | + |
| 138 | +## Usage Example |
| 139 | + |
| 140 | +### Start the Service |
| 141 | + |
| 142 | +```bash |
| 143 | +export S3_BUCKET="my-images" |
| 144 | +export AWS_ACCESS_KEY_ID="your-key" |
| 145 | +export AWS_SECRET_ACCESS_KEY="your-secret" |
| 146 | + |
| 147 | +./imgproxy-cache |
| 148 | +``` |
| 149 | + |
| 150 | +### Process an Image |
| 151 | + |
| 152 | +```bash |
| 153 | +# Request an image through the proxy |
| 154 | +curl http://localhost:8080/resize:fill:300:300/plain/https://example.com/cat.jpg > output.jpg |
| 155 | + |
| 156 | +# The processed image is: |
| 157 | +# 1. Returned immediately to your curl command |
| 158 | +# 2. Uploaded to S3 in the background |
| 159 | +``` |
| 160 | + |
| 161 | +### Check Logs |
| 162 | + |
| 163 | +``` |
| 164 | +2025/10/20 10:30:00 INFO Waiting for imgproxy to be ready... |
| 165 | +2025/10/20 10:30:01 INFO imgproxy is ready |
| 166 | +2025/10/20 10:30:15 INFO Uploaded to S3 path=/resize:fill:300:300/plain/https://example.com/cat.jpg bucket=my-images key=a3f8c9d2e1b4f7a6c8d9e2f1b3a4c5d6 |
| 167 | +``` |
| 168 | + |
| 169 | +## Example client code |
| 170 | +### HTML |
| 171 | +```html |
| 172 | +<img |
| 173 | + src="https://process-image-url" |
| 174 | + onerror="this.onerror=null; this.src='https://proxy-url/image-processing-params/original-image-url';" |
| 175 | +/> |
| 176 | +``` |
| 177 | + |
| 178 | +### Elixir |
| 179 | + |
| 180 | +```elixir |
| 181 | + @doc """ |
| 182 | + Renders an image with proxy URL transformation. |
| 183 | + The image URL will be transformed to: <image_proxy_url>/<dimensions>/<image_url> |
| 184 | + """ |
| 185 | + attr :src, :string, required: true |
| 186 | + attr :dimensions, :string, required: true |
| 187 | + attr :resize_mode, :string, default: "fit", values: ["fit", "fill"] |
| 188 | + attr :class, :string, default: nil |
| 189 | + attr :alt, :string, default: nil |
| 190 | + |
| 191 | + def image(assigns) do |
| 192 | + img_path = |
| 193 | + "/rs:#{assigns.resize_mode}:#{assigns.dimensions}:1/dpr:2/g:ce/" <> |
| 194 | + Base.encode64(assigns.src) <> ".webp" |
| 195 | + |
| 196 | + signature = |
| 197 | + :crypto.mac( |
| 198 | + :hmac, |
| 199 | + :sha256, |
| 200 | + Base.decode16!( |
| 201 | + System.get_env("IMGPROXY_KEY"), |
| 202 | + case: :lower |
| 203 | + ), |
| 204 | + Base.decode16!( |
| 205 | + System.get_env("IMGPROXY_SALT"), |
| 206 | + case: :lower |
| 207 | + ) <> img_path |
| 208 | + ) |
| 209 | + |> Base.url_encode64(padding: false) |
| 210 | + |
| 211 | + full_path = "/" <> signature <> img_path |
| 212 | + |
| 213 | + assigns = |
| 214 | + assigns |
| 215 | + |> assign(:proxy_src, "#{Application.get_env(:manage, :image_proxy_url)}#{full_path}") |
| 216 | + |> assign( |
| 217 | + :cached_src, |
| 218 | + Application.get_env(:manage, :image_cache_url) <> |
| 219 | + "/" <> |
| 220 | + (:crypto.hash(:md5, full_path) |
| 221 | + |> Base.encode16(case: :lower)) |
| 222 | + ) |
| 223 | + |
| 224 | + ~H""" |
| 225 | + <img |
| 226 | + src={@cached_src} |
| 227 | + class={@class} |
| 228 | + alt={@alt} |
| 229 | + onerror={"this.onerror=null; this.src='#{@proxy_src}';"} |
| 230 | + /> |
| 231 | + """ |
| 232 | + end |
| 233 | +``` |
| 234 | + |
| 235 | +## Development |
| 236 | + |
| 237 | +### Testing |
| 238 | + |
| 239 | +```bash |
| 240 | +go test -v ./... |
| 241 | +``` |
| 242 | + |
| 243 | +## Limitations & Considerations |
| 244 | + |
| 245 | +- **Memory Usage**: Entire response is buffered in memory before upload |
| 246 | +- **No Retry Logic**: Failed S3 uploads are not retried |
| 247 | +- **No Deduplication**: Same image can be uploaded multiple times |
| 248 | +- **No Cleanup**: Old/unused images are never deleted from S3 |
| 249 | +- **No Validation**: Uploads happen even if the same key already exists in S3 |
| 250 | + |
| 251 | +## License |
| 252 | + |
| 253 | +[MIT LICENSE](./LICENSE.md) |
0 commit comments