Open-Tinker

Note: This project is mostly LLM-generated with minimal human supervision. Use at your own risk.

Self-hosted Tinker API compatible training service for LoRA adapter training

Open-Tinker is a fully compatible implementation of the Tinker Training API, designed to be a drop-in replacement for the official Tinker service. It enables LoRA (Low-Rank Adaptation) training for large language models using the adapters library.

Features

• ✅ Full Tinker SDK Compatibility: Works seamlessly with the official Tinker Python SDK v0.14.0
• ✅ LoRA Training: Efficient adapter-based fine-tuning using the adapters library
• ✅ Async Operations: Celery-based task queue for background training
• ✅ Checkpoint Management: Save, load, and manage model checkpoints with tinker:// paths
• ✅ REST API: FastAPI-based service with automatic OpenAPI documentation
• ✅ Docker Ready: Easy deployment with Docker and docker-compose
• ✅ Production Ready: PostgreSQL database, Redis task queue, GPU support

Quick Start

Prerequisites

• Python 3.12+
• PostgreSQL (for database)
• Redis (for task queue)
• CUDA-capable GPU (recommended for training)
• uv package manager

Installation

1. Clone the repository

git clone https://github.com/your-org/open-tinker.git
cd open-tinker

1. Install dependencies with uv

uv sync

1. Set up environment

cp .env.example .env
# Edit .env with your configuration

1. Start PostgreSQL and Redis

Using Docker:

docker run -d --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:16
docker run -d --name redis -p 6379:6379 redis:7

1. Run the server

# Start the API server
uv run python -m open_tinker.main

# Or with uvicorn directly
uv run uvicorn open_tinker.main:app --host 0.0.0.0 --port 8000 --reload

1. Start the Celery worker (in a separate terminal)

uv run celery -A open_tinker.workers.celery_app worker --loglevel=info

The API will be available at http://localhost:8000. Visit http://localhost:8000/docs for the interactive API documentation.

Docker Deployment

Using docker-compose (Recommended)

docker-compose up -d

This will start:

• Open-Tinker API server
• Celery worker with GPU support
• PostgreSQL database
• Redis cache

Manual Docker Build

# Build the image
docker build -t open-tinker:latest .

# Run the container
docker run -d \
  --name open-tinker-api \
  --gpus all \
  -p 8000:8000 \
  -e DATABASE_URL=postgresql://postgres:postgres@db:5432/opentinker \
  -e REDIS_URL=redis://redis:6379/0 \
  open-tinker:latest

SDK Compatibility

Open-Tinker targets Tinker SDK v0.14.0. It implements the full Tinker Training API specification and is designed as a drop-in backend for the official SDK. Older SDK versions are not supported.

Usage with Tinker SDK

Open-Tinker is designed to work with the official Tinker SDK v0.14.0:

import tinker
from tinker import types

# Point the SDK to your Open-Tinker instance
service_client = tinker.ServiceClient(
    base_url="http://localhost:8000",
    api_key="tml-your-api-key",
)

# Check available models
capabilities = service_client.get_server_capabilities()
print("Available models:", [m.model_name for m in capabilities.supported_models])

# Create a LoRA training client
training_client = service_client.create_lora_training_client(
    base_model="Qwen/Qwen2.5-0.5B",
    rank=32
)

tokenizer = training_client.get_tokenizer()

# Create training data
datum = types.Datum(
    model_input=types.ModelInput.from_ints(tokenizer.encode("Hello world")),
    loss_fn_inputs={
        "target_tokens": types.TensorData.from_ints(tokenizer.encode(" world!")),
        "weights": types.TensorData.from_floats([1.0] * len(tokenizer.encode(" world!"))),
    }
)

# Forward-backward pass
result = training_client.forward_backward([datum], "cross_entropy").get()

# Optimizer step
training_client.optim_step(
    types.AdamParams(learning_rate=1e-4)
).get()

# Save checkpoint
checkpoint_path = training_client.save_state(name="checkpoint-001").get().path
print(f"Checkpoint saved: {checkpoint_path}")

# Create sampling client and generate text
sampling_client = training_client.save_weights_and_get_sampling_client()
prompt = types.ModelInput.from_ints(tokenizer.encode("Hello"))
response = sampling_client.sample(
    prompt=prompt,
    num_samples=1,
    sampling_params=types.SamplingParams(max_tokens=20, temperature=0.7),
).get()
print("Generated:", tokenizer.decode(response.sequences[0].tokens))

Configuration

Configuration is managed through environment variables (.env file). See .env.example for a complete template.

Variable	Default	Description
API_KEY_PREFIX	tml-	Required prefix for API keys
API_KEYS	[]	Explicit list of allowed API keys (empty = any key with valid prefix)
HOST	0.0.0.0	Server bind address
PORT	8000	Server port
DATABASE_URL	postgresql://postgres:postgres@localhost:5432/opentinker	PostgreSQL connection string
REDIS_URL	redis://localhost:6379/0	Redis connection string
CELERY_BROKER_URL	redis://localhost:6379/0	Celery broker URL
CELERY_RESULT_BACKEND	redis://localhost:6379/0	Celery result backend URL
CHECKPOINT_STORAGE_PATH	./checkpoints	Path for checkpoint storage
CHECKPOINT_STORAGE_TYPE	filesystem	Storage backend (filesystem or s3)
MODELS_CACHE_DIR	./models_cache	Path for model cache
SUPPORTED_MODELS	["Qwen/Qwen2.5-0.5B",...]	List of supported base models
CUDA_VISIBLE_DEVICES	0	GPU device(s) to use
LOG_LEVEL	INFO	Logging level

API Endpoints

Open-Tinker implements the full Tinker API specification:

Service Endpoints

• GET /api/v1/healthz - Health check
• GET /api/v1/get_server_capabilities - Get supported models
• POST /api/v1/create_session - Create a new session
• POST /api/v1/session_heartbeat - Keep a session alive

Training Endpoints

• POST /api/v1/create_model - Create new LoRA training model
• POST /api/v1/forward_backward - Execute forward/backward pass
• POST /api/v1/forward - Forward-only pass (no gradients)
• POST /api/v1/optim_step - Apply optimizer step
• POST /api/v1/save_weights - Save training checkpoint
• POST /api/v1/save_weights_for_sampler - Save weights for sampling
• POST /api/v1/load_weights - Load checkpoint
• POST /api/v1/unload_model - Unload model from memory
• POST /api/v1/get_info - Get model info
• POST /api/v1/weights_info - Get checkpoint info by tinker path

Sampling Endpoints

• POST /api/v1/create_sampling_session - Create sampling session
• POST /api/v1/asample - Generate text samples

Async / Futures

• POST /api/v1/retrieve_future - Poll for async operation result

Management Endpoints

• GET /api/v1/training_runs - List training runs
• GET /api/v1/training_runs/{training_run_id} - Get a training run
• GET /api/v1/training_runs/{training_run_id}/checkpoints - List checkpoints
• DELETE /api/v1/training_runs/{training_run_id}/checkpoints/{checkpoint_id} - Delete checkpoint
• GET /api/v1/training_runs/{training_run_id}/checkpoints/{checkpoint_id}/archive - Download checkpoint
• POST /api/v1/training_runs/{training_run_id}/checkpoints/{checkpoint_id}/publish - Publish checkpoint
• DELETE /api/v1/training_runs/{training_run_id}/checkpoints/{checkpoint_id}/publish - Unpublish checkpoint
• PUT /api/v1/training_runs/{training_run_id}/checkpoints/{checkpoint_id}/ttl - Set checkpoint TTL
• GET /api/v1/checkpoints - List all checkpoints
• GET /api/v1/sessions - List sessions
• GET /api/v1/sessions/{session_id} - Get session details
• GET /api/v1/samplers/{sampler_id} - Get sampler details

Telemetry

• POST /api/v1/telemetry - Accept SDK telemetry (no-op stub)

See the API documentation for complete details.

Development

Running Tests

# Install dev dependencies
uv sync --extra dev

# Run tests
uv run pytest

# Run with coverage
uv run pytest --cov=open_tinker --cov-report=html

Code Quality

This project uses Ruff for both formatting and linting.

# Format code
uv run ruff format src/ tests/

# Lint code
uv run ruff check src/ tests/

# Lint and auto-fix issues
uv run ruff check --fix src/ tests/

Architecture

┌─────────────────┐
│  Tinker SDK     │
│  (Client)       │
└────────┬────────┘
         │ HTTP/REST
         ▼
┌─────────────────────────────────────────┐
│         FastAPI Application             │
│  ┌─────────────────────────────────┐   │
│  │  API Routes (Training/Sampling) │   │
│  └─────────────┬───────────────────┘   │
│                │                         │
│  ┌─────────────▼───────────────────┐   │
│  │    Authentication Middleware     │   │
│  └─────────────┬───────────────────┘   │
└────────────────┼─────────────────────────┘
                 │
    ┌────────────┼────────────┐
    │            │            │
    ▼            ▼            ▼
┌────────┐  ┌────────┐  ┌─────────┐
│  DB    │  │ Redis  │  │ Celery  │
│  (PG)  │  │ (Queue)│  │ Workers │
└────────┘  └────────┘  └────┬────┘
                              │
                              ▼
                    ┌──────────────────┐
                    │  Training Engine │
                    │  (adapters lib)  │
                    └──────────────────┘
                              │
                              ▼
                    ┌──────────────────┐
                    │   GPU / Models   │
                    └──────────────────┘

Contributing

Contributions are welcome! Please read our contributing guidelines and code of conduct.

License

MIT License

Acknowledgments

• Tinker - Original API specification
• adapters - LoRA training backend
• FastAPI - Web framework
• Celery - Task queue

Support

• Documentation: http://localhost:8000/docs
• Issues: GitHub Issues