Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a6f747c
initial docker config
kernelism May 25, 2025
408264a
some updates to docker setup
kernelism May 26, 2025
4944a01
ffmpeg missing issue fix
kernelism May 26, 2025
06d8cc0
Add brave search tool with references addition to prompts (#32)
gautamp8 May 27, 2025
adcdbb9
Fix list rendering
satwikkansal May 27, 2025
a284f1d
Merge pull request #33 from mxtoai/fix-format
satwikkansal May 27, 2025
c8a83c6
addressed comments
kernelism May 27, 2025
dfc71b2
Use gpt-4 group for research related handles
gautamp8 Jun 2, 2025
7e6b549
Add linkedin data api tools via RapidAPI
gautamp8 Jun 3, 2025
59ae145
Fix output formatting, have factcheck to include additional summary, …
satwikkansal Jun 3, 2025
d54f231
Merge pull request #36 from mxtoai/rapid-api
satwikkansal Jun 3, 2025
08c3681
Merge branch 'master' into fix-output
satwikkansal Jun 3, 2025
6a1d640
Merge pull request #35 from mxtoai/fix-output
satwikkansal Jun 3, 2025
6b80921
Improve background research prompting (#37)
gautamp8 Jun 3, 2025
4823d80
Temporarily remove whitelist for reputed email providers (#38)
gautamp8 Jun 3, 2025
cc5b693
Fix logging issues, make logger more comprehensive
satwikkansal Jun 3, 2025
a4376af
Fix table formatting
satwikkansal Jun 3, 2025
e44fc23
Fix more markdown issue, center align tables
satwikkansal Jun 3, 2025
347d18d
Merge pull request #39 from mxtoai/fix-output
satwikkansal Jun 3, 2025
cf52eaf
Improve prompting for background and schedule handles
gautamp8 Jun 4, 2025
d278836
Perfect the prompts for all handles (#44)
gautamp8 Jun 4, 2025
2c09994
Merge branch 'docker' of https://github.com/mxtoai/mxtoai-backend int…
kernelism Jun 4, 2025
9e4cf0f
fixed docker volumes, rabbitmq connection issue and smolagents versio…
kernelism Jun 4, 2025
b899cc1
update README with env information and docker setup
kernelism Jun 4, 2025
79e9970
Fix issue with bold markers being detected as list items
satwikkansal Jun 4, 2025
47c1cb6
Run ruff
satwikkansal Jun 4, 2025
c9c6e1c
Add PDF export handle and tool
gautamp8 Jun 4, 2025
309067f
Merge pull request #45 from mxtoai/fix-output
satwikkansal Jun 4, 2025
2f157ac
Add pdf export handle tests
gautamp8 Jun 4, 2025
0536e14
Add proper cleanup of generated files
gautamp8 Jun 4, 2025
3b27017
Improve watermark formatting
gautamp8 Jun 4, 2025
c0c9f5c
Remove deep research not allowed
gautamp8 Jun 4, 2025
e51c58b
Add docs
gautamp8 Jun 4, 2025
cd577b9
Update mxtoai/tasks.py
satwikkansal Jun 5, 2025
cbf0f4e
Address review comments
gautamp8 Jun 5, 2025
02e0802
Merge pull request #46 from mxtoai/export-handles
satwikkansal Jun 5, 2025
0dd63d2
Fix other domain validation issue
gautamp8 Jun 5, 2025
ed4b758
Merge pull request #47 from mxtoai/validation-issue
satwikkansal Jun 5, 2025
561e54e
Trigger automatic verification for non-existant users with custom dom…
gautamp8 Jun 5, 2025
4580043
Improve schedule prompt
gautamp8 Jun 5, 2025
432d2ba
Formatting fixes and run ruff
satwikkansal Jun 5, 2025
5b1a4c4
Simplify formatting and add tests
satwikkansal Jun 6, 2025
57a9f7a
Do away with keyword based approach for section markdown qualification
satwikkansal Jun 6, 2025
d5e493a
Add independent search tools and guidelines (#50)
gautamp8 Jun 6, 2025
2d54a91
Add AWS postmaster to whitelist
satwikkansal Jun 6, 2025
3a9706c
Add AWS postmaster to whitelist
satwikkansal Jun 6, 2025
25f12da
Ignore rmq logs
satwikkansal Jun 6, 2025
472160a
Merge pull request #49 from mxtoai/fix-output
satwikkansal Jun 6, 2025
666c68f
Skip processing AWS system emails
satwikkansal Jun 7, 2025
60924a8
Merge pull request #58 from mxtoai/fix-output
satwikkansal Jun 7, 2025
0ff1aed
initial docker config
kernelism May 25, 2025
77701f2
some updates to docker setup
kernelism May 26, 2025
b50a53a
ffmpeg missing issue fix
kernelism May 26, 2025
356b1c6
addressed comments
kernelism May 27, 2025
b7773c4
fixed conflicts during merge
kernelism Jun 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# override this to your own model config toml
LITELLM_CONFIG_PATH=model.config.toml
LITELLM_CONFIG_PATH=./credentials/model.config.example.toml

# Azure OpenAI Configuration (Legacy)
AZURE_OPENAI_MODEL=your_model_name_here # e.g., o3-mini-deep-research
Expand All @@ -22,8 +22,10 @@ AWS_ACCESS_KEY_ID=your_aws_access_key_id
AWS_SECRET_ACCESS_KEY=your_aws_secret_access_key
SENDER_EMAIL=your_sender_email@domain.com

# JINA API Key
# External Services
JINA_API_KEY="YOUR_JINA_API_KEY" # Leave blank if not using deep research
BRAVE_SEARCH_API_KEY=""
RAPIDAPI_KEY=""

# LLM Routing Configuration
# GPT-4o-mini Instance 1
Expand Down Expand Up @@ -53,6 +55,7 @@ SUPABASE_URL=your_supabase_url
SUPABASE_KEY=your_supabase_key
SUPABASE_SERVICE_ROLE_KEY=your_supabase_service_role_key
WHITELIST_SIGNUP_URL=your_whitelist_signup_url # e.g., https://yourdomain.com/
FRONTEND_URL=https://mxtoai.com/

# Server Configuration
PORT=8000
Expand Down
92 changes: 90 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,58 @@ poetry run python run_api.py
```

5. Start the workers:

Using only single process and couple of threads for local development:

```bash
poetry run dramatiq mxtoai.tasks --processes 1 --threads 2 --watch ./.
```

### Docker Setup (Alternative Installation)

The project can also be run using Docker Compose, which provides an isolated environment with all required services.

1. Ensure you have Docker and Docker Compose installed on your system.

2. Build and start all services:
```bash
poetry run dramatiq mxtoai.tasks --watch ./.
docker compose up -d
```

3. Access the services:
- API Server: http://localhost:8000
- RabbitMQ Management: http://localhost:15672 (credentials: guest/guest)
- Redis: localhost:6379
- Ollama: localhost:11434 (optional)

#### Service Details
- **API Server**: FastAPI application running on port 8000
- **Worker**: Background task processor using Dramatiq
- **Redis**: Used for caching and session management
- **RabbitMQ**: Message broker for task queue
- **Ollama**: Optional LLM service (disabled by default)

#### Running with Ollama
To include the Ollama service (required for local LLM processing):
```bash
docker compose --profile ollama up -d
```

#### Stopping Services
```bash
# Stop all services
docker compose down

# Stop and remove all data volumes (this will delete all data)
docker compose down -v
```

#### Important Notes
- The Docker setup includes all required services (Redis, RabbitMQ) automatically
- Model configuration file (`model.config.toml`) should be placed in the `credentials/` directory
- All services are configured to restart automatically unless stopped manually
- Data persistence is enabled for Redis, RabbitMQ, and Ollama through Docker volumes

### Environment Variables

Copy the `.env.example` file to `.env` and update with your specific configuration:
Expand All @@ -110,6 +158,46 @@ LITELLM_CONFIG_PATH=model.config.toml
# Redis configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0
REDIS_PASSWORD=

# rabbitmq config
RABBITMQ_HOST=localhost
RABBITMQ_PORT=5672
RABBITMQ_USER=guest
RABBITMQ_PASSWORD=guest
RABBITMQ_VHOST=/
RABBITMQ_HEARTBEAT=60 # Default heartbeat interval in seconds

# server config
PORT=8000
HOST=0.0.0.0
LOG_LEVEL=INFO
IS_PROD=false
X_API_KEY=your_api_key

# supabase
SUPABASE_URL=your_supabase_url
SUPABASE_KEY=your_supabase_key
SUPABASE_SERVICE_ROLE_KEY=your_supabase_service_role_key
WHITELIST_SIGNUP_URL=your_whitelist_signup_url # e.g., https://yourdomain.com/

# open ai api key
AZURE_OPENAI_API_KEY=your_api_key_here

# Hugging Face Token
HF_TOKEN=your_huggingface_token

# AWS SES Configuration
AWS_REGION=your_aws_region # e.g., ap-south-1
AWS_ACCESS_KEY_ID=your_aws_access_key_id
AWS_SECRET_ACCESS_KEY=your_aws_secret_access_key
SENDER_EMAIL=your_sender_email@domain.com

# External services
JINA_API_KEY="YOUR_JINA_API_KEY" # Leave blank if not using deep research
BRAVE_SEARCH_API_KEY=""
RAPIDAPI_KEY=""

# Optional for research functionality
JINA_API_KEY=your-jina-api-key
Expand All @@ -122,7 +210,7 @@ AZURE_VISION_KEY=your-azure-vision-key
SERPAPI_API_KEY=your-serpapi-api-key
```

This project supports load balancing and routing across multiple models, so you can define as many models as you'd like. Copy `model.config.example.toml` to a toml file and update it with your preferred configuration. Update `.env` with the path your toml relative to root.
This project supports load balancing and routing across multiple models, so you can define as many models as you'd like. Copy `credentials/model.config.example.toml` to a toml file and update it in the same directory with your preferred configuration. Update `.env` with the path your toml relative to root.

A sample configuration looks like this:

Expand Down
90 changes: 90 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
version: "3.9"

services:
redis:
image: redis:7-alpine
container_name: redis
ports:
- ${REDIS_PORT:-6379}:${REDIS_PORT:-6379}
volumes:
- redis_data:/data
restart: unless-stopped
command: ["redis-server", "--requirepass", "${REDIS_PASSWORD:-changeme}"]

rabbitmq:
image: rabbitmq:3-management
container_name: rabbitmq
ports:
- ${RABBITMQ_PORT:-5672}:${RABBITMQ_PORT:-5672}
- ${RABBITMQ_MANAGEMENT_PORT:-15672}:${RABBITMQ_MANAGEMENT_PORT:-15672}
volumes:
- rabbitmq_data:/var/lib/rabbitmq
environment:
RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER:-guest}
RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD:-guest}
restart: unless-stopped
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "ping"]
interval: 30s
timeout: 10s
retries: 5

ollama:
image: ollama/ollama:latest
container_name: ollama
ports:
- ${OLLAMA_PORT:-11434}:${OLLAMA_PORT:-11434}
volumes:
- ollama_data:/root/.ollama
restart: unless-stopped
profiles:
- ollama

api_server:
build:
context: .
dockerfile: docker/api_server.dockerfile
container_name: api_server
env_file:
- .env
depends_on:
rabbitmq:
condition: service_healthy
redis:
condition: service_started
ports:
- "8000:8000"
environment:
- REDIS_URL=redis://:${REDIS_PASSWORD:-changeme}@redis:${REDIS_PORT:-6379}/0
- RABBITMQ_URL=amqp://${RABBITMQ_USER:-guest}:${RABBITMQ_PASSWORD:-guest}@rabbitmq:${RABBITMQ_PORT:-5672}/
- OLLAMA_URL=http://ollama:${OLLAMA_PORT:-11434}
- LITELLM_CONFIG_PATH=/app/credentials/model.config.toml
volumes:
- ./credentials/model.config.toml:/app/credentials/model.config.toml
command: ["poetry", "run", "uvicorn", "mxtoai.api:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

worker:
build:
context: .
dockerfile: docker/worker.dockerfile
container_name: worker
env_file:
- .env
depends_on:
rabbitmq:
condition: service_healthy
redis:
condition: service_started
environment:
- REDIS_URL=redis://:${REDIS_PASSWORD:-changeme}@redis:${REDIS_PORT:-6379}/0
- RABBITMQ_URL=amqp://${RABBITMQ_USER:-guest}:${RABBITMQ_PASSWORD:-guest}@rabbitmq:${RABBITMQ_PORT:-5672}/
- OLLAMA_URL=http://ollama:${OLLAMA_PORT:-11434}
- LITELLM_CONFIG_PATH=/app/credentials/model.config.toml
volumes:
- ./credentials/model.config.toml:/app/credentials/model.config.toml
command: ["poetry", "run", "dramatiq", "mxtoai.tasks", "--watch", "./."]

volumes:
rabbitmq_data:
ollama_data:
redis_data:
28 changes: 28 additions & 0 deletions docker/api_server.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM python:3.13-slim-bookworm

# System dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
build-essential \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*

# Set working directory
WORKDIR /app

# Install Poetry (latest)
RUN curl -sSL https://install.python-poetry.org | python3 - && \
ln -s /root/.local/bin/poetry /usr/local/bin/poetry

# Copy dependency files first (for cache)
COPY pyproject.toml poetry.lock ./

# Install dependencies (no virtualenv)
RUN poetry config virtualenvs.create false && poetry install --no-root --no-interaction --no-ansi

# Copy only the relevant application code
COPY mxtoai ./mxtoai
COPY run_api.py .

# Run the API via uvicorn
CMD ["poetry", "run", "uvicorn", "mxtoai.api:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
27 changes: 27 additions & 0 deletions docker/worker.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM python:3.13-slim-bookworm

# System dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
build-essential \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*

# Set working directory
WORKDIR /app

# Install Poetry (latest)
RUN curl -sSL https://install.python-poetry.org | python3 - && \
ln -s /root/.local/bin/poetry /usr/local/bin/poetry

# Copy dependency files first (for cache)
COPY pyproject.toml poetry.lock ./

# Install dependencies
RUN poetry config virtualenvs.create false && poetry install --no-root --no-interaction --no-ansi

# Copy only the relevant worker code
COPY mxtoai ./mxtoai

# Run the Dramatiq worker
CMD ["poetry", "run", "dramatiq", "mxtoai.tasks", "--watch", "./mxtoai"]
2 changes: 1 addition & 1 deletion docusaurus-site/docs/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Welcome to MXtoAI!

MXtoAI is an intelligent email assistant that processes users emails automatically. Users simply forward any email to our specialized handles (email IDs) and the appropriate actions are taken. Most of the actions are AI driven.

Forward emails to our general email handle `ask@mxtoai.com` or to one of our specialized handles. Get summaries, replies, and research without the complexity.
Forward emails to our general email handle `ask@mxtoai.com` or to one of our specialized handles. Get summaries, replies, research, and professional PDF exports without the complexity.

![Forwarding email to ask@mxtoai.com](/img/mxtoai-ask.gif)
*Example: Forwarding an email to ask@mxtoai.com with instructions.*
3 changes: 3 additions & 0 deletions docusaurus-site/docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@ Here's a brief overview of the available handles and their primary functions:
* `background@mxtoai.com`: Get background information on entities mentioned.
* `ask@mxtoai.com`: Ask specific questions about the email content.
* `schedule@mxtoai.com`: Extract scheduling information or propose meeting times.
* `pdf@mxtoai.com`: Export email content as a professional PDF document.

For detailed information on each handle, including aliases and key features, please refer to the [Email Handles section on our website](https://www.mxtoai.com/#usecases).

**Pro tip:** Add our email handles to your contacts for quicker access when forwarding emails.

**Note:** Any handle can export content as PDF by simply asking "convert to PDF" or "export as PDF" in your request.
26 changes: 25 additions & 1 deletion docusaurus-site/docs/why-use-mxtoai.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@
- Works anywhere - Gmail, Outlook, Apple Mail, even your phone
- Works with attachments - PDFs, images, spreadsheets all processed
- Works with email threads - understands context from conversation history
- **PDF Export** - any response can be converted to PDF, or use pdf@ to export content directly

As simple as it sounds, MXtoAI is very powerful. If you spend more than 10 minutes a day reading, understanding, or responding to emails, MXtoAI can probably save you time and mental energy. Whether you're dealing with work emails, personal correspondence, or staying informed, there's likely a use case that fits your daily routine. Here are some use-cases you can use it for.
As simple as it sounds, MXtoAI is very powerful. If you spend more than 10 minutes a day reading, understanding, or responding to emails, MXtoAI can probably save you time and mental energy. Whether you're dealing with work emails, personal correspondence, or staying informed, there's likely a use case that fits your daily routine. Plus, any content generated can be exported as a professional PDF for sharing, printing, or archiving. Here are some use-cases you can use it for.

## 📧 Daily Email Management

Expand Down Expand Up @@ -70,6 +71,10 @@ As simple as it sounds, MXtoAI is very powerful. If you spend more than 10 minut
- Forward to summarize@mxtoai.com - get key requirements and deadlines
- Forward to ask@mxtoai.com with "What's our competitive advantage here?"

**"I need to document compliance or regulatory communications"**
- Forward regulatory emails to pdf@mxtoai.com for permanent, clean records
- Perfect for audit trails and compliance documentation

## 🔍 Information Verification & Research

**"I get forwarded news and want to check if it's true"**
Expand All @@ -88,6 +93,20 @@ As simple as it sounds, MXtoAI is very powerful. If you spend more than 10 minut
- Forward competitor emails to background@mxtoai.com
- Get market position, recent moves, customer feedback

## 📄 PDF Export & Document Creation

**"I need to create professional documents from email content"**
- Forward newsletters to pdf@mxtoai.com - get clean, formatted PDF without email headers
- Perfect for sharing research, reports, or important communications

**"I want to convert AI responses to shareable documents"**
- Ask any handle to "generate a report and convert to PDF"
- Forward to ask@mxtoai.com with "Create a HN newsletter with top AI posts of the week and export as PDF"

**"I need to archive important email content professionally"**
- Forward contracts, agreements, or important announcements to pdf@mxtoai.com
- Get clean documents without email clutter - perfect for filing or legal records

## 🏠 Personal Life

**"I get complex medical/insurance emails"**
Expand Down Expand Up @@ -128,6 +147,10 @@ As simple as it sounds, MXtoAI is very powerful. If you spend more than 10 minut
- Forward to ask@mxtoai.com with "What are the practical applications?"
- Understand real-world relevance quickly

**"I want to compile research findings into shareable documents"**
- Forward multiple research emails to ask@mxtoai.com with "Synthesize findings and create PDF report"
- Perfect for thesis research or professional development

## 🛍️ Shopping & Consumer Decisions

**"I get overwhelmed by product comparison emails"**
Expand Down Expand Up @@ -199,6 +222,7 @@ As simple as it sounds, MXtoAI is very powerful. If you spend more than 10 minut
**Consultant:**
- "I forward client emails to background@mxtoai.com before meetings to understand industry challenges"
- "I forward RFPs to ask@mxtoai.com with 'What's the hidden agenda here?' to win more bids"
- "I forward research findings to ask@mxtoai.com with 'Create client presentation and export as PDF' for professional deliverables"

**Retiree:**
- "I forward Medicare emails to simplify@mxtoai.com - healthcare is confusing enough"
Expand Down
Loading
Loading