Skip to content
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a6f747c
initial docker config
kernelism May 25, 2025
408264a
some updates to docker setup
kernelism May 26, 2025
4944a01
ffmpeg missing issue fix
kernelism May 26, 2025
06d8cc0
Add brave search tool with references addition to prompts (#32)
gautamp8 May 27, 2025
adcdbb9
Fix list rendering
satwikkansal May 27, 2025
a284f1d
Merge pull request #33 from mxtoai/fix-format
satwikkansal May 27, 2025
c8a83c6
addressed comments
kernelism May 27, 2025
dfc71b2
Use gpt-4 group for research related handles
gautamp8 Jun 2, 2025
7e6b549
Add linkedin data api tools via RapidAPI
gautamp8 Jun 3, 2025
59ae145
Fix output formatting, have factcheck to include additional summary, …
satwikkansal Jun 3, 2025
d54f231
Merge pull request #36 from mxtoai/rapid-api
satwikkansal Jun 3, 2025
08c3681
Merge branch 'master' into fix-output
satwikkansal Jun 3, 2025
6a1d640
Merge pull request #35 from mxtoai/fix-output
satwikkansal Jun 3, 2025
6b80921
Improve background research prompting (#37)
gautamp8 Jun 3, 2025
4823d80
Temporarily remove whitelist for reputed email providers (#38)
gautamp8 Jun 3, 2025
cc5b693
Fix logging issues, make logger more comprehensive
satwikkansal Jun 3, 2025
a4376af
Fix table formatting
satwikkansal Jun 3, 2025
e44fc23
Fix more markdown issue, center align tables
satwikkansal Jun 3, 2025
347d18d
Merge pull request #39 from mxtoai/fix-output
satwikkansal Jun 3, 2025
cf52eaf
Improve prompting for background and schedule handles
gautamp8 Jun 4, 2025
d278836
Perfect the prompts for all handles (#44)
gautamp8 Jun 4, 2025
2c09994
Merge branch 'docker' of https://github.com/mxtoai/mxtoai-backend int…
kernelism Jun 4, 2025
9e4cf0f
fixed docker volumes, rabbitmq connection issue and smolagents versio…
kernelism Jun 4, 2025
b899cc1
update README with env information and docker setup
kernelism Jun 4, 2025
79e9970
Fix issue with bold markers being detected as list items
satwikkansal Jun 4, 2025
47c1cb6
Run ruff
satwikkansal Jun 4, 2025
c9c6e1c
Add PDF export handle and tool
gautamp8 Jun 4, 2025
309067f
Merge pull request #45 from mxtoai/fix-output
satwikkansal Jun 4, 2025
2f157ac
Add pdf export handle tests
gautamp8 Jun 4, 2025
0536e14
Add proper cleanup of generated files
gautamp8 Jun 4, 2025
3b27017
Improve watermark formatting
gautamp8 Jun 4, 2025
c0c9f5c
Remove deep research not allowed
gautamp8 Jun 4, 2025
e51c58b
Add docs
gautamp8 Jun 4, 2025
cd577b9
Update mxtoai/tasks.py
satwikkansal Jun 5, 2025
cbf0f4e
Address review comments
gautamp8 Jun 5, 2025
02e0802
Merge pull request #46 from mxtoai/export-handles
satwikkansal Jun 5, 2025
0dd63d2
Fix other domain validation issue
gautamp8 Jun 5, 2025
ed4b758
Merge pull request #47 from mxtoai/validation-issue
satwikkansal Jun 5, 2025
561e54e
Trigger automatic verification for non-existant users with custom dom…
gautamp8 Jun 5, 2025
4580043
Improve schedule prompt
gautamp8 Jun 5, 2025
432d2ba
Formatting fixes and run ruff
satwikkansal Jun 5, 2025
5b1a4c4
Simplify formatting and add tests
satwikkansal Jun 6, 2025
57a9f7a
Do away with keyword based approach for section markdown qualification
satwikkansal Jun 6, 2025
d5e493a
Add independent search tools and guidelines (#50)
gautamp8 Jun 6, 2025
2d54a91
Add AWS postmaster to whitelist
satwikkansal Jun 6, 2025
3a9706c
Add AWS postmaster to whitelist
satwikkansal Jun 6, 2025
25f12da
Ignore rmq logs
satwikkansal Jun 6, 2025
472160a
Merge pull request #49 from mxtoai/fix-output
satwikkansal Jun 6, 2025
666c68f
Skip processing AWS system emails
satwikkansal Jun 7, 2025
60924a8
Merge pull request #58 from mxtoai/fix-output
satwikkansal Jun 7, 2025
0ff1aed
initial docker config
kernelism May 25, 2025
77701f2
some updates to docker setup
kernelism May 26, 2025
b50a53a
ffmpeg missing issue fix
kernelism May 26, 2025
356b1c6
addressed comments
kernelism May 27, 2025
b7773c4
fixed conflicts during merge
kernelism Jun 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
version: "3.9"

services:
redis:
image: redis:7-alpine
container_name: redis
ports:
- "6379:6379"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add auth, env variable for REDIS_PASSWORD or something

volumes:
- redis_data:/data
restart: unless-stopped

rabbitmq:
image: rabbitmq:3-management
container_name: rabbitmq
ports:
- "5672:5672" # RabbitMQ main port
- "15672:15672" # RabbitMQ management UI
volumes:
- rabbitmq_data:/var/lib/rabbitmq
environment:
RABBITMQ_DEFAULT_USER: guest
RABBITMQ_DEFAULT_PASS: guest
restart: unless-stopped
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not hardcode this, it should be wired from .env file

healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "ping"]
interval: 30s
timeout: 10s
retries: 5

ollama:
image: ollama/ollama:latest
container_name: ollama
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
restart: unless-stopped
# Optional: Uncomment below to disable if not needed
# deploy:
# replicas: 0

api_server:
build:
context: .
dockerfile: docker/api_server.dockerfile
container_name: api_server
depends_on:
rabbitmq:
condition: service_healthy
redis:
condition: service_started
ports:
- "8000:8000"
environment:
- REDIS_URL=redis://redis:6379/0
- RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

username and passwords should be wired from .env

- OLLAMA_URL=http://ollama:11434
volumes:
- .:/app
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this?

command: ["poetry", "run", "python", "run_api.py"]

worker:
build:
context: .
dockerfile: docker/worker.dockerfile
container_name: worker
depends_on:
rabbitmq:
condition: service_healthy
redis:
condition: service_started
environment:
- REDIS_URL=redis://redis:6379/0
- RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672/
- OLLAMA_URL=http://ollama:11434
volumes:
- .:/app
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, this looks very permissive, why do we need this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only need this for model-config, nothing else, maybe put model-config under credentials directory and mount as volume.

command: ["poetry", "run", "dramatiq", "mxtoai.tasks", "--watch", "./."]

volumes:
rabbitmq_data:
ollama_data:
redis_data:

33 changes: 33 additions & 0 deletions docker/api_server.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# docker/api_server.Dockerfile

FROM python:3.13-slim-bookworm

# System deps
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
build-essential \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*

# Set workdir
WORKDIR /app

# Install Poetry
ENV POETRY_VERSION=2.1.3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not fix the poetry version, this can be omitted

RUN curl -sSL https://install.python-poetry.org | python3 - && \
ln -s /root/.local/bin/poetry /usr/local/bin/poetry

# Copy only dependency files to leverage Docker cache
COPY pyproject.toml poetry.lock ./

# Install dependencies (no venv)
RUN poetry config virtualenvs.create false && poetry install --no-root --no-interaction --no-ansi

# Copy the rest of the application
COPY . .
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is too permissive, let's copy only the modules that are needed i.e. /mxtoai


# Expose API port (change as needed)
EXPOSE 8000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not expose it here, should be done in docker ocmpose


# Run the API
CMD ["poetry", "run", "python", "run_api.py"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to run it via uvicorn, for example this is the command we use in prod uvicorn mxtoai.api:app --host 0.0.0.0 --port 9292 --workers 4

30 changes: 30 additions & 0 deletions docker/worker.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# docker/worker.Dockerfile

FROM python:3.13-slim-bookworm

# System deps
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
build-essential \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*

# Set workdir
WORKDIR /app

# Install Poetry
ENV POETRY_VERSION=2.1.3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as pervious comment

RUN curl -sSL https://install.python-poetry.org | python3 - && \
ln -s /root/.local/bin/poetry /usr/local/bin/poetry

# Copy only dependency files
COPY pyproject.toml poetry.lock ./

# Install dependencies
RUN poetry config virtualenvs.create false && poetry install --no-root --no-interaction --no-ansi

# Copy rest of the worker code
COPY . .
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as pervious comment


# Run Dramatiq worker (entrypoint can be adjusted)
CMD ["poetry", "run", "dramatiq", "mxtoai.tasks", "--watch", "./."]
14 changes: 4 additions & 10 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading