Bard

Bard is an offline, full-stack AI chatbot application designed as a showcase for AI engineering skills. It demonstrates the integration of a local Large Language Model (LLM) with a modern web frontend, robust backend, and persistent chat memory, all orchestrated with best practices in software engineering.

Features

• 💻 Modern Frontend – Built with React (Vite), TanStack Start, TailwindCSS, and Shadcn UI.
• 🔒 Offline – Secure and Local LLM via Ollama
• 💾 Persistent Chat – Session Handling and Conversation History via langchain-postgres
• 🚀 Chat Streaming – Chat completions are streamed from server for UX
• 🐳 Fully Containerized – Easy to setup and run with Docker Compose

Tech Stack

• Database: Postgres (Dockerized)
• LLM: Ollama (via LangChain)
• Backend: FastAPI (Python)
• Frontend: React (Vite, TanStack Start), TailwindCSS, Shadcn UI
• Deployment: Docker & Docker Compose

Documentation

Project documentation is available via a Jupyter Notebook

Usage

The application follows the typical workflow and user experience of most chat applications:

• select your Username, as this is used to identify each user's sessions and chats
• once on the chat screen, use the text input to send a message (you can shift + enter to create a new line, but clicking enter automatically sends the message!)
• you can view your past and current thread on the sidebar on the left (you can ctrl/cmd + b on your keyboard to show/hide the sidebar at any time)

Getting Started

Note

The project was develoepd and tested on a machine running Ubuntu 24.04 LTS with a x64 Ryzen 5 CPU, 16 GB RAM, and Nvidia GPU. However, it was tested to work in Windows 11 on the same system, then on MacOS Sequoia running on Apple Silicon (M2)

Requirements

• Docker
• NVIDIA Container Toolkit
• Python
• Node.js

Important

Please modify use docker-compose.yml if you do not have an Nvidia GPU. Otherwise, docker-compose.nvidia.yml has a modified ollama service with GPU capability provided you have an Nvidia GPU and NVIDIA Container Toolkit installed

Quickstart via Docker

1. Clone the repository

git clone https://github.com/reddiedev/bluedrive-chat 
cd bluedrive-chat

2. Load the default environment variables

cp .env.example .env
cp .env.example frontend/.env
cp .env.example backend/.env

3. Reserve Host Ports Please pause/stop any services running on the following ports to prevent port conflict. Otherwise, please update the .env files or the docker-compose files

• 3000 - Frontend React App
• 8000 - Backend FastAPI Server
• 11434 - Ollama API
• 5432 - Postgres Database

4. Start the application stack

docker compose down -v # remove old containers and volumes, if any

# recommended: download ollama models first 
docker compose up --build ollama

# you can view download progress by running the following commands in a separate terminal
docker exec -it bd_ollama ollama pull qwen3:0.6b
docker exec -it bd_ollama ollama pull gemma3:1b

# ollama CPU
docker compose up --build

# ollama Nvidia GPU
docker compose -f docker-compose.nvidia.yml up --build

On my machine, it takes roughly ~1 minute to build all services without cache, then around ~5 minutes to download all the models depending on your network speed.

5. On your browser, you can view the app at http://localhost:3000

Caution

The first run will take some time, as the models are being downloaded for Ollama. Additionally, please allocate ~10 GB of system space for the containers (the bulk of it is taken up by Ollama)

Building Locally

You can run the database and ollama services stand-alone for local testing

docker compose up -d --build database ollama

FastAPI Backend

1. Install Python 3.12 on your Local machine

python --version
>>> Python 3.12.7

For managing python instances, I usually prefer using a virtual env using uv

cd backend
uv venv 
source .venv/bin/activate
uv pip install -r requirements.txt

2. Install requirements then run the main.py app via Uvicorn

cd backend
pip install -r requirements.txt

uv run main.py
python main.py
python3 main.py

React Frontend

1. Install Node.js v20 and pnpm on your Local machine

node -v
>>> v20.18.0

npm install -g pnpm
pnpm setup # if you haven't used pnpm before

For managing Node environments, I prefer to use it via nvm

nvm i 20
nvm use 20

2. Install the requirements and use the .env example

cp .env frontend/.env
cd frontend
pnpm install --frozen-lockfile

3. Run the app

pnpm run dev

pnpm run build
pnpm run start

Tests

Please follow the same setup in FastAPI Backend

cd backend/tests
pytest -v

Acknowledgements

This project was heavenly inspired by open-webui as well as t3.chat

Troubleshooting

For any .env problems, please cd your terminal into the root directory, then run this command to load the .env to your terminal

export $(grep -v '^#' .env | xargs)