Merge branch '1.0.0'

Author: thomashacker

.gitignore

@@ -5,6 +5,7 @@ __pycache__
 .python-version
 *.egg-info
 venv
+venv*
 dist
 build
 ~
@@ -16,4 +17,6 @@ verba_config.json
 text-generation-inference
 test.py
 cache.txt
-.ruff_cache
+.ruff_cache
+*_secrets.json
+ollama

CHANGELOG.md

@@ -2,43 +2,101 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.0.0] - Beautiful Verba Update
+
+### Added
+
+- Added DaisyUI
+- Optimized frontend codebase
+- Fully Reworked Verba Design
+  - Fully Responsive, optimized for all screen sizes
+- Customization Capabilities
+  - Added Default, Darkmode, Weaviate themes
+  - Full text, color, image customization
+- Improve Chat Interface
+  - Better formatting of markdown + code
+  - Keep conversations saved in localBrowser storage
+  - Better Debugging by providing more information about current states
+- Improve Document Viewer Interface
+  - Add Pagination
+  - Add Sorting
+  - Use Aggregation for Filtering
+- Improve Status Overview
+  - Reworked Frontend + Optimize Code
+  - Sort status entries
+  - Improve Loading Speed by using Aggregation
+- Improve Component Selection for both Ingestion and RAG
+  - Added new configuraiton that will be passed between frontend and backend
+  - Cleaned codebase, merged interfaces and managers to single files
+  - Added clean endpoints for better code readability
+  - Reworked on interfaces
+- Added better console and logging for ingestion
+- More Configuration
+  - Enable/Disable Caching and Autocomplete Suggestions
+  - Improved verba_config.json
+- Ability to enable/disable caching + autosuggestions
+- Add Google Gemini as new Embedder and Generator
+- Added .CSV support (all file types available in Unstructured IO)
+- More test data
+- Add Ollama as Generator and Embedding Component
+- Add Support for Cohere R+
+- Improved WindowRetriever Context Generation
+- Show RAW Context in Frontend + Save in LocalStorage
+- Save Settings and Configuration in Weaviate
+
+### Changed
+
+- Changed to AppRouter framework
+- Changed frontend project structure
+- Changed backend project structure
+- Removed Llama Generator Component
+
+### Fixed
+
+- Using Accelerator Library
+
 ## [0.4.0] - 11.04.2024
 
 ### Added
+
 - Improved Docker Documentation
 - Improved Docker Settings
 - New Environment Variables for OpenAI proxies: OpenAI_BASE_URL (LiteLLM support) (https://github.com/weaviate/Verba/issues/56)
 - Increased version
 
 ### Changed
+
 - Removed spaCy from project
 
 ### Fixed
+
 - Python not working on version 3.12, 3.11, and 3.9
 - GitHub Links on README
 - Fix Docker Default Vectorizer (https://github.com/weaviate/Verba/issues/50)
 - Fix requirements.txt spelling error
 - Minor Bug fixes
 
-
 ## [0.3.1] - 15.11.2023
 
 ### Added
+
 - PDFReader powered by PyPDF2
 - TokenChunker powered by tiktoken
 - Ruff Linting (set as pre-commit)
 - Markdown Formatting for chat messages (https://github.com/weaviate/Verba/issues/48)
 
 ### Fixed
+
 - Added missing dependencies
 - Fixed restart bug
 - Fixed MiniLM Cuda to_device bug (https://github.com/weaviate/Verba/issues/41)
 - Fixed Config Issues (https://github.com/weaviate/Verba/issues/51)
-- Fixed Weaviate Embedded Headers for Cohere 
+- Fixed Weaviate Embedded Headers for Cohere
 
 ## [0.3.0] - 12.09.2023
 
 ### Added
+
 - Refactor modular architecture
 - Add ability to import data through the frontend, CLI, and script
 - Add Readers (SimpleReader, PathReader, GithubReader, PDFReader)
@@ -57,32 +115,37 @@ All notable changes to this project will be documented in this file.
 - Added technical docs and contribution guidelines
 
 ### Fixed
+
 - Error handling for data ingestion (handling chunk size)
 - Schmea handling on startup
 
 ### Changed
+
 - Removed Simple- and AdvancedEngine logic
 
 ## [0.2.3] - 05.09.2023
 
 ### Added
+
 - OpenAI API documentation example dataset
 
 ## [0.2.2] - 31.08.2023
 
 ### Release!
+
 - First version of Verba released! (many to come :)
 
 ### Added
+
 - Verba favicon
 
 ### Fixed
+
 - Add static files to package
 - Weaviate Embedded not shutting down
 
 ## [0.1.0] - 29.08.2023
 
 ### Added
-- Prepare Verba for first release
-
 
+- Prepare Verba for first release

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.10
+FROM python:3.11
 WORKDIR /Verba
 COPY . /Verba
 RUN pip install -e '.'

FRONTEND.md

@@ -1,15 +1,25 @@
-# 🎨 Verba - Frontend Documentation 🎨
+# Verba - Frontend Documentation
 
-Welcome to the Verba frontend documentation. Our frontend is a React application utilizing the robust Next.js framework, designed to offer a dynamic and interactive document search and display experience. Below are the instructions to help you set up and understand the structure of the frontend, which can be found within the [frontend](./frontend/) directory of our repository.
+Verba's Frontend is a [NextJS](https://nextjs.org/) application used together with [TailwindCSS](https://tailwindcss.com/) and [DaisyUI](https://daisyui.com/).
 
 ## 🚀 Setting Up the Frontend
 
 To get your local copy of the Verba frontend up and running, please follow these simple steps:
 
+1. Clone Repository
+
+```git
+
+git clone https://github.com/weaviate/Verba.git
+
+```
+
 1. **Node.js Requirement**:
-   - Confirm that Node.js version `>=18.16.0` is installed on your system. If you need to install or update Node.js, visit the official [Node.js website](https://nodejs.org/).
+
+   - Confirm that Node.js version `>=21.3.0` is installed on your system. If you need to install or update Node.js, visit the official [Node.js website](https://nodejs.org/).
 
 2. **Installation**:
+
    - Navigate to the frontend directory: `cd frontend`
    - Run `npm install` to install the dependencies required for the project.
 
@@ -19,25 +29,7 @@ To get your local copy of the Verba frontend up and running, please follow these
 
 ## 📦 Building Static Pages for FastAPI
 
-If you wish to serve the frontend through FastAPI, you need to build static pages:
+If you wish to serve and update the frontend through FastAPI, you need to build static pages:
 
 1. **Build Process**:
    - Execute `npm run build` to generate the static production build. The output will be directed to the FastAPI folder configured to serve the static content.
-
-## 🎨 Styling with Tailwind CSS
-
-Verba's frontend is styled with Tailwind CSS, a utility-first CSS framework, along with global CSS classes for a consistent and modern design language. Here's how styling is managed:
-
-- **Tailwind CSS**:
-  - We use Tailwind CSS for responsive design and to apply styles directly in the JSX code using utility classes.
-  - For custom styling, the `tailwind.config.js` file can be modified.
-
-- **Global Styles**:
-  - Global stylesheets are located in the `styles` directory. These are used for overarching styles that apply to the entire application.
-
-- **Assets**:
-  - All static assets such as images and animations are housed in the `public` directory.
-  - These assets are referenced directly in the codebase and are automatically optimized by Next.js.
-
-By following these guidelines, you'll be able to contribute to the frontend of Verba effectively. We value your contributions and are excited to see the creative ways you'll enhance the Verba experience.
-

PYTHON_TUTORIAL.md

@@ -0,0 +1,70 @@
+# Installing Python and Setting Up a Virtual Environment
+
+Before you can use Verba, you'll need to ensure that `Python >=3.10.0` is installed on your system and that you can create a virtual environment for a safer and cleaner project setup.
+
+## Installing Python
+
+Python is required to run Verba. If you don't have Python installed, follow these steps:
+
+### For Windows:
+
+Download the latest Python installer from the official Python website.
+Run the installer and make sure to check the box that says `Add Python to PATH` during installation.
+
+### For macOS:
+
+You can install Python using Homebrew, a package manager for macOS, with the following command in the terminal:
+
+```
+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
+```
+
+Then install Python:
+
+```
+brew install python
+```
+
+### For Linux:
+
+Python usually comes pre-installed on most Linux distributions. If it's not, you can install it using your distribution's package manager. You can read more about it [here](https://opensource.com/article/20/4/install-python-linux)
+
+## Setting Up a Virtual Environment
+
+It's recommended to use a virtual environment to avoid conflicts with other projects or system-wide Python packages.
+
+### Install the virtualenv package:
+
+First, ensure you have pip installed (it comes with Python if you're using version 3.4 and above).
+Install virtualenv by running:
+
+```
+pip install virtualenv
+```
+
+### Create a Virtual Environment:
+
+Navigate to your project's directory in the terminal.
+Run the following command to create a virtual environment named venv (you can name it anything you like):
+
+```
+python3 -m virtualenv venv
+```
+
+### Activate the Virtual Environment:
+
+- On Windows, activate the virtual environment by running:
+
+```
+venv\Scripts\activate.bat
+```
+
+- On macOS and Linux, activate it with:
+
+```
+source venv/bin/activate
+```
+
+Once your virtual environment is activated, you'll see its name in the terminal prompt. Now you're ready to install Verba using the steps provided in the Quickstart sections.
+
+> Remember to deactivate the virtual environment when you're done working with Verba by simply running deactivate in the terminal.