unit testing for deepsearcher (#237)

* finished testing agent and loader

* finished testing embeddings

* added llm testings

* added utils testing

* finalizing unit testing for all integrations

* added testing trigger commands in docs and CONTRIBUTING

* finalizing unit testing for deepsearcher

* handled env vars in tests

* using milvus lite to avoid connection failing issues

Author: jinhonglin-ryan

CONTRIBUTING.md

@@ -102,6 +102,53 @@ Syncing the environment manually is especially useful for ensuring your editor h
 For more detailed information about dependency locking and syncing, refer to the [offical Locking and syncing documentation](https://docs.astral.sh/uv/concepts/projects/sync/).
 
 
+## Running Tests
+
+Before submitting your pull request, make sure to run the test suite to ensure your changes haven't introduced any regressions.
+
+### Installing Test Dependencies
+
+First, ensure you have pytest installed. If you haven't installed the development dependencies yet, you can do so with:
+
+```shell
+uv sync --all-extras --dev
+```
+
+This will install all development dependencies and optional dependencies including pytest and other testing tools.
+
+### Running the Tests
+
+To run all tests in the `tests` directory:
+
+```shell
+uv run pytest tests
+```
+
+For more verbose output that shows individual test results:
+
+```shell
+uv run pytest tests -v
+```
+
+You can also run tests for specific directories or files. For example:
+
+```shell
+# Run tests in a specific directory
+uv run pytest tests/embedding
+
+# Run tests in a specific file
+uv run pytest tests/embedding/test_bedrock_embedding.py
+
+# Run a specific test class
+uv run pytest tests/embedding/test_bedrock_embedding.py::TestBedrockEmbedding
+
+# Run a specific test method
+uv run pytest tests/embedding/test_bedrock_embedding.py::TestBedrockEmbedding::test_init_default
+```
+
+The `-v` flag (verbose mode) provides more detailed output, showing each test case and its result individually. This is particularly useful when you want to see which specific tests are passing or failing.
+
+
 ## Developer Certificate of Origin (DCO)
 
 All contributions require a sign-off, acknowledging the [Developer Certificate of Origin](https://developercertificate.org/). 

docs/contributing/index.md

@@ -0,0 +1,159 @@
+# Contributing to DeepSearcher
+
+We welcome contributions from everyone. This document provides guidelines to make the contribution process straightforward.
+
+
+## Pull Request Process
+
+1. Fork the repository and create your branch from `master`.
+2. Make your changes.
+3. Run tests and linting to ensure your code meets the project's standards.
+4. Update documentation if necessary.
+5. Submit a pull request.
+
+
+## Linting and Formatting
+
+Keeping a consistent style for code, code comments, commit messages, and PR descriptions will greatly accelerate your PR review process.
+We require you to run code linter and formatter before submitting your pull requests:
+
+To check the coding styles:
+
+```shell
+make lint
+```
+
+To fix the coding styles:
+
+```shell
+make format
+```
+Our CI pipeline also runs these checks automatically on all pull requests to ensure code quality and consistency.
+
+
+## Development Environment Setup with uv
+
+DeepSearcher uses [uv](https://github.com/astral-sh/uv) as the recommended package manager. uv is a fast, reliable Python package manager and installer. The project's `pyproject.toml` is configured to work with uv, which will provide faster dependency resolution and package installation compared to traditional tools.
+
+### Install Project in Development Mode(aka Editable Installation)
+
+1. Install uv if you haven't already:
+   Follow the [offical installation instructions](https://docs.astral.sh/uv/getting-started/installation/).
+
+2. Clone the repository and navigate to the project directory:
+   ```shell
+   git clone https://github.com/zilliztech/deep-searcher.git && cd deep-searcher
+   ```
+3. Synchronize and install dependencies:
+   ```shell
+   uv sync
+   source .venv/bin/activate
+   ```
+   `uv sync` will install all dependencies specified in `uv.lock` file. And the `source .venv/bin/activate` command will activate the virtual environment.
+
+   - (Optional) To install all optional dependencies:
+      ```shell
+      uv sync --all-extras --dev
+      ```
+
+   - (Optional) To install specific optional dependencies:
+      ```shell
+      # Take optional `ollama` dependency for example
+      uv sync --extra ollama
+      ```
+   For more optional dependencies, refer to the `[project.optional-dependencies]` part of `pyproject.toml` file.
+
+
+
+### Adding Dependencies
+
+When you need to add new dependencies to the `pyproject.toml` file, you can use the following commands:
+
+```shell
+uv add <package_name>
+```
+DeepSearcher uses optional dependencies to keep the default installation lightweight. Optional features can be installed using the syntax `deepsearcher[<extra>]`. To add a dependency to an optional extra, use the following command:
+
+```shell
+uv add <package_name> --optional <extra>
+```
+For more details, refer to the [offical Managing dependencies documentation](https://docs.astral.sh/uv/concepts/projects/dependencies/).
+
+### Dependencies Locking
+
+For development, we use lockfiles to ensure consistent dependencies. You can use 
+```shell
+uv lock --check
+```
+to verify if your lockfile is up-to-date with your project dependencies.
+
+When you modify or add dependencies in the project, the lockfile will be automatically updated the next time you run a uv command. You can also explicitly update the lockfile using:
+```shell
+uv lock
+```
+
+While the environment is synced automatically, it may also be explicitly synced using uv sync:
+```shell
+uv sync
+```
+Syncing the environment manually is especially useful for ensuring your editor has the correct versions of dependencies.
+
+
+For more detailed information about dependency locking and syncing, refer to the [offical Locking and syncing documentation](https://docs.astral.sh/uv/concepts/projects/sync/).
+
+
+## Running Tests
+
+Before submitting your pull request, make sure to run the test suite to ensure your changes haven't introduced any regressions.
+
+### Installing Test Dependencies
+
+First, ensure you have pytest installed. If you haven't installed the development dependencies yet, you can do so with:
+
+```shell
+uv sync --all-extras --dev
+```
+
+This will install all development dependencies and optional dependencies including pytest and other testing tools.
+
+### Running the Tests
+
+To run all tests in the `tests` directory:
+
+```shell
+uv run pytest tests
+```
+
+For more verbose output that shows individual test results:
+
+```shell
+uv run pytest tests -v
+```
+
+You can also run tests for specific directories or files. For example:
+
+```shell
+# Run tests in a specific directory
+uv run pytest tests/embedding
+
+# Run tests in a specific file
+uv run pytest tests/embedding/test_bedrock_embedding.py
+
+# Run a specific test class
+uv run pytest tests/embedding/test_bedrock_embedding.py::TestBedrockEmbedding
+
+# Run a specific test method
+uv run pytest tests/embedding/test_bedrock_embedding.py::TestBedrockEmbedding::test_init_default
+```
+
+The `-v` flag (verbose mode) provides more detailed output, showing each test case and its result individually. This is particularly useful when you want to see which specific tests are passing or failing.
+
+
+## Developer Certificate of Origin (DCO)
+
+All contributions require a sign-off, acknowledging the [Developer Certificate of Origin](https://developercertificate.org/). 
+Add a `Signed-off-by` line to your commit message:
+
+```text
+Signed-off-by: Your Name <your.email@example.com>
+``` 

mkdocs.yml

@@ -60,6 +60,8 @@ nav:
       - "Development Mode": installation/development.md
     - "FAQ": 
       - "FAQ": faq/index.md
+  - Contribution Guide:
+    - "Contribution Guide": contributing/index.md
   - Usage:
     - "Usage": usage/index.md
     - "Quick Start": usage/quick_start.md

pyproject.toml

@@ -37,6 +37,7 @@ dev = [
     "mkdocs-jupyter>=0.25.0",
     "mkdocs-click>=0.8.1",
     "mkdocstrings[python]>=0.27.0",
+    "pytest>=8.3.5",
 ]
 
 [project.optional-dependencies]

tests/__init__.py

@@ -0,0 +1 @@
+# Tests for the deepsearcher package 

tests/agent/__init__.py

@@ -0,0 +1 @@
+# Tests for the agent module 

tests/agent/test_base.py

@@ -0,0 +1,142 @@
+import unittest
+from unittest.mock import MagicMock
+import numpy as np
+
+from deepsearcher.llm.base import BaseLLM, ChatResponse
+from deepsearcher.embedding.base import BaseEmbedding
+from deepsearcher.vector_db.base import BaseVectorDB, RetrievalResult, CollectionInfo
+
+
+class MockLLM(BaseLLM):
+    """Mock LLM implementation for testing agents."""
+    
+    def __init__(self, predefined_responses=None):
+        """
+        Initialize the MockLLM.
+        
+        Args:
+            predefined_responses: Dictionary mapping prompt substrings to responses
+        """
+        self.chat_called = False
+        self.last_messages = None
+        self.predefined_responses = predefined_responses or {}
+    
+    def chat(self, messages, **kwargs):
+        """Mock implementation of chat that returns predefined responses or a default response."""
+        self.chat_called = True
+        self.last_messages = messages
+        
+        if self.predefined_responses:
+            message_content = messages[0]["content"] if messages else ""
+            for key, response in self.predefined_responses.items():
+                if key in message_content:
+                    return ChatResponse(content=response, total_tokens=10)
+        
+        return ChatResponse(content="This is a test answer", total_tokens=10)
+    
+    def literal_eval(self, text):
+        """Mock implementation of literal_eval."""
+        # Default implementation returns a list with test_collection
+        # Override this in specific tests if needed
+        if text.strip().startswith("[") and text.strip().endswith("]"):
+            # Return the list as is if it's already in list format
+            try:
+                import ast
+                return ast.literal_eval(text)
+            except:
+                pass
+                
+        return ["test_collection"]
+
+
+class MockEmbedding(BaseEmbedding):
+    """Mock embedding model implementation for testing agents."""
+    
+    def __init__(self, dimension=8):
+        """Initialize the MockEmbedding with a specific dimension."""
+        self._dimension = dimension
+    
+    @property
+    def dimension(self):
+        """Return the dimension of the embedding model."""
+        return self._dimension
+    
+    def embed_query(self, text):
+        """Mock implementation that returns a random vector of the specified dimension."""
+        return np.random.random(self._dimension).tolist()
+    
+    def embed_documents(self, documents):
+        """Mock implementation that returns random vectors for each document."""
+        return [np.random.random(self._dimension).tolist() for _ in documents]
+
+
+class MockVectorDB(BaseVectorDB):
+    """Mock vector database implementation for testing agents."""
+    
+    def __init__(self, collections=None):
+        """
+        Initialize the MockVectorDB.
+        
+        Args:
+            collections: List of collection names to initialize with
+        """
+        self.default_collection = "test_collection"
+        self.search_called = False
+        self.insert_called = False
+        self._collections = []
+        
+        if collections:
+            for collection in collections:
+                self._collections.append(
+                    CollectionInfo(collection_name=collection, description=f"Test collection {collection}")
+                )
+        else:
+            self._collections = [
+                CollectionInfo(collection_name="test_collection", description="Test collection for testing")
+            ]
+    
+    def search_data(self, collection, vector, top_k=10, **kwargs):
+        """Mock implementation that returns test results."""
+        self.search_called = True
+        self.last_search_collection = collection
+        self.last_search_vector = vector
+        self.last_search_top_k = top_k
+        
+        return [
+            RetrievalResult(
+                embedding=vector,
+                text=f"Test result {i} for collection {collection}",
+                reference=f"test_reference_{collection}_{i}",
+                metadata={"a": i, "wider_text": f"Wider context for test result {i} in collection {collection}"}
+            )
+            for i in range(min(3, top_k))
+        ]
+    
+    def insert_data(self, collection, chunks):
+        """Mock implementation of insert_data."""
+        self.insert_called = True
+        self.last_insert_collection = collection
+        self.last_insert_chunks = chunks
+        return True
+    
+    def init_collection(self, dim, collection, **kwargs):
+        """Mock implementation of init_collection."""
+        return True
+    
+    def list_collections(self, dim=None):
+        """Mock implementation that returns the list of collections."""
+        return self._collections
+    
+    def clear_db(self, collection):
+        """Mock implementation of clear_db."""
+        return True
+
+
+class BaseAgentTest(unittest.TestCase):
+    """Base test class for agent tests with common setup."""
+    
+    def setUp(self):
+        """Set up test fixtures for agent tests."""
+        self.llm = MockLLM()
+        self.embedding_model = MockEmbedding(dimension=8)
+        self.vector_db = MockVectorDB()