feat: get function definitions from service (#6)

* add list functions endpoint

* update dependencies

* fix splitter functions

* add dockerfile

* number of service workers as env variable

* add config file and api_key

* add more precise input validation

* add api key to list_functions

* updated repository name

* modify config module

* add pre-commit fixes

* add pre-commit fixes

* update requirements.txt and Dockerfile

* add pre-commit fixes

* add default config path

* provisional extended example in readme file

* add config at default location

* add example entry to index

* Update README.md

Co-authored-by: Roberto Pastor Muela <[email protected]>

* modify readme and pre-commit files

---------

Co-authored-by: Roberto Pastor Muela <[email protected]>

Author: laurasgkadri98

.flake8

@@ -0,0 +1,7 @@
+[flake8]
+exclude = venv, __init__.py, doc/_build
+select = W191, W291, W293, W391, E115, E117, E122, E124, E125, E225, E231, E301, E303, E501, F401, F403
+count = True
+max-complexity = 10
+max-line-length = 120
+statistics = True

.gitignore

@@ -35,3 +35,9 @@ dist/
 .vscode/
 *.sublime-project
 *.sublime-workspace
+
+# Ignore config.yaml globally
+config.yaml
+
+# Do not ignore config.yaml in the configs/ directory
+!configs/config.yaml

.pre-commit-config.yaml

@@ -0,0 +1,43 @@
+repos:
+- repo: https://github.com/adamchainz/blacken-docs
+  rev: 1.16.0
+  hooks:
+  -   id: blacken-docs
+      additional_dependencies:
+      - black==23.12.1
+
+- repo: https://github.com/psf/black
+  rev: 23.12.1
+  hooks:
+  - id: black
+
+- repo: https://github.com/pycqa/isort
+  rev: 5.13.2
+  hooks:
+  - id: isort
+
+- repo: https://github.com/PyCQA/flake8
+  rev: 6.1.0
+  hooks:
+  - id: flake8
+
+- repo: https://github.com/pycqa/pydocstyle
+  rev: 6.3.0
+  hooks:
+  - id: pydocstyle
+    args: ["--ignore=D205,D100,D213,D203"]
+    exclude: examples
+    additional_dependencies: [toml]
+
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.5.0
+  hooks:
+  - id: check-merge-conflict
+  - id: debug-statements
+  - id: check-yaml
+  - id: trailing-whitespace
+
+- repo: https://github.com/python-jsonschema/check-jsonschema
+  rev: 0.27.3
+  hooks:
+    - id: check-github-workflows

README.md

@@ -7,10 +7,13 @@ Welcome to Allie FlowKit Python. This repository hosts Python functions similar
 - [Objectives](#objectives)
 - [How it works](#how-it-works)
 - [Getting started](#getting-started)
-  - [Prerequisites](#prerequisites)
-  - [Installation](#installation)
-  - [Usage](#usage)
+    - [Run locally](#run-locally)
+        - [Prerequisites](#prerequisites)
+        - [Installation](#installation)
+        - [Usage](#usage)
+    - [Run as a Docker container](#run-as-a-docker-container)
 - [Adding custom functions](#adding-custom-functions)
+  - [Example](#example)
 - [Example functions](#example-functions)
 - [Contributing](#contributing)
 
@@ -38,13 +41,17 @@ Allie Flowkit Python supports these actions:
 
 ## Getting started
 
-### Prerequisites
+Allie FlowKit Python can be run locally or as a Docker container. Follow the instructions below to get started.
+
+### Run locally
+
+#### Prerequisites
 
 - Python 3.7 or later
 - pip (Python package installer)
 - A running instance of the Allie Flowkit
 
-### Installation
+#### Installation
 
 1. Clone the repository:
     ```sh
@@ -57,62 +64,142 @@ Allie Flowkit Python supports these actions:
     pip install -r requirements.txt
     ```
 
-### Usage
+#### Usage
 
 1. Start the service:
-   ```sh
-   uvicorn app.app:app --host 0.0.0.0 --port 8000 --workers 1
-   ```
-   You can specify the host, port, and number of workers as needed. The service exposes the functions as REST APIs on the specified port. The default is 8000.
+    ```sh
+    uvicorn app.app:app --host 0.0.0.0 --port 50052 --workers 1
+    ```
+    You can specify the host, port, and number of workers as needed.
+
+2. The service will expose the functions as REST APIs on the specified port (default: 8000).
+
+3. Integrate these APIs into your Allie workflows as needed.
 
-2. Integrate these APIs into your Allie workflows as needed.
+### Run as a Docker container
+
+1. Build the Docker container image with the following command:
+
+```bash
+    docker build -f docker/Dockerfile . -t allie-flowkit-python:latest
+```
+
+2. Run the Docker container and expose the port on your desired endpoint. You can also specify the number of workers as needed:
+
+```bash
+    docker run -d -e WORKERS=5 --rm --name allie-flowkit-python -p 50052:50052 allie-flowkit-python:latest
+```
 
 ## Adding custom functions
 
-1. **Create a function.**
+1. **Create a New Function:**
    - Add your function code as an endpoint to a new Python file in the `app/endpoints` directory.
    - Use the `app/endpoints/splitter.py` file and its endpoints as an example.
-   - Be explicit about the input and output of the function as they are used by the Allie agent
-   to call the function.
-
-2. **Add the models for the function.**   
-   - Add the models for the input and output of the function in the `app/models` directory.
-   - Use the `app/models/splitter.py` file its models as an example.
-   
-2. **Add the endpoints to the service.**
-   
-   - Import your module in the `app/app.py` file.
-   - Add the router to the service:
-     ```python
-     app.include_router(splitter.router, prefix="/custom_module", tags=["custom_module"])
-     ```
-
-**Example**
- ```python
- from fastapi import FastAPI, APIRouter
- from app.models.custom_model import CustomRequest, CustomResponse
-
- app = FastAPI()
- router = APIRouter()
-
- @router.post('/custom_function', response_model=CustomResponse)
- async def custom_function(request: CustomRequest) -> CustomResponse:
-     """Endpoint for custom function.
-
-     Parameters
-     ----------
-     request : CustomRequest
-        Object containing the input data required for the function.
-        
-     Returns
-     -------
-     CustomResponse
-        Object containing the output data of the function.
-     """
-     # Your custom processing logic here
-     result = ...
-     return result
- ```
+   - Explicitly define the input and output of the function using Pydantic models, as these will be used by the Allie Agent to call the function.
+
+2. **Add the models for the function:**
+   - Create the models for the input and output of the function in the `app/models` directory.
+   - Use the `app/models/splitter.py` file and its models as an example.
+
+3. **Add the endpoints to the service:**
+   - Import your module in the `app/app.py` file and add the router to the service.
+
+4. **Add the function to the function map:**
+    - Add your function to the `function_map` dictionary in the `app/app.py` file.
+
+### Example´
+
+1. **Create a new file for all your custom functions:**
+- In the `app/endpoints` directory, create a new Python file named `custom_endpoint.py`.
+
+2. **Create the models for the custom function:**
+- In the `app/models` directory, create a new Python file named `custom_model.py`.
+
+    **custom_model.py**:
+    ```python
+    from pydantic import BaseModel
+
+
+    class CustomRequest(BaseModel):
+        """Model for the input data required for the custom function.
+
+        Parameters
+        ----------
+        BaseModel : pydantic.BaseModel
+            Base model for the request.
+
+        """
+
+        input_data: str
+
+
+    class CustomResponse(BaseModel):
+        """Model for the output data of the custom function.
+
+        Parameters
+        ----------
+        BaseModel : pydantic.BaseModel
+            Base model for the response.
+
+        """
+
+        output_data: str
+    ```
+
+3. **Define your custom function:**
+- Add your function to ``custom_endpoint.py``, explicitly defining the input and output using Pydantic models.
+
+    **custom_endpoint.py**:
+    ```python
+    from fastapi import FastAPI, APIRouter
+    from app.models.custom_model import CustomRequest, CustomResponse
+
+
+    @router.post("/custom_function", response_model=CustomResponse)
+    async def custom_function(request: CustomRequest) -> CustomResponse:
+        """Endpoint for custom function.
+
+        Parameters
+        ----------
+        request : CustomRequest
+            Object containing the input data required for the function.
+
+        Returns
+        -------
+        CustomResponse
+            Object containing the output data of the function.
+
+        """
+        # Your custom processing logic here
+        result = ...
+        return result
+    ```
+
+4. **Import the module and add the router to the service:**
+- Import the module in the ``app/app.py`` file and add the router to the service.
+
+    **app.py**:
+    ```python
+    from app.endpoints import custom_endpoint
+
+    app.include_router(splitter.router, prefix="/splitter", tags=["splitter"])
+    app.include_router(
+        custom_endpoint.router, prefix="/custom_endpoint", tags=["custom_endpoint"]
+    )
+    ```
+
+5. **Add the function to the function map:**
+- Add your function to the ``function_map`` dictionary in the ``app/app.py`` file.
+
+    **app.py**:
+    ```python
+    function_map = {
+        "split_ppt": splitter.split_ppt,
+        "split_pdf": splitter.split_pdf,
+        "split_py": splitter.split_py,
+        "custom_function": custom_endpoint.custom_function,
+    }
+    ```
 
 ## Example functions
 

app/__init__.py

@@ -1 +1 @@
-# app/__init__.py
+"""App package responsible for creating the FastAPI app."""

app/app.py

@@ -1,7 +1,42 @@
-from fastapi import FastAPI
+from typing import List
+
+from app.config._config import CONFIG
 from app.endpoints import splitter
+from app.fastapi_utils import extract_endpoint_info
+from app.models.functions import EndpointInfo
+from fastapi import FastAPI, Header, HTTPException
 
 app = FastAPI()
 
 # Include routers from all endpoints
 app.include_router(splitter.router, prefix="/splitter", tags=["splitter"])
+
+# Map of function names to function objects
+function_map = {
+    "split_ppt": splitter.split_ppt,
+    "split_pdf": splitter.split_pdf,
+    "split_py": splitter.split_py,
+}
+
+
+# Endpoint to list all enpoint information
+@app.get("/", response_model=List[EndpointInfo])
+async def list_functions(api_key: str = Header(...)) -> List[EndpointInfo]:
+    """List all available functions and their endpoints.
+
+    Parameters
+    ----------
+    api_key : str
+        The API key for authentication.
+
+    Returns
+    -------
+    List[EndpointInfo]
+        A list of EndpointInfo objects representing the endpoints.
+
+    """
+    # Check if the API key is valid
+    if api_key != CONFIG.flowkit_python_api_key:
+        raise HTTPException(status_code=401, detail="Invalid API key")
+
+    return extract_endpoint_info(function_map, app.routes)

app/config/__init__.py

@@ -0,0 +1,3 @@
+"""Configuration package for the application."""
+
+from app.config._config import CONFIG