consolidate requirements files using a `requirements.in`

[EmilienM]

Description:

Currently, the Dockerfile installs dependencies in two separate steps from two distinct requirements.txt files:

COPY ./common/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY ./huggingface/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

This approach makes it difficult to manage and consolidate dependencies, especially when resolving version conflicts or updating packages consistently.

Suggestion

To simplify dependency management and improve maintainability, I propose introducing a requirements.in file (or one per component if separation is really needed). This would allow us to:

• Declare top-level dependencies in one or more .in files
• Use a tool like pip-compile from [pip-tools](https://github.com/jazzband/pip-tools) to generate a unified, locked requirements.txt