Implement Phase 2 - Backend Database Infrastructure

[andrade94]

Description:

This pull request implements Phase 2 of the backend development, focusing on establishing the core database infrastructure for both PostgreSQL and MongoDB as outlined in next-implementations.md. It sets up the necessary models, schemas (utilizing existing ones for MongoDB), and connection utilities required for subsequent data storage and retrieval operations.

Significant effort was also dedicated to debugging and resolving Python import path issues and configuration differences encountered when running the backend application both locally and within the Docker Compose environment.

Changes Made:

• MongoDB Connection Utilities:
  • Created backend/app/db/mongo_utils.py containing asynchronous functions (get_mongo_client, close_mongo_connection, get_mongo_database, get_mongo_collection) to manage the Motor client connection using settings from config.py.
• FastAPI Integration:
  • Modified backend/app/main.py to:
    • Integrate mongo_utils.py functions into FastAPI's startup and shutdown events for MongoDB connection lifecycle management.
    • Update the /health check endpoint to use the new MongoDB utility for verifying the connection status.
• PostgreSQL Models (SQLModel):
  • Ensured the application utilizes the existing, detailed SQLModel definitions located in individual files:
    • backend/app/db/models/political_entity.py
    • backend/app/db/models/social_media_account.py
    • backend/app/db/models/entity_relationship.py
  • Deleted a redundant, combined model file (political_models.py) that was initially created.
• MongoDB Schemas (Pydantic):
  • Ensured the application utilizes the existing, comprehensive Pydantic schemas defined in backend/app/db/schemas/mongodb.py.
  • Deleted redundant, simplified schema files (post_schema.py, comment_schema.py, analysis_schema.py) that were initially created.
• Import Path & Environment Debugging:
  • Added missing __init__.py files to backend/ and backend/app/db/ to ensure proper package recognition by Python.
  • Corrected import paths within mongo_utils.py (from backend.app... to app...) to be compatible with the Docker container's working directory.
  • Removed temporary sys.path modifications from main.py used during local debugging.
  • Diagnosed MongoDB connection issues when running locally vs. Docker (hostname resolution localhost vs. mongodb) and confirmed the Docker setup works correctly with service names.
• Documentation:
  • Created an updated database-architecture.md file in the project root directory, reflecting the actual codebase structure (individual SQL model files, correct MongoDB schema path, and Pydantic definitions).

Testing:

• Attempted local server startup (uvicorn / python -m uvicorn) which revealed and led to the fixing of Python import path issues.
• Attempted local server startup which revealed MongoDB connection hostname issues (localhost vs. mongodb service name).
• Successfully built and started the full application stack using docker compose up --build -d.
• Verified successful application startup and database connections (MongoDB, Pinecone) within the Docker environment by inspecting the backend container logs via docker compose logs --follow backend.
• Confirmed the /health endpoint reports correctly via logs after startup.

Next Steps:

• Implement repository layers to interact with the newly defined models and schemas.
• Develop API endpoints that utilize these repositories for CRUD operations.
• Consider standardizing configuration (e.g., in .env.example) for running locally vs. Docker, especially regarding database hostnames.