Enhance RAG system and PDF processing capabilities

- Introduced flags to track generation state and stop generation in the chat interface.
- Updated chunk size in RAG configuration to improve retrieval accuracy.
- Enhanced PDF text extraction with a new `EnhancedPDFProcessor`, providing fallback mechanisms for error handling.
- Improved system prompt structure for better clarity and response generation.
- Adjusted retrieval logic in the RAG system to ensure high-similarity chunks are prioritized and filtered effectively.
- Updated tests to validate new PDF processing features and ensure robust error handling.

Author: clstaudt