Merge pull request #33 from AKKI0511/implement-sentiment-analysis-with-litellm

Add LiteLLM sentiment analysis

Author: AKKI0511

LLM.md

@@ -241,13 +241,34 @@ sharpe = sharpe_ratio(returns, risk_free_rate=0.02)
 3. Test individual components
 4. Verify configuration parameters
 
+## LLM Sentiment Analysis
+
+LiteLLM provides a unified interface for scoring text sentiment. Enable it through `config/features_config.yaml`:
+
+```yaml
+sentiment:
+  enabled: true
+  provider: openai
+  model: gpt-3.5-turbo
+  api_key_env_var: OPENAI_API_KEY
+```
+
+Set the corresponding API key:
+
+```bash
+export OPENAI_API_KEY="sk-..."
+```
+
+Switch providers by editing `provider`, `model`, and `api_key_env_var`. The data pipeline automatically adds a `sentiment_score` column during the `generate_sentiment` step.
+
+For CLI and Python examples see [docs/llm-sentiment.md](docs/llm-sentiment.md).
+
 ## Future Development Areas
 
 ### High Priority
 - Real-time data streaming
 - Advanced risk management
 - Multi-timeframe support
-- LLM integration for sentiment analysis
 
 ### Medium Priority
 - GPU acceleration

README.md

@@ -27,6 +27,7 @@ poetry run quanttradeai evaluate -m models/trained/AAPL
 - **📊 Multi-source Data Fetching** - YFinance, AlphaVantage with intelligent caching
 - **🔧 Advanced Feature Engineering** - 20+ technical indicators and custom features
 - **🤖 Ensemble ML Models** - Voting Classifier with LR, RF, XGBoost
+- **🗞️ LLM Sentiment Analysis** - Swapable provider/model via LiteLLM
 - **⚡ Hyperparameter Optimization** - Optuna-based automatic tuning
 - **📈 Comprehensive Backtesting** - Risk management and performance metrics
 - **🎯 Production Ready** - Model persistence, CLI interface, configuration management
@@ -129,6 +130,26 @@ X, y = classifier.prepare_data(df_labeled)
 classifier.train(X, y)
 ```
 
+### Sentiment Analysis
+
+LiteLLM powers provider-agnostic sentiment scoring. Configure it in
+`config/features_config.yaml` and set the API key in your environment:
+
+```yaml
+sentiment:
+  enabled: true
+  provider: openai
+  model: gpt-3.5-turbo
+  api_key_env_var: OPENAI_API_KEY
+```
+
+```bash
+export OPENAI_API_KEY="sk-..."
+```
+
+Switching providers is as simple as updating the YAML config to point to a
+different `provider`/`model` pair supported by LiteLLM.
+
 ## 📊 Performance Metrics
 
 | Metric | Description |

config/features_config.yaml

@@ -52,6 +52,17 @@ feature_combinations:
       - ['close', 'sma_20']
       - ['high', 'low']
 
+# Sentiment Analysis (LLM-based)
+# Configure LiteLLM provider/model and API key env var. Set `enabled: true` to
+# generate a numeric sentiment score (-1 negative, 1 positive) for any text
+# column named "text".
+sentiment:
+  enabled: false
+  provider: openai        # e.g., openai, anthropic, huggingface, ollama
+  model: gpt-3.5-turbo    # model name for the chosen provider
+  api_key_env_var: OPENAI_API_KEY  # environment variable holding the API key
+  extra: {}               # Additional LiteLLM params (e.g., base_url, temperature)
+
 # Feature Selection
 feature_selection:
   method: 'recursive'  # Options: recursive, lasso, random_forest
@@ -74,6 +85,7 @@ pipeline:
     - generate_technical_indicators
     - generate_volume_features
     - generate_custom_features
+    - generate_sentiment
     - handle_missing_values
     - remove_outliers
     - scale_features

docs/README.md

@@ -7,6 +7,7 @@ Welcome to the QuantTradeAI documentation! This comprehensive machine learning f
 ### Core Documentation
 - **[API Documentation](api/)** - Comprehensive reference for all public APIs, functions, and components
 - **[Quick Reference Guide](quick-reference.md)** - Common usage patterns and examples
+- **[LLM Sentiment Analysis](llm-sentiment.md)** - Configure LLM-based sentiment scoring
 - **[Main README](../README.md)** - Project overview and getting started guide
 
 ## 🚀 Quick Start
@@ -240,4 +241,4 @@ For questions and support:
 
 ---
 
-This documentation provides comprehensive coverage of the QuantTradeAI framework. Start with the [Quick Reference Guide](quick-reference.md) for common usage patterns, then refer to the [API Documentation](api/) for detailed function references.
+This documentation provides comprehensive coverage of the QuantTradeAI framework. Start with the [Quick Reference Guide](quick-reference.md) for common usage patterns, then refer to the [API Documentation](api/) for detailed function references.

docs/configuration.md

@@ -140,6 +140,23 @@ feature_combinations:
     - ['high', 'low']
 ```
 
+### Sentiment Features
+
+```yaml
+sentiment:
+  enabled: true
+  provider: openai  # e.g. openai, anthropic, huggingface, ollama
+  model: gpt-3.5-turbo
+  api_key_env_var: OPENAI_API_KEY
+  extra: {}
+```
+
+Set the API key before running:
+
+```bash
+export OPENAI_API_KEY="sk-..."
+```
+
 ### Preprocessing
 
 ```yaml
@@ -170,6 +187,7 @@ pipeline:
     - generate_technical_indicators
     - generate_volume_features
     - generate_custom_features
+    - generate_sentiment
     - handle_missing_values
     - remove_outliers
     - scale_features

docs/llm-sentiment.md

@@ -0,0 +1,69 @@
+# LLM-Powered Sentiment Analysis
+
+QuantTradeAI can attach numeric sentiment scores to any dataset that includes a text column. The feature uses [LiteLLM](https://github.com/BerriAI/litellm) so you can swap providers and models without changing code.
+
+## Setup
+
+1. **Install QuantTradeAI (if not already installed)**
+   ```bash
+   poetry install
+   ```
+2. **Configure `features_config.yaml`**
+   ```yaml
+   sentiment:
+     enabled: true
+     provider: openai        # e.g. openai, anthropic, huggingface, ollama
+     model: gpt-3.5-turbo    # model name for the chosen provider
+     api_key_env_var: OPENAI_API_KEY
+     extra: {}               # optional LiteLLM params
+   ```
+3. **Set the API key**
+   ```bash
+   export OPENAI_API_KEY="sk-..."
+   ```
+
+### Switching Providers
+Change `provider`, `model`, and `api_key_env_var` in the YAML config. Example for Anthropic:
+```yaml
+sentiment:
+  enabled: true
+  provider: anthropic
+  model: claude-instant-1
+  api_key_env_var: ANTHROPIC_API_KEY
+```
+No code changes are required—update the YAML and corresponding environment variable.
+
+## Usage
+
+### Command Line
+```bash
+# with sentiment enabled in features_config.yaml
+export OPENAI_API_KEY="sk-..."
+poetry run quanttradeai train
+```
+The training pipeline loads sentiment settings automatically and adds a `sentiment_score` column.
+
+### Python
+```python
+import pandas as pd
+from quanttradeai.data import DataProcessor
+
+# DataFrame must contain a 'text' column
+raw = pd.DataFrame({"text": ["Stock surges on earnings", "Lawsuit worries investors"]})
+processor = DataProcessor("config/features_config.yaml")
+features = processor.process_data(raw)
+print(features[["text", "sentiment_score"]])
+```
+
+## Input & Output
+- **Input**: DataFrame with a `text` column
+- **Output**: New `sentiment_score` column with values in `[-1, 1]`
+- Added via the `generate_sentiment` step in the feature pipeline
+
+## Troubleshooting
+| Problem | Fix |
+|--------|-----|
+| `API key environment variable ... is not set` | Ensure the variable from `api_key_env_var` is exported |
+| Unsupported provider/model | Verify the provider/model pair is supported by LiteLLM |
+| `sentiment enabled but 'text' column not found` | Provide a `text` column in your input DataFrame |
+| Network/API errors | Check connectivity, credentials, or provider status |