|
1 | | -# AI Newsletter Digest |
| 1 | +# Cosmic Digest |
2 | 2 |
|
3 | | -An automated AI-powered newsletter system that curates personalized news digests from RSS feeds, tracks prices, and delivers customized summaries via email. |
| 3 | +A personal "signal over noise" digest that pulls stories from RSS feeds, scores them against your interests, tracks a small price watchlist, and emails a compact 3-day brief via Mailgun. |
4 | 4 |
|
5 | | -## Features |
| 5 | +- Built for automation: run on a schedule, keep lightweight state, and get the digest in your inbox. |
| 6 | +- Built for personalization: relevance scoring via topics/regions/keywords + recency boost. |
| 7 | +- Built for extensibility: a clean pipeline you can swap pieces in/out (better scoring, LLM summaries, real price APIs). |
6 | 8 |
|
7 | | -- ?? **RSS Feed Ingestion**: Aggregates news from multiple RSS sources |
8 | | -- ?? **Personalized Relevance Scoring**: Filters news based on your topics, regions, and keywords |
9 | | -- ?? **Trend Detection**: Identifies developing stories by comparing recent vs. previous periods |
10 | | -- ?? **Price Tracking**: Monitors product prices and provides buy/hold recommendations |
11 | | -- ?? **AI-Powered Summaries**: Uses OpenAI to generate concise, relevant news digests |
12 | | -- ?? **Email Delivery**: Sends digests via Mailgun |
| 9 | +## Screenshots |
| 10 | +Add your images to `docs/screenshots/` and replace these placeholders. |
13 | 11 |
|
14 | | -## Tech Stack |
15 | | - |
16 | | -- **.NET 10** (C# 14.0) |
17 | | -- **OpenAI API** for AI-powered summaries |
18 | | -- **Mailgun** for email delivery |
19 | | -- **CodeHollow.FeedReader** for RSS parsing |
20 | | -- **AngleSharp** for web scraping |
21 | | -- **DotNetEnv** for environment configuration |
22 | | - |
23 | | -## Prerequisites |
24 | | - |
25 | | -- .NET 10 SDK |
26 | | -- OpenAI API key |
27 | | -- Mailgun account (free sandbox tier works for testing) |
28 | | - |
29 | | -## Setup |
30 | | - |
31 | | -1. **Clone the repository** |
32 | | - ```bash |
33 | | - git clone https://github.com/yourusername/ai-newsletter.git |
34 | | - cd ai-newsletter |
35 | | - ``` |
36 | | - |
37 | | -2. **Install dependencies** |
38 | | - ```bash |
39 | | - dotnet restore |
40 | | - ``` |
41 | | - |
42 | | -3. **Configure environment variables** |
43 | | - |
44 | | - Copy `.env.example` to `.env` and fill in your credentials: |
45 | | - ```bash |
46 | | - cp .env.example .env |
47 | | - ``` |
48 | | - |
49 | | - Edit `.env` with your values: |
50 | | - - `OPENAI_API_KEY`: Your OpenAI API key |
51 | | - - `MAILGUN_API_KEY`: Your Mailgun API key |
52 | | - - `MAILGUN_DOMAIN`: Your Mailgun domain (sandbox or verified) |
53 | | - - `MAIL_TO`: Recipient email address |
54 | | - - `MAIL_FROM`: Sender email address (optional) |
55 | | - |
56 | | -4. **Configure your preferences** |
57 | | - |
58 | | - Customize these in `.env`: |
59 | | - - `PREF_TOPICS`: Topics you're interested in |
60 | | - - `PREF_REGIONS`: Regions to focus on |
61 | | - - `PREF_KEYWORDS`: Specific keywords to prioritize |
62 | | - - `RSS_FEEDS`: Comma-separated list of RSS feed URLs |
63 | | - - `PRICE_WATCH`: Products to track (format: `name|url|currency`) |
64 | | - |
65 | | -## Usage |
66 | | - |
67 | | -Run the newsletter generator: |
68 | | - |
69 | | -```bash |
70 | | -dotnet run |
71 | | -``` |
72 | | - |
73 | | -The application will: |
74 | | -1. Load existing state from `/data/state.json` |
75 | | -2. Fetch fresh news from configured RSS feeds |
76 | | -3. Score and filter articles based on your preferences |
77 | | -4. Detect trending topics |
78 | | -5. Update tracked prices |
79 | | -6. Generate a personalized digest |
80 | | -7. Send it via email |
81 | | -8. Save updated state |
| 12 | + |
| 13 | + |
| 14 | + |
| 15 | + |
82 | 16 |
|
83 | | -## Configuration |
| 17 | +## How It Works |
| 18 | +1. Ingest RSS feeds into a rolling cache (`/data/state.json`). |
| 19 | +2. Score items for relevance (topics/regions/keywords + recency) and select the top stories. |
| 20 | +3. Detect "developing stories" by comparing title tokens in the last N days vs the previous window. |
| 21 | +4. Fetch prices for a watchlist (currently a naive HTML scan; designed to be replaced with APIs). |
| 22 | +5. Compose a Markdown digest and send it via Mailgun. |
84 | 23 |
|
85 | | -### RSS Feeds |
86 | | -Add RSS feeds as a comma-separated list: |
87 | | -``` |
88 | | -RSS_FEEDS=https://feeds.bbci.co.uk/news/rss.xml,https://www.reuters.com/rssFeed/worldNews |
89 | | -``` |
90 | | - |
91 | | -### Personalization |
92 | | -``` |
93 | | -PREF_TOPICS=ai,hardware,gaming,geopolitics,macroeconomy |
94 | | -PREF_REGIONS=US,EU,Middle East |
95 | | -PREF_KEYWORDS=AMD,NVIDIA,SSD,GPU |
96 | | -``` |
97 | | - |
98 | | -### Price Tracking |
99 | | -Track products with semicolon-separated entries: |
100 | | -``` |
101 | | -PRICE_WATCH=GPU RTX 4090|https://example.com/gpu|USD;SSD 4TB|https://example.com/ssd|USD |
| 24 | +## Tech Stack |
| 25 | +- Runtime: .NET 10 (`net10.0`), C# |
| 26 | +- Integrations: Mailgun HTTP API |
| 27 | +- Content ingestion: `CodeHollow.FeedReader` (RSS) |
| 28 | +- HTML parsing/scraping: `AngleSharp` (price fetcher) |
| 29 | +- Config: `DotNetEnv` (`.env` for local dev) |
| 30 | +- Containerization: Docker |
| 31 | +- CI: GitHub Actions (.NET build workflow) |
| 32 | +- Optional/experimental: OpenAI .NET SDK (see `NewsAi.cs`) |
| 33 | + |
| 34 | +## Sample Output |
| 35 | +```md |
| 36 | +## Developing stories (what changed) |
| 37 | +- **ai chips** - now: 9, prev: 2, delta: +7 |
| 38 | + |
| 39 | +## Worldwide but relevant to you |
| 40 | +- [Headline...](https://...) |
| 41 | + |
| 42 | +## Price trends (watchlist) |
| 43 | +- **Example Product** - HOLD. Price above recent lows/avg. Latest: 123.45 USD on 2025-01-01 |
102 | 44 | ``` |
103 | 45 |
|
104 | | -## Deployment |
105 | | - |
106 | | -### Docker Support |
107 | | -A Dockerfile is included for containerized deployment. |
| 46 | +## Quickstart (Local) |
| 47 | +**Prereqs:** .NET 10 SDK, Mailgun API key + domain. |
108 | 48 |
|
109 | 49 | ```bash |
110 | | -docker build -t ai-newsletter . |
111 | | -docker run --env-file .env ai-newsletter |
| 50 | +cd Cosmic-Digest |
| 51 | +dotnet restore |
112 | 52 | ``` |
113 | 53 |
|
114 | | -### Fly.io |
115 | | -This project is ready for deployment on Fly.io. Set your secrets: |
| 54 | +Create your `.env`: |
| 55 | +- PowerShell: `Copy-Item .env.example .env` |
| 56 | +- macOS/Linux: `cp .env.example .env` |
116 | 57 |
|
| 58 | +Run: |
117 | 59 | ```bash |
118 | | -fly secrets set OPENAI_API_KEY=your_key |
119 | | -fly secrets set MAILGUN_API_KEY=your_key |
120 | | -fly secrets set MAILGUN_DOMAIN=your_domain |
121 | | -fly secrets set MAIL_TO=your_email |
122 | | -# ... set other secrets |
| 60 | +dotnet run --project ai-newsletter.csproj |
123 | 61 | ``` |
124 | 62 |
|
125 | | -## Project Structure |
| 63 | +Note: state persists to `/data/state.json`. On Windows, this maps to `C:\\data\\state.json` (root of the current drive). For a cleaner setup, prefer Docker (below) or change the path in `StateStore.cs`. |
126 | 64 |
|
| 65 | +## Configuration |
| 66 | +Set these in `.env` (see `.env.example`): |
| 67 | + |
| 68 | +| Variable | What it does | |
| 69 | +| --- | --- | |
| 70 | +| `MAILGUN_API_KEY` | Mailgun API key | |
| 71 | +| `MAILGUN_DOMAIN` | Mailgun domain (sandbox or verified) | |
| 72 | +| `MAIL_TO` | Recipient email | |
| 73 | +| `MAIL_FROM` | Sender email (defaults to `bot@{MAILGUN_DOMAIN}`) | |
| 74 | +| `RSS_FEEDS` | Comma-separated RSS feed URLs | |
| 75 | +| `PREF_TOPICS` | CSV topics you care about | |
| 76 | +| `PREF_REGIONS` | CSV regions you care about | |
| 77 | +| `PREF_KEYWORDS` | CSV high-signal keywords | |
| 78 | +| `PRICE_WATCH` | `name|url|currency` entries separated by `;` | |
| 79 | +| `OPENAI_API_KEY` | Only needed if you wire in LLM summaries | |
| 80 | + |
| 81 | +## Quickstart (Docker) |
| 82 | +```bash |
| 83 | +cd Cosmic-Digest |
| 84 | +docker build -t cosmic-digest . |
| 85 | +docker run --rm --env-file .env -v cosmic-digest-data:/data cosmic-digest |
127 | 86 | ``` |
128 | | -ai-newsletter/ |
129 | | -??? Program.cs # Main application logic |
130 | | -??? NewsAi.cs # OpenAI integration for summaries |
131 | | -??? Models.cs.cs # Data models |
132 | | -??? StateStore.cs # State persistence |
133 | | -??? RssIngestor.cs # RSS feed parsing |
134 | | -??? Relevance.cs # Scoring and trend detection |
135 | | -??? PriceTracker.cs # Price monitoring |
136 | | -??? DigestComposer.cs # Markdown digest generation |
137 | | -??? .env.example # Environment template |
138 | | -``` |
139 | | - |
140 | | -## Security Notes |
141 | | - |
142 | | -?? **Never commit your `.env` file to GitHub!** It contains sensitive API keys. |
143 | | - |
144 | | -- The `.gitignore` is configured to exclude `.env` |
145 | | -- Use `.env.example` as a template |
146 | | -- For production, use environment variables or secure secret management |
147 | | - |
148 | | -## Mailgun Free Tier Limitations |
149 | 87 |
|
150 | | -If using Mailgun's free sandbox domain: |
151 | | -- You must add authorized recipients in the Mailgun dashboard |
152 | | -- Recipients must confirm their email address |
153 | | -- Consider upgrading or using a verified domain for production |
154 | | - |
155 | | -## License |
156 | | - |
157 | | -MIT License - feel free to use and modify as needed. |
158 | | - |
159 | | -## Contributing |
160 | | - |
161 | | -Contributions are welcome! Please open an issue or submit a pull request. |
| 88 | +## Project Highlights (Portfolio Notes) |
| 89 | +- Pipeline-style design: each stage is a small, testable unit (ingestion -> scoring -> trends -> prices -> compose -> send). |
| 90 | +- State management: rolling cache for news + time-series price history in a single JSON file. |
| 91 | +- Trend detection: windowed comparison to surface "what changed" instead of just "what happened". |
162 | 92 |
|
163 | 93 | ## Roadmap |
| 94 | +- [ ] Wire in LLM summaries for the selected headlines (optional mode) |
| 95 | +- [ ] Add scheduling (cron / hosted job) |
| 96 | +- [ ] Replace naive price scraping with API-backed integrations |
| 97 | +- [ ] Add web dashboard for configuration and history |
| 98 | +- [ ] Swap JSON file state for a database when needed |
164 | 99 |
|
165 | | -- [ ] Add more sophisticated AI summarization |
166 | | -- [ ] Support additional email providers (SendGrid, AWS SES) |
167 | | -- [ ] Improve price tracking with API integrations |
168 | | -- [ ] Add web dashboard for configuration |
169 | | -- [ ] Implement scheduling (cron/periodic runs) |
170 | | -- [ ] Add database support for larger state storage |
171 | | - |
172 | | -## Author |
| 100 | +## Security |
| 101 | +- Never commit `.env` (API keys). Use `.env.example` as a template. |
| 102 | +- For production, prefer platform secrets (GitHub Actions/Fly/your host). |
173 | 103 |
|
174 | | -Built with ?? for personalized news consumption |
| 104 | +## License |
| 105 | +MIT |
0 commit comments