"Turning the noise of the Spanish market into high-purity insights."
In a world of information overload, this project acts as a Digital Refinery. It automatically monitors the pulse of the Spanish news cycle and the tech job market (specifically for my upcoming move to Bilbao), using AI to distinguish between "Market Noise" and "Market Mood."
For the non-techies: Imagine a high-end restaurant in the 22@ district.
Step 1: The Ingestion (The Head Chef) 👨🍳
Our system goes to the markets (LinkedIn, InfoJobs, Google News) every 6 hours to find the freshest headlines.Step 2: The Transformation (The Bouncer) 🕴️
Using dbt, we run 13 different quality tests. We remove duplicates and fix messy dates. If the data isn't "Pure" (per 1 Timothy 4:12), it doesn't get served.Step 3: The Visualization (The Waiter) 🍷
A clean Streamlit dashboard serves the final insights on a silver platter.| Component | Technology | Role |
|---|---|---|
| Automation | Mage.ai | The heartbeat of the factory. |
| Integrity | dbt | Ensuring every row is tested and unique. |
| Storage | PostgreSQL | Fast, sustainable, local-first database. |
| Interface | Streamlit | Turning data into a story. |
"Why did I build this instead of just using a spreadsheet?"
- ⚡ Speed: Manual checking takes hours. This takes milliseconds.
- ⚖️ Integrity: Spreadshots have human bias. Our dbt-tests ensure 100% logic consistency.
- 🌍 Scalability: This system can monitor all of Spain, not just the North, with zero extra effort.
- 🌱 Sustainability: Built with a Lean Docker footprint to minimize compute waste.
If you want to see the pipes under the floorboards:
- Clone the Repo
- Start the Container:
docker-compose up -d
- Check the Dashboard:
localhost:8501
Built with ☕ in 🌼 BCN ... Relocating to 🌲 Bilbao soon.This project represents a commitment to Ethics in Engineering built in the winter of 2026 in Barcelona as a strategic bridge to the industrial powerhouses of Northern Spain.