Welcome! I’m a Cloud Data Engineer specializing in Python automation, web scraping, and high-volume data extraction. I build production-ready pipelines, interactive bots, and user interfaces to streamline data workflows.
- Languages & Frameworks: Python 🐍, SQL 🗄️, FastAPI 🌐, Streamlit ⚡
- Data Engineering: ETL/ELT workflows, pandas 🐼
- Web Scraping & Automation: Playwright 🎭, Selenium 🕷️, Scrapy 🧱, BeautifulSoup 🍜, Requests 🔗
- Anti-Bot & CAPTCHA Handling: 2Captcha, custom scripts, dynamic challenge bypass
- Bot Development & GUI Applications: Interactive bots and lightweight UIs using Streamlit and Flask
- API Integration: REST APIs, OpenAI, DeepSeek, DeepInfra, OAuth2
- PDF & Document Parsing: PyMuPDF 📄, pdfplumber, OpenAI Vision
- Orchestration & Scheduling: Apache Airflow 🌬️, GitHub Actions ⏱️, cron 🕑, bash scripts
- Cloud Platforms:
- AWS: S3 🪣, EC2 🖥️
- Azure: Blob Storage ☁️
- GCP: BigQuery
- Output Formats: Excel, CSV, Parquet, JSON, Google Sheets
- 🔄 ETL/ELT Pipeline Development – End-to-end data workflows, scheduling & monitoring
- 🤖 Bot Development & GUI Applications – Build interactive bots and user-friendly interfaces for data extraction with Streamlit or Flask
- 🔍 Web Scraping & Data Extraction – Large-scale, reliable extraction from websites, APIs & documents
- 🛡️ Advanced CAPTCHA & Anti-Bot Solutions – Bypass dynamic CAPTCHA and anti-bot measures for consistent data access
- 🔗 API Integration – Connect & automate data from any REST endpoint
- ⚙️ Custom API Development – Deploy robust data services with Flask
- 📄 PDF & Text Extraction – Structured parsing from complex or multilingual files
- ☁️ Cloud Automation – Orchestrate workflows with Airflow, Data Factory or GitHub Actions
-
📄 U.S. Legislative Data Pipeline (2024–2025)
Scraped 30+ state legislature sites (web & PDF), scheduled via Airflow, outputs to Excel. -
🤖 AI-Powered Bots & Interfaces
Developed custom automation bots and Streamlit dashboards for end-user data insights. -
📊 Real-Time Data Connectors
Built scheduled connectors for product, real estate & news APIs—transforming raw feeds for ML. -
🛒 E-commerce Scrapers
Extracted Amazon, Walmart & vendor data at scale, delivering clean datasets for BI.
Let’s connect if you need robust, production-ready data engineering, automation, and interactive solutions for your AI, analytics, or business projects!