This is a section with links to data sources. During my data engineer coaching we need to find good data sets to work with. So, I started this section to make it easier to find good sources.
I've taken these links from articles and blog posts. Why not only link the articles? You know, these posts can go away at any time. I want to keep the links to the platforms either way.
I haven't had the chance to check each link myself. Please let me know if something isn't right.
You can find the articles on the bottom of this section to read more. They include even more data sources I haven't had time to add to this list.
- Student Favorites
- Content Marketing
- Crime
- Drugs
- Education
- Entertainment
- Environmental And Weather Data
- Financial And Economic Data
- General And Academic
- Government And World
- Health
- Human Rights
- Labor And Employment Data
- Politics
- Retail
- Social
- Source Articles and Blog Posts
- Travel And Transportation
- Various Portals
In my Coaching program my students learn by doing a project. And the foundation of every project is selecting a dataset. That can be an API or a file source, depending a lot on the student's goals and interests.
Working out goals for the dataset, figuring out the data modeling, creating the architecture and building it. It's a fun way to learn and get better at Data Engineering.
Here's a list of my student's favorite datasets and APIs
Learn more about the Coaching program: click here
- Fraud detection
- Industrial equipment monitoring
- Energy demand & generation
- Online Retail
- Brazilian E-commerce
- Beijing Air Quality
- NYC Taxi
- Bike sharing Bluebikes
- Bike sharing Divvy Bikes
- Weather API
- Bluesky API
- Guardian news API
- Football API
- Amazon Public Data Sets
- Datasets Subreddit
- Enigma Public
- FiveThirtyEight
- Google Scholar
- Pew Research
- The Upshot by New York Times
- UNData
- Bureau of Justice Statistics
- FBI Crime Statistics
- National Archive of Criminal Justice Data
- Uniform Crime Reporting Statistics
- Drug Data and Database by First Databank
- Drug War Facts
- National Institute on Drug Abuse
- U.S. Food and Drug Administration
- United Nations Office on Drugs and Crime
- Academic Rights Press
- BFI Film Forever
- BLS: Arts, Entertainment, and Recreation
- IFPI
- Million Song Dataset
- Statista: Film Industry
- Statista: Music Industry
- Statista: Video Game Industry
- The Numbers
- Environmental Protection Agency
- International Energy Agency Atlas
- National Center for Environmental Health
- National Climatic Data Center
- National Weather Service
- Weather Underground
- WeatherBase
- Federal Reserve Economic Database
- Financial Data Finder at OSU - Missing link.
- Global Financial Data
- Google Finance
- Google Public Data Explorer
- IMF Economic Data
- National Bureau of Economic Research
- OpenCorporates
- The Atlas of Economic Complexity
- U.S. Bureau of Economic Analysis
- U.S. Securities and Exchange Commission
- UN Comtrade Database
- Visualizing Economics
- World Bank Doing Business Database
- World Bank Open Data
- Data.gov
- European Union Open Data Portal
- Gapminder
- Land Matrix (Transnational Land Database)
- OECD Aid Database
- Open Data Network
- The CIA World Factbook
- The World Bank’s World Development Indicators
- U.S. Census Bureau
- UNDP’s Human Development Index
- America’s Health Rankings
- Centers for Disease Control and Prevention
- Health & Social Care Information Centre
- Health Services Research Information Central
- HealthData.gov
- Medicare Hospital Quality
- MedicinePlus
- National Center for Health Statistics
- SEER Cancer Incidence
- World Health Organization
- Amnesty International
- Human Rights Data Analysis Group
- The Armed Conflict Database by Uppsala University
- Bureau of Labor Statistics
- Department of Labor
- Employment by U.S. Census
- U.S. Small Business Administration
- California Field Poll
- Crowdpac
- Gallup
- Open Secrets
- Rand State Statistics
- Real Clear Politics
- Roper Center for Public Opinion Research
- US Voter Files Note only some states are free, and most do not allow voter files to be used for commercial purposes - this map allows you to see the rules/cost for each state.
- Facebook Graph API
- Google Trends
- SocialMention - Missing link.
- Bureau of Transportation Statistics
- Monthly Tourism Statistics – U.S. Travelers Overseas
- Search the World
- SkiftStats
- U.S. Travel Association
- Ckan
- Dataverse
- DBpedia
- freeCodeCamp Open Data
- Kaggle
- LODUM
- Open Data Ipact Map
- Open Data Kit
- Open Data Monitor
- Plenar.io
- UCI Machine Learning Repository
- Yelp Open Datasets
- 100+ of the Best Free Data Sources For Your Next Project
- 15 Great Free Data Sources for 2016
- 20 Awesome Sources of Free Data
- 30+ Free Data Sources Every Company Should Be Aware Of
- 50 Amazing Free Data Sources You Should Know
- 50 Best Open Data Sources Ready to be Used Right Now
- 70 Amazing Free Data Sources You Should Know
- Big Data: 33 Brilliant And Free Data Sources Anyone Can Use
- These Are The Best Free Open Data Sources Anyone Can Use