Skip to content

Latest commit

 

History

History
9 lines (5 loc) · 2.11 KB

File metadata and controls

9 lines (5 loc) · 2.11 KB

MLS Email Notification Scraper

Mainly I want a system that will allow me to better filter MLS results while house hunting. Right now, most MLS Systems rely on data to be properly "tagged" or "marked" for their filters to work. The problem is that many listings are not properly marked, and thus are able to bypass many of the filters being used. For example, many listings for homes in 55+ communities are not tagged as such, but they will explicitly say "55+" in the description. Even when filtering out 55+ listings, these ones will still get through, wasting time for those wishing to explicitly ignore those listings.

From my understanding, there is no single API that can provide all the MLS data for the entire country, so if you wanted to use MLS data directly, you would need to spend thousands of dollars to integrate with each MLS individually, which for individual use is not feasible. So instead, I propose using the Email notifications, and scraping those as they come in to filter against. Realistically this will only be usable for developers or power users, but still.

The user would create a separate email account specifically for signing up to the various MLS systems (RedFin, Zillow, etc.), and configure their searches and filters within these systems, enabling notification emails from these systems to send new results as soon as they are available. Then this script would use the GMail API to, once per day, check for new emails, and use these emails to scrape data (or load the listing page to scrape data). Then, compare this data against a set of filters that will look at more than just "tags" like checking the description. If a listing is able to pass all filters, then forward the notification email on to the users actual email address. If setup as a web site (e.g. using Laravel), this information could even be saved to a custom database, allowing the user to only browse listings that actually match their criteria.

This system could also tie to some API to provide information about interest rates, and potentially increase the rate of checking and forwarding emails if the interest rates fall below a certain threshold.