Add robots.txt to notify search engines

[figureone]

Depending on your preference, you can either disallow all search engine indexing, or allow it and point search engines to a generated sitemap they can use. A robots.txt file that prevents search engine indexing looks like:

User-agent: *
Disallow: /

Here's a simple guide for Jekyll if you decide to allow indexing:

http://pixelcog.com/blog/2013/jekyll-from-scratch-core-architecture/#sitemaps-and-robotstxt