Cloud: robots.txt and basic auth

Author: AugustMiller

docs/cloud/domains.md

@@ -18,6 +18,8 @@ To find your preview domain, click the globe icon next to any environment in the
 Changing the handle of a project or environment may take a moment to update in our routing layer. If you have other services that rely on a consistent hostname (say, for the delivery of webhooks), consider temporarily pointing a [subdomain](#subdomains) at the environment.
 :::
 
+We automatically inject a `robots.txt` for preview domains (and non-production [environments](environments.md)) that disallows all crawling. If you want to prevent regular users from accessing your site’s front-end, consider using [basic HTTP authentication](/5.x/reference/config/app.md#basic-authentication).
+
 ## Adding a Domain
 
 In your Cloud project, navigate to **Domains**, then click **New domain**.

docs/cloud/environments.md

@@ -70,14 +70,13 @@ Craft Console uses the production environment’s preview domain (or a custom do
 
 ### robots.txt
 
-For any environment that is not marked as production, Craft Cloud will serve requests to `robots.txt` with this body:
+For any environment that is not marked as production, Cloud injects a `robots.txt` with this body, to prevent indexing:
 
 ```
 User-agent: *
 Disallow: /
 ```
-
-So they don’t get indexed by search engines. Production environments will allow the application to dictate the contents of `robots.txt`.
+Production environments allow the application to dictate the contents of `robots.txt`. This is typically handled by creating `templates/robots.txt.twig` and populating it with static text, or using a plugin like [SEOMatic](plugin:seomatic) that expose an editing interface in the control panel.
 
 ## Deleting an Environment