chore: update robots.txt to disallow additional AI user agents

Signed-off-by: rajput-hemant <[email protected]>

Author: rajput-hemant

public/robots.txt

@@ -29,4 +29,25 @@ User-agent: ChatGPT-User
 Disallow: /
 
 User-agent: omgili
+Disallow: /
+
+User-agent: meta-externalagent
+Disallow: /
+
+User-agent: Amazonbot
+Disallow: /
+
+User-agent: Perplexity-User
+Disallow: /
+
+User-agent: PerplexityBot
+Disallow: /
+
+User-agent: Applebot-Extended
+Disallow: /
+
+User-agent: ClaudeBot
+Disallow: /
+
+User-agent: FacebookBot
 Disallow: /