From 17e6e9e0d1835ef6881540b2f30c241cf97a8ff1 Mon Sep 17 00:00:00 2001
From: Badri Sunderarajan <badrihippo@disroot.org>
Date: Sun, 23 Mar 2025 01:13:20 +0530
Subject: [PATCH] Add robots.txt, disallowing /article/ and /api/

That's because /article/ and /api/ are dummy paths that can be
used for an AI tarpit like Nepenthes!
---
 robots.txt | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 robots.txt

diff --git a/robots.txt b/robots.txt
new file mode 100644
index 0000000..38fdb15
--- /dev/null
+++ b/robots.txt
@@ -0,0 +1,9 @@
+User-agent: *
+Sitemap: https://snipettemag.com/sitemap.xml
+Disallow: /ghost/
+Disallow: /email/
+Disallow: /members/api/comments/counts/
+Disallow: /r/
+Disallow: /webmentions/receive/
+Disallow: /article/
+Disallow: /api/