Badri Sunderarajan
17e6e9e0d1
That's because /article/ and /api/ are dummy paths that can be used for an AI tarpit like Nepenthes!
9 lines
217 B
Text
9 lines
217 B
Text
User-agent: *
|
|
Sitemap: https://snipettemag.com/sitemap.xml
|
|
Disallow: /ghost/
|
|
Disallow: /email/
|
|
Disallow: /members/api/comments/counts/
|
|
Disallow: /r/
|
|
Disallow: /webmentions/receive/
|
|
Disallow: /article/
|
|
Disallow: /api/
|