buyfakett/kazoottt-blog
1
0
Fork 0
mirror of https://github.com/KazooTTT/kazoottt-blog.git synced 2025-06-22 10:11:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: add robots.txt

Browse Source
This commit is contained in:
KazooTTT
2024-11-23 23:24:44 +08:00
parent 5a1eaa6884
commit fbb70a3e90
1 changed files with 17 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
public/robots.txt Normal file
View File

@ -0,0 +1,17 @@
User-agent: *
Allow: /
# Sitemaps
Sitemap: https://blog.kazoottt.top/sitemap-index.xml
# Common crawl optimizations
Disallow: /api/
Disallow: /_astro/
Disallow: /404
Disallow: /500
# Allow crawling of content
Allow: /posts/
Allow: /categories/
Allow: /tags/
Allow: /blog/