Skip to content

Commit 96da25b

Browse files
authored
Update robots.txt (#168)
This should ignore URLs with query params. Source: https://stackoverflow.com/questions/9149782/ignore-urls-in-robot-txt-with-specific-parameters
1 parent 8aa5914 commit 96da25b

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

docs/robots.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,5 @@ layout: null
66

77
# Allow crawling of all content
88
User-agent: *
9-
Disallow:
9+
Disallow: /*?*
1010
Sitemap: {{ site.url }}/sitemap.xml

0 commit comments

Comments
 (0)