text robots.txt

# The asterisk (*) in the User-agent field is shorthand for "all robots"
User-agent: *
Disallow: /admin/
Allow: /
Sitemap: http://www.example.com/sitemap.xml

Example robots.txt file - specifying all User Agents, allow, disallow and URL to the sitemap.

Updated: Tuesday 5th October 2010, 11:44pm

There are 0 comments

snippets.slickhouse.com

text robots.txt

Languages