text robots.txt

# The asterisk (*) in the User-agent field is shorthand for "all robots"
User-agent: *
Disallow: /admin/
Allow: /
Sitemap: http://www.example.com/sitemap.xml
Example robots.txt file - specifying all User Agents, allow, disallow and URL to the sitemap.

Updated: Tuesday 5th October 2010, 04:21pm

Share Snippet

There are 0 comments

Leave a comment of your own

Comments are currently closed.