# A standard robots.txt file that allows all crawlers access to everthing. User-agent: * Disallow: