# Robots.txt user-agent: * Crawl-delay: 30 Disallow: /01 Disallow: /016 Disallow: /017 Disallow: /02 Disallow: /03 Disallow: /04 Disallow: /05 Disallow: /06 Disallow: /07 Disallow: /fb Disallow: /free Disallow: /cgi-bin