User-agent: * Disallow: /cda/ Disallow: /cgi-bin/ Disallow: /design/ Disallow: /doc_lib/ Disallow: /downloads/ Disallow: /editx/ Disallow: /edit-x/ Disallow: /edx/ Disallow: /liveresponse2/ Disallow: /package/ Disallow: /package_dev/ Disallow: /sig/ Disallow: /support/ Disallow: /smarty/ Disallow: /clients/ Disallow: /scripts/ Crawl-delay: 10 Request-rate: 1/5 # maximum rate is one page every 5 seconds Visit-time: 0600-0845 # only visit between 6:00 AM and 8:45 AM UT (GMT)