User-Agent: WWW::CheckSite::Spider Disallow: /search.htm User-Agent: * Disallow: /unref/ User-Agent: * Disallow: /admin/ User-Agent: * Disallow: /pt/