# Allow Select User Agents to Crawl Site User-agent: googlebot User-agent: Oncrawl User-agent: slurp User-agent: bingbot User-agent: duckduckbot User-agent: SEMrushBot User-agent: applebot User-agent: facebookexternalhit User-agent: ia_archiver User-agent: linkedinbot User-agent: BrightEdge Crawler User-agent: brightedge Allow: / Disallow: /calendar/action* Disallow: /events/action* Disallow: /cdn-cgi* Disallow: /wp-content/uploads/2023/09/dirparts/* Allow: /*.css Allow: /*.js Disallow: /*? Crawl-delay: 3 # Disallow All Other User Agents from Crawling Site User-agent: * Disallow: /