User-agent: * Disallow: / Allow: /w3c Allow: /accessibilite User-agent: ia_archiver Allow: / User-agent: Twitterbot Disallow: User-agent: Applebot Allow: / User-agent: DuckDuckBot Allow: /