AI2Bot Crawler Simulator
Test how the Allen Institute's AI2Bot sees your website. Contribute to AI research and academic datasets.
Test your site with AI2Bot
Enter your URL to see how AI2Bot views your website.
What is AI2Bot?
AI2Bot is the web crawler operated by the Allen Institute for AI (AI2), a nonprofit AI research institute. AI2 creates datasets and models used throughout the AI research community, including popular resources like Semantic Scholar. Their crawler collects data for research corpora and academic AI development.
Why Allow AI2Bot?
AI2Bot robots.txt Configuration
Control how AI2Bot accesses your website using robots.txt directives. Add these rules to your robots.txt file at the root of your domain.
Allow AI2Bot
# Allow AI2Bot (Allen Institute) User-agent: AI2Bot Allow: /
Block AI2Bot
# Block AI2Bot User-agent: AI2Bot Disallow: /
User-Agent String: AI2Bot/1.0 (+http://www.allenai.org)
AI2Bot FAQ
Related Crawler Simulators
Make Your Site Crawlable
JavaScript websites often have indexing issues. LovableHTML pre-renders your SPA into crawler-friendly HTML so AI2Bot and other bots can read your content.
