Facebook Crawler Simulator
Test how Meta's crawler fetches your pages for link previews on Facebook, Instagram, and Threads. Optimize your Open Graph tags.
Test your site with Facebook Crawler
Enter your URL to see how Facebook Crawler views your website.
What is Facebook Crawler?
Facebook's crawler (facebookexternalhit) fetches your web pages to generate link previews when content is shared on Facebook, Instagram, Threads, and Messenger. It reads Open Graph meta tags to display titles, descriptions, and images. The Meta-ExternalAgent crawler is also used for content understanding across Meta's platforms.
Why Allow Facebook Crawler?
Facebook Crawler robots.txt Configuration
Control how Facebook Crawler accesses your website using robots.txt directives. Add these rules to your robots.txt file at the root of your domain.
Allow Facebook Crawler
# Allow Facebook/Meta crawlers User-agent: facebookexternalhit Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: FacebookBot Allow: /
Block Facebook Crawler
# Block Facebook/Meta crawlers (not recommended) User-agent: facebookexternalhit Disallow: / User-agent: Meta-ExternalAgent Disallow: /
User-Agent String: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext)
Facebook Crawler FAQ
Related Crawler Simulators
Make Your Site Crawlable
JavaScript websites often have indexing issues. LovableHTML pre-renders your SPA into crawler-friendly HTML so Facebook Crawler and other bots can read your content.
