Social Crawlers & Troubleshooting Guide

Resolving common Open Graph issues: crawler cache refreshes, blank preview cards, dimension mismatches, and robots.txt blocks.

Diagnose and resolve the most frequent issues encountered when sharing links on social media platforms and messaging apps.

1. Social Platform Crawler Cache is Not Refreshing

Symptom: You updated your page title, description, or image parameters, but platforms like Facebook, LinkedIn, Twitter/X, or Slack still show the old preview card.

Cause: Social media crawlers cache scraped HTML metadata and images aggressively for days to weeks to reduce bandwidth and server load. Merely purging your own CDN cache will not force third-party social platforms to re-crawl your site.

Fix:

  1. Change the image URL: Add or increment the version parameter (&v=...) in your og:image tag (e.g. &v=2 or &v=20261022).
  2. Force re-scraping using official platform debuggers:
    • Facebook: Facebook Sharing Debugger — paste your page URL and click Scrape Again.
    • LinkedIn: LinkedIn Post Inspector — submit your URL to purge LinkedIn’s internal cache.
    • Twitter / X: Post the link in a private direct message or test tweet draft; Twitter refreshes cards when a new image URL is detected.
    • Telegram: Send the link to the official @WebpageBot on Telegram to trigger a cache refresh.

2. Preview Card is Completely Blank or Missing the Image

Common Causes & Fixes:

  • Insecure Protocol (http:// instead of https://): Modern platforms (especially Facebook and Slack) drop images served over unencrypted HTTP. Always ensure image URLs use https://.
  • Missing twitter:card tag: Twitter/X requires <meta name="twitter:card" content="summary_large_image">. Without this tag, Twitter falls back to a small square icon or plain text link.
  • Blocked by robots.txt: Check your robots.txt file to ensure social crawler user agents (e.g. facebookexternalhit, Twitterbot, LinkedInBot, Slackbot) are not disallowed.
  • Client-Side Rendering (SPA) Delay: If your metadata is injected purely via client-side JavaScript (useEffect or onMounted), simple scrapers like WhatsApp or Facebook will only see empty initial HTML. Ensure your framework uses Server-Side Rendering (SSR) or Static Site Generation (SSG).

3. First Share Shows Small Card, Subsequent Shares Show Large Card

Symptom: The very first time a link is posted to Facebook or LinkedIn, the card displays as a small thumbnail; only on subsequent shares does it display as a full-bleed banner.

Cause: The scraper did not know the image dimensions in advance. When og:image:width and og:image:height are missing, the crawler must asynchronously download and measure the image before deciding whether it qualifies for a large card.

Fix: Always specify explicit dimension tags in your <head>:

<meta property="og:image:width" content="1200" />
<meta property="og:image:height" content="630" />

4. Redirect Chains & Timeouts

Symptom: Scrapers report target_unreachable or fail to render preview cards.

Cause: Crawlers often have strict redirect limits (usually 2 or 3 hops) and short fetch deadlines (often 3 to 5 seconds). If your page or image URL goes through multiple HTTP redirects or temporary tracking URLs, scrapers may abort.

Fix: Ensure canonical URLs point directly to the final destination without intermediary redirect hops.

5. Diagnostic Tools

Before sharing links publicly, use our developer diagnostic tools:

  • Page Inspector: Simulates crawler requests (Facebook, Twitter/X, LinkedIn, Slack), traces redirect chains, checks required tags, and highlights issues.
  • Image Inspector: Validates image dimensions, byte size, format, and displays three standard crop frames.

Last verified against official platform crawler documentation on 2026-10-22.