Indexability checker for search and AI engines

The Indexability Checker analyzes the signals that affect whether search engines and AI tools can discover, crawl, and index your important pages. WebPixie checks robots.txt rules, llms.txt availability, robots.txt sitemap references, meta robots tags, X-Robots-Tag headers, canonical URLs, hreflang declarations, and link rel attributes like nofollow, ugc, and sponsored. The Indexability Checker surfaces blocking directives, canonical conflicts, missing multilingual signals, and related crawlability issues; a consolidated per-page indexable/blocked verdict is coming soon. This helps you catch accidental noindex rules, restrictive robots.txt directives, missing sitemap references, and other technical SEO issues during review. The results pair well with the Link Crawler, which finds crawlable URLs and link-level signals across your site. For homepage-specific headers, redirects, cookies, and meta checks, use the Main Page Analyzer.

Uptime affects SEO by keeping your pages reachable for search engine crawlers and real users. A brief outage may not cause an immediate ranking change, but repeated downtime can interrupt crawling, reduce trust in page availability, waste crawl opportunities, and create poor user signals when visitors land on errors. Uptime monitoring helps you detect access problems quickly, including timeouts, unexpected status codes, and missing expected content. For marketers, this matters most on high-value pages such as campaign landing pages, product pages, checkout flows, and content that receives frequent organic traffic. WebPixie pairs uptime checks with retry logic to reduce false positives before alerting your team. For search visibility, combine uptime checks with the Indexability Checker and Link Crawler to catch blocked pages, broken links, and crawlability issues.

Yes. The Indexability Checker validates and analyzes text files such as robots.txt and llms.txt line by line, giving each file a valid or invalid verdict and flagging syntax it cannot parse. This lives in the TXT Files view, where you can see how a directive is interpreted instead of guessing whether a rule is written correctly. It also checks whether a sitemap is referenced in robots.txt, so a missing reference shows up before it costs you crawl coverage. On the Main Page view it reads the canonical URL and its alternates, which together indicate how search engines and AI crawlers are likely to treat the page. Indexability is one of the five sub-scores in your composite WebPixie Site Score, alongside header, SSL, DNS, and domain analysis. For link-level crawl health, pair it with the Link Crawler, and compare plan limits on the pricing page.

WebPixie alerts you when a monitored page becomes unreachable or fails the conditions you configured. No monitoring tool can guarantee that a search engine has not attempted to crawl during an outage, but fast detection helps reduce the duration and SEO risk of availability problems. Uptime monitoring checks for timeouts, connection failures, unexpected status codes, redirects, and missing expected content, then uses retry logic to reduce false positives before opening an incident. Alerts are sent by email on every plan, with Slack and webhooks available on eligible plans, so the right team can act quickly. Confirmed outages can be tracked through incident management, including severity, affected resource, and resolution timing. For search-specific risk, combine outage alerts with the Indexability Checker to catch crawl-blocking configuration problems.

They answer different questions. The Link Crawler is about link health: starting from your homepage it follows internal and external links and reports each one's status code, response time, redirect status, and whether it loads successfully. The Indexability Checker is about crawl eligibility: it validates text files such as robots.txt and llms.txt, reads canonical URLs and alternates on the Main Page view, and contributes a sub-score to your composite WebPixie Site Score. A link can be healthy yet blocked from search engines, and a page can be indexable yet link to broken destinations, so the two signals do not replace each other. The crawler deliberately ignores robots.txt while following links, because it must fetch a page to confirm it works, while indexability is where the search-eligibility verdict belongs. Many teams run both and compare plan limits on the pricing page.