Free SEO tool · AI & prompts
AI Crawler Checker
Check whether ChatGPT, Claude, Perplexity, Gemini and other AI crawlers are allowed or blocked by your robots.txt.
Reads your robots.txt and llms.txt and tells you which AI bots can reach your home page.
| Crawler | Company | Used for | Access |
|---|
Why it matters for AI search
ChatGPT, Perplexity, Claude and Gemini answer questions using pages their crawlers can read. If your robots.txt blocks them, your business is less likely to be mentioned or linked in AI answers.
Training bots vs search bots
- Search and user bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot) fetch pages to answer questions and link to sources. Keep these allowed if you want AI visibility.
- Training bots (GPTBot, CCBot, Google-Extended, ClaudeBot) collect content to train models. Blocking them is a business choice.
Next step: create an llms.txt file that points AI tools to your most important pages.
Frequently asked questions
Does blocking Google-Extended affect Google Search?
No. Google-Extended only controls use of your content for Gemini and AI training. Googlebot still crawls your site for Search.
Is the robots.txt check enough?
It shows your rules. Some firewalls and CDNs also block bots, so check your Cloudflare or hosting bot settings too.
What is llms.txt?
A simple text file that lists your key pages for AI tools. It is new, optional and easy to add.
More free tools
Meta Tag Generator
Write title and description tags that fit Google limits.
ON-PAGESERP Preview Tool
See how your page will look in Google results.
ON-PAGESchema Markup Generator
Create FAQ, LocalBusiness, Article and other JSON-LD schema.
ON-PAGESlug Generator
Turn titles into clean, SEO-friendly URL slugs.
CONTENTWord Counter
Count words, characters and reading time.
CONTENTCharacter Counter
Count characters and check limits for Google, X, Instagram and more.
CONTENTKeyword Density Checker
Spot over-used and missing keywords on a page.
CONTENTCase Converter
Switch text to UPPER, lower, Title or Sentence case.
CONTENTLorem Ipsum Generator
Generate placeholder text in paragraphs, sentences or words.
TECHNICALWebsite SEO Checker
Paste a URL, get the top issues holding it back.
TECHNICALPage Speed Checker
Test speed and Core Web Vitals on mobile and desktop.
TECHNICALBroken Link Checker
Find dead links on any page, internal and external.
TECHNICALHTTP Status Code Checker
Check status codes for up to 100 URLs at once.
TECHNICALRedirect Checker
Trace every redirect hop from a URL to its final page.
TECHNICALDomain Age Checker
Find a domain’s age, expiry date, registrar and WHOIS details.
TECHNICALRobots.txt Generator
Create a clean robots.txt without syntax errors.
TECHNICALRobots.txt Tester
Check if a URL is allowed or blocked for any crawler.
TECHNICALXML Sitemap Generator
Crawl a site and build a sitemap.xml file.
LOCAL SEOGoogle Review Link Generator
Make a direct Google review link and QR code.
LOCAL SEONAP Consistency Checker
Check your name, address and phone match across listings.
LOCAL SEOLocal Keyword Generator
Combine services and cities into local search keywords.
AI & PROMPTSllms.txt Generator
Create an llms.txt file that guides AI tools to your best pages.
SOCIAL MEDIAInstagram Hashtag Generator
Get niche, keyword and local hashtags for your next post.
SOCIAL MEDIAInstagram Bio Generator
Write a clear Instagram bio for your business in 150 characters.
SOCIAL MEDIAInstagram Caption Generator
Ready captions with hooks, calls to action and hashtags.
AI & PROMPTSAI Prompt Generator for Marketers
Build ready ChatGPT prompts for SEO, ads, emails and social.
AI & PROMPTSChatGPT Prompt Library
Copy-ready ChatGPT prompts for SEO, content, ads and email.
CONTENTContent Brief Generator
Enter a keyword, get titles, headings, questions and a full brief.
UTILITYImage Converter (JPG, PNG, WebP)
Convert images between JPG, PNG and WebP in your browser.
UTILITYFavicon Generator
Make a favicon set from an image, text or emoji.
UTILITYInvoice Generator
Make GST-ready invoices for freelancers and small businesses.
UTILITYPrivacy Policy Generator
Create a clear privacy policy for your website or app.
UTILITYTerms and Conditions Generator
Create terms and conditions for your website or online store.
UTILITYBulk URL Opener
Open many links or websites at once, in batches.
UTILITYImage Compressor
Shrink JPG, PNG and WebP images for a faster site.
UTILITYSocial Bookmarking Sites List
35+ high DA bookmarking sites. Filter, open in bulk or suggest one.
Tools show the problems. We fix them.
Send us your URL and we will reply with your three biggest issues, free.