Free tools / robots.txt checker
Does your robots.txt let AI shoppers in?
Enter your store. We read its robots.txt and show which AI shopping agents and AI training crawlers can open your products, cart and checkout.
Free. Instant. Nothing stored.
AI shopping agents
These fetch your pages when a person asks them to find or buy something. You want these allowed.
AI training crawlers
These collect pages to train AI models. Blocking them is your choice and doesn’t affect shopping agents.
Show the robots.txt we read
Firewalls, bot protection and CAPTCHAs can still stop an agent that robots.txt allows. The free scan sends a real AI agent through your store, from product to checkout, and shows where it gets stuck.
How to read the result
- Allowed The agent may fetch that kind of page.
- Blocked A Disallow rule covers that path, so well-behaved agents stay out.
- Named Your robots.txt has rules written specifically for this agent; otherwise the agent follows your
User-agent: *rules.
Fix it in two minutes
Read robots.txt for AI shopping agents for copy-paste rules that let shopping agents in and keep training crawlers out. On WooCommerce, Tallygator Pro makes the change for you and undoes it in one click.
Questions
What does this check?
It reads your store’s robots.txt the way AI agents do (the RFC 9309 rules: the most specific matching rule wins) and tells you, for each AI shopping agent and AI training crawler, whether it may open your home page, shop, product, cart and checkout pages.
What’s the difference between shopping agents and training crawlers?
Shopping agents fetch your pages when a person asks them to find or buy something, like ChatGPT-User or Claude-User. Training crawlers collect pages to train AI models, like GPTBot or CCBot. Most stores want the first and many block the second. You can do both; see our robots.txt guide.
My robots.txt allows everything. Am I fine?
For robots.txt, yes. But robots.txt is only the polite request. Firewalls, bot protection and CAPTCHAs can still block agents that robots.txt allows. Our free scan sends a real agent through your store to check all of it.
Do you store what I check?
No. We fetch your robots.txt, show you the result and keep nothing except a short-lived rate limit counter.