AI search visibility
Whether to let AI tools read your website at all
Somewhere in the last couple of years your website probably took a position on this without anybody asking you. Worth finding out which one.
Every website has a small text file on it that tells automated readers whether they are welcome. ChatGPT, Google AI, Perplexity and the rest each identify themselves when they arrive, and that file either lets them in or turns them away.
A lot of website platforms started turning them away by default. A lot of agencies started doing it as a paid service, usually framed as protecting your content. For a newspaper, that argument holds. For a contractor, it is close to shooting yourself in the foot.
What turning them away actually costs you
If those tools cannot read your site, they cannot name you. They will still answer the customer asking who does furnace repair in your town. They will just answer using a competitor who left the door open, or using an old directory listing that gets your details wrong and takes the call.
And think about what you would be protecting. It is a page describing furnace repair. Nobody is going to read it instead of hiring somebody to fix their furnace.
The version worth watching for is an agency selling you AI visibility from a website that turns those same tools away. It happens, and it is a fast way to find out whether the person pitching you has done this before.
How to check yours in ten minutes
Type your website address into a browser with /robots.txt on the end, so it reads something like yourcompany.com/robots.txt. You will get a plain page of text. It is not pretty and it is not meant to be.
Read down it looking for lines saying Disallow underneath a name you do not recognise. Names worth spotting are GPTBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot. If those are being turned away and you never asked for that, somebody made the decision for you and it is worth reversing.
One honest caveat. Letting them in only helps if there is something worth reading when they get there. Access matters and it is nowhere near enough on its own.
- Open your own site address with /robots.txt on the end
- Look for Disallow lines naming GPTBot, ClaudeBot, PerplexityBot or Google-Extended
- Run the same check on any agency pitching you AI services
- Once they are allowed in, make sure your words are genuinely in the page
The rest of the cluster
| Topic | What it covers |
|---|---|
| Getting named in the answer | A search gives ten names. An AI answer gives two. What decides which two. |
| What GEO really means | Some of what gets sold under this name is real. Some is old work with a new invoice. |
| Labelling your facts | A standard way to label your phone, hours and services so software reads them right. |
| Laying out a page | Answer first, proof in the same section, and nothing important hidden until you click. |
| Checking what AI says about you | An afternoon of asking the questions your customers ask, and writing down what comes back. |
| Checking your existing labels | After a few plugins, most sites quietly claim things nobody meant. The four usual problems. |
Back to AI search visibility, or see what publishing primary data looks like in practice on our contractor website research.