How to rank in Claude
Claude can search the web and cite what it finds. Its results are widely reported to come from Brave Search, which means the work that gets you into Claude is different from the work that gets you into Google or ChatGPT.
By the AIO Automatic team · Last updated
Where does Claude get its web search results?
Claude's web search draws on Brave Search, an independent index separate from Google and Bing. Anthropic lists Brave among its providers, and Claude's citations closely match Brave results. If Brave has not indexed your pages, Claude is unlikely to cite them, even when you rank well on Google.
Brave builds its index with its own crawler and anonymized signals about which pages people actually visit. Pages with real traffic and links get discovered faster.
Which Anthropic crawlers should you allow?
Allow Claude-SearchBot, Claude-User and ClaudeBot. Claude-SearchBot improves the quality of search results, Claude-User fetches pages when a user asks Claude to read them, and ClaudeBot collects training data. Blocking the first two removes you from live answers; blocking ClaudeBot only limits what future models learn.
- Add explicit Allow groups for each Anthropic user agent in robots.txt.
- Serve content as raw HTML: Claude's fetchers do not execute JavaScript.
- Publish llms.txt with a clean summary and links to your key pages.
| User agent | Purpose | If you block it |
|---|---|---|
| Claude-SearchBot | Improves Claude's search result quality | Your pages are less likely to surface in Claude search answers |
| Claude-User | Fetches pages when a user asks Claude to | Claude cannot read your page on request |
| ClaudeBot | Collects content for model training | Future Claude models learn less about you |
How do you get your pages into Brave Search?
Search site:yourdomain.com on search.brave.com to see what Brave has indexed. Missing pages usually need stronger discovery: internal links from indexed pages, an up-to-date sitemap, external links, and real visits. Then re-check after each content release, because Claude can only cite what Brave already holds.
Treat Brave like a third search engine in your reporting. A page can be indexed in Google and Bing and still be missing from Brave.
What content does Claude cite?
Claude favors pages that answer the question plainly and precisely, with clear headings, definitions, numbers and sourced claims. It is cautious about promotional copy. Write answer-first sections, cite your data sources, keep your brand facts identical across the web, and avoid vague superlatives that Claude cannot verify elsewhere.
See how to get AI to recommend your brand for identity and corroboration.
Frequently asked questions
Does ranking on Google help in Claude?
Indirectly. Claude's live citations come from Brave Search, so Google rankings do not transfer automatically. The authority and links behind good Google rankings do help Brave discover and trust you.
Does Claude use Bing?
Claude's web search is associated with Brave Search rather than Bing. Cover Bing for ChatGPT and Copilot, and Brave for Claude.
Is Claude-Web still a crawler name?
Older robots.txt files reference Claude-Web and anthropic-ai. Anthropic now documents ClaudeBot, Claude-User and Claude-SearchBot, so make sure those three are covered.