Crawler directory · User-triggered AI fetcher

ChatGPT-User: user agent, robots.txt rules, and how to allow or block it

ChatGPT-User fetches a page when a person using ChatGPT asks it to read a URL or a tool call needs live content. It is triggered by a user request, not by a crawl schedule.

Operator: OpenAI
Feeds: ChatGPT live browsing
Type: User-triggered AI fetcher
robots.txt token: ChatGPT-User

ChatGPT-User user agent

Requests from ChatGPT-User carry a User-Agent header like this (example — vendors change version numbers, so match on the token ChatGPT-User, not the whole string):

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot

To see how often it visits, filter your access log with grep -i "ChatGPT-User" access.log. A User-Agent can be faked, so treat it as a claim, not proof.

Official documentation: platform.openai.com. Vendors change their crawlers, so check it for the current details.

Should you allow or block ChatGPT-User?

Blocking stops the assistant reading your page when a person asks it to. That person then gets no answer from your site, so most sites that want AI visibility leave it allowed.

Heads up: OpenAI notes that because these requests are initiated by a user, robots.txt rules may not apply to them. To control access reliably, use server-level rules rather than robots.txt alone.

robots.txt rules for ChatGPT-User

Put these in the robots.txt at the root of your domain.

Block ChatGPT-User

User-agent: ChatGPT-User
Disallow: /

Explicitly allow ChatGPT-User

User-agent: ChatGPT-User
Allow: /

Block only part of your site

User-agent: ChatGPT-User
Disallow: /private/
Allow: /

robots.txt states your policy; it does not enforce it. A CDN or firewall can block a bot your robots.txt allows, which is why it is worth testing rather than assuming.

Check what your site does today

The free AI bot checker reads your robots.txt and tells you whether GPTBot, ClaudeBot, PerplexityBot and nine other AI crawlers are allowed, blocked or crawling by default, then runs a live GPTBot fetch to prove your content is actually served.

Check my site free →

Questions, answered

What is ChatGPT-User?+

ChatGPT-User fetches a page when a person using ChatGPT asks it to read a URL or a tool call needs live content. It is triggered by a user request, not by a crawl schedule.

How do I block ChatGPT-User in robots.txt?+

Add these two lines to the robots.txt at the root of your domain: User-agent: ChatGPT-User Disallow: / User-agent tokens are matched case-insensitively. The rule applies from the crawler's next visit; it does not delete anything already collected. OpenAI notes that because these requests are initiated by a user, robots.txt rules may not apply to them. To control access reliably, use server-level rules rather than robots.txt alone.

Will blocking ChatGPT-User affect my Google rankings?+

No. ChatGPT-User is separate from Googlebot, so blocking it does not change how Google Search indexes or ranks you. Blocking stops the assistant reading your page when a person asks it to. That person then gets no answer from your site, so most sites that want AI visibility leave it allowed.

What does the ChatGPT-User user agent look like?+

An example User-Agent header is: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot. Vendors change version numbers, so match on the token ChatGPT-User rather than the whole string. A User-Agent can be spoofed, so verify important traffic against the vendor's published details before trusting it.

Related crawlers

More on the trade-offs in the AI bot policies guide and the robots.txt guide.