How to recognize it

User agents, as they reach our logs:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot

  • Browser prefix: all variants start with a full Chrome-on-Mac string, so a rule that looks for “bot” at the start of the user agent misses them.

  • robots.txt marker: the second string carries it. OpenAI says robots.txt fetches may use it.

  • Versions: 1.0 and 1.4 are both in use.

  • Matching: use the token OAI-SearchBot.

  • robots.txt token: OAI-SearchBot

  • IP list (JSON): searchbot.json

  • Reverse DNS: none published by OpenAI.

What it does

OAI-SearchBot finds pages to show in ChatGPT’s search features. OpenAI says sites that opt out of it are not shown in ChatGPT search answers, though they can still appear as navigational links. It is independent of GPTBot. When a site allows both, OpenAI may use one crawl for both purposes.

What you gain by allowing it

  • Appearing, with a link, in ChatGPT search answers, and the clicks that come from them.

What you lose

By allowing it

  • Little load. On our two sites it made 179 verified requests in 8 days, most of them for robots.txt.
  • If you also allow GPTBot, OpenAI may use the same crawl for training. To keep search visibility without training, allow OAI-SearchBot and disallow GPTBot.

By blocking it

  • Your pages drop out of ChatGPT search answers, except as navigational links.
  • It does not stop ChatGPT-User, which fetches pages a user asks for and is governed separately.

How to verify it

Match the IP against searchbot.json. The one-liner on the GPTBot card works with this URL. Cloudflare flags the real bot as verified in the category “Search Engine Crawler”, not “AI Crawler”; in our data the flag matched the IP list on every request.

Gotchas and history

  • OpenAI says robots.txt changes take about 24 hours to reach its search systems.
  • It reads robots.txt for GPTBot too. GPTBot never requests robots.txt on our sites, while marked OAI-SearchBot requests arrive seconds before its crawls; see GPTBot never fetched our robots.txt.
  • On a new domain it may fetch nothing but robots.txt for a while; see OAI-SearchBot only fetched robots.txt.
  • Its name is one of 28 that a scanner on Google Cloud cycles through, so user agent counts are mostly fake (see below).

What we observed

Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).

  • 963 requests carried an OAI-SearchBot user agent:
    • 179 (19%) came from OpenAI’s list, all from Microsoft’s network (AS8075);
    • 777 of the 784 fakes came from the Google Cloud scanner;
    • Cloudflare’s verified-bot flag matched the list on all 963.
  • 132 of the 179 verified requests (74%) were for /robots.txt, spread over all four hostnames; 53 of them carried the robots.txt marker. The other 47 fetched pages.
  • 7 verified requests got a 4xx from the origin server.

About OpenAI

  • GPTBot collects content for training foundation models. Disallowing it opts the site out of training only.
  • OAI-SearchBot finds pages for search results in ChatGPT. Disallowing it removes the site from those results. OpenAI gives about 24 hours for a robots.txt change to take effect there.
  • ChatGPT-User opens pages when a user asks ChatGPT or a custom GPT to. It does not crawl on its own, and OpenAI says robots.txt rules may not apply to it.
  • OAI-AdsBot visits only pages submitted as ads, to check the landing page.

OpenAI states that each setting is independent. When a site allows both GPTBot and OAI-SearchBot, OpenAI may use one crawl for both. The common mix-up is blocking GPTBot to stay out of ChatGPT: that only stops training, while search visibility depends on OAI-SearchBot.

Sources