← AI resource directoryai-discovery-and-registry

ai-discovery-and-registry · prepared

OpenAI search crawler access and discovery controls

Official-source lifecycle facts for controlling OpenAI search crawling without confusing crawler access, ChatGPT search eligibility, navigational links, training use, citation, or user-triggered retrieval.

version 1.0.0freshness currentobserved 2026-08-26T15:20:00Z

resource.discovery.openai-search-crawler-controls
sha256:5d45949de922576f9f8742488b1fb2550fee32ba31ff1dde3373ecf5251efdd3

Open canonical machine JSON →

Evidence-backed facts

  1. OAI-SearchBot and GPTBot robots.txt controls are independent: a publisher can allow the search crawler while disallowing the training crawler. informative
  2. OAI-SearchBot is the automatic crawler used to surface websites in ChatGPT search results; publishers should allow it in robots.txt and permit OpenAI's published search-bot IP ranges when they want search eligibility. informative
  3. Allowing OAI-SearchBot is an access and eligibility signal, not evidence that a URL has been indexed, cited in an answer, or used by a person or Agent; those outcomes require separate observations. informative
  4. ChatGPT-User represents certain user-triggered page visits, is not an automatic web crawler, and is not the control used to determine whether content may appear in ChatGPT search. informative