Three kinds of crawler
Search and answer crawlers (for example OAI-SearchBot and PerplexityBot) fetch pages that engines cite in live answers. Blocking one removes your site from that engine's sources.
User-triggered fetchers (for example ChatGPT-User) fetch a page when a person asks an assistant about it.
Training crawlers (for example GPTBot) collect data for future models. Blocking them is a legitimate choice and does not remove you from live answers.
How a rule is decided
A bot obeys the group that names it, otherwise the wildcard group, otherwise nothing applies. The longest matching rule wins, and Allow wins a tie. A missing robots.txt allows everything.
robots.txt is one layer. A CDN or firewall can still refuse a crawler that robots.txt allows.
Check yours
The free AI crawler check reads a site's robots.txt and shows each crawler's status. It also checks llms.txt and what a crawler sees without running JavaScript.