Definition
robots.txt
robots.txt is a file at your site root that tells crawlers, including AI bots, which paths they may or may not access. Blocking an AI crawler here makes your content invisible to that answer engine, so review it before assuming you have a content problem.
It controls access, not meaning, that's the difference from llms.txt. Related: crawlability.
Should I block AI bots in robots.txt?
Only if you intentionally don't want to appear in AI answers. To be recommended by ChatGPT, Gemini or Perplexity, the relevant crawlers must be allowed to access your public content.