# unformentoo.org # # Everything here is public and meant to be read, quoted and checked. There # is nothing to hide behind a disallow, and the register exists precisely so # that people can verify the claims rather than take them on trust. # # The site was reachable on Google and almost nowhere else, which is what # happens when only one engine has been told you exist. The sitemap is # declared below, and publishes push URLs over IndexNow, which Bing, Yandex, # Seznam and Naver share between them. User-agent: * Allow: / Disallow: # Crawlers that read the site to answer questions are welcome too. This is a # reference source: being quoted accurately by a model, with the register one # click away, is the point rather than a cost. llms.txt is written for them. User-agent: GPTBot Allow: / Disallow: User-agent: ClaudeBot Allow: / Disallow: User-agent: PerplexityBot Allow: / Disallow: User-agent: Google-Extended Allow: / Disallow: User-agent: CCBot Allow: / Disallow: User-agent: Applebot-Extended Allow: / Disallow: # The crawlers above are the ones that read pages to train on. These are the # ones that read a page because someone just asked a question, which is the # traffic this site is actually written for. A named group replaces the # wildcard for that crawler, so each one has to say Allow in full. User-agent: OAI-SearchBot Allow: / Disallow: User-agent: ChatGPT-User Allow: / Disallow: User-agent: Google-CloudVertexBot Allow: / Disallow: User-agent: Applebot Allow: / Disallow: User-agent: Amazonbot Allow: / Disallow: User-agent: meta-externalagent Allow: / Disallow: User-agent: Bingbot Allow: / Disallow: Sitemap: https://unformentoo.org/sitemap.xml