AI Training Opt-Out in robots.txt: What Small Businesses Should Do

On October 30, 2026, something unusual happened in the AI industry: Google, OpenAI, Microsoft, and Anthropic all agreed on one thing. They announced support for a shared robots.txt signal that lets website owners opt out of AI training in a single line of code.

The signal is called __usage_ai-training, and it comes from an IETF draft written by Kevin Kelley of Anthropic. For small business owners, this is a significant moment. Until now, blocking AI companies from using your content to train their models required writing separate rules for each company’s crawler. Now, one rule can cover them all.

But should your small business actually use it? Let’s break down what the signal does, how to add it, and whether opting out makes sense for your website.

What Is the AI Training Opt-Out Signal?

Every website has a file called robots.txt that tells search engines and web crawlers what they can and cannot access. AI companies have their own crawlers — Google uses Google-Extended, OpenAI uses GPTBot, and Microsoft has its own bots — and until now, you had to write a separate rule for each one.

The new __usage_ai-training token changes that. Instead of blocking crawlers one at a time, you add a single group to your robots.txt file that tells every supporting company: “Do not use my content for AI model training.”

Here is what the robots.txt entry looks like:

User-agent: __usage_ai-training
Disallow: /

This tells all participating AI companies that your entire site should not be used for training purposes. You can also be more specific, blocking only certain directories while allowing others.

It is important to understand what this signal does not do. It does not block crawling. It does not remove your site from search results. It does not affect your rankings in Google Search or Bing. It only addresses how your content is used — specifically, whether it ends up in the training data for future AI models.

Which Companies Support It?

Four major AI companies have announced support, each with a different timeline:

• Anthropic: Proposed the signal through the IETF draft. Support is being built and will complement its existing crawler controls.
• OpenAI: Plans to implement the signal “in the coming months.” Its existing GPTBot rules still apply in the meantime.
• Microsoft: Will honor the signal from February 1, 2027. It replaces the older NoArchive and NoCache tags Microsoft has used since 2023.
• Google: Plans to support it in early 2027. Until then, Google-Extended remains the way to block Google’s AI training crawler.

One notable absence is Meta. Andy Stone, Meta’s communications head, voiced general support for a common signal on X, but Meta has not committed to honoring it.

How It Differs from Blocking AI Search and AI Overviews

This is where things get nuanced, and where many small business owners get confused.

The __usage_ai-training signal is only about training. It tells AI companies not to feed your content into the datasets they use to build and improve their models. It says nothing about whether AI-powered search features — like Google’s AI Overviews or Microsoft Copilot — can reference your pages when answering a user’s question in real time.

These are two different things:

• Training: Your content becomes part of the AI’s permanent knowledge. The model “learns” from it. This is what the new signal controls.
• Grounding or retrieval: An AI search tool reads your page live to answer a specific question and links back to you. This is controlled by separate settings in Google Search Console and Bing Webmaster Tools.

For most small businesses, the grounding and retrieval side is actually beneficial. When someone asks an AI assistant about your industry and it pulls from your page, you get visibility and often a link. Blocking that is a different decision from blocking training.

Should Your Small Business Opt Out of AI Training?

The answer depends on what your website does for your business.

If You Sell Products or Services

For most small businesses that use their website to attract customers — a restaurant, a consulting firm, an agency, a local service provider — opting out of AI training is probably not worth it.

Here is why: when AI models are trained on content about your industry, they learn about businesses like yours. If someone asks an AI chatbot “What does a managed IT service include?” or “How do AI chatbots help small businesses?”, the model’s answer is shaped by the content it was trained on. If your well-written blog posts were part of that training data, the model is more likely to understand and reference your category accurately.

Opting out means future AI models may know less about what you do. They will still answer questions about your industry, but the answers will come from everyone else’s content instead of yours.

If Your Content Is the Product

For content-driven businesses — online courses, premium newsletters, review sites, publishers — the calculation is different. If your revenue depends on people visiting your pages and your writing style is a key differentiator, you may have a legitimate reason to opt out.

When an AI model trains on your content, it can generate text that covers the same topics in a similar way, potentially reducing the need for someone to visit your site. In this case, the opt-out signal is a reasonable protective measure.

A Practical Middle Ground

You do not have to make an all-or-nothing choice. The robots.txt signal supports path-level rules. You could block training on your premium guides while allowing it on your general marketing pages:

User-agent: __usage_ai-training
Disallow: /premium-guides/
Allow: /blog/
Allow: /services/

This lets AI models learn about your business and industry from your blog while keeping your most valuable content out of training datasets.

How to Add the Signal to Your WordPress Site

If you decide to opt out, here is how to add the signal. Most WordPress SEO plugins, like Yoast SEO or Rank Math, let you edit your robots.txt file from the dashboard.

Go to your SEO plugin settings, find the robots.txt editor, and add the lines shown above. If your plugin does not have a robots.txt editor, you can create or edit the file manually in your site’s root directory through your hosting file manager.

After adding the rules, verify by visiting yourdomain.com/robots.txt in your browser to make sure the new lines appear correctly.

Keep in mind that the signal will only take effect once each company actually implements support. Google and Microsoft have announced early 2027 as their target. In the meantime, if you want to block specific crawlers immediately, you can add separate rules for Google-Extended and GPTBot alongside the new token.

What to Do Before You Decide

Before changing anything, measure your baseline. Open Google Search Console and check your performance in the generative AI report, if available. Note your current impressions, clicks, and rankings. Do the same in Bing Webmaster Tools.

This gives you a point of comparison. If you add the opt-out signal and see no change in traffic, you will know the trade-off was minimal. If you see a drop, you can remove the signal and reassess.

The Bottom Line

The shared __usage_ai-training signal is a step forward for website owners who want more control over how AI companies use their content. For the first time, one robots.txt rule can communicate your preference to Google, OpenAI, Microsoft, and Anthropic at once.

For most small businesses that sell products or services, the smart move is to keep AI training on and focus on making your content so useful that AI models reference your expertise. For content-driven businesses, the opt-out signal offers a practical, standardized way to protect your most valuable work.

Either way, the fact that the biggest names in AI agreed on a common signal is a win for every website owner. You now have a clearer, simpler way to make your choice known.

Sources

• OpenAI — Helping Websites Express AI Training Preferences
• The Next Web — Google, Microsoft, OpenAI and Anthropic Back One AI Training Opt-Out
• Martin Cid Magazine — Robots.txt AI Training Opt-Out

Try it free: take the 3-minute AI readiness quiz, see what missed calls cost you with the AI savings calculators, try the in-browser RAG chatbot demo or listen to the AI receptionist demo.

Frequently asked questions

Waseem Hassan

Founder & CEO, NexaIntellect AI

AI/ML engineer building LLM agents, RAG systems and automation for growing businesses.

About the author