Specoria
Menu
TR Free test
← All insights
Infrastructure7 min read

Bot or buyer? How shopping agents identify themselves to your store

For years, stores built walls against bots. Now some bots arrive on the customer's behalf. Signed agents, the card networks' agent protocols, and why bot protection is now a sales channel setting.

For a long time, e-commerce teams lumped all bots into one bucket: crawlers scraping prices, bots tying up inventory, scripts testing stolen cards. The answer to all of them was the same: block them, slow them down, send them to a verification challenge.

Today, some of the bots arriving at your store are different. An agent that a customer has told to "find an espresso machine that ships in two days with 30-day returns, and add it to the cart" is there on that customer's behalf. A firewall that stops this agent at the door hasn't stopped a scraper. It has stopped a sale.

In this article, we look at how agents have started to identify themselves, where the card networks come in, and how stores should rethink their bot rules.

More than half of traffic is no longer human

According to a Cloudflare report published in July 2026, more than half of internet traffic now comes from non-human sources, for the first time. Retail looks similar: citing Adobe data, Visa reported that AI-driven traffic to US retail sites grew by more than 4,700% in a year.

Not all of this traffic comes for the same reason. Some crawlers collect content to train models, some read pages for search answers, and some are agents carrying out a single user's task. For a store, these three are worth very different things.

The default is now "block"

Since July 1, 2025, Cloudflare has asked every newly added domain at signup whether it wants to allow AI crawlers; by default, the site owner is in control. The company says it handles traffic for about a fifth of the web, so this isn't just one settings change. It's a new starting point for a large part of the web.

AI companies have also started splitting their crawlers by purpose. According to OpenAI's documentation:

  • GPTBot collects content for model training.
  • OAI-SearchBot crawls pages for ChatGPT's search answers. Sites that block it don't appear in ChatGPT search answers.
  • ChatGPT-User opens pages at a user's request and doesn't crawl automatically.

These settings are independent of each other. You can opt out of training and stay open to search and to requests made on a user's behalf. A single "block all AI bots" switch shuts out all three.

Identity takes a signature, not just a user agent

The classic way for a bot to identify itself is to put its name in the request (the user-agent header). The problem: anyone can spoof that header. IP address ranges, meanwhile, can change or be shared with other services. Cloudflare considers both methods unreliable for the same reasons.

The new approach is a cryptographic signature. Web Bot Auth, which Cloudflare announced in May 2025, is a thin layer built on the HTTP Message Signatures standard (RFC 9421). The agent signs each request with its own key, and a header added to the request points to the domain where the public key is published. By verifying the signature, a site or CDN can confirm that the request really comes from the agent it claims to be. The approach is also being discussed as a draft at the IETF.

In August 2025, Cloudflare went a step further and defined a class of signed agents. The distinction: verified bots work on behalf of a single company, while signed agents are directed by individual users. The first group included ChatGPT agent, Block's Goose, Browserbase and Anchor Browser.

Enter the card networks: agent identity meets payments

Who the agent is matters. So does who it's shopping for, and with what authority. The card networks have moved to answer that question:

  • Mastercard Agent Pay, April 2025. Mastercard announced "agentic tokens," built on its existing card tokenization, so that agents can make payments.
  • Visa Trusted Agent Protocol, October 2025. Developed with Cloudflare, this open protocol lets a trusted agent pass three kinds of information to a merchant: that the agent intends to view or buy a specific product, whether the customer has an account with the merchant, and, optionally, payment information.
  • Web Bot Auth becomes common ground. According to Cloudflare's announcement the same day, Mastercard is adding Web Bot Auth to Agent Pay, and American Express will use it in its own agentic commerce program.

Put this together with AP2, which we covered in our first article, and the direction is clear: the agent identifies itself with a signature, proves the user's authorization with a signed mandate, and pays with a token the card network recognizes. The store's job is to recognize that identity and act on it.

What it means for stores: bot rules are a sales setting

  1. Decide by purpose. Make separate decisions for training crawlers, search crawlers and agents acting on a user's behalf. Commercially, the last group is usually the most valuable.
  2. Allow by identity, not by name. If your CDN or security provider recognizes verified bots and signed agents, write your rules around that verification. An allow rule that only checks the user-agent name opens the door to every bot that spoofs it.
  3. Don't send verified agents to a challenge. A human verification check (CAPTCHA) or a strict rate limit on product, stock and shipping pages ends the agent's task right there. Set separate rate limits for verified agents.
  4. Read your logs. Check your server and CDN logs to see which agents request which pages and what responses they get. Agent requests to product pages that get a 403 or 429 are lost sales you can't see.
  5. Ask your providers. Find out what your e-commerce platform, payment provider and CDN are planning for Web Bot Auth, signed agents and the card networks' agent programs. Most of these decisions are made in their dashboards, not yours.

The bottom line

For a long time, bot management was purely a security setting. In agentic commerce, the same setting also opens or closes a sales channel. The stores that win in the years ahead won't be the ones that block all bots or let them all in. They'll be the ones that recognize an agent that proves who it is, and treat it like a customer.

When Specoria measures how your store looks to shopping agents with real buyer tasks, it also records access problems: it shows which agent couldn't reach which page, and how that affects your selection rate. Start with a free readiness report.

Sources