Guides
  1. forwarding

Forwarding AI crawler traffic

The Full Agentic Stack relies on you sending AI-crawler requests to your
vanity host. You keep them off your canonical pages (no scraping of
ad-supported HTML, no skew in analytics) and we serve a clean, policy-aware
copy.

The rule, in one sentence

When the request's User-Agent matches the AI-bot regex, 302-redirect to
https://oasy.<your-domain> — with no path component.

The path you redirect from is preserved in the Referer header; we
reconstruct the canonical article URL on our side and look up the right row.

The regex

Copy from the MCP page → ForwardTrafficCard, or
use this:

(ChatGPT-User|PerplexityBot|Perplexity-User|GPTBot|OAI-SearchBot|anthropic-ai|Claude-Web|ClaudeBot|Claude-User|Google-Extended|Gemini|Applebot-Extended|meta-externalagent|meta-externalfetcher|CCBot|cohere-ai|YouBot|Diffbot|Timpibot|Amazonbot|Bytespider)

The destination

https://oasy.<your-domain> — exactly the vanity host you CNAMEd during
verification. No path — Vercel WAF and most others treat the redirect
target as literal text.

Per-platform setup

We have copy-paste configs for the popular options. Pick yours:

Once it's in place, click Check forwarding on the indigo card on the
MCP page. We fetch your homepage server-side
with a GPTBot/1.2 UA and confirm the 3xx points at oasy.<your-domain>.

What gets logged

Every redirected hit becomes an McpCall row in your database with tool
_oasy:get_article or _oasy:index. They appear in the same Recent Calls
table on /mcp as native MCP traffic — same dashboards, same per-agent
breakdown.

What about PAYWALL articles on this path?

The WAF redirect carries no OAuth identity (rotating egress IPs, no DCR
flow), so per-end-user metering doesn't apply. Instead PAYWALL articles
served on this path use the agent payment flow —
HTTP 402 + Stripe MPP. Agents with wallets (Stripe SPT) pay per article;
agents without get a clean error.