If your app calls OpenAI, Anthropic, or Workers AI directly from application code, you have no shared choke point to cache repeat calls, cap runaway spend, or see which feature is burning through your token budget. Cloudflare’s AI Gateway is built to sit in that gap. It is a proxy that every model call passes through, and it adds caching, rate limiting, logging, and a single consolidated invoice across providers without you standing up your own infrastructure.
Cloudflare describes the product plainly: “AI Gateway sits between your application and the AI APIs that your application makes requests to (like OpenAI) – so that we can cache responses, limit and retry requests, and provide analytics to help you monitor and track usage,” according to Cloudflare’s official announcement. The pitch is low friction too: “AI Gateway’s core features available today are offered for free, and all it takes is a Cloudflare account and one line of code to get started,” per Cloudflare’s pricing documentation.
This tutorial walks through building a working setup from scratch: creating the gateway, wiring it into a Cloudflare Worker, routing calls to both Workers AI and third-party providers, turning on caching and rate limits, adding retry logic the gateway doesn’t provide out of the box, tagging spend by feature, locking down access, and shipping it. By the end you’ll have a deployed Worker and a gateway dashboard showing real request data, plus a punch list of the mistakes that trip up most first-time setups.
Most teams reach for a tool like this after the fact, once a support chatbot or an internal search feature has already run up a surprise invoice. The pattern is familiar. Someone wires a feature straight to a provider’s SDK, ships it, and it works fine at low volume. Then usage climbs, the same handful of questions get asked over and over, a retry loop fires during a provider outage and multiplies the bill, and nobody has a clean way to answer “which feature is costing us this much.” A gateway sitting in front of every call fixes that by giving you one place to cache, throttle, and tag traffic instead of scattering that logic across every service that happens to call a model.
What Cloudflare AI Gateway Actually Does
An AI gateway is not a model, and it doesn’t generate anything itself. It’s a routing and control layer that sits between your code and whichever inference API you’re calling. Instead of your app hitting api.openai.com directly, it hits Cloudflare’s endpoint, which forwards the request, applies whatever rules you’ve configured, and returns the provider’s response unchanged (unless it served the answer from cache).
That middle position is what makes caching, rate limiting, and unified logging possible without touching your app’s business logic. Cloudflare’s general-availability announcement frames caching as the headline cost lever: “Caching: Enable custom caching rules and use Cloudflare’s cache for repeat requests instead of hitting the original model provider API, helping you save on cost and latency,” according to Cloudflare’s GA post. That single feature is often the difference between a support chatbot that stays cheap and one that reprocesses the same five questions a thousand times a day.
The gateway supports two access patterns. You can call it through a Workers AI binding inside a Cloudflare Worker, which gives you the shortest path if you’re already on the Workers platform. Or you can hit its universal HTTP endpoint from any client, in any language, without touching Workers at all. Both paths land in the same gateway, the same cache, and the same analytics dashboard, so you can mix them as your architecture grows.
It’s also worth being clear about what the gateway is not. It’s not a model host, it doesn’t run inference itself unless you’re specifically calling Workers AI. It’s not a fine-tuning platform, and it doesn’t rewrite or moderate your prompts. Think of it closer to a reverse proxy purpose-built for LLM traffic, similar in spirit to how a CDN sits in front of a web server, except tuned for token-based billing, provider-specific request formats, and cache keys built from prompts instead of URLs.
Prerequisites: Accounts, Tools, and Versions You Need
You don’t need an enterprise Cloudflare plan for any of this. The core AI Gateway features run on the free tier. Here’s what to have ready before Step 1:
- A Cloudflare account (the free plan is enough to complete this whole tutorial)
- Node.js 20 or newer and npm 10 or newer installed locally
- The Wrangler CLI, run through
npxso you always pull the current release instead of a pinned, possibly stale, global install - An OpenAI or Anthropic API key if you want to route third-party calls (not required if you’re only testing against Workers AI’s own models)
- Basic comfort with JavaScript or TypeScript and a terminal
- About 45 minutes: 20 for the base gateway and Worker, the rest for caching, rate limits, and retry logic
One clarification worth making early: your Cloudflare account ID and your gateway ID are two different strings you’ll need throughout this tutorial. The account ID sits in the right-hand sidebar of almost every dashboard page. The gateway ID is the name you pick when you create the gateway in Step 1. Keep both handy in a scratch file.
It’s also worth deciding up front whether you’re routing only Workers AI models or mixing in third-party providers, since that changes how many secrets you’re managing. A Workers AI-only setup needs nothing beyond the binding in your wrangler.jsonc. Add OpenAI or Anthropic to the mix and you’re now responsible for storing and rotating those provider keys as Worker secrets, on top of whatever gateway-level token you generate once authentication is turned on in Step 11. Neither path is hard, but picking one before you start saves you from re-plumbing secrets halfway through.
Step 1-3: Create the Gateway and Scaffold Your Worker
Step 1. Log into the Cloudflare dashboard, open the AI section, and select AI Gateway. Click Create Gateway and name it something you’ll recognize in logs later, such as tutorial-gateway. Gateway names are permanent identifiers used in URLs, so avoid anything you’ll want to rename in six months.
Step 2. Copy your Account ID from the dashboard sidebar and save it next to your new gateway ID. You’ll combine both into the universal endpoint pattern later: https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}.
Step 3. Scaffold a new Worker project on your machine:
npx wrangler init ai-gateway-tutorial
cd ai-gateway-tutorial
# choose "Hello World Worker" when prompted, TypeScript or JavaScript, your callWrangler will generate a working project with a default wrangler.jsonc file and a minimal fetch handler. Leave the sample code in place for now. You’ll replace the body in Step 5.
Step 4-5: Bind Workers AI and Send Your First Request
Step 4. Add an AI binding to your wrangler.jsonc so the Worker can call Workers AI without manually managing an API key:
{
"ai": {
"binding": "AI"
}
}Regenerate types so your editor understands the new env.AI object:
npx wrangler typesStep 5. Replace the Worker’s fetch handler with a call that routes through your gateway by name:
export default {
async fetch(request, env) {
const result = await env.AI.run(
'@cf/meta/llama-3.2-3b-instruct',
{
messages: [
{ role: 'user', content: 'Explain what an API is in one short sentence' },
],
max_tokens: 80,
},
{
gateway: {
id: 'tutorial-gateway',
metadata: { feature: 'api-explanation' },
},
},
)
return Response.json({
answer: result.response,
logId: env.AI.aiGatewayLogId,
})
},
}Run it locally and hit it with curl:
npx wrangler dev
# in a second terminal
curl http://localhost:8787You should get back something like this:
{
"answer": "An API is a defined way for software programs to communicate with each other.",
"logId": "01K..."
}Switch back to the AI Gateway dashboard and refresh the Analytics tab for tutorial-gateway. That single request should already show up with its metadata tag, confirming the binding is routed correctly before you add any third-party providers.
Step 6: Route OpenAI and Anthropic Calls Through the Universal Endpoint
Workers AI bindings are convenient, but most teams also need to route calls to OpenAI or Anthropic through the same gateway so all their model spend lands on one dashboard. Cloudflare’s universal endpoint accepts calls from any HTTP client, not just Workers, using the pattern https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}.
Store your provider key and, if you’ve enabled gateway authentication (covered in Step 11), your gateway token as Worker secrets rather than hardcoding them:
npx wrangler secret put OPENAI_API_KEY
npx wrangler secret put CF_AIG_TOKENThen call the provider through the gateway instead of calling it directly. Note the two separate headers: Authorization authenticates you to OpenAI, while cf-aig-authorization authenticates you to the gateway itself:
const response = await fetch(
'https://gateway.ai.cloudflare.com/v1/ACCOUNT_ID/GATEWAY_ID/openai/chat/completions',
{
method: 'POST',
headers: {
Authorization: `Bearer ${env.OPENAI_API_KEY}`,
'cf-aig-authorization': `Bearer ${env.CF_AIG_TOKEN}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages: [
{ role: 'user', content: 'Explain DNS in one sentence' },
],
}),
},
)The two-header pattern trips people up because it looks redundant at first. It isn’t: drop the cf-aig-authorization header once gateway authentication is on, and the request never reaches OpenAI, it gets rejected by Cloudflare first. Both requests, the Workers AI binding call from Step 5 and this direct OpenAI call, now show up under the same gateway ID, giving you one place to see combined spend across providers.
The provider path in the URL, openai in the example above, tells the gateway which upstream API to forward to and which request or response shape to expect, since OpenAI, Anthropic, and other providers don’t share an identical request format. Swap that path segment and adjust the body to match the target provider’s API, and the same gateway, same caching rules, and same rate limits apply without any additional setup on the Cloudflare side. That’s the main advantage of centralizing on one gateway instead of writing a separate wrapper per provider: the operational rules (cache, limits, logging) stay consistent even as the set of providers behind them changes.
Step 7: Turn On Caching to Cut Repeat-Call Costs
Caching is where most of the cost savings live. When a cache hit occurs, the gateway serves the stored response straight from Cloudflare’s network instead of forwarding the call to the provider, cutting both the bill and the round-trip time. Cloudflare’s own product page puts it directly: “Serve requests directly from Cloudflare’s cache instead of the original model provider for faster requests and cost savings,” according to Cloudflare’s AI Gateway documentation, which does not specify an exact latency-reduction percentage for cache hits.
Add a cacheTtl (in seconds) to the same gateway object you used in Step 5:
gateway: {
id: 'tutorial-gateway',
cacheTtl: 300,
metadata: { feature: 'api-explanation' },
}For any call that must never be cached, such as anything involving a timestamp, a one-time code, or content that should vary run to run, override it explicitly:
gateway: {
id: 'tutorial-gateway',
skipCache: true,
}A cache hit in AI Gateway is an exact match on the request payload, not a semantic match. Two prompts that mean the same thing but differ by a trailing space or a reordered field count as different requests. Normalize your request bodies before sending them if you want cache hit rates that reflect the app’s real repeat-question rate rather than string formatting noise.
| Cache and gateway limit | Value |
|---|---|
| Maximum cacheable request size | 25 MB per request |
| Maximum cache TTL | 1 month |
| Custom metadata entries per request | 5 |
| Gateways per account (Free plan) | 10 |
| Gateways per account (Workers Paid) | 20 |
Figures above are from Cloudflare’s AI Gateway limits reference. Requests over the 25 MB ceiling simply aren’t cached, they’re forwarded every time, so if you’re sending large document context windows through the gateway, don’t expect a cache discount on those calls.
Step 8: Set Rate Limits and Handle the 429 from Unified Billing
Rate limiting protects you from a runaway loop or an abusive client hammering your endpoint and running up a provider bill in minutes. AI Gateway supports both sliding and fixed window limiting, configurable per gateway. If you manage gateways as infrastructure, the Terraform provider exposes it directly:
resource "cloudflare_ai_gateway" "tutorial" {
account_id = var.cloudflare_account_id
id = "tutorial-gateway"
collect_logs = true
authentication = true
rate_limiting_interval = 60
rate_limiting_limit = 100
}Setting either field to 0 disables rate limiting for that gateway. There’s a separate, non-configurable limit that only applies to calls billed through Cloudflare’s Unified Billing (meaning you’re using Cloudflare-managed provider credentials rather than your own key). Exceed it and the gateway responds with a 429, per Cloudflare’s limits documentation:
{
"success": false,
"errors": [
{ "code": 3040, "message": "Rate limit exceeded for Unified Billing requests" }
]
}That specific limit does not apply to bring-your-own-key (BYOK) traffic, the pattern used in Step 6. If a production path needs guaranteed high throughput, route it through your own provider key rather than Unified Billing credits, and rely on the gateway-level rate_limiting_limit field to protect it instead.
Choosing sensible values for rate_limiting_interval and rate_limiting_limit depends on what you’re protecting against. A limit meant to stop a runaway client-side loop, someone’s browser tab stuck retrying a failed request, can be tight, since legitimate traffic rarely bursts that hard. A limit meant to cap worst-case spend during a genuine traffic spike needs to be set closer to your actual peak load plus a margin, or you’ll end up throttling real users during your busiest hour. Start conservative, watch the Analytics tab for a week of real traffic, then widen the limit based on what normal usage actually looks like rather than a guess made before launch.
Step 9-10: Add Retry Logic and Attach Metadata for Cost Tracking
Step 9. AI Gateway caches, limits, logs, and consolidates billing, but it does not automatically fail over to a backup provider when one goes down. That logic has to live in your Worker. Here’s a minimal pattern that tries OpenAI first and falls back to Anthropic, keeping both calls under the same gateway ID so they land in one log stream:
async function callWithFallback(env, prompt) {
const base = `https://gateway.ai.cloudflare.com/v1/${env.CF_ACCOUNT_ID}/${env.CF_GATEWAY_ID}`
try {
const primary = await fetch(`${base}/openai/chat/completions`, {
method: 'POST',
headers: {
Authorization: `Bearer ${env.OPENAI_API_KEY}`,
'cf-aig-authorization': `Bearer ${env.CF_AIG_TOKEN}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: prompt }],
}),
})
if (!primary.ok) throw new Error(`primary provider status ${primary.status}`)
return await primary.json()
} catch (err) {
const backup = await fetch(`${base}/anthropic/v1/messages`, {
method: 'POST',
headers: {
'x-api-key': env.ANTHROPIC_API_KEY,
'anthropic-version': '2023-06-01',
'cf-aig-authorization': `Bearer ${env.CF_AIG_TOKEN}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'claude-sonnet-4.5',
max_tokens: 200,
messages: [{ role: 'user', content: prompt }],
}),
})
return await backup.json()
}
}Step 10. Tag each call with metadata so spend can be attributed to a feature, customer tier, or environment instead of showing up as one undifferentiated total. Remember the 5-entries-per-request ceiling from the limits table above, so pick tags deliberately:
gateway: {
id: 'tutorial-gateway',
metadata: {
feature: 'support-chatbot',
plan: 'pro',
environment: 'production',
},
}Cloudflare’s September 8, 2026 changelog update made this more useful: monthly usage invoices now show a single consolidated cost per model with a standardized name, for example a line item reading anthropic/claude-haiku-4.5: $0.16, instead of separate input and output rows per call. Combined with your own metadata tags, that gives you a cost-per-feature view without building a separate billing pipeline.
Step 11-12: Lock Down Authentication and Ship to Production
Step 11. Anyone who guesses your gateway ID can send requests through it unless you turn on gateway authentication. In the dashboard, open your gateway’s settings and enable Authentication, then generate a token. Every call must now include the cf-aig-authorization: Bearer <token> header shown in Steps 6 and 9, or the gateway rejects it before it ever reaches OpenAI or Anthropic. Rotate that token the same way you’d rotate a provider key, on a schedule, not just after an incident.
Step 12. Deploy and confirm everything end to end:
npx wrangler deployOpen the gateway’s Analytics tab and watch requests, cache hit rate, and per-metadata spend populate as real traffic hits the deployed Worker. If you configured Unified Billing, check the invoice view for the consolidated per-model line items described above.
Here’s the complete Worker combining everything above, the binding, caching, metadata, and a fallback path, as a single reference file:
export default {
async fetch(request, env) {
const gateway = {
id: 'tutorial-gateway',
cacheTtl: 300,
metadata: { feature: 'support-chatbot', environment: 'production' },
}
try {
const result = await env.AI.run(
'@cf/meta/llama-3.2-3b-instruct',
{ messages: [{ role: 'user', content: 'Explain what an API is in one short sentence' }], max_tokens: 80 },
{ gateway },
)
return Response.json({ answer: result.response, logId: env.AI.aiGatewayLogId, source: 'workers-ai' })
} catch (err) {
const backupUrl = `https://gateway.ai.cloudflare.com/v1/${env.CF_ACCOUNT_ID}/${gateway.id}/openai/chat/completions`
const backup = await fetch(backupUrl, {
method: 'POST',
headers: {
Authorization: `Bearer ${env.OPENAI_API_KEY}`,
'cf-aig-authorization': `Bearer ${env.CF_AIG_TOKEN}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'Explain what an API is in one short sentence' }],
}),
})
const data = await backup.json()
return Response.json({ answer: data, source: 'openai-fallback' })
}
},
}Verify the Setup: What a Working Gateway Looks Like
Before moving on to cost tuning, confirm the whole chain actually behaves the way you configured it. Run the same request twice in a row against the caching example from Step 7. The first call should take the normal round trip to the provider or to Workers AI. The second, identical call should come back noticeably faster and should be flagged as a cache hit in the gateway’s request log:
curl http://localhost:8787
curl http://localhost:8787
# second call returns the same "answer" text, and the matching
# log entry in the AI Gateway dashboard shows cached: trueNext, confirm the rate limit is actually enforced instead of just configured. Fire more requests than your rate_limiting_limit allows inside the configured interval and confirm you get the 429 shown in Step 8, not a silently dropped request or a provider-side error. If you see a provider error instead of a Cloudflare 429, the limit isn’t wired to the path you’re testing, usually because the traffic is going through BYOK rather than Unified Billing, and BYOK isn’t covered by that specific limit.
Finally, pull up the gateway’s Analytics tab and confirm the metadata tags from Step 10 show up as filterable fields, not just as opaque JSON buried in a log line. If you can filter the dashboard down to just feature: support-chatbot and see a request count and rough cost for that slice alone, the cost-attribution half of this setup is working as intended.
Do this verification pass before you consider the setup done, not after something breaks in production. A gateway that’s silently misconfigured, caching nothing, rate limiting nothing, tagging nothing, still returns correct answers to every request, so there’s no functional signal telling you it isn’t earning its keep. The only way to catch that is to deliberately check each control the way described above, ideally as part of your deploy checklist rather than a one-time exercise.
What Cloudflare AI Gateway Costs: Free Tier vs Paid
The core gateway (routing, caching, rate limiting, logging) costs nothing on its own, per Cloudflare’s pricing reference. What scales with usage is Workers platform consumption, log volume, and, if you opt into it, the Unified Billing convenience fee. None of these numbers include the actual token cost you pay OpenAI, Anthropic, or Workers AI for generating responses, since that’s billed separately by whichever provider you’re routing to.
| Item | Free / Workers Free | Workers Paid |
|---|---|---|
| Core gateway features (cache, limits, logs) | $0 | $0 |
| AI Gateway requests included | 10 million/month, then $0.05 per extra 1M | 10 million/month, then $0.05 per extra 1M |
| Logs stored | 100,000 total across all gateways | 10,000,000 per gateway |
| Gateways per account | 10 | 20 |
| Base plan cost | $0/month | $5/month |
| Unified Billing fee on purchased credits | 5% | 5% |
There’s also a time-limited promotion worth flagging if you’re reading this close to publication: Cloudflare’s AI Gateway changelog lists discounted Unified Billing pricing for GPT-5.6 Sol, $2.50 per 1M input tokens versus a standard $5, $15 per 1M output tokens versus $30, and $0.25 per 1M cache-read tokens versus $0.50. The promotion runs through September 18, 2026, after which pricing reverts to the standard rate. If you’re benchmarking cost for a new project this month, run the numbers again after that date.
The practical question most teams actually need answered is when the 5% Unified Billing fee is worth paying versus managing your own provider keys. Unified Billing buys you one invoice, one payment method, and no separate provider account to manage, which matters most when you’re moving fast or running several small provider accounts you’d otherwise have to reconcile by hand. At meaningful volume, that 5% starts to add up to real money, and BYOK removes it entirely at the cost of you managing each provider relationship, key rotation, and invoice separately. There’s no universal answer here. It depends on your monthly spend and how much you value one combined bill over the discount of skipping the fee.
Common Pitfalls When Setting Up Cloudflare AI Gateway
Most of the mistakes teams make with AI Gateway aren’t exotic. They come from carrying over assumptions from a different kind of proxy, a CDN, a database cache, or a hand-rolled retry wrapper, and expecting this one to behave the same way. The list below covers the ones that show up repeatedly once a gateway moves from a weekend experiment into something a whole team depends on.
- Expecting automatic failover. The gateway caches, limits, and logs, but it won’t switch providers for you when one is down. Build that in your Worker, as in Step 9.
- Mixing up BYOK and Unified Billing rate limits. The documented rate limit that returns a 429 applies specifically to Unified Billing traffic. Bring-your-own-key calls aren’t covered by it.
- Sending oversized payloads and expecting a cache discount. Anything over 25 MB per request is never cached, it’s forwarded to the provider every single time.
- Provisioning a gateway per feature. That burns through the 10 (Free) or 20 (Paid) per-account ceiling fast. Use metadata tags inside fewer gateways instead.
- Forgetting the second header. Once gateway authentication is on, a valid provider key alone isn’t enough. Missing
cf-aig-authorizationgets the request rejected before it reaches the provider. - Treating cache TTL like CDN freshness. A cache hit returns the identical stored completion, not a refreshed one. Anything that must vary per call needs
skipCache: true.
Troubleshooting Cloudflare AI Gateway: 9 Errors and Fixes
Even a correctly configured gateway throws errors once real traffic and real edge cases show up. The table below covers the errors that come up most often while building and running the setup from this tutorial, along with the specific cause and fix for each one, rather than generic “check your configuration” advice.
| Symptom | Likely cause | Fix |
|---|---|---|
| HTTP 429 on Unified Billing calls | Exceeded the Unified Billing rate limit | Raise rate_limiting_limit, or move high-volume traffic to BYOK |
| HTTP 403 with a valid provider key | Gateway authentication is on, header missing | Add cf-aig-authorization: Bearer <token> to every call |
| Cache hit rate stuck near zero | Request bodies vary slightly between calls | Normalize payloads (whitespace, field order) before sending |
| “AI is not defined” in Worker code | Binding name mismatch between wrangler.jsonc and code | Confirm the binding name matches your env.AI reference exactly |
| 404 on the universal endpoint URL | Typo in account ID or gateway ID | Re-copy both IDs directly from the dashboard |
| Logs stop appearing after heavy testing | Free plan hit the 100,000 log cap | Upgrade to Workers Paid, or prune test traffic before it counts against the cap |
| Unexpected 5% line item on the invoice | Unified Billing credit purchase fee | Switch high-volume paths to BYOK if the fee matters at your scale |
| Gateway field ignored by wrangler dev | Stale cached Wrangler version via npx | Run npx wrangler@latest dev to force the current release |
| Terraform apply fails on billing mode | workers_ai_billing_mode only supports “postpaid” today | Remove the field or set it explicitly to “postpaid” |
Advanced Tips for Running Cloudflare AI Gateway in Production
Once the base setup is running, a few habits keep it manageable as traffic grows. Separate gateways by environment, staging and production, rather than sharing one, so a load test never pollutes production analytics or eats into your log cap. Lean on metadata tags plus the standardized per-model invoice lines Cloudflare shipped on September 8, 2026 to build a cost-per-feature view without a separate billing pipeline.
For spend alerts, pair a Worker Cron Trigger with a scheduled check against your gateway’s analytics, since the gateway itself doesn’t hard-cap spend or page you when a threshold is crossed mid-month, that logic has to live on your side. For latency-sensitive endpoints, resist the urge to disable caching gateway-wide just because a handful of calls need to always be fresh. Set a short default cacheTtl and override individual calls with skipCache instead, so the majority of repeat traffic still benefits.
If you’re approaching the 20-gateway ceiling on a paid plan because every team spun up its own, consolidate by folding team or feature identity into metadata tags inside a smaller number of shared gateways. You lose nothing on the analytics side, since metadata is filterable in the dashboard, and you get back headroom for genuinely separate use cases like a staging environment or a customer-isolated deployment.
It also pays to revisit your fallback logic periodically rather than writing it once and forgetting it. Provider model names change, pricing tiers shift, and a fallback that pointed at a sensible backup model six months ago can quietly become the expensive path if that model’s pricing moved while your primary provider’s didn’t. Treat the fallback branch in Step 9 as part of your regular cost review, not just as disaster-recovery code that only runs during an outage.
Cloudflare AI Gateway vs LiteLLM, Helicone, and Portkey
Cloudflare isn’t the only option for putting a control layer in front of LLM calls, and the right pick depends more on where you already run infrastructure than on any single feature. Cloudflare’s own docs don’t publish head-to-head benchmarks against these tools, so treat the table below as a category comparison rather than a scored bake-off.
| Tool | Deployment model | Best fit |
|---|---|---|
| Cloudflare AI Gateway | Edge proxy, no server to run yourself | Teams already on Workers who want caching, limits, and one invoice with zero added infrastructure |
| LiteLLM | Self-hosted proxy and SDK | Teams that want an open-source router they fully control across many providers |
| Helicone | Hosted observability layer | Teams prioritizing prompt-level analytics and evaluation over gateway plumbing |
| Portkey | Hosted AI gateway product | Teams wanting a dedicated gateway with enterprise routing and guardrail features |
If your stack is already built on Cloudflare Workers, the AI Gateway is close to a zero-cost default, since you get caching and rate limiting without adding a new service to operate. If you need deep multi-provider routing logic that lives outside any single cloud vendor, a self-hosted option like LiteLLM keeps you portable at the cost of running and patching it yourself.
None of these tools are mutually exclusive in practice. It’s common to see a team run Cloudflare AI Gateway at the edge for caching and rate limiting, while also piping logs into a dedicated observability tool for prompt-level debugging that a general-purpose gateway isn’t built to do. Pick the gateway layer based on where your traffic already terminates, and add a specialized observability tool on top only once you’ve outgrown what the built-in analytics dashboard shows you.
Frequently Asked Questions
Is Cloudflare AI Gateway free to use?
The core feature set, caching, rate limiting, and logging, is free with just a Cloudflare account, per Cloudflare’s pricing page. You still pay for Workers usage beyond the free allowances and for the underlying provider or Workers AI tokens you consume.
Does Cloudflare AI Gateway store the full text of my prompts?
Logging is a core feature, and the documented caps are count-based rather than time-based: 100,000 logs total on the Free plan and 10 million per gateway on Workers Paid. If prompts include sensitive data, handle redaction at the application layer rather than assuming a specific retention window.
Can I use AI Gateway without writing a Cloudflare Worker?
Yes. The universal endpoint pattern shown in Step 6 accepts calls from any HTTP client in any language. The Workers AI binding used in Step 5 is a convenience for teams already on the Workers platform, not a requirement.
What happens if I exceed the Unified Billing rate limit?
The gateway returns an HTTP 429, per Cloudflare’s limits reference. That specific limit doesn’t apply to bring-your-own-key traffic, so high-throughput production paths often route through BYOK instead.
How many gateways can I create per account?
Ten on the Free plan and 20 on Workers Paid, according to Cloudflare’s documented limits. Most teams do better consolidating by metadata tag than provisioning a new gateway per feature.
Does turning on caching change my API responses?
A cache hit returns the exact completion the provider generated the first time, not a refreshed one. Anything that must differ per call, timestamps, one-time codes, or randomized text, should set skipCache: true for that request.
Is there a promotional pricing window worth knowing about right now?
Yes. Cloudflare’s AI Gateway changelog lists discounted Unified Billing pricing for GPT-5.6 Sol running through September 18, 2026, after which it reverts to standard rates. Re-check current pricing if you’re benchmarking cost after that date.
How does AI Gateway compare to building my own proxy Worker?
It replaces the caching, rate limiting, logging, and billing-consolidation code you’d otherwise write and maintain yourself, in exchange for less granular control than a fully custom proxy would give you.
