AI gateway platforms differ mainly in three places: where routing rules run, what happens after a provider fails, and who owns the model bill. Portkey, LiteLLM, Cloudflare AI Gateway, OpenRouter, Helicone, Kong AI Gateway, and Vercel AI Gateway can all place one endpoint in front of multiple models, but they do not offer the same control model. Some sit in front of provider contracts you already hold; others sell managed model access and choose an upstream on your behalf. That split affects failover behavior, spend limits, data handling, and how easily the application can move later.
For a self-hosted gateway with detailed budgets and routing, LiteLLM is the most direct shortlist. Portkey suits teams that want a standalone managed control plane with routing, observability, and governance. Cloudflare AI Gateway fits applications that benefit from edge-managed traffic rules, while OpenRouter is stronger when broad model access and provider-level failover matter more than owning the gateway runtime. Helicone joins gateway traffic with request-level observability, Kong extends an existing API management estate, and Vercel AI Gateway is the lowest-friction option for applications already built around Vercel and the AI SDK.
Verification note: Product and pricing details were checked on July 31, 2026. Model availability, plan limits, regional access, and billing rules can change.
Table of Contents
AI Gateway Comparison by Control and Billing Model
The fastest way to narrow the field is to compare control ownership, not model count. A platform that uses your provider keys behaves differently from one that resells inference through credits. The first preserves provider contracts and negotiated rates; the second can simplify procurement and failover but places another billing layer in the request path.
| Platform | Control Model | Routing Depth | Failover Scope | Cost Control | Deployment | Natural Fit |
|---|---|---|---|---|---|---|
| Portkey | Independent AI control plane | Rules, load balancing, retries, circuit breaking, canaries | Configured providers, keys, and models | Usage tracking, caching, virtual-key limits, plan-dependent budgets | Managed, hybrid, private, or self-hosted options | Teams needing gateway operations and governance together |
| LiteLLM | Open-source proxy using your infrastructure and keys | Cost, latency, usage, weighted, and custom routing | Deployments, regions, providers, and model groups | Budgets and rate limits by key, user, team, provider, model, or tag | Self-hosted; enterprise support available | Platform teams that want direct configuration control |
| Cloudflare AI Gateway | Managed edge control plane | Visual or JSON flows with conditions, splits, limits, and model nodes | Retries, model fallback, and route branches | Rate limits, budget branches, caching, and unified logs | Cloudflare-managed | Edge applications and teams already using Cloudflare |
| OpenRouter | Hosted model marketplace and routing network | Provider order, price ceilings, throughput and latency preferences, model routers | Automatic provider failover; model fallback is separately configured | Central credits, provider pricing visibility, usage limits | Hosted | Broad model access without managing many provider accounts |
| Helicone | Gateway plus observability platform | Provider selection, cheapest-route preference, custom fallback chains | Provider and billing-path fallback | Cost dashboards, alerts, request/token/cost rate limits, caching | Managed or self-hosted | Teams that want request debugging beside routing |
| Kong AI Gateway | AI traffic layer built on Kong Gateway | Priority, lowest latency, lowest usage, semantic and standard load balancing | Retries and fallback across upstream targets | Token quotas, usage analytics, caching, showback and chargeback options | Konnect, hybrid, Kubernetes, or self-managed | Organizations already standardizing APIs through Kong |
| Vercel AI Gateway | Managed inference gateway tied closely to Vercel | Availability, cost, latency, and application-defined model routing | Same-model failover across providers, plus configured model routes | Per-key budgets, unified spend reporting, provider-rate billing | Vercel-managed | Vercel applications using AI SDK or compatible endpoints |
Gateway, Router, and Model Marketplace Are Different Product Classes
An AI gateway is a traffic and policy layer between an application and one or more model endpoints. It can normalize authentication, apply limits, record usage, retry failed calls, and choose an upstream. A model router adds selection logic that decides which model or provider should handle a request. A hosted marketplace also supplies access and billing, so the application may not need direct commercial relationships with every upstream provider.
These products should not be confused with end-user interfaces. A team that wants staff to chat with several models in a browser is comparing multi-model chat apps, not gateway infrastructure. A user deciding between ChatGPT, Claude, Gemini, and similar services has a separate choice among general-purpose AI assistants. Gateways serve applications, agents, and internal developer platforms rather than replacing those user-facing tools.
The Billing Boundary Changes the Architecture
BYOK gateways usually keep provider invoices and negotiated commitments with the customer. Managed routing networks can consolidate billing and route around provider failures without exposing every upstream key to the application. Neither model is automatically better. The better fit depends on procurement, data policy, committed cloud spend, and whether the team wants to operate the data plane.
Routing Controls That Affect Production Behavior
A long provider list does not prove that a gateway can route production traffic safely. The useful comparison starts with the request contract, the failure conditions, and the cost data used by the router.
Request Compatibility
- OpenAI-compatible chat and responses endpoints
- Streaming behavior and tool-call event formats
- Structured output and JSON schema support
- Embeddings, image, audio, reranking, and batch endpoints
- Provider-specific parameters that survive normalization
Policy and Failure Controls
- Timeouts, retries, cooldowns, and circuit breaking
- Provider-level versus model-level fallback
- Regional and data-retention restrictions
- Per-user, team, key, model, or project budgets
- Routing based on price, latency, load, metadata, or prompt content
Cost-aware routing also needs accurate price metadata. Public token prices may not match a customer’s Azure agreement, cloud commitment, private endpoint rate, cached-token discount, or internal chargeback rule. A gateway that supports custom pricing can produce better showback reports than one that assumes public list prices. The same concern applies to new models: stale pricing data can make a “lowest cost” route more expensive than the dashboard suggests.
Platform Profiles by Operating Model
Portkey: Managed Control Plane With Deep Routing Policies
Portkey combines an AI gateway with observability, prompt management, guardrails, and governance. Its gateway supports automatic retries, circuit breakers, load balancing, canary traffic, request timeouts, and budget limits. [Product documentation]
The platform is a good fit when routing rules need to live outside application code and the same team also wants request logs, policy controls, and prompt operations. Portkey offers a free developer tier, a listed Production plan, and custom enterprise arrangements; recorded-log allowances, retention, private deployment, and granular budget controls vary by plan. [Official pricing]
Watch for: budget enforcement is not identical across every plan, and model-cost controls depend on supported or custom pricing records. Teams with negotiated provider rates should configure pricing adjustments before treating dashboard totals as finance-grade showback.
LiteLLM: Self-Hosted Routing and Budget Control
LiteLLM is an open-source gateway and proxy that exposes an OpenAI-compatible interface across many model providers. Its router handles load balancing, retries, cooldowns, timeouts, and fallbacks across deployments and providers. [Routing documentation]
Cost controls are unusually granular for a self-hosted option. Budgets and rate limits can be assigned to virtual keys, users, teams, providers, models, and tags, while spend tracking can be grouped across organizational units. [Budget documentation]
The open-source gateway is free to self-host, while enterprise features and support are quoted according to deployment scale. [Official pricing] LiteLLM makes the most sense when a platform team is comfortable operating the proxy, database, cache, secrets, telemetry, upgrades, and availability layer itself.
Cloudflare AI Gateway: Visual Routing at the Edge
Cloudflare AI Gateway provides analytics, logging, caching, rate limiting, retries, and model fallback through Cloudflare’s managed network. Its Dynamic Routing feature can build flows in a visual editor or JSON, with conditions, traffic splits, rate or budget limits, and fallback branches. [Dynamic routing]
A current integration detail matters for new deployments: Cloudflare has deprecated its older Universal Endpoint and directs new integrations toward the OpenAI-compatible endpoint plus Dynamic Routing for retries, fallbacks, and conditional flows. [Migration notice]
Core gateway features are available on all plans without a separate AI Gateway charge, although inference, Workers features, log export, and other connected services can create their own costs. [Official pricing] This option is especially practical when Cloudflare already handles the application edge, authentication, or Workers deployment.
OpenRouter: Managed Model Access With Provider Failover
OpenRouter is a hosted model marketplace and routing network rather than a self-hosted policy proxy. One API provides access to hundreds of models and multiple upstream providers, with provider selection and pricing exposed through a common interface. [Product documentation]
The failover distinction is important: provider failover for the selected model is automatic, while switching to a different model requires an explicit model fallback configuration. [Failover documentation] This can preserve model identity during an upstream outage, but it does not guarantee that every provider implements every parameter or data policy in exactly the same way.
OpenRouter passes through underlying model prices and charges a fee when credits are purchased; enterprise and bring-your-own-key arrangements have separate rules. [Official pricing] It is a natural fit for teams that value one commercial access layer and broad model choice more than operating their own gateway data plane.
Helicone: Gateway Routing Joined to Request Observability
Helicone combines an OpenAI-compatible gateway with request logs, sessions, traces, cost tracking, caching, prompt management, and user-level properties. Its provider routing can select among providers offering the requested model, prefer lower-cost routes, and move to another provider when rate limits, timeouts, or server errors occur. [Provider routing]
Helicone also supports rate policies measured by requests, tokens, cost, or custom properties. [Rate-limit documentation] This makes it useful when debugging and unit economics are part of the gateway decision, not a separate observability purchase.
Hosted plans and self-hosting are available, while gateway token billing can use managed credits or provider keys. The pricing page should be checked against expected request volume, seat count, and retention needs rather than judging the gateway only by token markup. [Official pricing]
Kong AI Gateway: AI Traffic Inside an Existing API Platform
Kong AI Gateway extends Kong Gateway with AI-specific proxying, governance, quotas, analytics, caching, and routing. AI Proxy Advanced supports priority, lowest-latency, lowest-usage, round-robin, consistent-hashing, and semantic load-balancing patterns, together with retry and fallback behavior. [Load-balancing documentation]
Kong is most persuasive when the organization already uses Kong for API authentication, policy enforcement, Kubernetes ingress, hybrid gateways, or developer access. The AI layer can then share operational patterns with the wider API estate instead of becoming a separate proxy maintained by another team.
Pricing is more layered than a simple per-seat plan. Konnect control planes, request volume, gateway mode, AI plugins, and the number of proxied models can affect the total, while fully self-hosted enterprise deployments use custom pricing. [Official pricing]
Vercel AI Gateway: Managed Routing for Vercel Applications
Vercel AI Gateway provides one model endpoint, unified billing, observability, automatic retries, and failover across providers. It can optimize for availability, cost, or latency, and it can keep the same model available by moving a request to another provider when the first provider degrades. [Product documentation]
Per-key budgets and unified spend reporting are useful for agents and user-facing applications where a loop or traffic spike must stop at a known threshold. Vercel charges provider list rates for tokens, supports credits and bring-your-own-key flows, and meters some higher-tier capabilities separately. [Official pricing]
The strongest reason to choose Vercel is operational fit: applications already using Vercel deployments, project identity, and AI SDK can add the gateway with less infrastructure work. Teams that require a self-hosted data plane or a provider-neutral platform outside Vercel should compare LiteLLM, Portkey, or Kong first.
Failover Is a Policy, Not a Checkbox
A gateway can report “fallback support” while protecting only one failure domain. Production planning should separate provider failover, model failover, regional failover, billing-path failover, and application-level recovery. The safest route depends on what failed and whether the replacement can honor the original request contract.
| Failure Event | Safer Response | Hidden Risk | Control to Verify |
|---|---|---|---|
| One provider endpoint returns 429 or 5xx | Retry another provider serving the same model | Parameter, region, or data-retention differences between providers | Provider allowlist, retry codes, timeout, and routing order |
| The selected model is unavailable everywhere | Move to a tested model fallback | Different tool behavior, output style, context limit, or schema support | Explicit fallback model chain and compatibility tests |
| A prompt exceeds the next model’s context window | Reject, trim, summarize, or route to a larger-context model | Silent truncation can change the answer or remove instructions | Token preflight and context-aware routing |
| Managed credits are exhausted | Use approved provider keys or return a controlled billing error | Unexpected provider invoice or a route outside procurement policy | Billing-path order and hard budget ceiling |
| A region or data policy becomes unavailable | Route only to approved regional endpoints | Cross-region failover may violate internal data rules | Region tags, provider restrictions, and no-retention requirements |
| A streaming request fails after partial output | Return a controlled error or restart with idempotency protection | Duplicate charges, repeated tool actions, or conflicting partial text | Streaming retry policy and application-side deduplication |
Model substitution should be tested as an application change, even when the gateway performs it automatically. A fallback model may accept the same JSON body yet differ in tool choice, structured output adherence, refusal behavior, context handling, or maximum completion length. Routing tests should cover representative prompts and failure responses rather than checking only whether the HTTP request returns 200.
Cost Control Depends on the Billing Path
Gateway cost control usually comes from four mechanisms: hard budgets, rate limits, cheaper routing, and caching. They solve different problems. A hard budget stops spend after a threshold. A rate limit controls request velocity. A router changes where traffic goes. Caching avoids an upstream call when reuse is acceptable.
Questions to Ask Before Trusting the Cost Dashboard
- Whose price is recorded? Public list price, a managed marketplace rate, or the organization’s negotiated contract?
- What resets the budget? Calendar month, rolling period, fixed duration, or no automatic reset?
- What happens at the limit? Reject the request, route to a cheaper model, use another key, or continue without logging?
- Which dimensions can be limited? User, team, project, API key, model, provider, region, or custom metadata?
- Are cache reads billed and attributed correctly? Provider cache discounts and gateway response caching are separate mechanisms.
- How quickly are new model prices updated? A new model with missing pricing can bypass a cost-based rule or appear as zero spend.
Cost-first model routing should also have a quality floor. Sending every request to the cheapest model can reduce spend while raising retries, human review, or failed tool calls. A more reliable design defines eligible models for each task, then optimizes price or latency within that approved set.
Selection Paths for Common AI Architectures
Direct Provider APIs Can Still Be the Better Choice
Calling a provider directly can remain sensible for a low-volume application using one model, one region, and one billing account with no uptime requirement beyond the provider’s service. A gateway becomes easier to justify when the application needs two or more providers, centralized keys, team budgets, model experiments, regional policy, shared observability, or a tested outage path.
Practical selection rule: first decide who should own the data plane and provider contracts. Then test one real workload against a provider outage, a rate-limit response, a budget threshold, and a model substitution. The platform that handles those four events with the least application-specific code is usually the better operational fit.