Tools · LLM gateways & spend
LLM gateways and AI spend controls
An LLM gateway, also sold as an AI gateway, is a proxy between a company's applications and the model providers. The products here keep provider keys in one place, route requests across models, and enforce rate limits, token quotas or budget caps per team, user or key, with logs of token use and spend. A team needs one once several applications or agents call model providers directly, because each then holds its own key and runs up its own bill, and nobody can say afterward who spent what or put a ceiling on it.
LLM gateways & spend 11
-
Agent Router (vendor site, opens in a new tab)
generally availableAgentic AI Foundation
Open-source control plane, formerly Envoy AI Gateway, that turns declared providers, credentials, routing, tools and token limits into Envoy configuration, so those controls sit in the traffic path instead of in agent code.
-
AI gateway in Azure API Management (vendor site, opens in a new tab)
generally availableAPI Management policies for LLM APIs, AI agents and MCP servers, including token rate limits and quotas per consumer, semantic caching, and monitoring of token use.
-
Apigee AI gateway (vendor site, opens in a new tab)
generally availableGoogle Cloud on the vendor map
Uses Apigee as a gateway in front of LLM APIs across clouds, with policy-based model access, token limit enforcement for cost control, token usage monitoring, and LLM auditing and logging.
-
Bifrost (vendor site, opens in a new tab)
generally availableOpen-source AI gateway for LLM, MCP and agent traffic with budgets per team or virtual key, audit logs, SSO-based access control and output guardrails.
-
Cloudflare AI Gateway (vendor site, opens in a new tab)
generally availableRoutes requests across model providers with caching, fallback routing, rate limiting and guardrails that keep sensitive data from being sent, and records logs, token counts and spend.
-
Concentrate LLM Gateway (vendor site, opens in a new tab)
generally availableConcentrate AI on the vendor map
Sends model calls through one API that handles provider keys, routing and fallbacks, and adds spend tracking, token allocation, role-based access, SSO, data redaction and audit logs.
-
F5 AI Gateway (vendor site, opens in a new tab)
generally availableCombines a model gateway, an MCP gateway and guardrails under one policy model, with token metering, budget enforcement, routing and caching. It keeps a central registry of agents and tools and an audit trail of agent-to-tool access.
-
LiteLLM (vendor site, opens in a new tab)
generally availableOpen-source, self-hosted gateway that puts more than 140 model providers behind one OpenAI-compatible key, with virtual keys, spend tracking and budget caps, routing and guardrails.
-
OpenRouter Guardrails (vendor site, opens in a new tab)
generally availablePer-workspace rules for OpenRouter organizations, assigned to members or API keys, that set spending limits, restrict which models and providers can be used, and enforce data privacy policies.
-
Prisma AIRS AI Gateway (vendor site, opens in a new tab)
generally availablePalo Alto Networks on the vendor map
Gateway to more than 3,000 LLMs, MCP servers and tools that tracks usage, users, tokens and cost, applies rate limits, spending rules and access controls for approved models and tools, and inspects agent traffic for attacks.
-
TrueFoundry AI Gateway (vendor site, opens in a new tab)
generally availableRoutes traffic to more than 1,600 models with rate limits per user, service or endpoint, cost or token quotas, role-based access and guardrails. Deploys in a VPC, on-premises or air-gapped.
Recent launches
- Hard spend caps for agent workloads in Cloud Billing (Google) · announced
- Rippling AI Gateway · announced
Announced in the last 90 days. Each launch on New tools links the vendor's own announcement.
The briefing
New AI agent governance tools, by email
The briefing's New Tools section covers the latest governance launches, each linked to the vendor's announcement, next to the news that decides which ones matter. At least once a week, five-minute read, free.
You confirm by email. Unsubscribe in one click.
Questions to ask vendors
Neutral questions for a request for proposal (RFP) or a security questionnaire. No product here is named as the answer.
-
Can spend be attributed per team, per application, and per agent, rather than only per provider account?
-
Are budget limits hard caps that stop traffic, or alerts that arrive after the money is gone?
-
Where do provider keys live, how are they rotated, and who can read them?
-
What is the security track record and patch turnaround? Ask specifically how fast known exploited vulnerabilities get fixed in your deployment model.
In September 2026 the Cybersecurity and Infrastructure Security Agency (CISA) added a LiteLLM gateway flaw, CVE-2026-59822, to its Known Exploited Vulnerabilities catalog.
-
How much latency does the proxy add, and what happens to traffic when the gateway itself is down?
-
If we leave, does our code keep working? Gateways that mimic a standard API shape are cheaper to exit.
More questions, grouped by topic: Questions to ask AI agent security vendors.
Related incidents
- Attackers stole a METR API key and burned about $600,000 in AI credits
- Microsoft documented live attacks on exposed LiteLLM gateways
- An unauthenticated Bifrost AI gateway flaw ran commands from a single HTTP request
- GitLab patched a 9.9-severity AI Gateway flaw that let Duo Agent Platform users run commands
Every record in the AI agent incident database links the sources that reported it.
Missing a product, or found an entry that is out of date? Write to [email protected]. A submission is read, never paid for, and does not guarantee a listing. The glossary defines the terms used here.