Google Cloud’s API Gateway now supports streaming connections, solving a long-standing gap for real-time applications and AI APIs. This means you can put streaming endpoints — like chat completions, live dashboards, or collaborative tools — behind the same API Gateway that handles your traditional REST calls, with identical security and monitoring.
What streaming changes for your APIs
Previously, API Gateway only worked with request-response APIs where the client sends a request and waits for a complete response. Streaming endpoints — where data flows continuously in one or both directions — had to bypass the gateway entirely. This forced teams to maintain two parallel paths:
- Governed traffic (with authentication, rate limiting, and logging) for traditional APIs
- Ungoverned streaming endpoints with no centralized control
Now you can route both types through API Gateway. The gateway handles streaming protocols just like regular HTTP, applying the same policies you’ve already defined. Your API specification doesn’t need special annotations or changes — the same OpenAPI document describes both streaming and non-streaming operations.
Supported streaming patterns
API Gateway supports four common streaming approaches:
| Protocol | Use case | Example |
|---|---|---|
| Server-Sent Events (SSE) | Push updates from server to client | AI model token streaming |
| HTTP response streaming | Progressive response delivery | Large file downloads |
| WebSockets | Full-duplex communication | Chat applications |
| gRPC streaming | Efficient binary streams | Microservice communication |
Tip
Server-Sent Events (SSE) is particularly important for AI applications — it’s the standard way models stream tokens to clients, used by most OpenAI-compatible APIs.
How to enable streaming
During the public preview, you enable streaming when creating a new gateway with a single flag:
gcloud api-gateway gateways create GATEWAY_ID \
--api=API_ID \
--api-config=CONFIG_ID \
--location=REGION \
--enable-streaming
Existing gateways can’t be converted — you’ll need to create a new one with streaming enabled. Google plans to make streaming the default behavior when the feature reaches general availability.
Predictable hostnames
Streaming isn’t the only improvement. Gateways now get deterministic hostnames following this pattern:
GATEWAY_ID-PROJECT_NUMBER.REGION.gateway.dev
This replaces the previous random hash in the URL, which caused problems when:
- Pre-configuring clients before the gateway exists
- Rebuilding identical environments
- Documenting endpoints for partners
Now you can generate the exact URL in advance and know it won’t change unless you modify the gateway ID, project, or region.
Why this matters for AI applications
Streaming isn’t optional for modern AI applications:
- Users expect to see responses appear incrementally, not wait for completion
- Agents need to report intermediate steps during multi-turn conversations
- Tool-calling protocols often rely on long-lived connections
- Without streaming, you’re forced to buffer entire responses before sending
Important
Latency metrics work differently for streaming connections. The gateway reports the entire stream duration, not time-to-first-byte, which will change how your dashboards look.
Model routing and AI gateways
API Gateway’s streaming support integrates with its model routing feature, letting you expose a single OpenAI-compatible endpoint that dispatches to different models (Gemini, Claude, or OpenAI) based on the request. This combines with Google Cloud’s broader AI gateway capabilities:
| Gateway type | Best for | Key features |
|---|---|---|
| API Gateway | Lightweight API management | Streaming, model routing |
| Apigee | Full lifecycle API management | Advanced policies, monetization |
| Agent Gateway | Securing agent connections | Tool access control, MCP support |
What's still coming
During the public preview, a few features aren’t available yet but will arrive before general availability:
- Custom domains via Serverless NEGs
- Cloud console UI for streaming configuration
- Terraform support
Getting started
If you already use API Gateway with a streaming-capable backend (like an SSE-emitting AI model), the quickest test is to:
- Create a new gateway with
--enable-streaming - Point it at your existing backend
- Verify events arrive incrementally through the gateway
No changes to your API spec or backend code are required.
Frequently asked questions
Can I convert an existing gateway to support streaming?
No — you’ll need to create a new gateway with the streaming flag enabled. Existing gateways continue working as before.
Do I need to modify my OpenAPI spec for streaming endpoints?
No. The same API definition works for both streaming and non-streaming operations. The gateway detects streaming based on the actual traffic.
How does this compare to Apigee for AI APIs?
API Gateway is lighter-weight and focused on routing traffic to models. Apigee adds full API lifecycle management, monetization, and advanced policies but requires more configuration.
This update removes a major limitation for anyone building real-time applications or AI APIs on Google Cloud. By unifying streaming and traditional APIs under one gateway, it simplifies infrastructure while maintaining governance — exactly what growing teams need as they scale their AI deployments.
Want to try all of this hands-on? Start with the free Claude Code from Zero course.
Source
Based on Google Cloud’s announcement, “Streaming comes to API Gateway: real-time and AI-ready APIs, without changing your spec”. Written for people learning to build with these tools.