Update AI Routing

Endpoint

PUT /apiops/projects/{projectName}/apiProxies/{apiProxyName}/ai-routing/

Warning

This endpoint returns HTTP 400 for API proxies whose type is not AI — aiRouting can only be set on AI proxies. To create a new AI proxy in one call, use Create AI API Proxy — this endpoint only updates the routing config of a proxy that already exists.

Authentication

Requires a Personal API Access Token.

Permission: API_MANAGEMENT × MANAGE (consistent with the mcp-routing and a2a-routing endpoints).

Header

Authorization: Bearer YOUR_TOKEN

Request

Headers

Header	Value	Required
Authorization	Bearer {token}	Yes
Content-Type	application/json	Yes

Path Parameters

Parameter	Type	Required	Description
projectName	string	Yes	Project name
apiProxyName	string	Yes	AI API Proxy name

Request Body

The body is the full AiRouting object — it replaces the proxy's current AI routing configuration.

Full JSON Body Example

{
  "llmProviderRefName": "openai-prod",
  "modelId": "gpt-4o-mini",
  "clientModelPresentBehavior": "OVERWRITE",
  "clientModelAbsentBehavior": "KEEP_ABSENT",
  "streaming": false,
  "timeoutSeconds": 60,
  "connectTimeoutSeconds": 10,
  "streamTimeoutSeconds": 120,
  "retryCount": 0,
  "failoverStrategy": "SEQUENTIAL",
  "providerFailoverChain": [],
  "conditionalRoutes": [],
  "primaryPool": [],
  "enumAiRoutingAlgorithm": "ROUND_ROBIN",
  "circuitBreakerEnabled": false,
  "proxyEnabled": false,
  "mTLSSettings": null,
  "blockAnonymousRequests": false,
  "anonymousBudget": null
}

Field Reference — Primary Routing

Field	Type	Description
llmProviderRef	string	Primary LLM provider connection id. Use llmProviderRefName instead when referencing by name
llmProviderRefName	string	Name-based sidecar for llmProviderRef. Resolved to an id within the project when present
modelId	string	Primary catalog model id (for example gpt-4o-mini, deepseek-chat)
clientModelPresentBehavior	string	What to do when the client's request body already has a model field — USE_CLIENT, OVERWRITE, or ERROR. Default OVERWRITE when unset
clientModelAbsentBehavior	string	What to do when the client's request body has no model field — KEEP_ABSENT, ADD_ROUTING_MODEL, or ERROR. Default KEEP_ABSENT when unset
streaming	boolean	Enable SSE streaming responses. Default false
timeoutSeconds	integer	Per-call LLM timeout in seconds. Default 60

USE_CLIENT is not honored on Azure OpenAI

Gemini, Vertex, and Bedrock carry the effective model in the URL path, which the gateway builds itself — attribution and the model that answers always agree. An Azure OpenAI deployment endpoint pins the deployment in the endpoint the operator configured, so the request body's model field cannot change which model actually answers — it could only shift the attribution (the catalog entry that rate limits, quota, semantic cache, cost calculation, and traffic logs are booked against) to a value the client supplies. Apinizer therefore does not apply clientModelPresentBehavior: "USE_CLIENT" on an AZURE_OPENAI provider: the effective model stays the configured modelId and no catalog check is run against the client's value. The upstream request carries the configured modelId in its body, exactly as it does under OVERWRITE — the client's value only stops being a reason to reject the request. OVERWRITE, ERROR, and clientModelAbsentBehavior are unaffected.

A failover entry without its own modelId inherits the configured modelId above, not the client's model, and re-evaluates USE_CLIENT against its own provider's catalog.

Field Reference — Timeouts, Limits, and Retry

Field	Type	Description
connectTimeoutSeconds	integer	Connection establish timeout (DNS + TCP + TLS). Default 10
streamTimeoutSeconds	integer	SSE stream idle timeout. Default 120
maxResponseBytes	long	Maximum response body size; null or ≤ 0 = unlimited
maxChunks	long	Maximum SSE chunks; null or ≤ 0 = unlimited
retryCount	integer	Pre-stream connection retry count for transient errors. Default 0
retryDelayType	string	NO_DELAY, FIXED, EXPONENTIAL_BACKOFF
retryFixedDelayMs	integer	Fixed retry delay in ms. Default 500
retryInitialDelayMs	integer	Initial backoff delay in ms. Default 500
retryMaxDelayMs	integer	Maximum backoff delay in ms. Default 10000
retryBackoffMultiplier	number	Backoff multiplier. Default 2.0

Field Reference — Failover and Load Balancing

Field	Type	Description
failoverStrategy	string	SEQUENTIAL, ROUND_ROBIN, or CONDITIONAL
providerFailoverChain	array	Ordered failover entries — each with llmProviderRef/llmProviderRefName, optional modelId, credentialRef, timeoutMs, orderIndex, displayName
failoverStatusCodeList	array[integer]	HTTP status codes eligible for failover. Empty = default 5xx/408/429
failoverRetryCount	integer	Inner retry count per failover entry. Default 1
conditionalLatencyThresholdMs	integer	For CONDITIONAL strategy — proactive latency threshold in ms
primaryPool	array	N-primary load-balancing pool — each entry with llmProviderRef/llmProviderRefName, optional modelId, weight, displayName
enumAiRoutingAlgorithm	string	ROUND_ROBIN, WEIGHTED, RANDOM, LEAST_COST, PREFIX_AFFINITY, LRU
leastCostOutputRatio	number	Output-to-input cost weight for LEAST_COST. Default 0.25
affinityKeyHeader	string	Header for PREFIX_AFFINITY affinity key. Default session_id
affinityLoadFactor	number	Bounded-load factor for PREFIX_AFFINITY. Default 1.25

Field Reference — Conditional Routes

Field	Type	Description
conditionalRoutes	array	Ordered route variants — each with name, condition, optional llmProviderRef/llmProviderRefName, modelId, semanticUtterances, semanticThreshold
semanticEmbeddingProviderRef	string	Embedding provider id for semantic route matching
semanticEmbeddingModelId	string	Embedding model id. Default text-embedding-3-small

Field Reference — Circuit Breaker

Field	Type	Description
circuitBreakerEnabled	boolean	Enable circuit breaker. Default false
errorWindow	integer	Error observation window in seconds. Default 10
errorThresholdValue	integer	Error threshold. Default 1
enumErrorThresholdType	string	COUNT or PERCENTAGE
sleepWindow	integer	Open-circuit sleep window in seconds. Default 10
halfOpenEnabled	boolean	Enable half-open probe. Default false

Field Reference — Outbound Proxy, mTLS, and SSL

Field	Type	Description
proxyEnabled	boolean	Route outbound LLM calls through an HTTP proxy
proxyHost	string	Outbound proxy host
proxyPort	integer	Outbound proxy port
proxyAuthorizationNeeded	boolean	Proxy requires authentication
proxyUsername	string	Proxy username
proxyPassword	string	Proxy password (encrypted at rest; blank preserves existing on update)
mTLSSettings	object	Client mTLS settings for provider connections
disableSslValidation	boolean	Accept self-signed certificates (lab only)

Field Reference — Anonymous Traffic and Trace

Field	Type	Description
blockAnonymousRequests	boolean	Reject unauthenticated requests with HTTP 401. Default false
anonymousBudget	object	Token/cost budget override for anonymous requests
traceMode	string	AI trace mode override — METRIC, FULL, etc.
enableAiTrace	boolean	AI trace lifecycle flag
aiTraceEnvironmentName	string	Environment where AI trace was started
aiTraceCustomQueryId	string	CustomQuery filter id for AI trace
aiTraceMaxDurationSeconds	integer	Maximum AI trace duration. Default 300

Behavior

• llmProviderRefName (and name sidecars on failover entries, primary pool entries, and conditional routes) are resolved to connection ids within the project before save.
• Every environment the proxy is currently deployed to is marked redeploy required — redeploy to activate the new configuration.
• AI proxies do not use the classic routing object. Settings endpoints that target classic routing (circuit breaker, connection, mTLS, addresses, …) return HTTP 400 for AI proxies and point here instead.

Response

Success Response (200 OK)

{
  "result": {
    "success": true
  }
}

Error Responses

Status	Condition
400	Blank apiProxyName, empty body, proxy not found, or proxy is not an AI proxy
400	A USD budget covering the proxy would meter a model that has no input price in the model catalog (error key aiQuota.usdBudgetModelPriceMissing, parameters modelCount / models). Raised when the body's aiAppBudget or the routing changed; an unchanged re-save is not re-checked. Add the price to the model catalog or use a token-based limit.
401	Invalid or missing token
500	Unexpected server error

cURL Example

curl -X PUT \
  "https://demo.apinizer.com/apiops/projects/MyProject/apiProxies/my-ai-proxy/ai-routing/" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "llmProviderRefName": "openai-prod",
    "modelId": "gpt-4o-mini",
    "streaming": false
  }'

Related Documentation

• Create AI API Proxy
• Update MCP Routing
• Update A2A Routing
• API Proxies
• AI Gateway