You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**container_port** | **int** | Internal Dynamo Frontend listener port for the non-root runtime. The public endpoint remains on HTTPS; port 8788 is unavailable when backend_protocol is GRPC. | [optional][default to 8000]
21
-
**initial_replicas** | **int** | | [optional][default to 1]
22
-
**gpus_per_worker** | **int** | | [optional][default to 1]
18
+
**min_replicas** | **int** | | [optional][default to 1]
19
+
**max_replicas** | **int** | | [optional][default to 1]
**session_affinity** | **bool** | Enable best-effort sticky routing via the `X-Session-Id` request header. Requests carrying the same header value land on the same pod, improving KV cache reuse for agentic workloads. Requests without the header are routed at random. Affinity is NOT durable: scaling, rollouts, restarts, or readiness-probe transitions will remap sessions to different pods. Do not use for irreplaceable in-pod state. | [optional][default to False]
container_port: Optional[Annotated[int, Field(le=65535, strict=True, ge=1024)]] =Field(default=8000, description="Internal Dynamo Frontend listener port for the non-root runtime. The public endpoint remains on HTTPS; port 8788 is unavailable when backend_protocol is GRPC.")
error_messages.append(f"Error! Input type `{type(v)}` is not `GetDynamoDeploymentResponse`")
82
-
else:
83
-
returnv
84
-
85
85
iferror_messages:
86
86
# no match
87
87
raiseValueError("No match found when setting the actual_instance in DeploymentResponse with anyOf schemas: GetCServeV3DeploymentResponse, GetDynamoDeploymentResponse, GetInferenceV3DeploymentResponse. Details: "+", ".join(error_messages))
0 commit comments