Skip to content

cld2labs/Granite-3.3-8b-Instruct#95

Open
arpannookala-12 wants to merge 2 commits into
opea-project:mainfrom
cld2labs:cld2labs/Granite-3.3-8b-Instruct
Open

cld2labs/Granite-3.3-8b-Instruct#95
arpannookala-12 wants to merge 2 commits into
opea-project:mainfrom
cld2labs:cld2labs/Granite-3.3-8b-Instruct

Conversation

@arpannookala-12
Copy link
Copy Markdown
Contributor

@arpannookala-12 arpannookala-12 commented Apr 21, 2026

Summary

  • Adds model card for Granite-3.3-8b-Instruct (IBM) under third_party/Dell/model-deployment/Granite-3.3-8b-Instruct/
  • Adds Helm-based deployment guide for deploying Granite-3.3-8b-Instruct via vLLM on CPU (Xeon) with Keycloak OIDC and APISIX ingress

@arpannookala-12
feat: add Granite-3.3-8b-Instruct model card and deployment guide for…
f8a6359 
… Dell EI

Signed-off-by: arpannookala-12 <ganesh.arpan.nookala@cloud2labs.com>
@alexsin368 alexsin368 self-requested a review April 29, 2026 04:17
@alexsin368
Copy link
Copy Markdown
Collaborator

alexsin368 commented May 18, 2026 •
edited
Loading

Model deployed successfully. Getting this error when testing inference:

<html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx</center>
</body>
</html>

vLLM pod shows no errors. The ingress-nginx shows an upstream error:

2026/05/18 22:29:08 [error] 119156#119156: *4289737 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 172.17.23.1, server: api.example.com, request: "POST /granite-3.3-8b-instruct-vllmcpu/v1/completions HTTP/2.0", upstream: "http://10.233.104.80:9080/granite-3.3-8b-instruct-vllmcpu/v1/completions", host: "api.example.com"
172.17.23.1 - - [18/May/2026:22:29:08 +0000] "POST /granite-3.3-8b-instruct-vllmcpu/v1/completions HTTP/2.0" 504 160 "-" "curl/7.81.0" 1236 60.001 [auth-apisix-auth-apisix-gateway-80] [] 10.233.104.80:9080 0 60.000 504 b474a6afa4833fdbcbc732d064a67169

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexsin368 alexsin368 Awaiting requested review from alexsin368

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@arpannookala-12 @alexsin368