Repository navigation
Conversation
|
Hi @BwL1289. Thanks for your PR. I'm waiting for a validatedpatterns member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
CC @atherr |
|
🤖 Sat Nov 14 20:22:46 - The preview is ready at: |
|
hey @BwL1289, thanks for the PR! I don't think that google group (hybrid-cloud-patterns@googlegroups.com) is used anymore. We're actively revising the requirements for each tier so I'll make sure we update the process for requesting a technical review. For now, if you're happy with it, we can use this PR as the platform for the review of both the pattern repo and the docs. Regarding the naming: I'd just drop the I'm happy to move this repo into validatedpatterns-sandbox if you like and make you a maintainer of that repo. This is NOT required though and if you're happy with the pattern sitting in your repo that's fine. Regarding the pattern itself: I personally installed this pattern on AWS (OCP 4.22.22) and can confirm that installation worked without a hitch and argo had all resources synced and healthy. (You should be proud of this!!) Following the guidance from the README and these docs I was able to verify the inference server was healthy and functional. Overall, I think you did an awesome job. There are some things I'd like to understand better though:
|
| # The directory name, the aliases, and the GitHub links in this page set use the | ||
| # current repository name, rhvp-vllm-ray-inference-server. That name can change, | ||
| # pending two open questions to the Validated Patterns team: whether the "rhvp-" | ||
| # prefix is acceptable under the Red Hat trademark guidelines, and whether the | ||
| # repository moves to the validatedpatterns-sandbox organization. If either | ||
| # changes, rename the directory, the aliases, the image folder, and every link | ||
| # in both pages together. |
There was a problem hiding this comment.
Remove this (answered in main conversation on this PR)
| # in both pages together. | ||
| title: vLLM Ray Inference Server | ||
| date: 2026-11-08 | ||
| validated: false |
There was a problem hiding this comment.
I don't think the validated field actually exists so you can delete this
| # partners: left out on purpose. The pattern deploys software from NVIDIA (the | ||
| # GPU Operator and the CUDA base image) and from the Ray and vLLM projects, but | ||
| # no partner organization takes part in the pattern. A partners entry suggests a | ||
| # partnership, and the site prints a partner trademark notice for it. Add an | ||
| # entry only when a partner organization joins the pattern. |
There was a problem hiding this comment.
Remove the comment here. If you are happy with it, I think we can have Eugo as a partner here. If you add Eugo here, you'll automatically show up in the partners filter in the side bar of https://validatedpatterns.io/patterns/
| # partners: left out on purpose. The pattern deploys software from NVIDIA (the | |
| # GPU Operator and the CUDA base image) and from the Ray and vLLM projects, but | |
| # no partner organization takes part in the pattern. A partners entry suggests a | |
| # partnership, and the site prints a partner trademark notice for it. Add an | |
| # entry only when a partner organization joins the pattern. | |
| partners: | |
| - Eugo |
| include::modules/comm-attributes.adoc[] | ||
| :vllm-ray-pattern: vLLM Ray Inference Server pattern | ||
|
|
||
| // The pattern name can change. See the comment at the top of the front matter. |
Add an overview page and a getting-started page for the rhvp-vllm-ray-inference-server sandbox pattern, plus its architecture diagram.
…artner, apply review notes The pattern repository is now named vllm-ray-inference-server. Move both page folders and the image folder to that name, and update the aliases, the architecture image path, the GitHub links, the clone and cd commands, the secrets file name and the notebook name to match. List Eugo, the maintainer, under partners, so the overview page prints the partner trademark notice. Apply the review notes: drop the front matter comments about a possible rename and the left-out partners entry, and drop validated: false.
… speed The pattern's namespaces are now vllm-inference (the server) and vllm-notebook (the notebook). Update the oc commands, the in-cluster server URL, the notebook access text and the leftover-volume check on both pages, and the namespace labels in the architecture diagram, whose notebook build box now names the BuildConfig and ImageStream notebook. The overview page's notebook entry also lists the single-stream decode speed that the notebook measures.
… products - Background: say that the pattern does not replace or integrate with the Red Hat AI products, such as Red Hat OpenShift AI, which parts are pinned and which are not, the one-time install step and what OpenShift GitOps does not do after it, and what the notebook measures. Drop the claims that each rebuild installs the same software stack and that the pattern uses upstream open source components only. - Prerequisites: use a cluster that does not already run the Node Feature Discovery Operator or the NVIDIA GPU Operator. The pattern installs both itself and can conflict with existing installations. It has not been tested on a cluster that runs Red Hat OpenShift AI.
…e Server background
e0daf7f to
d98e693
Compare
…Server page Add a "Partner results: Eugo" section before Support: the start-up time and start-up reliability of Eugo's next vLLM image (release pending) against the upstream vLLM image, and a link to the matching section of the pattern README.
Summary
This pull request adds the vLLM Ray Inference Server pattern to the site in the sandbox tier.
The pattern deploys an OpenAI-compatible LLM server on one OpenShift cluster. One pod runs Ray Serve LLM with the vLLM engine on one NVIDIA GPU. The cluster builds the server image with a BuildConfig from hash-pinned PyPI wheels on the NVIDIA CUDA UBI 9 image. OpenShift GitOps deploys the server, the NVIDIA GPU Operator and Node Feature Discovery, Vault with the External Secrets Operator for an optional Hugging Face token, and a JupyterLab notebook that times the server.
tier: sandbox,validated: false)We also emailed hybrid-cloud-patterns@googlegroups.com on 2026-11-08 to ask for the informal technical and
architecture reviews that the sandbox tier asks for. If that list is no longer read, or if it is easier, this pull
request can carry those reviews.
Files
content/patterns/rhvp-vllm-ray-inference-server/_index.adoccontent/patterns/rhvp-vllm-ray-inference-server/getting-started.adocstatic/images/rhvp-vllm-ray-inference-server/architecture.svgSandbox tier requirements
./pattern.sh make installfrommainat1a781d1, with default settings, on a new installer-provisioned AWS cluster (OpenShift 4.22.18, us-east-1, three m5.2xlarge nodes, no cluster changes). The command's own Argo CD health check passed, with all seven Argo CD applications Synced and Healthy. The pattern added the GPU node, the cluster built both images, the server answered the OpenAI API, and the notebook ran against it with no errors. An earlier install, on another new AWS cluster, found one bug (OpenShift's builder records an emptyCMDas/bin/sh -c null), which1a781d1fixes (README, What this pattern was verified against).pattern-metadata.yamlarchitecture.svgin this pull request_index.adocNotes for reviewers
questions are open with the Validated Patterns team: whether the
rhvp-prefix is acceptable under the Red Hattrademark guidelines, and whether the repository moves to
validatedpatterns-sandbox. A comment at the top of thefront matter says which values to change together.
partnersentry. The pattern uses NVIDIA, Ray, and vLLM software, but no partner organization takes part init. The front matter explains this in a comment. Leaving the field out also keeps the partner trademark notice off
the page.
page uses an image. The SVG uses plain SVG text, with no embedded HTML.
AGENTS.mdfile in the patternrepository.
Checks
hugo, run in the container image thatmake builduses, builds the site with these files and exits 0. The twopages report no warning other than the site-wide notice that the optional
rougegem is missing.