Skip to content

feat: scaffold load balancing worker as a template repo#5

Open
TimPietruskyRunPod wants to merge 1 commit into
mainfrom
template-repo-scaffolding
Open

feat: scaffold load balancing worker as a template repo#5
TimPietruskyRunPod wants to merge 1 commit into
mainfrom
template-repo-scaffolding

Conversation

@TimPietruskyRunPod

Copy link
Copy Markdown

Makes worker-load-balancing a proper template repo, mirroring the structure of worker-template (the queue-balanced one) while staying a load balancing worker.

Context: requested by Owen — turn this into a template repo similar to worker-template.

What changed

File What
.runpod/hub.json (new) Runpod Hub listing config. Sets "endpointType": "LB" so the Hub deploys this as a load balancer, not a queue worker. Exposes PORT/PORT_HEALTH as configurable env vars. GPU defaults (ADA_24) to match worker-template.
.runpod/tests.json (new) Sample input for Hub validation.
README.md Rewritten in template style: banner → description → Hub badge → "Use this template" → Customizing → Testing Locally → Deploying → Further Info. Keeps the LB-specific port config + curl examples.
public/worker-load-balancing_banner.png (new) Placeholder banner — swap for an official design asset when available.
LICENSE (new) MIT, matching worker-template.
app.py Added template-guidance comments (where to load your model, /ping status codes, adding routes). Logic unchanged.
.gitignore Expanded to the standard Python set.
requirements.txt Bumped fastapi/uvicorn/pydantic off the 2023 pins (pydantic==1.10.7) that crash on modern Python.

Verified

Booted the server locally on the updated deps — /ping200, /generate, /stats, and / all respond correctly. Both JSON configs parse.

Follow-ups (can't be done in code)

  • Flip the Template repository toggle in repo Settings → General (enables the "Use this template" button).
  • Hub badge/listing goes live once the repo is published to the Runpod Hub.
  • Confirm the Hub tests.json runner exercises LB endpoints correctly (LB workers don't take queue job input) — the file is schema-valid and harmless either way.

mirror the worker-template structure so this repo works as a github
template and a runpod hub listing, while staying a load balancing worker:

- add .runpod/hub.json with endpointType LB and PORT/PORT_HEALTH config
- add .runpod/tests.json for hub validation
- rewrite README in template style (banner, hub badge, use-this-template)
- add generated banner under public/ and MIT LICENSE
- add template guidance comments to app.py
- expand .gitignore to the standard python set
- bump fastapi/uvicorn/pydantic off the 2023 pins that break on modern python
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant