ppocrvl-docker

RunPod Serverless endpoint for PaddleOCR-VL-0.9B served by vLLM.

Built to be benchmarked against two existing endpoints under identical conditions:

	model	stack	repo
this	PaddleOCR-VL-0.9B	vLLM	LinklyAI/ppocrvl-docker
VLM baseline	GLM-OCR	vLLM	LinklyAI/ocr-docker
pipeline baseline	PP-OCRv6 medium	paddle	LinklyAI/ppocrv6-docker

Same vLLM base image and same 1900 px long-edge cap as the GLM-OCR endpoint; same request/response shape as the PP-OCRv6 endpoint, so one benchmark harness drives all three.

Request

{"input": {"image_base64": "<base64 png/jpeg/pdf>"}}

Accepts image_url, image_base64, file_url, file_base64, a bare image, or an OpenAI-style messages array. PDFs are detected by magic bytes and rendered page by page at MAX_IMAGE_SIDE. An optional prompt overrides OCR_PROMPT per request — the model card documents OCR:, Table Recognition:, Formula Recognition: and Chart Recognition:.

Response carries markdown, per-page results with token counts, and a meta block recording the concurrency and image cap the worker actually ran with.

Tuning

Everything below is an environment variable, so changing a benchmark rung does not require a rebuild:

var	default	note
CONCURRENCY	1	jobs the handler accepts at once
MAX_NUM_SEQS	max(CONCURRENCY, 4)	vLLM running sequences; must be ≥ CONCURRENCY or the measured concurrency is fake
MAX_IMAGE_SIDE	1900	long-edge cap; visual tokens scale with pixel count
MAX_NUM_BATCHED_TOKENS	16384	per the official vLLM recipe
GPU_MEMORY_UTILIZATION	0.90	matches the GLM-OCR endpoint
MAX_MODEL_LEN	16384
MAX_OUTPUT_TOKENS	4096	per page
OCR_PROMPT	OCR:

Weights (revision pinned) are baked into the image and HF_HUB_OFFLINE=1, so a cold worker never reaches HuggingFace.

Measured results

See benchmark/ for the harness and LinklyAI/linkly-ai-workspace (specs/robin/159-cloud-ocr/PaddleOCR-VL-自部署评测.md) for the write-up.

Headline, on one RTX 4090 at $1.10/hr against three real single-page documents:

concurrency	image mean	image $/1k pages
1	2.047 s	$0.626
2	2.889 s	$0.613
4	3.362 s	$0.382

Cost keeps falling with concurrency — the knee was not reached. For comparison, PP-OCRv6 medium on the same samples and card reaches $0.180/1k pages at concurrency 8, and GLM-OCR $0.236.

Known issue. The endpoint entered a state where workers report running while the container holds no processes and serves no jobs, first seen during the concurrency-4 PDF rung. Ruled out: corrupt samples, vLLM launch parameters, and a platform-wide fault (a sibling PP-OCRv6 endpoint served 178 requests with zero failures over the same window). Root cause is still open — investigate before relying on this endpoint.