Truncate generated container names to the Kubernetes 63-character limit - #57
Draft
Mohit-Ak wants to merge 1 commit into
Draft
Truncate generated container names to the Kubernetes 63-character limit#57Mohit-Ak wants to merge 1 commit into
Mohit-Ak wants to merge 1 commit into
Conversation
Init container names are built from the connection name, so a long connection produced a name longer than the 63 characters Kubernetes allows for a DNS label and the pod was rejected as invalid. Cap the generated name, trimming only the caller-supplied part so the prefix and the unique suffix survive, and apply the same cap in sanitize_container_name. Refs polyaxon/polyaxon#1529
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Polyaxon builds init container names from the connection name so it's obvious what each container is for, but nothing caps the result. Kubernetes container names are DNS labels and top out at 63 characters, so a long connection name produces an invalid pod spec and the deployment is rejected outright:
generate_container_namejust did"{}-{}".format(prefix, suffix)with no length awareness, andsanitize_container_nameonly handled underscores and casing. With a 53-character connection name the artifacts init container came out at 89 characters.This isn't specific to the artifacts store from the report — every init container goes through the same helper, so the git, custom, tensorboard, dockerfile and file init containers all have the same failure mode. I checked each of those call sites in
_k8s/converter/base/init.pyand_docker/converter/base/init.py; they all pass a user-controlled connection name as the suffix.What the fix does
generate_container_namenow computes how much room the caller-supplied suffix can actually have — total budget minus the prefix, the joining dashes and the 10-character unique value — and trims only that part. The prefix stays intact so names remain readable and greppable, and the uuid fragment stays intact so distinct runs never collide. Theunique=Falsepath gets the same treatment against the prefix alone.sanitize_container_nameapplies the same cap, which covers the_patch_containerpaths in the k8s and docker converters where a name arrives from elsewhere.Trimming can leave a trailing dash mid-word, which would itself be an invalid DNS label, so
_trimstrips stray dashes off the ends.Short names are unaffected — the budget only binds when the name would have been too long, so existing behaviour and existing test expectations don't move.
Testing
New
cli/tests/test_containers/test_names.pycovers the reported case and the siblings: the length cap across prefixes and bothuniquemodes, uniqueness under truncation, DNS-label validity, and the sanitize path. There are also two cases asserting short names come through byte-for-byte unchanged, so the truncation can't silently start rewriting normal names.To confirm the tests actually pin the bug rather than passing for free, I reverted
names.pyto its original contents and ran them with the limit inlined:With the fix, all 7 pass. Full suite:
Also spot-checked that truncation doesn't cost uniqueness: 500 generations from the same long connection name yielded 500 distinct results.
blackandisortflagnames.py, but they flag the pristine file on master identically (an import blank line I didn't touch, plus import ordering) — the same is true of several neighbouring modules, so that's pre-existing and I left it alone rather than adding unrelated churn.Refs polyaxon/polyaxon#1529