feat: ship language data, report usage, stop inspect leaking keys - #21
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Subsystem C, the last of three. A (providers and transport) and B (the pipeline) are merged.
Three changes that share a theme: the gem knew things it never told anyone, and told one thing it should not.
The usage event
Every provider built a
Translation::Usageand nothing read it. It is now afifth instrumentation event,
usage.translation_diff, besidetranslate,cache,requestandrate_limit:providercharactersbilled_charactersreportedmodelreportedis what makes a nil honest: three of the six providers reportbilling, so a subscriber summing
billed_characterswithout checking itgets a total that is quietly too low.
The language registry
One JSON file per provider under
data/languages/, captured live from thevendors on 2026-09-10, each naming the date and the endpoint it came from.
Translatorconsults it before every translation and raisesUnsupportedLanguageErrorbefore any request is made.Data ships for DeepL, Google, Azure and ModernMT. Amazon ships none — its
list needs signed credentials — and LibreTranslate ships none either, because
it is self-hosted, so the language set belongs to the instance. A provider
with no data refuses nothing:
Languages.supports?answers nil, and nil isnot a refusal.
Codes match downcased on the primary subtag, so
en-GBandenmatch eachother. Deliberately permissive: the registry catches a wrong language, not a
wrong regional variant. One extra rule handles a mismatch the vendors create
themselves — ModernMT publishes
peswhere applications writefa— and itholds in one direction only, because a vendor publishing
zhhas promisednothing about
cmn.rake languages:refreshre-fetches the lists. A provider whose fetch fails,or which answers with an empty list, keeps its previous file, and failures
are reported rather than silently blanking good data.
Breaking: validation is on by default, so an application translating a
pair the shipped data does not list will begin raising. Two escapes:
assume_supported: trueper call,config.validate_languages = falseglobally. Both are named in the error message.
Redacted inspect
Configuration#inspectandProvider#inspectprinted API keys verbatim,walking around every protection the earlier subsystems built. They now print
[FILTERED], and the filtered set is derived from the provider registryrather than from a list someone has to remember to update. A credential
embedded in a URL — the usual shape of a managed Redis URL — has its
userinfo redacted while the host stays readable.
Along the way
Translator's dispatch seam moved into its ownDispatcher: batching,throttling, the wire call and three of the five events.
Translatorkeepsdocument assembly, provider resolution, source-language settling, validation
and the sentence cache. The
Metrics/ClassLengthexclude it had acquired isgone.
Cache keys are untouched.
assume_supported:is a reserved keyword, so itnever reaches
@optionsand never reaches a provider's payload — anupgrading application keeps its cache.
567 runs, 0 failures. Rubocop clean across 94 files.
Not in this branch
No price list. No vendor exposes prices through an API, a list price is wrong
for exactly the people who count money, and a stale number in money is worse
than a stale sentence in a README. The character count is here; the rate is
the reader's own.