Skip to content

pricing: a model name can decide an endpoint's unit when two host globs tie #1189

Description

@huang195

Found by an independent review of #1153 at 80348bf; deferred there by the author's call.

#1153 makes a billing unit host-wide: Table.unitFor(endpoint) takes the unit of the best-ranked row whose host covers the endpoint, and bestRow only admits rows in that unit. unitFor ranks with specificity.beats, which falls through to the model fields when two host patterns tie. So when two blocks in different units have globs of equal host rank that both cover an endpoint, a model name decides the endpoint's unit — and the other block's models go unpriced there.

Measured at 80348bf, blocks *.bob.ibm.com (credits, premium-ide) and api.*.ibm.com (USD, one model):

USD model "gpt-4o"                  unit(api.bob.ibm.com)=credits  premium-ide=configured  gpt-4o=none
USD model "gpt-4o-mini-2024-07-18"  unit(api.bob.ibm.com)=USD      premium-ide=none        gpt-4o-mini=configured

At a66911e both models priced in both cases. Config.entries' agreement check compares identical host strings only, so neither config is refused.

A related, deliberate-but-undocumented case: *.bob.ibm.com (credits) plus a more specific api.us-east.bob.ibm.com block with no unit: makes that host USD, and premium-ide unpriced there, with no startup error — a block that omits unit: declares USD for its hosts.

Shape of a fix

Rank the endpoint-unit lookup on provenance and host specificity only, tie-breaking on the host pattern, so no model name can move a unit; and document that the most specific block covering a host decides its unit, including a block that omits unit:.

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions