A model call's cost is the provider's reported charge; the catalog price is estimated_cost

CHANGELOG.md

@@ -2,6 +2,28 @@
 
 User-visible changes to Imp are recorded here.
 
+## Unreleased
+
+### Fixed
+
+- Breaking: `Imp.Core.LMResponse.cost`, and the `:cost` on a
+  `:model_response` event, is what the provider reported charging, as its
+  docs said, and `nil` when the provider reported no charge. It was ReqLLM's
+  catalog estimate for every model in ReqLLM's catalog, so every spend total
+  built on it counted an estimate as money spent, and OpenRouter's own charge,
+  which OpenRouter responses carry unasked, was overridden by it. OpenRouter
+  calls now report OpenRouter's charge; one made with the caller's own
+  provider key reports OpenRouter's fee plus the upstream charge, or `nil`
+  when the upstream charge is missing. A call to any catalog-priced provider
+  that reports no charge (Anthropic, OpenAI, Google, Groq, xAI and others)
+  now has a `nil` cost where it had the estimate. The estimate is the new `estimated_cost` field on
+  `Imp.Core.LMResponse` and `:estimated_cost` on the `:model_response` event;
+  `billing` is the breakdown behind that estimate, as it always was, and its
+  docs now say so. Migration: a host that sums `cost` treats `nil` as an
+  unknown charge, not a free one; a host that wants the old number for calls
+  with no reported charge reads `estimated_cost` for them, knowing it is an
+  estimate.
+
 ## 0.6.0 — 2026-09-28
 
 ### Security

decisions.md

@@ -8,6 +8,7 @@ not necessarily when it was made.
 
 | Date | Decision | Source and reason | Status | Retires when |
 | --- | --- | --- | --- | --- |
+| 2026-09-28 | A model call's `cost` (`Imp.Core.LMResponse`, the `:model_response` event) is only a charge the provider reported, and `nil` when it reported none; ReqLLM's catalog price is `estimated_cost`, never `cost`. | deepfates/imp#252. `lib/imp/core.ex` `LMResponse` moduledoc, `test/model_response_cost_test.exs`. Hosts sum `cost` as money spent (Dwell's daily cap); the catalog price differs from the charge when prices change, when routing picks another endpoint, and by ReqLLM rounding each line item to a millionth of a dollar. | In force. | Does not retire. |
 | 2026-09-28 | Imp follows semantic versioning. Before 1.0, a release that changes what a caller receives or can rely on bumps the minor version (`0.5` to `0.6`), and a release of fixes that change nothing a caller relies on bumps the patch; `{:imp, "~> 0.x"}` then never takes a breaking release unasked. | Owner, 2026-09-28. `CHANGELOG.md` marks each breaking change "Breaking:" with its migration, and `RELEASE_NOTES.md` names them. | In force. | Does not retire. |
 | 2026-09-26 | `Imp.Optimizer.GEPA` with no `:execution_profile` runs DSPy's GEPA (`:gepa_v0_1_4_merge`, or `:gepa_v0_1_4` with `use_merge: false`), and its reflection records are DSPy's; Imp's own search is `execution_profile: :beam_native`, chosen explicitly. | Owner, 2026-09-26. An upstream name promises upstream semantics; the published GEPA results (`research/RESULTS.md` R3, R5) used the pinned profile, as `:gepa_v0_1_4`; and no reason for the BEAM-native default was ever recorded. `lib/imp/optimizer/gepa.ex` moduledoc, `test/gepa_agent_reflection_test.exs`. | In force. | Does not retire. |
 | 2026-09-25 | Prompts and parsed values use neutral spellings: types in words (`string`, `integer`, `true or false`, `one of: a, b`, `list of strings`, `object`, declared once in `Imp.Adapter.FieldType`), values in JSON (`null`, `true`, `["x", "y"]`), in every adapter, ReAct, and the MIPROv2, GEPA and SIMBA prompts. A non-string answer for a string field is kept as its JSON text; a null is no value. Parity with DSPy means the same fields, order and constraints in the prompt, the same fields and types accepted, and the same errors raised, not DSPy's text or Python's `str()` spelling. | The owner's intention, 2026-09-25: Python spellings (`Literal[...]`, `True`, `None`, `repr(Example)`) read as foreign in an Elixir library and teach the model nothing the neutral words do not. `test/adapter_type_wording_test.exs`, `test/upstream_exam/adapters_test.exs`; the golden trace and the MIPROv2 proposer differentials compare prompts after putting DSPy's spellings into Imp's words (`Imp.DSPyWording`). | In force. | Does not retire. |

lib/imp/core.ex

@@ -75,25 +75,52 @@ defmodule Imp.Core do
     @moduledoc """
     Provider-neutral LM response: normalized outputs, usage, cost, and raw data.
 
-    `cost` is the provider's reported total for this call in USD as a
-    non-negative float, or `nil` when the provider reported nothing Imp can
-    read as a number. Providers report that total in several shapes — a bare
-    number, a string, a `Decimal`, or a cost breakdown map carrying a `total`
-    — and Imp reads the number out of all of them here, so a host reading a
-    call's money never has to learn a provider library's private shape.
-
-    `billing` is the provider's cost breakdown map, untouched, when the
-    provider reported one, and `nil` otherwise. It is the detail behind `cost`
-    (line items, input and output splits); its shape belongs to the provider,
-    so it is evidence to inspect rather than a contract to depend on.
+    `cost` is what the provider says it charged for this call, in USD, as a
+    non-negative float, or `nil` when the provider reported no charge. `nil`
+    means the charge is unknown, not that the call was free. OpenRouter reports
+    its charge, unasked, as the `"cost"` field of its usage object;
+    ReqLLM carries that field through unchanged, and Imp reads it from there.
+    A provider whose response carries no charge — the Anthropic, OpenAI and
+    Google APIs called directly among them — gives a `nil` cost. On an
+    OpenRouter call made with the caller's own provider key (`"is_byok"`),
+    `cost` is OpenRouter's fee plus the upstream charge it reports in
+    `"cost_details"`, and `nil` when that upstream charge is missing. An LM
+    client other than ReqLLM reports its charge as `:cost` in its response
+    metadata.
+
+    `estimated_cost` is ReqLLM's estimate for the call: the reported token
+    counts priced from its model catalog, as a non-negative float, or `nil`
+    when the catalog has no price for the model or the call was streamed. It
+    is a different number from
+    the charge whenever prices have changed, the provider routed to an endpoint
+    with other prices, or the provider prices caching and reasoning differently
+    from the catalog, so a host that falls back on it when `cost` is `nil` is
+    choosing to count an estimate as spend.
+
+    Either figure may arrive as a bare number, a string, a `Decimal` or a
+    breakdown map carrying a `total`, and Imp reads the number out of each, so
+    a host reading a call's money never has to learn a provider library's
+    private shape.
+
+    `billing` is the breakdown behind `estimated_cost` — ReqLLM's catalog line
+    items and input, output and reasoning splits — untouched, when there is
+    one, and `nil` otherwise. Its shape belongs to ReqLLM, so it is evidence to
+    inspect rather than a contract to depend on.
     """
 
-    defstruct outputs: [], usage: %{}, cost: nil, billing: nil, metadata: %{}, raw: nil
+    defstruct outputs: [],
+              usage: %{},
+              cost: nil,
+              estimated_cost: nil,
+              billing: nil,
+              metadata: %{},
+              raw: nil
 
     @type t :: %__MODULE__{
             outputs: list(),
             usage: map(),
-            cost: number() | nil,
+            cost: float() | nil,
+            estimated_cost: float() | nil,
             billing: map() | nil,
             metadata: map(),
             raw: term()
@@ -118,15 +145,15 @@ defmodule Imp.Core do
   def response(raw) do
     with {:ok, outputs, metadata} <- split_outputs(raw) do
       usage = response_usage(metadata)
-
-      reported = reported_cost(metadata, usage)
+      estimate = Map.get(usage, :cost)
 
       {:ok,
        %LMResponse{
          outputs: outputs,
          usage: usage,
-         cost: cost_number(reported),
-         billing: billing_breakdown(reported),
+         cost: cost_number(reported_cost(metadata, usage)),
+         estimated_cost: cost_number(estimate),
+         billing: billing_breakdown(estimate),
          metadata: metadata,
          raw: raw
        }}
@@ -267,12 +294,34 @@ defmodule Imp.Core do
     end
   end
 
+  # In ReqLLM's usage map the atom keys are ReqLLM's own: its usage step
+  # prices the token counts from the model catalog and stores that estimate as
+  # `:cost` and `:total_cost`. The provider's wire fields that ReqLLM does not
+  # interpret stay under their string keys, so OpenRouter's charge is
+  # `"cost"`. OpenRouter includes it without being asked (its
+  # `usage: %{include: true}` request option is not needed for it). A client
+  # that is not ReqLLM reports its charge as `:cost` in its own metadata.
+  #
+  # With the caller's own provider key (`"is_byok"`), OpenRouter's `"cost"` is
+  # only its fee; the provider billed the key separately, and OpenRouter
+  # reports that as `"cost_details"."upstream_inference_cost"`. Counting the fee
+  # alone would understate spend, so without the upstream figure the charge
+  # is unknown.
+  defp reported_cost(_metadata, %{"is_byok" => true} = usage) do
+    upstream = usage |> Map.get("cost_details") |> map_value(:upstream_inference_cost, nil)
+
+    case Map.get(usage, "cost") do
+      fee when is_number(fee) and is_number(upstream) -> fee + upstream
+      _incomplete -> nil
+    end
+  end
+
   defp reported_cost(metadata, usage) do
-    map_value(usage, :cost, map_value(metadata, :cost, nil))
+    Map.get(usage, "cost", map_value(metadata, :cost, nil))
   end
 
-  # A cost breakdown is the provider's own map. Anything else a provider
-  # reports as a cost is a value, not a breakdown, so there is nothing to keep.
+  # A cost breakdown is ReqLLM's map. Anything else under `:cost` is a value,
+  # not a breakdown, so there is nothing to keep.
   defp billing_breakdown(%_struct{}), do: nil
   defp billing_breakdown(reported) when is_map(reported), do: reported
   defp billing_breakdown(_reported), do: nil

lib/imp/lm.ex

@@ -17,16 +17,20 @@ defmodule Imp.LM do
   is recorded the same way, with the usage the provider reported at the end of
   the stream.
 
-  The response event's metadata carries the
-  money for that call in `:cost`: the provider's reported total in USD as a
-  non-negative float, or `nil` when the provider reported nothing Imp can read
-  as a number. A host summing spend reads that number and nothing else.
-
-  Providers report the total as a bare number, a string, a `Decimal` or a cost
-  breakdown map, and Imp reads the number out of all four. When the provider
-  reported a breakdown, that map is also on the event as `:billing`, unchanged;
-  when it reported none, there is no `:billing` key. A breakdown's shape is the
-  provider's, so treat it as evidence to inspect, not as a contract.
+  The response event's metadata carries the money for that call as two
+  numbers, each a non-negative float in USD or `nil`, as on
+  `Imp.Core.LMResponse`. `:cost` is what the provider reported charging, and
+  `nil` when it reported no charge, which means the charge is unknown rather
+  than zero. `:estimated_cost` is ReqLLM's catalog price for the reported
+  tokens, and `nil` when the catalog has no price for the model or the call
+  was streamed. A host summing
+  money spent sums `:cost`; one that falls back on `:estimated_cost` for calls
+  with no reported charge is counting an estimate, and should know it.
+
+  When ReqLLM priced the call, the breakdown behind `:estimated_cost` is also
+  on the event as `:billing`, unchanged; otherwise there is no `:billing` key.
+  A breakdown's shape is ReqLLM's, so treat it as evidence to inspect, not as
+  a contract.
   """
 
   @typedoc """
@@ -219,6 +223,7 @@ defmodule Imp.LM do
                   model: request.config.model,
                   usage: response.usage,
                   cost: response.cost,
+                  estimated_cost: response.estimated_cost,
                   response: response.metadata
                 },
                 response.billing

priv/public_api.json

@@ -3230,6 +3230,7 @@
       "struct_fields": [
         "billing",
         "cost",
+        "estimated_cost",
         "metadata",
         "outputs",
         "raw",