diff --git a/CHANGELOG.md b/CHANGELOG.md index 36b74a25..c6f859b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- `ActiveAgent::Base.rendered_instructions` renders an agent's instructions + outside a generation, for a dashboard mirroring the class and for tests + asserting what a model is told. Both otherwise reached a private renderer + through `send`. +- `ActionAgent::AgentSync` mirrors host agent classes into dashboard `Agent` + records, setting the `agent_class_name` that `AgentRelease` already expects a + host to have written. The code owns what an agent is (name, description, + instructions, tools — rewritten every sync); the operator owns how it runs + (provider, model, status — set on create and preserved), so a model chosen in + the dashboard survives the next deploy. +- `ActionAgent.run_host_agent_classes` (default `false`) runs an agent that + mirrors a host class as that class, rather than as one rebuilt from the + record's `tools` and `instructions` columns. Dashboard-authored agents, which + name no class, keep using the dynamic runtime either way; a class name that no + longer resolves falls back to it rather than failing the run. + +### Fixed + +- A Rails enum is offered to the model as its names (`{type: "string", enum: + [...]}`) instead of the integer backing it. `SchemaGenerator` reads enums from + inclusion validators and never consulted `defined_enums`, so a `status` column + reached the model as a bare integer with no labels — leaving every host to + explain the mapping in prose. +- A filter value outside an enum is rejected, naming the valid values, instead + of matching no rows. `status: "pending"` returned `{count: 0}`, which an agent + reports as a fact — indistinguishable from "none match". Same reasoning as the + unknown-operator rejection in `range_predicates!`. +- An enum is no longer offered the range form. Its integer backing is a + declaration-order artefact, so `status: {gt: 1}` was a meaningless filter that + still returned a confident count. + ## [1.6.3] - 2026-09-18 Releases `activeagent` and `actionagent` 1.6.3 from one tag. diff --git a/actionagent/app/services/action_agent/agent_execution_service.rb b/actionagent/app/services/action_agent/agent_execution_service.rb index 6fc8a47a..82a2fc06 100644 --- a/actionagent/app/services/action_agent/agent_execution_service.rb +++ b/actionagent/app/services/action_agent/agent_execution_service.rb @@ -500,6 +500,21 @@ def generate! tool_definitions = tool_schemas service = self + # A dashboard-authored agent has no Ruby class — it is rows: a tool + # selection, instructions typed in the builder. That is the common case + # and the runtime below builds a class for it. + # + # An agent mirrored from host code is the other case: the class exists, + # already declares its own tools (SchemaTools rosters, delegations) and + # renders its own instructions, and none of that is reachable through + # `tools` + `instructions` columns. Running the real class keeps the + # dashboard evaluating what production runs, instead of a rebuilt + # lookalike. Both runtimes stay; which one applies is decided by whether + # the class resolves. + if (host_class = resolved_host_class) + return run_host_class(host_class, actor: actor, action: action, run_trace_id: run_trace_id) + end + agent_class = Class.new(ActiveAgent::Base) do # SolidAgent persists contexts under self.class.name; anonymous # classes would fail its agent_name presence validation. @@ -677,6 +692,40 @@ def record_tool_spans(root_span, response) end end + # The host class this agent mirrors, when it names one that resolves to a + # runnable ActiveAgent::Base subclass. Anything else — no class name, a + # class that no longer exists, a name that resolves to something else — is + # nil, and the dynamic runtime handles the record as before. + # + # @return [Class, nil] + def resolved_host_class + return nil unless ActionAgent.run_host_agent_classes + + name = @agent_record.agent_class_name.presence + return nil if name.blank? + + klass = name.safe_constantize + klass if klass.is_a?(Class) && klass < ActiveAgent::Base + end + + # Runs the host's own class. Its tools, delegations and instructions come + # from the code, so the engine supplies only what is the run's business: + # the caller, and the trace to correlate against. + def run_host_class(klass, actor:, action:, run_trace_id:) + generation = klass.as(actor).public_send(action, **host_action_arguments(klass, action)) + generation.prompt_options[:trace_id] = run_trace_id if generation.respond_to?(:prompt_options) + generation.generate_now + end + + # A code agent's action takes named arguments (`ask(question:)`), so the + # run's message is passed under the action's own keyword rather than as a + # bare message the signature would reject. + def host_action_arguments(klass, action) + contract = klass.try(:delegation_contracts)&.dig(action.to_sym) + keyword = contract&.try(:parameters)&.keys&.first + keyword ? { keyword.to_sym => user_text } : {} + end + def provider_available?(name) # The gem's mock provider is a test double: accepted only in the test # environment so app runs can never store fabricated output. diff --git a/actionagent/app/services/action_agent/agent_sync.rb b/actionagent/app/services/action_agent/agent_sync.rb new file mode 100644 index 00000000..ad5fe303 --- /dev/null +++ b/actionagent/app/services/action_agent/agent_sync.rb @@ -0,0 +1,134 @@ +# frozen_string_literal: true + +module ActionAgent + # Mirrors host ActiveAgent classes into Agent records, so the dashboard can + # run, evaluate and release the agents an app already has in code. + # + # The engine reads an agent's identity from the class rather than asking the + # host to restate it: name, description and tool roster all come from the + # class, and re-running the sync updates each record in place. `AgentRelease` + # already expects `agent_class_name` to be set by "whatever syncs its + # ActiveAgent classes into Agent records" — this is that, so a host no longer + # has to write it. + # + # The split is deliberate and is the reason this is safe to run on deploy: + # + # * **The code owns what an agent is** — name, description, instructions, + # tools. Rewritten on every sync, so it cannot drift from the class. + # * **The operator owns how it runs** — provider, model, status. Set once on + # create and never touched again, so a model chosen in the dashboard + # survives the next deploy. + # + # ActionAgent::AgentSync.call(RecordAgent.all, owner: owner) + # + # @see AgentRelease which cuts a version per synced agent + class AgentSync + Row = Struct.new(:agent, :created, :skipped, keyword_init: true) + Result = Struct.new(:rows, :errors, keyword_init: true) do + def success? = errors.blank? + def agents = rows.filter_map(&:agent) + def created = rows.select(&:created) + def skipped = rows.select(&:skipped) + end + + # @param agents [Array] ActiveAgent::Base subclasses + # @param owner [Object] the record agents and API keys scope to + # @param provider [String, Symbol, nil] defaults to the class's own + # @param model [String, nil] defaults to the class's own + def self.call(agents, owner:, provider: nil, model: nil) + new(agents, owner: owner, provider: provider, model: model).call + end + + def initialize(agents, owner:, provider: nil, model: nil) + @agents = Array(agents) + @owner = owner + @provider = provider + @model = model + end + + # @return [Result] + def call + return Result.new(rows: [], errors: "An owner is required: the engine scopes agents to one.") if @owner.nil? + + rows = Agent.transaction { @agents.map { |klass| upsert(klass) } } + Result.new(rows: rows, errors: nil) + rescue ActiveRecord::RecordInvalid => e + Result.new(rows: [], errors: e.record.errors.full_messages.join(", ")) + end + + private + + def upsert(klass) + unless klass.respond_to?(:prompt_options) + return Row.new(agent: nil, created: false, skipped: "#{klass} is not an ActiveAgent::Base subclass") + end + + provider = resolved_provider(klass) + model = resolved_model(klass) + if provider.blank? || model.blank? + return Row.new(agent: nil, created: false, skipped: "#{klass} has no provider/model configured") + end + + agent = Agent.find_or_initialize_by(slug: self.class.slug_for(klass)) + created = agent.new_record? + if created + agent.owner = @owner + agent.provider = provider + agent.model = model + agent.status = :active + end + + agent.assign_attributes( + name: klass.name.titleize, + agent_class_name: klass.name, + description: description_for(klass), + instructions: instructions_for(klass), + tools: tool_names_for(klass) + ) + agent.save! + Row.new(agent: agent, created: created, skipped: nil) + end + + # "TicketAgent" -> "ticket-agent", the slug an MCP client sees as + # run_ticket-agent. A namespaced class flattens its separators, because + # Agent validates slugs as /\A[a-z0-9\-_]+\z/ — "Billing::TicketAgent" + # becomes "billing-ticket-agent". + def self.slug_for(klass) + klass.name.underscore.tr("/", "-").tr("_", "-") + end + + # An agent's own description if it declares one (a delegation contract is + # where an agent says what it answers), else its titleized name. + def description_for(klass) + contract = klass.try(:delegation_contracts)&.values&.first + contract&.try(:description).presence || klass.name.titleize + end + + # The rendered instructions, so the dashboard record runs on the same text + # the class does rather than a hand-maintained copy. + # + # An agent whose instructions are assembled rather than rendered straight + # from its own template — filled from assigns it computes, or falling back + # to a template it shares with sibling agents — says so by defining + # `dashboard_instructions_text`. That is asked first, because only the + # class knows how its own prompt is built. + def instructions_for(klass) + return klass.dashboard_instructions_text.presence if klass.respond_to?(:dashboard_instructions_text) + + klass.try(:rendered_instructions).presence + end + + def tool_names_for(klass) + names = klass.try(:tool_names) + Array(names).map(&:to_s) + end + + def resolved_provider(klass) + (@provider || klass.prompt_options[:service]).to_s.downcase.presence + end + + def resolved_model(klass) + (@model || klass.prompt_options[:model]).presence + end + end +end diff --git a/actionagent/lib/action_agent.rb b/actionagent/lib/action_agent.rb index c37e035f..ad28e936 100644 --- a/actionagent/lib/action_agent.rb +++ b/actionagent/lib/action_agent.rb @@ -256,6 +256,21 @@ def deprecator # @return [Boolean] attr_accessor :execution_enabled + # Whether a run of an agent that mirrors a host class executes that class, + # instead of the class the engine builds from the record's `tools` and + # `instructions` columns. + # + # Off by default: it changes what a run of a mirrored agent executes, and + # a host that has tuned its dashboard records around the dynamic runtime + # should opt in deliberately. Dashboard-authored agents — the ones with no + # `agent_class_name` — are unaffected either way. + # + # On, a mirrored agent runs its real tools, delegations and instructions, + # so an evaluation scores the agent production runs rather than a + # flattened copy of it. + # @return [Boolean] + attr_accessor :run_host_agent_classes + # Whether the "Ask ActiveAgents" assistant is available. # # The assistant is a tool for developing and CI-ing agents: it sends @@ -544,6 +559,7 @@ def reset! @provider_credentials_resolver = nil @sandbox_backends = {} @execution_enabled = true + @run_host_agent_classes = false @assistant_enabled = nil @scenario_evaluation_adapter_resolver = nil diff --git a/actionagent/test/agent_sync_test.rb b/actionagent/test/agent_sync_test.rb new file mode 100644 index 00000000..da6be0b9 --- /dev/null +++ b/actionagent/test/agent_sync_test.rb @@ -0,0 +1,89 @@ +# frozen_string_literal: true + +require "test_helper" + +# Mirroring host agent classes into Agent records — the step that lets the +# dashboard run, evaluate and release the agents an app already has in code. +class AgentSyncTest < ActiveSupport::TestCase + class AssembledAgent < ApplicationAgent + generate_with :mock, model: "mock-1" + + def self.dashboard_instructions_text = "assembled for the dashboard" + + def ask + prompt(message: "hi") + end + end + + class InvoiceAgent < ApplicationAgent + generate_with :mock, model: "mock-1" + + def ask + prompt(message: "hi") + end + end + + setup do + ActionAgent::Agent.delete_all + end + + teardown do + ActionAgent::Agent.delete_all + end + + test "creates a record carrying the class name, so releases can resolve it" do + result = ActionAgent::AgentSync.call([ InvoiceAgent ], owner: nil_owner) + + assert result.success?, result.errors + agent = result.agents.sole + assert_equal "AgentSyncTest::InvoiceAgent", agent.agent_class_name + assert_equal "agent-sync-test-invoice-agent", agent.slug + assert_equal "mock", agent.provider + assert_equal "mock-1", agent.model + end + + test "re-running updates in place rather than duplicating" do + ActionAgent::AgentSync.call([ InvoiceAgent ], owner: nil_owner) + + assert_difference -> { ActionAgent::Agent.count }, 0 do + ActionAgent::AgentSync.call([ InvoiceAgent ], owner: nil_owner) + end + end + + test "the operator's provider and model survive a re-sync" do + agent = ActionAgent::AgentSync.call([ InvoiceAgent ], owner: nil_owner).agents.sole + agent.update!(model: "operator-choice") + + ActionAgent::AgentSync.call([ InvoiceAgent ], owner: nil_owner) + + # The code owns what an agent is; the operator owns how it runs. A model + # picked in the dashboard must survive the next deploy's sync. + assert_equal "operator-choice", agent.reload.model + end + + test "an agent that assembles its own instructions is asked for them" do + result = ActionAgent::AgentSync.call([ AssembledAgent ], owner: nil_owner) + + # A record agent's prompt comes from a template it shares with siblings, + # filled from assigns it computes — the class is the only thing that knows + # how to build it, so the sync asks rather than rendering a template that + # would come back empty. + assert_equal "assembled for the dashboard", result.agents.sole.instructions + end + + test "a class that is not an agent is skipped, not raised" do + result = ActionAgent::AgentSync.call([ String ], owner: nil_owner) + + assert result.success?, result.errors + assert_empty result.agents + assert_match "not an ActiveAgent::Base subclass", result.skipped.sole.skipped + end + + private + + # Agent has no owner presence validation; the engine scopes by whatever the + # host configures, and these tests exercise the sync itself. + def nil_owner + ActionAgent::Agent.new + end +end diff --git a/actionagent/test/host_agent_class_execution_test.rb b/actionagent/test/host_agent_class_execution_test.rb new file mode 100644 index 00000000..1ab9f5c1 --- /dev/null +++ b/actionagent/test/host_agent_class_execution_test.rb @@ -0,0 +1,89 @@ +# frozen_string_literal: true + +require "test_helper" + +# Two runtimes, and the rule for choosing between them. +# +# A dashboard-authored agent is rows — a tool selection and instructions typed +# into the builder — and the engine builds a class for it at run time. An +# agent mirrored from host code already *is* a class, with its own tools, +# delegations and instructions that `tools` + `instructions` columns cannot +# express. Both have to keep working. +class HostAgentClassExecutionTest < ActiveSupport::TestCase + class MirroredAgent < ApplicationAgent + generate_with :mock, model: "mock-1" + + def ask + prompt(message: "hi") + end + end + + setup do + ActionAgent::Agent.delete_all + @original = ActionAgent.run_host_agent_classes + end + + teardown do + ActionAgent.run_host_agent_classes = @original + ActionAgent::Agent.delete_all + end + + test "a dashboard-authored agent names no class, so the dynamic runtime keeps it" do + ActionAgent.run_host_agent_classes = true + agent = ActionAgent::Agent.create!( + name: "Builder Made", provider: "mock", model: "mock-1", agent_class_name: nil + ) + + assert_nil service_for(agent).send(:resolved_host_class), + "an agent with no class must fall to the dynamic runtime" + end + + test "a mirrored agent resolves its host class when the flag is on" do + ActionAgent.run_host_agent_classes = true + agent = ActionAgent::Agent.create!( + name: "Mirrored", provider: "mock", model: "mock-1", + agent_class_name: "HostAgentClassExecutionTest::MirroredAgent" + ) + + assert_equal MirroredAgent, service_for(agent).send(:resolved_host_class) + end + + test "the flag is off by default, so existing hosts are untouched" do + refute @original, "run_host_agent_classes must default to off" + + ActionAgent.run_host_agent_classes = false + agent = ActionAgent::Agent.create!( + name: "Mirrored", provider: "mock", model: "mock-1", + agent_class_name: "HostAgentClassExecutionTest::MirroredAgent" + ) + + assert_nil service_for(agent).send(:resolved_host_class) + end + + test "a class name that no longer resolves falls back rather than failing" do + ActionAgent.run_host_agent_classes = true + agent = ActionAgent::Agent.create!( + name: "Stale", provider: "mock", model: "mock-1", agent_class_name: "DeletedAgent" + ) + + # A renamed or removed class must not take the dashboard down with it. + assert_nil service_for(agent).send(:resolved_host_class) + end + + test "a class that is not an agent is not run as one" do + ActionAgent.run_host_agent_classes = true + agent = ActionAgent::Agent.create!( + name: "Not An Agent", provider: "mock", model: "mock-1", agent_class_name: "String" + ) + + assert_nil service_for(agent).send(:resolved_host_class) + end + + private + + def service_for(agent) + ActionAgent::AgentExecutionService.allocate.tap do |service| + service.instance_variable_set(:@agent_record, agent) + end + end +end diff --git a/lib/active_agent/concerns/view.rb b/lib/active_agent/concerns/view.rb index 9c0f6408..61a983c2 100644 --- a/lib/active_agent/concerns/view.rb +++ b/lib/active_agent/concerns/view.rb @@ -12,6 +12,29 @@ module View include ActionView::Layouts end + class_methods do + # The agent's rendered instructions, outside a generation. + # + # Two surfaces need the text an agent would run on without running it: a + # dashboard that mirrors the class (ActionAgent::AgentSync) and a test + # asserting what the model is told. Both otherwise reach a private + # renderer through `send`, which couples them to internals that can move + # without notice. + # + # TicketAgent.rendered_instructions # => "You are the Ticket agent..." + # + # TicketAgent.rendered_instructions(topic: "tickets") + # + # @param template [String] template name, default "instructions" + # @param assigns [Hash] instance variables the template reads + # @return [String, nil] nil when the agent has no such template + def rendered_instructions(template: "instructions", **assigns) + agent = new + assigns.each { |name, value| agent.instance_variable_set(:"@#{name}", value) } + agent.send(:view_render_template, template) + end + end + # Builds template lookup paths supporting both flat and nested directory structures. # # Templates are searched in priority order: diff --git a/lib/active_agent/schema_tools.rb b/lib/active_agent/schema_tools.rb index 55bd6531..a69fe8ed 100644 --- a/lib/active_agent/schema_tools.rb +++ b/lib/active_agent/schema_tools.rb @@ -332,6 +332,8 @@ def permitted_filters!(arguments) "`#{key}` is not a filterable attribute. Allowed filters: #{filterable.join(", ")}" end + validate_enum_value!(column, value) + memo[column] = value end @@ -386,6 +388,35 @@ def range_predicates!(column, value) end end + # A value outside a Rails enum reaches `where` as an unmatched name and + # returns zero rows — the same silent-nothing that {.range_predicates!} + # rejects for an unknown operator, and just as bad here: an agent reads + # "0 tickets under review" as a fact rather than a mistyped filter. + # + # The range form is not offered on an enum (see {.range_filterable?}), so + # a Hash here is a filter that cannot mean anything. + # + # @api private + # @raise [UnpermittedAttribute] on a value the enum does not define + def validate_enum_value!(column, value) + values = enum_values_for(column) + return if values.blank? + + if value.is_a?(Hash) + raise UnpermittedAttribute, + "`#{column}` is an enum and cannot be compared as a range. " \ + "Allowed values: #{values.keys.join(", ")}" + end + + # The schema offers names only, so an integer here came from a model + # ignoring it. Accepted anyway: it is unambiguous, and a host calling + # the tool directly in Ruby reasonably passes the backing value. + return if values.key?(value.to_s) || values.value?(value) + + raise UnpermittedAttribute, + "`#{value}` is not a valid `#{column}`. Allowed values: #{values.keys.join(", ")}" + end + # Projects a record down to the declared return columns. # # The projection happens in SQL (+select+) as well as here, but the Ruby @@ -455,14 +486,44 @@ def filter_properties properties = schema[:schema][:properties] filterable.index_with do |column| + next enum_property(column) if enum_column?(column) + scalar = (properties[column] || { type: "string" }).deep_dup range_filterable?(column) ? with_range_form(column, scalar) : scalar end end + # A Rails enum is declared on the model, not through the inclusion + # validator SchemaGenerator reads, so without this a `status` column + # reaches the model as a bare integer: no labels, no constraint. The + # names are the model's own API (`Ticket.open`, `status: "open"`), so + # they are what the tool offers. + # + # @api private + def enum_property(column) + { type: "string", enum: enum_values_for(column).keys, description: "#{column.to_s.humanize} field" } + end + + # @api private + def enum_column?(column) + enum_values_for(column).present? + end + + # @api private + def enum_values_for(column) + @model.defined_enums[column.to_s] || {} + end + # Dates, times and numbers are the columns a question like "overdue" or # "more than 10" actually needs a comparison on. + # + # An enum is backed by an integer but is not ordered in any sense a + # question can use: `status > 1` is an artefact of declaration order, so + # offering the range form on one invites a meaningless filter that still + # returns a confident count. def range_filterable?(column) + return false if enum_column?(column) + RANGE_FILTERABLE_TYPES.include?(@model.type_for_attribute(column).type) end diff --git a/test/dummy/app/models/post.rb b/test/dummy/app/models/post.rb index 7ed82220..28a71c66 100644 --- a/test/dummy/app/models/post.rb +++ b/test/dummy/app/models/post.rb @@ -3,6 +3,11 @@ class Post < ApplicationRecord belongs_to :user + # A Rails enum, for the schema-tools tests: declared on the model rather + # than through an inclusion validator, which is the case SchemaGenerator + # cannot see and SchemaTools has to read from defined_enums. + enum :state, { draft: 0, review: 1, live: 2 }, prefix: true + validates :title, presence: true, length: { maximum: 255 } validates :content, presence: true diff --git a/test/dummy/app/views/rendered_instructions_probe_agent/instructions.md.erb b/test/dummy/app/views/rendered_instructions_probe_agent/instructions.md.erb new file mode 100644 index 00000000..0f12c368 --- /dev/null +++ b/test/dummy/app/views/rendered_instructions_probe_agent/instructions.md.erb @@ -0,0 +1 @@ +You are the probe agent<%= " for #{@topic}" if @topic %>. diff --git a/test/dummy/db/migrate/002_create_posts.rb b/test/dummy/db/migrate/002_create_posts.rb index 26897f45..e19e1bf8 100644 --- a/test/dummy/db/migrate/002_create_posts.rb +++ b/test/dummy/db/migrate/002_create_posts.rb @@ -8,6 +8,9 @@ def change t.references :user, foreign_key: true t.boolean :published, default: false t.datetime :published_at + # Backs Post's `enum :state` — the schema-tools tests need a real Rails + # enum, which is the case SchemaGenerator cannot read from validators. + t.integer :state, default: 0 t.timestamps end diff --git a/test/dummy/db/schema.rb b/test/dummy/db/schema.rb index b9452945..818d6716 100644 --- a/test/dummy/db/schema.rb +++ b/test/dummy/db/schema.rb @@ -532,6 +532,7 @@ t.datetime "created_at", null: false t.boolean "published", default: false t.datetime "published_at" + t.integer "state", default: 0 t.string "title", null: false t.datetime "updated_at", null: false t.integer "user_id" diff --git a/test/integration/solid_agent/runs_test.rb b/test/integration/solid_agent/runs_test.rb index 31e7d1b3..76f5bd01 100644 --- a/test/integration/solid_agent/runs_test.rb +++ b/test/integration/solid_agent/runs_test.rb @@ -98,20 +98,30 @@ class SolidAgentRunsTest < SolidAgentIntegrationTest generation = AgentGeneration.sole - # The mock provider's model prices at zero by design, so the assertion - # that matters is that estimation runs and stays consistent. + # Usage has to reach the record before there is anything to price. + assert_operator generation.input_tokens + generation.output_tokens, :>, 0 + + # The mock provider's model prices at zero by design: 0.0 rather than nil + # is the proof that estimation ran over the recorded tokens. + assert_equal 0.0, generation.estimated_cost + + # A real model prices above zero. Which table answers — solid_agent's + # static rates or RubyLLM's registry — depends on whether ruby_llm has + # been loaded elsewhere in this process (its provider tests load it when + # OPENAI_API_KEY is set), and the two disagree on this model. So no dollar + # figure is pinned through the tables; solid_agent's own suite owns those. + generation.update!(model: "claude-sonnet-5", input_tokens: 12_000, output_tokens: 800) + + assert_operator generation.estimated_cost, :>, 0 assert_equal( - SolidAgent::ModelPricing.estimate( - model: generation.model, - input_tokens: generation.input_tokens, - output_tokens: generation.output_tokens - ), + SolidAgent::ModelPricing.estimate(model: "claude-sonnet-5", input_tokens: 12_000, output_tokens: 800), generation.estimated_cost ) - assert_in_delta 0.048, SolidAgent::ModelPricing.estimate( - model: "claude-sonnet-5", input_tokens: 12_000, output_tokens: 800 - ), 0.0005 + # Explicit rates bypass every table, so the arithmetic itself is stable. + assert_in_delta 0.048, + generation.estimated_cost(input_price_per_million: 3.0, output_price_per_million: 15.0), + 1e-9 end test "one trace id joins the run, the conversation and the generation" do diff --git a/test/rendered_instructions_test.rb b/test/rendered_instructions_test.rb new file mode 100644 index 00000000..a27d9d2f --- /dev/null +++ b/test/rendered_instructions_test.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +require "test_helper" + +# Rendering an agent's instructions without running it: what a dashboard that +# mirrors the class needs (ActionAgent::AgentSync), and what a test asserting +# what the model is told needs. Both otherwise reach a private renderer +# through `send`. +class RenderedInstructionsProbeAgent < ActiveAgent::Base +end + +class RenderedInstructionsTest < ActiveSupport::TestCase + test "renders the agent's own instructions template" do + assert_equal "You are the probe agent.", RenderedInstructionsProbeAgent.rendered_instructions + end + + test "assigns reach the template" do + # The hub case: instructions describing a roster the class computes. + assert_equal( + "You are the probe agent for tickets.", + RenderedInstructionsProbeAgent.rendered_instructions(topic: "tickets") + ) + end + + test "returns nil when the agent has no instructions template" do + agent = Class.new(ActiveAgent::Base) do + def self.name = "NoInstructionsTemplateAgent" + end + + assert_nil agent.rendered_instructions + end +end diff --git a/test/schema_tools_test.rb b/test/schema_tools_test.rb index b03b5b5e..cb72606f 100644 --- a/test/schema_tools_test.rb +++ b/test/schema_tools_test.rb @@ -36,6 +36,15 @@ class UnscopedPostTools < ActiveAgent::SchemaTools returns :id, :title end + # A Rails enum, which is declared on the model rather than through the + # inclusion validator SchemaGenerator reads. + # Post declares `enum :state` (test/dummy/app/models/post.rb). + class EnumPostTools < ActiveAgent::SchemaTools + model Post + filterable :state, :published + returns :id, :title + end + setup do Post.delete_all Profile.delete_all if defined?(Profile) @@ -107,6 +116,54 @@ class UnscopedPostTools < ActiveAgent::SchemaTools assert_equal "boolean", properties[:active][:type] end + test "a Rails enum is offered as its names, not its integer backing" do + definition = EnumPostTools.tool_definitions.find { |d| d[:name] == "find_posts" } + state = definition[:parameters][:properties][:state] + + # Without this the column reaches the model as {type: "integer"}: no + # labels, so the only way a model learns "review" means 1 is prose in the + # instructions, written by hand in every host app. + assert_equal "string", state[:type] + assert_equal [ "draft", "review", "live" ], state[:enum] + end + + test "an enum is not offered the range form" do + definition = EnumPostTools.tool_definitions.find { |d| d[:name] == "find_posts" } + state = definition[:parameters][:properties][:state] + + # `state > 1` is an artefact of declaration order, not a question anyone + # can ask. Offering it invites a filter that returns a confident number. + refute state.key?(:anyOf), "an enum must not advertise comparisons" + end + + test "an undefined enum value is rejected rather than matching nothing" do + result = EnumPostTools.call("count_posts", state: "pending") + + # The silent alternative is {count: 0}, which an agent reports as a fact: + # "no posts are pending" reads identically to "pending is not a state". + # Same shape as the unknown-column error, so the model can correct itself. + refute result.key?(:count), "an undefined enum value must not return a count" + assert_match "pending", result[:error] + assert_match "draft, review, live", result[:error] + end + + test "an enum still accepts the names and values it defines" do + # setup seeds two posts, both at the column default (draft). + Post.create!(title: "Live one", content: "x", user: @alice, state: "live") + + assert_equal({ count: 2 }, EnumPostTools.call("count_posts", state: "draft")) + assert_equal({ count: 1 }, EnumPostTools.call("count_posts", state: "live")) + # The integer backing still works, so a stored value round-trips. + assert_equal({ count: 1 }, EnumPostTools.call("count_posts", state: 2)) + end + + test "a range predicate on an enum is rejected" do + result = EnumPostTools.call("count_posts", state: { "gt" => 1 }) + + refute result.key?(:count), "a comparison on an enum must not return a count" + assert_match "cannot be compared as a range", result[:error] + end + test "only filterable columns appear as find parameters" do definition = UserTools.tool_definitions.find { |d| d[:name] == "find_users" } properties = definition[:parameters][:properties]