Repository navigation
v4 - #2406
Open
Luca Forstner (lforst) wants to merge 26 commits into
Open
v4#2406Luca Forstner (lforst) wants to merge 26 commits into
Luca Forstner (lforst) wants to merge 26 commits into
Conversation
ref: [SDK-226](https://linear.app/braintrustdata/issue/SDK-226) Replace Zod-inferred public types with explicit TypeScript types to reduce declaration size and downstream compile cost.
ref [#1841](#1841) Removes Plugins and unsubscribe functionality. Reasoning: Plugins are a bit of an unnecessary API to be exposed because they just attach stuff and un-attach stuff. The user can very easily just manage this by themselves. Also, unsubscribe is literally called from nowhere in the SDK.
Ref [#1841](#1841) `flushAsync` grinded my gears a bit a) as a user b) as a maintainer. Why?: - As a user, it added a quite intransparent API that is hard to wrap your head around. - As a user, it added a generic type to otherwise very non-generic APIs that made the LSP types incredibly hard to navigate - As a maintainer, it required a generic type to be passed around to literally everywhere - As a maintainer, it required us to have a weird `flushAsync` option to every tracing API In reality, what you can do instead, and which is very simple to understand, is simply do `await flush()` or `await logger.flush()` right after a span is ended. Not hard, very few if no drawbacks.
ref [#1841](#1841) - Replaced the root `braintrust` export surface with an explicit allowlist, removing roughly 140 named exports—primarily internal helpers, implementation classes, schemas, constants, and low-level types. - Stopped named-exporting most SDK-owned request/result types. Retained types are primarily user extension points, including `StartSpanArgs`, `Span`, tracing context types, eval callbacks, reporters, template plugins, and durable-eval stores. - Removed package subpaths `braintrust/browser`, `braintrust/util`, and `braintrust/webpack-loader`; the util bundle is no longer built or published. - Kept the webpack loader private: `braintrust/next` now resolves its built loader directly instead of exposing it as a package entrypoint. - Removed public implementation/testing exports such as `_exportsForTestingOnly`, `_internalIso`, `_internalGetGlobalState`, `_internalSetInitialState`, `SpanImpl`, `NoopSpan`, `TestBackgroundLogger`, `ObjectFetcher`, and ID-generator classes. - Removed public runtime schemas and serialization internals, including prompt, stream-chunk, attachment-reference, logs-overflow, and propagation schemas/constants. - Removed lower-level project/building APIs such as `graph`, `Project`, `CodeFunction`, `CodePrompt`, `PromptBuilder`, `ScorerBuilder`, and `ToolBuilder`; the `projects` facade remains. - Removed miscellaneous convenience and implementation exports including `log`, `summarize`, `traceable`, `newId`, `permalink`, `deepCopyEvent`, template-registry internals, span fetchers, and logs-overflow helpers. - Removed deprecated APIs: - `BraintrustMiddleware` and `wrapAISDKModel` - `wrapMastraAgent` - `withExperiment` and `withDataset` - `toAIStreamResponse` - deprecated bundler aliases `esbuildPlugin`, `rollupPlugin`, `vitePlugin`, and `webpackPlugin` - deprecated `useDiagnosticChannelCompatShim` - `Experiment.close()`, `Dataset.close()`, and `Span.close()` - `EvalHooks.meta`, `EvalResultWithSummary.toString()`, and `Score.error` - Removed legacy aliases and call shapes: - `initExperiment` - string-first `init()` and `initDataset()` overloads - dataset `useOutput` mode and the `output` record alias - deprecated event `inputs` and `datasetRecordId` fields - `wrapOpenAIv4` - Changed `EvalResultWithSummary` from an exported runtime class to a plain `{ summary, results }` result shape. - Restricted public span parenting to opaque `PropagationContext`; serialized span-slug strings remain internal plumbing. The public type is still named `StartSpanArgs`. - Added `configureContextManager()` and `ContextManager.getCurrentSpanStore()` as the supported context integration mechanism, replacing global symbols and direct state mutation. - Moved instrumentation-only APIs off the root: - `configureInstrumentation`, `InstrumentationConfig`, and Eve/Flue instrumentation remain under `braintrust/instrumentation`. - `registerOtelFlush` now lives there as the narrow OTEL hook. - Simplified OTEL integration: - Uses `configureContextManager()` instead of internal globals. - No longer overrides ID generation or V4 span serialization. - Replaced `contextFromSpanExport(string)` with `contextFromSpan(span)`. - `parentFromHeaders()` now returns opaque W3C propagation context. - Environment and `.env.braintrust` handling is self-contained and Edge-safe. - Removed the OTEL-specific exported ID generator. - Migrated Temporal propagation from private serialized span headers to standard `traceparent`, `tracestate`, and `baggage`, covering client→workflow, child workflow, and same-/cross-worker activity parenting. - Reworked `@braintrust/browser` to install a browser `ContextManager` through the public configuration API instead of mutating core isomorph internals. - Updated OpenAI Agents and the Vercel adapter to avoid `braintrust/util` and the exported `BraintrustStreamChunk`; the adapter now derives its stream chunk type from `BraintrustStream`. - Canonicalized instrumentation config keys by removing typed aliases such as `vercel`, `google`, `cursor`, `bedrock`, and `awsBedrock`; environment spellings still map to canonical keys. - Updated smoke tests, integration tests, Deno/browser coverage, and e2e scenarios to use only the curated public surface; added explicit tests for the allowed root types and published package entrypoints.
Integrate main through 5353f61 while preserving the v4 public API reductions, explicit public types, engine requirement, and removal of legacy CLI, plugin, flush, Flue, and Eve interfaces. Port provider updates into the internal instrumentation consumers, retain the new OpenAI Batch helpers, and reconcile dataset and OTEL changes with the v4 APIs. Keep Eve 0.34 pinned coverage and update the latest provider lane with tracePolicy support and matching replay fixtures.
Ref [SDK-226](https://linear.app/braintrustdata/issue/SDK-226/reduce-zod-derived-types-in-public-typescript-sdk-declarations) - Replace broad, backend-generated SDK types with smaller, explicit contracts, reducing the type complexity users’ editors and TypeScript compilers must process. - Bundle only the runtime validators the SDK needs. Previously, importing generated schemas also pulled unrelated backend validators into client bundles. - Simplify prompt and function types and remove internal cache-serialization methods from the public API, while preserving supported inputs and autocomplete. - Keep generated definitions as compatibility references, with tests checking type compatibility, validation behavior, autocomplete, and their exclusion from production bundles. All 1,536 replay tests pass.
- Installing the JavaScript SDK previously also installed the `bt` CLI. Remove its launcher, binary dependencies, and automatic downloader so the SDK installs independently. - Update documentation to explain separate CLI installation and use `bt` directly in examples. - Verify package contents, build, and tracing and evaluation replay tests, including successful delivery to Braintrust. Ref https://linear.app/braintrustdata/issue/SDK-321
Contributor
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If this PR is still relevant, please leave a comment, push an update, or remove the stale label. Thank you for your contributions! |
chore: Merge main into v4
…reAttachments` logger option for all integrations (#2519) Ref [#1841](#1841) Targets `v4` (#2406). - Make automatic attachment capture opt-in across all integrations using the existing `BRAINTRUST_CAPTURE_ATTACHMENTS` environment variable; previously, integrations applied this setting inconsistently. - Add `initLogger({ captureAttachments: true | false })` to override the environment setting. Loggers created with `setCurrent: false` keep their own policy, independent of the global logger. - When capture is disabled, omit inline media and skip unnecessary decoding, copying, and buffering. Preserve text, metadata, metrics, remote URLs, and explicitly logged attachments. - Keep the owning logger’s policy throughout asynchronous calls and streaming responses, with unit, integration, and replay tests covering these behaviors. Validation: build, type checks, lint, and 628 focused unit tests pass. The affected replay e2e scenarios pass across three runs; production forwarding and generated Braintrust links were verified. The full replay suite also hits existing v4 failures in the Git timestamp fixture and HarnessAgent fixture; both were reproduced on an untouched v4 checkout.
) I fucked up with the vite plus change because we made it more treeshakable but completely fumbled because people had to add an import for auto instrumentation when using bundlers. We now make it so that the bundlers inject the import statement directly. - Bundler integrations now load `braintrust/auto` automatically. Previously, they added tracing hooks but required a separate import to actually collect traces. - Apply this to both standard instrumentation and provider-specific patches, preserving strict mode and source maps.
Co-authored-by: Raphael Fakhri <153192858+RaphaelFakhri@users.noreply.github.com> Co-authored-by: Andrew Kent <31153155+realark@users.noreply.github.com> Co-authored-by: Abhijeet Prasad <abhijeet@braintrustdata.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: alexcastgonz <82875882+pcbeingused333@users.noreply.github.com> Co-authored-by: theanecdote <sungeun.jp@icloud.com> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: David Elner <david.elner@braintrustdata.com>
Merge `main` at `734e9585` into `v4`, with conflicts resolved against the v4 instrumentation, platform, and Vite+ refactors. MCP tracing and LiveKit audio configuration use v4's explicit registration paths, and the Eve and Pi fixes keep the v4 attachment-capture behavior. Validation: - Workspace build and type checks, lint, dead-code checks, and changeset validation pass. - Focused SDK and audio unit tests pass. - All 498 tests across the seven affected e2e scenarios pass in three consecutive cassette replay runs. - E2e helper tests pass three times; production forwarding and generated trace-summary links are verified against traces in `tmp-luca`. Use a merge commit to preserve the shared history with `main`. `v4` currently requires linear history, so that rule must be disabled for this merge while keeping the required approval and CI checks enabled. --------- Co-authored-by: alexcastgonz <82875882+pcbeingused333@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: David Elner <david.elner@braintrustdata.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: starfolkai[bot] <259249677+starfolkai[bot]@users.noreply.github.com> Co-authored-by: Abhijeet Prasad <abhijeet@braintrustdata.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.