-
Notifications
You must be signed in to change notification settings - Fork 0
docs: add experimentation guides and Coding Agent Mode workflows #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cosmic-flood
wants to merge
2
commits into
main
Choose a base branch
from
new-experimentation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file added
BIN
+124 KB
content/docs/experimentation/assets/experimentations/bayesian-analysis-results.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+105 KB
...nt/docs/experimentation/assets/experimentations/coding-agent-recommendation.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+62.8 KB
content/docs/experimentation/assets/experimentations/coding-agent-setup.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+59.8 KB
...ocs/experimentation/assets/experimentations/configure-primary-and-guardrail.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+95.3 KB
content/docs/experimentation/assets/experimentations/copy-coding-agent-prompt.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+67.3 KB
content/docs/experimentation/assets/experimentations/create-bayesian-run.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+54.7 KB
...ent/docs/experimentation/assets/experimentations/create-checkout-experiment.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+53.9 KB
...ocs/experimentation/assets/experimentations/create-simplified-checkout-flag.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+107 KB
content/docs/experimentation/assets/experimentations/decision-and-learning.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+86.3 KB
content/docs/experimentation/assets/experimentations/flag-targeting-50-50.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+108 KB
content/docs/experimentation/assets/experimentations/intent-and-hypothesis.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+59.4 KB
content/docs/experimentation/assets/experimentations/observation-window.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+102 KB
...ent/docs/experimentation/assets/experimentations/saved-exposure-and-metrics.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+64.2 KB
content/docs/experimentation/assets/layers/layer-after-window-ends.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+47.3 KB
content/docs/experimentation/assets/metrics/create-checkout-metric.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+49.6 KB
content/docs/experimentation/assets/metrics/numeric-aggregation-options.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+68.7 KB
content/docs/experimentation/assets/metrics/saved-checkout-metrics.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| --- | ||
| title: Layers | ||
| description: Create a Layer, assign a Run to a bucket range, and understand current traffic reservations. | ||
| --- | ||
|
|
||
| import { ImageZoom } from 'fumadocs-ui/components/image-zoom'; | ||
| import createCheckoutLayer from './assets/layers/create-checkout-layer.jpg'; | ||
| import editCheckoutLayer from './assets/layers/edit-checkout-layer.jpg'; | ||
| import runLayerAssignment from './assets/layers/run-layer-assignment.jpg'; | ||
| import layerRunAllocation from './assets/layers/layer-run-allocation.jpg'; | ||
| import layerAfterWindowEnds from './assets/layers/layer-after-window-ends.jpg'; | ||
|
|
||
| ## Overview | ||
|
|
||
| A Layer is a shared bucket space used to decide which users are eligible for a Run's analysis. Experiments can use separate ranges in that space to coordinate mutually exclusive analysis populations. | ||
|
|
||
| This tutorial creates **Checkout experience** with key `checkout-experience`, then assigns **Simplify checkout / run-1** to the first half of the Layer. It uses the same **First Project / Dev** environment as [Metrics](./metrics.mdx) and [Experiments](./experimentations.mdx). | ||
|
|
||
| You can create the Layer before the experiment exists. To complete the association steps, prepare the experiment and its Bayesian Run using [Create a Bayesian Run](./experimentations.mdx#create-a-bayesian-run). | ||
|
|
||
| ## Create the Layer | ||
|
|
||
| 1. Confirm the project and environment in the header. | ||
| 2. Open **Layers** under **Release Decision** and click **New layer**. | ||
| 3. Enter **Name**: `Checkout experience` and **Key**: `checkout-experience`. | ||
| 4. Check **Assignment unit**. In the current UI it is read-only and fixed to `user.keyId`. | ||
| 5. Enter **Description**: `Shared analysis space for checkout experiments.` | ||
| 6. Click **Create layer**. | ||
|
|
||
| <ImageZoom src={createCheckoutLayer} alt="New Checkout experience Layer with checkout-experience key and fixed user.keyId assignment unit" className="rounded-lg" /> | ||
|
|
||
| The Layer appears as **Active**. Until a Run reserves a range during a current observation window, the allocation bar shows **No allocation**. | ||
|
|
||
| **Assignment unit** identifies the user consistently for bucketing. In the .NET SDK example, `FbUser.Builder("customer-1234")` supplies the user key that corresponds to `user.keyId`. Reuse that ID for exposure and outcome events. All Runs in the Layer use the same assignment unit. | ||
|
|
||
| ## Edit and verify the Layer | ||
|
|
||
| 1. In the Layer row, click **Edit**. | ||
| 2. Confirm the saved name and assignment unit. The key cannot be changed after creation. | ||
| 3. Change **Description** to `Shared analysis space for checkout experiments, assigned by stable user ID.` | ||
| 4. Click **Save changes**, wait for the editor to close, and refresh. | ||
| 5. Reopen **Edit** to confirm the description persisted, then close it with **Cancel**. | ||
|
|
||
| <ImageZoom src={editCheckoutLayer} alt="Reopened Layer editor showing its saved description and read-only key and assignment unit" className="rounded-lg" /> | ||
|
|
||
| Changing this description does not allocate traffic. The bucket range belongs to a Run. | ||
|
|
||
| ## Associate a Run with the Layer | ||
|
|
||
| 1. Open **Experiments → Simplify checkout → Measuring** and select `run-1`. | ||
| 2. Under **Experiment traffic assignment**, click **Edit assignment**. Creating the Run alone does not save a Layer association. | ||
| 3. Confirm the variation roles: **Control** is the flag variation named **Control** (`false`), and the checked **Treatment** is the variation named **Treatment** (`true`). | ||
| 4. Under **Layer eligibility**, select **Checkout experience (checkout-experience)**. | ||
| 5. Keep **Assignment unit** as `user.keyId`. Set **Bucket start** to `0` and **Bucket end** to `50`. | ||
| 6. Under **Analysis sampling**, keep **Control** and **Treatment** at `100%`. Leave **Audience filters** empty for this example. | ||
| 7. Click **Save changes**. Refresh the experiment and reopen **Edit assignment** to verify the saved values. | ||
|
|
||
| <ImageZoom src={runLayerAssignment} alt="Run assignment with Checkout experience, bucket range 0 to 50, and 100 percent analysis sampling for both variants" className="rounded-lg" /> | ||
|
|
||
| The saved summary shows **Active range 0%–50% · Width 50%**. The range includes its start and excludes its end: `[0, 50)`. | ||
|
|
||
| ## Distinguish eligibility, serving, and sampling | ||
|
|
||
| These settings answer different questions: | ||
|
|
||
| | Setting | What it controls | This example | | ||
| | --- | --- | --- | | ||
| | Layer range | Which users are eligible for the Run's analysis, based on stable bucketing. | First 50% of the `checkout-experience` Layer. | | ||
| | Feature flag variation rollout | Which variation the application actually serves when it evaluates the flag. | `simplified-checkout` serves Control and Treatment at 50/50. | | ||
| | Analysis sampling | How much of each eligible variation's traffic is retained for analysis. | 100% for both variants. | | ||
|
|
||
| Assigning a Run to a Layer does not change the flag's rollout. Users outside this Run's Layer range can still receive either flag variation; their data is excluded from this Run's analysis. Do not prefilter SDK users yourself to imitate the Layer. | ||
|
|
||
| For example, this walkthrough evaluated the flag for **4,000 demonstration users**. The analysis included **1,982** users after Layer eligibility: **994 Control** and **988 Treatment**. A 50% range does not guarantee exactly half of a finite user population or identical group sizes. | ||
|
|
||
| ## Check current reservations | ||
|
|
||
| 1. Return to **Layers** and search for `checkout-experience` using **Filter by name or key**. | ||
| 2. Check **Experiment runs** for **Simplify checkout**, `run-1`, and `0–50%`. | ||
| 3. While the Run's observation window covers the current time, inspect the allocation bar and **Allocation status**. | ||
|
|
||
| While `run-1` was collecting events with **No fixed end**, the Layer showed **50% reserved · 50% free** and **No conflicts**: | ||
|
|
||
| <ImageZoom src={layerRunAllocation} alt="Checkout experience while run-1 is ongoing, showing its 0 to 50 percent reservation and No conflicts" className="rounded-lg" /> | ||
|
|
||
| Interpret the labels as follows: | ||
|
|
||
| - **Reserved** is the part of the Layer occupied by Runs whose observation windows cover the current time. | ||
| - **Free** is the remaining unreserved space. | ||
| - **No conflicts** is the allocation status reported for this configuration. Here, only the tutorial Run is associated with this Layer. | ||
| - **No allocation** means no range is currently reserved; it does not mean all historical Run associations were removed. | ||
|
|
||
| For additional experiments, choose non-overlapping ranges and inspect **Allocation status** before using them. This walkthrough does not create a conflicting Run. | ||
|
|
||
| ## Understand what happens when the window ends | ||
|
|
||
| After event collection, set an end time as shown in [Experiments](./experimentations.mdx#finish-the-demonstration-window). Once the observation window has ended: | ||
|
|
||
| 1. Refresh **Layers** and search for `checkout-experience` again. | ||
| 2. Confirm that **Simplify checkout / run-1** remains associated with `0–50%`. | ||
| 3. Confirm that the current allocation bar now shows **No allocation**, **0% reserved · 100% free**, and **No conflicts**. | ||
|
|
||
| <ImageZoom src={layerAfterWindowEnds} alt="Checkout experience after the observation window ends, with no current allocation and its Run association preserved" className="rounded-lg" /> | ||
|
|
||
| The finished Run still uses its saved Layer range when analyzing events in its observation window. The allocation bar describes current reservations, not the number of users previously analyzed. Ending the window also does not turn off the feature flag. | ||
|
|
||
| ## Next step | ||
|
|
||
| Continue with [collecting and analyzing experiment data](./experimentations.mdx#collect-events-and-analyze). Keep the Layer range, flag rollout, and analysis sampling settings consistent throughout the observation window. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "title": "Experimentation", | ||
| "pages": [ | ||
| "metrics", | ||
| "layers", | ||
| "experimentations", | ||
| "native-data-warehouse" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| --- | ||
| title: Metrics | ||
| description: Create reusable metrics, find their experiment runs, and report checkout events with SDKs and APIs. | ||
| --- | ||
|
|
||
| import { ImageZoom } from 'fumadocs-ui/components/image-zoom'; | ||
| import createCheckoutMetric from './assets/metrics/create-checkout-metric.jpg'; | ||
| import editCheckoutMetric from './assets/metrics/edit-checkout-metric.jpg'; | ||
| import numericAggregationOptions from './assets/metrics/numeric-aggregation-options.jpg'; | ||
| import savedCheckoutMetrics from './assets/metrics/saved-checkout-metrics.jpg'; | ||
| import filterByMetricKey from './assets/metrics/filter-by-metric-key.jpg'; | ||
| import filterByExperiment from './assets/metrics/filter-by-experiment.jpg'; | ||
|
|
||
| ## Overview | ||
|
|
||
| A metric defines the behavior or numeric value you want to measure. Experiments reference these reusable definitions as a **Primary** metric or a **Guardrail**. Creating a definition does not start collecting events: your application must also report the matching metric key. | ||
|
|
||
| This tutorial creates two metrics for a checkout experiment: | ||
|
|
||
| | Name | Key | Type | Aggregation | Purpose | | ||
| | --- | --- | --- | --- | --- | | ||
| | Checkout completed | `checkout-completed` | Binary conversion | Once per user | Measure whether an exposed user completes checkout. | | ||
| | Checkout errors | `checkout-errors` | Numeric value | Count all | Measure how many checkout errors an exposed user encounters. | | ||
|
|
||
| You need access to **Metrics** in a project environment. The screenshots use **First Project / Dev**. Keep the same environment when creating the feature flag, metrics, Layer, and experiment. | ||
|
|
||
| ## Create a conversion metric | ||
|
|
||
| 1. Confirm the project and environment in the page header, then open **Metrics** under **Release Decision**. | ||
| 2. Click **New metric**. | ||
| 3. Enter **Name**: `Checkout completed` and **Key**: `checkout-completed`. The key is the event name your application will send. | ||
| 4. Choose **Type**: **Binary conversion**. **Aggregation** is fixed to **Once per user** for this type. | ||
| 5. Enter **Description**: `Users who complete checkout.` | ||
| 6. Click **Create metric**. | ||
|
|
||
| <ImageZoom src={createCheckoutMetric} alt="New metric form with Checkout completed, the checkout-completed key, Binary conversion, and Once per user" className="rounded-lg" /> | ||
|
|
||
| The new metric appears as **Active** in the list. Before an experiment references it, **Experiment runs** shows **No experiment runs**. | ||
|
|
||
| ## Edit and verify the definition | ||
|
|
||
| 1. Click **Edit** in the **Checkout completed** row. | ||
| 2. Confirm that the saved name, type, and aggregation are correct. The **Key** is read-only after creation. | ||
| 3. Change **Description** to `Users who complete checkout after seeing the checkout experience.` | ||
| 4. Click **Save changes** and wait for the editor to close. | ||
| 5. Refresh the page and reopen **Edit**. Confirm that the new description remains. | ||
|
|
||
| <ImageZoom src={editCheckoutMetric} alt="Reopened Checkout completed editor showing its saved description and immutable metric key" className="rounded-lg" /> | ||
|
|
||
| Use **Cancel** to close the editor without further changes. Keep an event key's meaning consistent while collecting experiment data. | ||
|
|
||
| ## Add the numeric guardrail | ||
|
|
||
| 1. Click **New metric** again. | ||
| 2. Enter **Name**: `Checkout errors` and **Key**: `checkout-errors`. | ||
| 3. Choose **Type**: **Numeric value** and **Aggregation**: **Count all**. | ||
| 4. Enter **Description**: `Number of checkout errors after exposure.` | ||
| 5. Click **Create metric**, then refresh the list to verify both definitions. | ||
|
|
||
| Numeric metrics expose the following aggregation choices: | ||
|
|
||
| <ImageZoom src={numericAggregationOptions} alt="Numeric value metric form with Once per user, Count all, Sum values, and Average values in the aggregation menu" className="rounded-lg" /> | ||
|
|
||
| <ImageZoom src={savedCheckoutMetrics} alt="Saved Checkout completed and Checkout errors definitions in the Metrics list" className="rounded-lg" /> | ||
|
|
||
| ## Choose the right type and aggregation | ||
|
|
||
| Analysis first calculates each eligible exposed user's contribution, then compares the variants. An exposed user with no matching metric event contributes zero. | ||
|
|
||
| | Type | Supported aggregation | What one user contributes | Example | | ||
| | --- | --- | --- | --- | | ||
| | Binary conversion | Once per user only | 1 if at least one matching event occurred; otherwise 0. | Did the user complete checkout? | | ||
| | Numeric value | Once per user | 1 if at least one matching event occurred; otherwise 0. It does not take the first event's numeric value. | Did the user encounter any checkout error? | | ||
| | Numeric value | Count all | The number of matching events. | Three error events contribute 3. | | ||
| | Numeric value | Sum values | The sum of the events' numeric values. | Orders worth 10 and 20 contribute 30. | | ||
| | Numeric value | Average values | The mean of that user's event values. | Checkout durations of 10 and 20 seconds contribute 15 seconds. | | ||
|
|
||
| **Average values** gives each exposed user a contribution; it is not a pooled average of all events. For example, if one user reports values 10 and 20 and a second exposed user reports no events, their contributions are 15 and 0. The variant's mean is 7.5. | ||
|
|
||
| For **Checkout errors**, send one event for each actual error. Sending `checkout-errors` with a value of `0` still creates an event and therefore increases **Count all**. If no error occurred, do not send that error event. | ||
|
|
||
| ## Find metrics and their experiment runs | ||
|
|
||
| 1. In **Metrics**, use **Filter by name, key, or experiment**. Enter `checkout-errors` to find the guardrail by its key. | ||
| 2. Clear the field to restore the list. You can also search by metric name, such as `Checkout completed`. | ||
| 3. Complete the metric binding and Run creation in [Experiments](./experimentations.mdx#bind-the-flag-and-metrics). | ||
| 4. Return to **Metrics** and enter `Simplify checkout` in the filter. | ||
| 5. Check **Experiment runs**: **Checkout completed** is **Primary**, **Checkout errors** is **Guardrail**, and both reference `run-1`. | ||
|
|
||
| <ImageZoom src={filterByMetricKey} alt="Metrics filtered by the checkout-errors key, showing the guardrail's run-1 association" className="rounded-lg" /> | ||
|
|
||
| <ImageZoom src={filterByExperiment} alt="Metrics filtered by Simplify checkout, with Primary and Guardrail associations to run-1" className="rounded-lg" /> | ||
|
|
||
| Associations appear after they have been saved in the experiment. A matching name alone does not establish a relationship. | ||
|
|
||
| ## Set direction in the experiment | ||
|
|
||
| The definition and its experiment bindings have different jobs: | ||
|
|
||
| | Setting | Where to edit it | Checkout example | | ||
| | --- | --- | --- | | ||
| | Name, key, description, type, aggregation | **Metrics** | `checkout-completed`, Binary conversion, Once per user | | ||
| | Primary metric improvement direction | Experiment **Exposure → Edit metrics → Direction** | **Higher is better** for checkout completion | | ||
| | Guardrail warning direction | Experiment **Exposure → Edit metrics → Alert if** | **Increases** for checkout errors; the saved binding shows **Increase is bad** | | ||
|
|
||
| For a guardrail that should increase, **Alert if → Decreases** produces **Decrease is bad**. Choose the direction for that experiment; editing a shared metric definition is not how you change one experiment's direction. | ||
|
|
||
| ## Report metric events | ||
|
|
||
| All FeatBit SDKs can report metric events. You can also use APIs such as the [Track Insights API](../api-guides/track-insights-api.md) to send events directly. See the [SDK overview](../sdk/overview.md) for the available SDKs. | ||
|
|
||
| Here is a [.NET example](https://github.com/featbit/featbit-dotnet-sdk) using an initialized `client`. The `orderCompleted` and `checkoutError` conditions represent outcomes from your checkout flow. | ||
|
|
||
| ```csharp | ||
| var user = FbUser.Builder("customer-1234").Build(); | ||
|
|
||
| // Evaluate the flag when the user enters checkout. | ||
| var simplifiedCheckout = client.BoolVariation("simplified-checkout", user, false); | ||
|
|
||
| // When the order is completed: | ||
| if (orderCompleted) | ||
| client.Track(user, "checkout-completed"); | ||
|
|
||
| // For each checkout error: | ||
| if (checkoutError) | ||
| client.Track(user, "checkout-errors", 1.0); | ||
| ``` | ||
|
|
||
| Use the metric keys defined above, the same user ID and environment for evaluation and events, and timestamps within the Run's observation window. After events are processed, click **Analyze latest data** in the Run to see the results. | ||
|
|
||
| ## Next step | ||
|
|
||
| Create the [Checkout experience Layer](./layers.mdx), then use these two metrics in the [complete experiment walkthrough](./experimentations.mdx). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| --- | ||
| title: Native Data Warehouse | ||
| description: Contact FeatBit about using your own data source for experimentation. | ||
| --- | ||
|
|
||
| If you would like to use your own data source instead of having FeatBit record experiment data, contact FeatBit. FeatBit will provide a solution based on your requirements. | ||
|
|
||
| Email [contact@featbit.co](mailto:contact@featbit.co), or open the [FeatBit website](https://www.featbit.co/) and choose **Support** to find the available contact channels. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🛡️ Analyzed with Security Review | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
CVE-2026-50016 pnpm 10.33.4 10.34.0 affected fixed version💡 Result:
<search_synthesis>
CVE-2026-50016 is a security vulnerability in the pnpm package manager that allows for path traversal via transitive dependency aliases [1][2]. Affected versions include: - All versions prior to 10.34.0 [1][2] - Versions 11.0.0 and above, but prior to 11.4.0 [1][2] The vulnerability is fixed in pnpm version 10.34.0 and version 11.4.0 [1][2]. This flaw allows a malicious registry package to include specially crafted dependency aliases containing path traversal segments [2][3]. During the
pnpm installprocess, these aliases can cause pnpm to replace project paths with symbolic links (symlinks) pointing to attacker-controlled directories [1][2]. This can occur even when runningpnpm install --ignore-scripts, posing a risk to project file integrity and potentially enabling arbitrary code execution [1][2][3].</search_synthesis>
<source_evidence>
Citations:
🌐 Web query:
site:github.com/pnpm/pnpm/security/advisories/GHSA-hwx4-2j3j-g496 affected versions 10.33.4 10.34.0💡 Result:
<search_synthesis>
The versions 10.33.4 and 10.34.0 are associated with a security advisory for pnpm regarding a transitive dependency alias path traversal vulnerability (GHSA-hwx4-2j3j-g496) [1]. According to the advisory, the vulnerability affects pnpm versions prior to 10.34.0 [1]. Version 10.34.0 is listed as a patched version, meaning it addresses the security issue present in earlier versions, including 10.33.4 [1]. Users are advised to upgrade to a patched version to mitigate this risk [1].
</search_synthesis>
<source_evidence>
Citations:
Path Traversal
Reachability: External
Exploitability: Moderate
CWE: CWE-23
Update pnpm to a patched version. pnpm 10.33.4 is affected by CVE-2026-50016. Use pnpm 10.34.0 or a patched 11.x release starting at 11.4.0. Update
package.jsonand both README installation commands.🤖 Prompt for AI Agents