Skip to content

fix(E2E): Lift the E2E organisation's plan limits on SaaS - #8918

Merged
khvn26 merged 2 commits into
mainfrom
fix/e2e-audit-log-saas
Nov 9, 2026
Merged

khvn26 merged 2 commits into
mainfrom
fix/e2e-audit-log-saas

Conversation

@khvn26

@khvn26 khvn26 commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

In this PR, we:

  1. Skip the test added in fix(Diff display): Segment rules diff showing up as [Object object] #8910 when running against SaaS.
  2. Fix the reason for the test to fail in the first place: a seeded enterprise org's subscription cache was not updated to reflect the lifted limits, which included the audit log visibility limit. This will allow us to unskip the test in a follow-up PR after the next release.

How did you test this code?

Ran the E2E Segment 5 test against SaaS to confirm green; ran the original test against a properly seeded organisation to confirm green.

On SaaS, plan limits such as audit log visibility come from the
subscription information cache, which the seed left on free plan
limits. The E2E organisation therefore saw no audit logs.

Upgrade the cache to enterprise limits when one exists.
Until the seed fix is deployed, the E2E organisation can't see audit
logs on SaaS, which blocks the production frontend deploy.
@vercel

vercel Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Building Building Preview Nov 9, 2026 5:12pm UTC
flagsmith-frontend-preview Ready Ready Preview Nov 9, 2026 5:12pm UTC
flagsmith-frontend-staging Ready Ready Preview Nov 9, 2026 5:12pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Explain Change →

Walkthrough

The seed data now upgrades the organisation’s subscription-information caches to Enterprise using the subscription’s seat and API-call limits. Unit tests check the SaaS cache values and verify that no cache is created without the SaaS marker. The frontend adds an API-based SaaS check and uses it to skip the audit-log segment test in SaaS mode.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: Low · up to 41913

The cache upgrade restores seeded audit visibility, but some deployment configurations can still fail the tests, and SaaS continues to skip the audit regression assertion. Merge risk is low, with these bounded test limitations requiring owner awareness or follow-up.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codecov

codecov Bot commented Nov 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.90%. Comparing base (a002217) to head (41913d9).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8918   +/-   ##
=======================================
  Coverage   98.90%   98.90%           
=======================================
  Files        1698     1699    +1     
  Lines       71517    71537   +20     
=======================================
+ Hits        70733    70753   +20     
  Misses        784      784           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@khvn26
khvn26 marked this pull request as ready for review November 9, 2026 17:18
@khvn26
khvn26 requested review from a team as code owners November 9, 2026 17:18
@khvn26
khvn26 requested review from emyller and talissoncosta and removed request for a team November 9, 2026 17:18
@github-actions github-actions Bot removed the fix label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-8918 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api-test:pr-8918 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api:pr-8918 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-private-cloud:pr-8918 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-frontend:pr-8918 Finished ✅ Results ✅

@github-actions github-actions Bot added the fix label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor
✅ private-cloud · depot-ubuntu-latest-16 — run #22978 (attempt 2)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  4 passed

Details

stats  4 tests across 4 suites
duration  42.3 seconds
commit  41913d9
info  🔄 Run: #22978 (attempt 2)

🗂️ Previous results
❌ private-cloud · depot-ubuntu-latest-16 — run #22978 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

failed  1 failed
passed  1 passed

Details

stats  2 tests across 2 suites
duration  1 minute, 8 seconds
commit  41913d9
info  📦 Artifacts: View test results and HTML report
🔄 Run: #22978 (attempt 1)

Failed tests

firefox › tests/project-permission-test.pw.ts › Project Permission Tests › Project-level permissions control access to features, environments, audit logs, and segments @enterprise

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #22978 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  37.7 seconds
commit  41913d9
info  🔄 Run: #22978 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #22978 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  35.7 seconds
commit  41913d9
info  🔄 Run: #22978 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #22978 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  31.9 seconds
commit  41913d9
info  🔄 Run: #22978 (attempt 1)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 0895202d-b866-4891-81e0-144d7b80c157
Commits

Reviewing files that changed from the base of the PR and between 13539af and 41913d9.

Files selected for processing: 5

  • api/e2etests/e2e_seed_data.py
  • api/tests/unit/e2etests/__init__.py
  • api/tests/unit/e2etests/test_unit_e2etests_seed_data.py
  • frontend/e2e/helpers/utils.playwright.ts
  • frontend/e2e/tests/segment-test.pw.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Comment thread frontend/e2e/tests/segment-test.pw.ts Resolved
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Visual Regression

24 screenshots compared. See report for details.
View full report

@khvn26
khvn26 merged commit ca7a393 into main Nov 9, 2026
70 of 71 checks passed
@khvn26
khvn26 deleted the fix/e2e-audit-log-saas branch November 9, 2026 17:53

This branch was successfully deployed

2 active deployments
Preview – flagsmith-frontend-staging — 41913d9f Deployed Nov 9, 2026 by vercel[bot]
Preview – flagsmith-frontend-preview — 41913d9f Deployed Nov 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants