Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/actions/setup-hugo/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
name: 'Setup Hugo with caches'
description: 'Install Hugo + Bun, cache dependencies and build artifacts'
inputs:
hugo-version:
description: 'Hugo version'
default: '0.149.1'
environment:
description: 'Hugo build environment'
default: 'development'
runs:
using: 'composite'
steps:
- uses: peaceiris/actions-hugo@v3
with:
hugo-version: ${{ inputs.hugo-version }}
extended: true

- uses: oven-sh/setup-bun@v2

- name: Cache Bun dependencies
uses: actions/cache@v5
with:
path: |
~/.bun/install/cache
node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lockb', '**/package.json') }}
restore-keys: ${{ runner.os }}-bun-

- run: bun install --frozen-lockfile
shell: bash

- name: Cache Hugo build
uses: actions/cache@v5
with:
path: |
/tmp/hugo_cache
resources/_gen
_dest
key: ${{ runner.os }}-hugo-${{ inputs.environment }}-${{ hashFiles('hugo.toml', 'hugo.dev.toml', 'postcss.config.js', 'package.json', '**/bun.lockb', 'themes/**', 'layouts/**', 'content/**') }}
restore-keys: |
${{ runner.os }}-hugo-${{ inputs.environment }}-
${{ runner.os }}-hugo-

- name: Build Hugo site
run: PATH="./node_modules/.bin:$PATH" bin/hugo-build
shell: bash
env:
HUGO_CACHEDIR: /tmp/hugo_cache
ENVIRONMENT: ${{ inputs.environment }}
2 changes: 1 addition & 1 deletion .github/workflows/sync-and-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: 3.4.2
ruby-version: '4.0'
bundler-cache: true

- name: Run sync script
Expand Down
24 changes: 0 additions & 24 deletions .github/workflows/test-unit.yml

This file was deleted.

88 changes: 88 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
name: Tests

on:
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
inputs:
screenshots:
description: 'Run screenshot tests (slow, requires Hugo build)'
type: boolean
default: false
update-baselines:
description: 'Re-record screenshot baselines and commit'
type: boolean
default: false

permissions:
contents: write
pull-requests: write

concurrency:
group: tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
unit:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v6

- uses: ruby/setup-ruby@v1
with:
ruby-version: '4.0'
bundler-cache: true

- uses: ./.github/actions/setup-hugo

- run: bundle exec rake test:unit

screenshots:
name: Screenshot Tests
if: ${{ inputs.screenshots || inputs.update-baselines }}
runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.head_ref || github.ref }}

- uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/test.yml | head -50

Repository: jetthoughts/jetthoughts.github.io

Length of output: 1369


🏁 Script executed:

grep -n "@" .github/workflows/test.yml | grep "uses:"

Repository: jetthoughts/jetthoughts.github.io

Length of output: 298


Pin external action refs to immutable commit SHAs instead of @master.

Lines 26 and 40 reference moving branches which can break reproducibility and increase supply-chain risk. Replace @master with specific commit SHAs for both setup-ruby-and-dependencies and upload-screenshots actions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/test.yml at line 26, The workflow uses mutable refs for
external actions; update both occurrences of
snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
and snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master to
use immutable commit SHAs: locate the two action usages in the workflow and
replace the `@master` tag with the exact commit SHA for each action (obtain the
SHA from the corresponding repository commit you trust), commit the updated
workflow, and verify the workflow runs successfully.

with:
ruby-version: '4.0'
cache-apt-packages: true

- uses: ./.github/actions/setup-hugo

- name: Run screenshot tests
if: ${{ !inputs.update-baselines }}
run: bundle exec rake test
env:
SCREENSHOT_DRIVER: vips

- name: Record baselines
if: ${{ inputs.update-baselines }}
run: FORCE_SCREENSHOT_UPDATE=true bundle exec rake test
env:
SCREENSHOT_DRIVER: vips

- name: Commit updated baselines
if: ${{ inputs.update-baselines }}
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add test/fixtures/screenshots/
git diff --staged --quiet || git commit -m "chore: update screenshot baselines [ci skip]"
git push

- name: Upload screenshot report
if: failure()
uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
with:
name: screenshots
report-path: test/fixtures/screenshots
pr-comment: 'true'
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ hugo.linux
*.diff.png
*.base.png
*.attempt_*.png
snap_diff_report.html

# Capybara
capybara-*.png
Expand Down
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ gem "ostruct"

gem "sanity-ruby"

gem "capybara-screenshot-diff", github: "donv/capybara-screenshot-diff"
gem "capybara-screenshot-diff", github: "snap-diff/snap_diff-capybara"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
gem "simplecov", require: false
gem "vips"

Expand Down
6 changes: 3 additions & 3 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
GIT
remote: https://github.com/donv/capybara-screenshot-diff.git
revision: 62419f7432afa4e0e06cb9e099bc386f92335bb2
remote: https://github.com/snap-diff/snap_diff-capybara.git
revision: 4070438ec9579be86faf8a5580783889fbc8431f
specs:
capybara-screenshot-diff (1.11.0)
capybara-screenshot-diff (1.12.0)
capybara (>= 2, < 4)

GEM
Expand Down
6 changes: 6 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ namespace :test do
t.pattern = "test/**/*_test.rb"
end

Rake::TestTask.new(:unit) do |t|
t.libs << "test"
t.libs << "lib"
t.pattern = "test/unit/**/*_test.rb"
end

Rake::TestTask.new(:critical) do |t|
t.libs << "test"
t.libs << "lib"
Expand Down
Binary file modified test/fixtures/screenshots/linux/desktop/404.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/about_page/_footer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/about_page/_values.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/about_us.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/blog/index.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/blog/post.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/careers.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/careers/_footer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/careers/_offers.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/clients.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/clients/_clients.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/clients/_footer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/fixtures/screenshots/linux/desktop/contact_us.png
Binary file modified test/fixtures/screenshots/linux/desktop/homepage.png
Binary file modified test/fixtures/screenshots/linux/desktop/homepage/_clients.png
Binary file modified test/fixtures/screenshots/linux/desktop/homepage/_footer.png
Binary file modified test/fixtures/screenshots/linux/desktop/homepage/_services.png
Binary file modified test/fixtures/screenshots/linux/desktop/nav/services.png
Binary file modified test/fixtures/screenshots/linux/desktop/nav/use_cases.png
Binary file modified test/fixtures/screenshots/linux/desktop/privacy-policy.png
Binary file modified test/fixtures/screenshots/linux/desktop/services/_footer.png
Binary file modified test/fixtures/screenshots/linux/desktop/services/_overview.png
Binary file modified test/fixtures/screenshots/linux/desktop/services/_services.png
Binary file modified test/fixtures/screenshots/linux/desktop/services/_use-cases.png
Binary file modified test/fixtures/screenshots/linux/desktop/use-cases/_footer.png
Binary file modified test/fixtures/screenshots/linux/desktop/use-cases/_overview.png
Binary file modified test/fixtures/screenshots/linux/mobile/404.png
Binary file modified test/fixtures/screenshots/linux/mobile/about_page/values.png
Binary file modified test/fixtures/screenshots/linux/mobile/about_us.png
Binary file modified test/fixtures/screenshots/linux/mobile/blog/index.png
Binary file modified test/fixtures/screenshots/linux/mobile/blog/post.png
Binary file modified test/fixtures/screenshots/linux/mobile/careers.png
Binary file modified test/fixtures/screenshots/linux/mobile/clients.png
Binary file modified test/fixtures/screenshots/linux/mobile/contact_us.png
Binary file modified test/fixtures/screenshots/linux/mobile/free_consultation.png
Binary file modified test/fixtures/screenshots/linux/mobile/homepage.png
Binary file modified test/fixtures/screenshots/linux/mobile/nav/hamburger_menu.png
2 changes: 1 addition & 1 deletion test/support/setup_capybara.rb
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,6 @@ def register_chrome_driver(name, device_metrics)
Capybara.current_driver = Capybara.javascript_driver
Capybara.disable_animation = true
Capybara.threadsafe = false
Capybara.default_max_wait_time = 10
Capybara.default_max_wait_time = 5

Capybara.save_path = "tmp/capybara/"
70 changes: 7 additions & 63 deletions test/support/setup_snap_diff.rb
Original file line number Diff line number Diff line change
@@ -1,80 +1,24 @@
# Screenshot Testing Configuration
#
# Environment Variables for controlling screenshot behavior:
# - SCREENSHOT_TOLERANCE: Override global tolerance (default: 0.01 = 1%)
# - SCREENSHOT_STABILITY_TIME: Override stability wait time (default: 1.0s)
# - FORCE_SCREENSHOT_UPDATE: Set to 'true' to regenerate all baseline screenshots
# - CAPYBARA_SCREENSHOT_DIFF_FAIL_ON_DIFFERENCE: Set to 'false' to update baselines
#
# Usage Examples:
# SCREENSHOT_TOLERANCE=0.02 bundle exec rake test # 2% tolerance
# SCREENSHOT_STABILITY_TIME=2.0 bundle exec rake test # 2s stability time
# FORCE_SCREENSHOT_UPDATE=true bundle exec rake test # Regenerate baselines
# Environment Variables:
# - SCREENSHOT_STABILITY_TIME: Override stability wait time (default: 0.1s)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
# - FORCE_SCREENSHOT_UPDATE: Set to 'true' to regenerate all baselines
#

require "capybara_screenshot_diff/minitest"
require "capybara_screenshot_diff/reporters/html"

Capybara::Screenshot.save_path = "test/fixtures/screenshots"
Capybara::Screenshot.add_os_path = true
Capybara::Screenshot.window_size = nil

# Screenshot stability and tolerance settings
#
# stability_time_limit: Time to wait for the page to be stable before taking screenshot
# Increased from 0.25s to 1.0s to reduce flakiness from animations/loading states
Capybara::Screenshot.stability_time_limit = ENV.fetch("SCREENSHOT_STABILITY_TIME", "0.1").to_f

# tolerance: Allowed difference percentage between baseline and current screenshot
# Increased from 0.005 (0.5%) to 0.01 (1%) to handle cross-platform rendering differences
# across different environments, versions, and operating systems
Capybara::Screenshot::Diff.tolerance = ENV.fetch("SCREENSHOT_TOLERANCE", "0.005").to_f
Capybara::Screenshot.disable_animations = true
Capybara::Screenshot.root = Dir.pwd

Capybara::Screenshot::Diff.driver = :vips
Capybara::Screenshot::Diff.perceptual_threshold = 2.0
Comment thread
pftg marked this conversation as resolved.
Capybara::Screenshot::Diff.delayed = true
Capybara.default_max_wait_time = 5

# Add global setup hook to disable animations for all screenshot tests
Capybara.configure do |config|
# Ensure screenshots are taken with consistent rendering
config.default_set_options = {clear: :backspace}
end

# Add after_action callback to prepare pages for screenshots if needed
module ScreenshotPreparation
def self.disable_animations_globally(page)
page.execute_script(<<~JS) if page.driver.browser.respond_to?(:execute_script)
if (!document.querySelector('#screenshot-test-animations-disabled')) {
const style = document.createElement('style');
style.id = 'screenshot-test-animations-disabled';
style.innerHTML = `
*, *::before, *::after {
animation-duration: 0s !important;
animation-delay: 0s !important;
transition-duration: 0s !important;
transition-delay: 0s !important;
animation-play-state: paused !important;
-webkit-animation-duration: 0s !important;
-webkit-animation-delay: 0s !important;
-webkit-transition-duration: 0s !important;
-webkit-transition-delay: 0s !important;
}
`;
document.head.appendChild(style);
}
JS
end
end

# Environment variable controls for screenshot behavior
#
# CAPYBARA_SCREENSHOT_DIFF_FAIL_ON_DIFFERENCE=false: Useful for updating screenshot baselines
if ENV["CAPYBARA_SCREENSHOT_DIFF_FAIL_ON_DIFFERENCE"] == "false"
Capybara::Screenshot::Diff.fail_on_difference = false
end

# FORCE_SCREENSHOT_UPDATE=true: Regenerate all baseline screenshots
if ENV["FORCE_SCREENSHOT_UPDATE"] == "true"
Capybara::Screenshot::Diff.fail_on_difference = false
end

Capybara::Screenshot.root = Dir.pwd
13 changes: 6 additions & 7 deletions test/system/desktop_site_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ def test_homepage
# Move mouse to (0,0) to prevent menu hover state causing flakiness
page.driver.browser.action.move_to_location(0, 0).perform

# Wait for animations to complete
sleep 0.5
# Anchor on hero content before screenshot
assert_selector ".hero, [data-section=hero], main", wait: 2

assert_screenshot "homepage", tolerance: 0.03
end
Expand Down Expand Up @@ -57,7 +57,7 @@ def test_visit_blog_post
def test_blog_post
visit "/blog/red-flags-watch-for-in-big-pr-when-stop-split-or-rework-development-productivity/"

assert_stable_screenshot "blog/post", tolerance: 0.03, stability_time_limit: 5
assert_stable_screenshot "blog/post", tolerance: 0.03, stability_time_limit: 1
end

def test_about_us
Expand Down Expand Up @@ -128,7 +128,7 @@ def test_services
verify_section_for("services", "overview", css: '[data-node="i8x1zs2grf9h"]')
verify_section_for("services", "services", css: '[data-node="nhf6l2ycmzoe"]')
verify_section_for("services", "technologies")
verify_section_for("services", "use-cases")
verify_section_for("services", "use-cases", stability_time_limit: 1)
verify_section_for("services", "testimonials-header", css: '[data-node="1a4igunq3xvj"]')
verify_section_for("services", "cta-contact_us", css: '[data-node="ohd51ixf3842"]')
verify_section_for("services", "footer", css: "footer")
Expand Down Expand Up @@ -307,10 +307,9 @@ def test_about_page_sections

private

def verify_section_for(page_name, section_id, css: nil)
def verify_section_for(page_name, section_id, css: nil, **options)
scroll_to find(css || "##{section_id}")
# Ruby hash-based config handles tolerance automatically
assert_screenshot "#{page_name}/_#{section_id}"
assert_screenshot "#{page_name}/_#{section_id}", **options
end

def within_top_bar
Expand Down
Loading