Merge updates#1
Open
OhxSteve wants to merge 442 commits into
Open
Conversation
* Adds cucumber integration * Adds documentation how to use * Adds documentation how to use
* chore: add readme note about retina devices * chore: formatting * chore: wording
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
If a spec or test has a very long name or is nested deeply,
the temporary path name formed by `Tmpname` (which preserves the
directory names but removes the / separators) may exceed the 255
character filename limit present on many Linux filesystems.
This causes an exception like the one below when Vips tries to
write the temporary file. Since
1) temporary file is ultimately moved to the target path, which
doesn't exceed the length limit (since most of the length is in nested
directory strcuture) and
2) `Tmpname` has some logic to avoid name collisions anyway
shortening the `prefix` passed to `Tmpname` to stay well below 255
characters resolves this error.
```
1) email preferences with EMAIL_PREFERENCES doc preferences #edit should not display any doc preference link for non-imaging backend users
Failure/Error: raise Vips::Error
Vips::Error:
/tmp/homecircleciprojectspecscreenshotslinuxfeatures-email_notification_preferences_specemail_preferences_with_EMAIL_PREFERENCES_doc_preferences_edit_should_not_display_any_doc_preference_link_for_non-imaging_backend_users01_non_imaging_no_pref_link.committed.png20200101-2583-3jy9w5.png: unable to open for write
unix error: File name too long
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/ruby-vips-2.1.4/lib/vips/operation.rb:228:in `build'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/ruby-vips-2.1.4/lib/vips/operation.rb:483:in `call'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/ruby-vips-2.1.4/lib/vips/image.rb:592:in `write_to_file'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff/drivers/vips_driver.rb:112:in `block in save_image_to'
# /usr/local/lib/ruby/3.1.0/tmpdir.rb:144:in `create'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff/drivers/vips_driver.rb:111:in `save_image_to'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff/image_compare.rb:138:in `save'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff/image_compare.rb:134:in `annotate_and_save'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff/image_compare.rb:114:in `different?'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff/test_methods.rb:115:in `assert_image_not_changed'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff.rb:99:in `block in track_failures'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff.rb:98:in `map'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff.rb:98:in `track_failures'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/capybara-screenshot-diff-1.6.3/lib/capybara/screenshot/diff.rb:87:in `block in included'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:457:in `instance_exec'
# /home/circleci/project/vendor/bundle/ruby/3.1.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:457:in `instance_exec'
```
Verify that tests run after checkout. Allow running tests without VIPS installed. Retry test once since JRuby runs sometimes time out. --------- Co-authored-by: Uwe Kubosch <[email protected]>
…ence report. Fixes #84
* Auto-fix standardrb errors * Adds rails 7.1 to CI/CD * Drops unreachable code
…#88) This prevents creating a region that is one pixel too small due to rounding down.
The VCS checkout compares against committed baselines. Without committing first, the second run has nothing to compare against. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…t.html
Default output path changed from tmp/snap_diff/index.html to
{root}/{save_path}/snap_diff_report.html in PR #174. Updated all
docs and CI action to match.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Use Open3.capture3 instead of capture2 to capture stderr from git rev-parse, preventing noise when tests run in temp dirs. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* docs: add test/fixtures/images/README.md (T2) Document fixture images and generated artifacts for contributors. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat: add snap_diff:clean rake task for diff artifact pruning (P3.3) bin/rake snap_diff:clean removes diff artifacts (*.diff.png, *.heatmap.diff.png, snap_diff_report.html) from tmp/ and doc/screenshots/ while keeping baselines. Addresses tmp/ bloat — 262 files accumulated after a development session. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Overlapping globs (*.diff.png matches *.base.diff.png and *.heatmap.diff.png) produced duplicate paths, causing the "Removed N" count to over-report. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
- Opening line sells outcome ("Stop shipping UI bugs") with competitive
positioning vs Percy/Chromatic/BackstopJS
- Renamed "HTML Reporter" to "Web UI for Reviewing Screenshot Changes"
with GitHub Actions PR comment integration shown inline
- Promoted Web UI to its own section (was buried in 1 line)
- Replaced tuning table with single configure block
- Reframed Troubleshooting as "Common Questions"
- Added Diff.compare usage showing actual return API
- Folded ruby-vips into Quick Start Gemfile
Based on 3-reviewer brutal honesty review (Rails dev, DevOps, OSS maintainer).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Bug fix: - Composite action checked for `index.html` but gem generates `snap_diff_report.html` — report was never found in CI README improvements (from 3-reviewer round 2): - Add prerequisites note (Capybara + browser driver required) - Add non-Rails setup section (Hugo/Jekyll/static sites) - Convert Quick Start to numbered steps with directory tree output - Reformat FAQ with collapsible details for scannability - Add "Will this slow down my tests?" FAQ entry - Rename "Advanced Topics" to "Docs" (Framework Setup isn't advanced) - Remove Docker Testing from docs list (internal dev concern) - Fix Requirements to show non-Rails is supported Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add `#summary` method to HTML reporter that prints screenshot count and report path after every test run: - "3 screenshots compared, 1 failure. Report: doc/screenshots/snap_diff_report.html" - "5 screenshots compared, no failures." - Nothing printed when no screenshots were taken Follows the SimpleCov pattern: always show status so developers know the reporter is active and where to find results. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
- Fix double summary evaluation: assign to local variable (was called twice) - Remove respond_to? guard: only HTML reporters exist, fail fast if not - Always warn on reporter errors (was silenced unless DEBUG) - Remove dead success_rate method (zero callers) - Fix fragile test assertion: check output_path instead of string "report" Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: use Minitest.after_run for report summary, not at_exit Root cause: at_exit hooks run in LIFO order. Our at_exit was registered after minitest's autorun, so it fired before tests finished — reporter had total=0 because no assertions had been processed yet. Fix: Use Minitest.after_run (runs after all tests complete) when minitest is available, falling back to at_exit for RSpec/Cucumber. Also remove screenshots&.clear from verify_screenshots! ensure block — clearing now happens in registry.reset after notify_reporters, so assertions are available for the reporter before being cleared. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: keep base image when screenshots differ for reporter Previously, base_image_path was deleted after comparison when images differed. This meant the HTML reporter couldn't show the baseline image — it appeared as a broken "Base" alt text. The base image is a VCS checkout that gets re-created on next run, so keeping it has no side effects. The reporter now shows both baseline and current images side-by-side. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * docs: document .gitignore patterns for diff artifacts - Add .gitignore block to README Quick Start - Add .gitignore section to CI integration docs - Add generated artifacts table to organization docs - Add *.base.png to snap_diff:clean rake task - Add *.base.png to test fixtures README Files to gitignore: *.diff.png, *.base.png, *.diff.webp, *.base.webp, snap_diff_report.html. Only commit baseline screenshots. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: register both at_exit and Minitest.after_run hooks Always register at_exit (works for RSpec/Cucumber) AND additionally Minitest.after_run when Minitest is loaded (correct ordering for Minitest). The @Finalized guard in #finalize prevents double execution. Previously the else branch meant RSpec/Cucumber users got no at_exit hook when Minitest was also loaded in the same process. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: avoid double finalize call for Minitest Restore if/else — Minitest.after_run when Minitest is loaded, at_exit otherwise. Nobody runs Minitest and RSpec in the same process, so the @Finalized double-execution guard is unnecessary overhead. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: register framework-native hooks for RSpec and Cucumber - Minitest: Minitest.after_run (runs after all tests) - RSpec: RSpec.configure { |c| c.after(:suite) } (runs after suite) - Cucumber/other: at_exit fallback at_exit fires in LIFO order, so it runs BEFORE the test framework finishes when registered after it. Framework-native hooks guarantee correct ordering — reporter has data when finalize runs. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: move finalize_reporters! to framework adapters Extract finalize_reporters! into CapybaraScreenshotDiff module and call it from each framework's native "after suite" hook: - Minitest: Minitest.after_run - RSpec: config.after(:suite) - Cucumber: AfterAll html.rb no longer detects frameworks — it just registers at_exit as fallback for unknown frameworks. Each adapter owns its hook. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: skip at_exit when framework adapter handles finalization Minitest, RSpec, and Cucumber adapters call finalize_reporters! via native hooks. The at_exit fallback in html.rb was firing as a wasted no-op for those frameworks (hitting @Finalized guard every time). Now only registers at_exit when no known framework is loaded. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: adopt SimpleCov's external_at_exit pattern Framework adapters set external_at_exit = true and register native hooks (Minitest.after_run, RSpec after(:suite), Cucumber AfterAll). The at_exit block checks the flag and skips when a framework adapter handles finalization. This eliminates both the double-call problem and the LIFO ordering problem cleanly — same pattern SimpleCov uses successfully. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * docs: add custom framework finalize_reporters! and vips install notes - Document how to call finalize_reporters! for custom test frameworks - Add brew/apt install commands for libvips in Requirements Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: remove external_at_exit flag, rely on @Finalized guard The external_at_exit flag was overkill — @Finalized in #finalize already prevents double work. Simpler: at_exit always registers, framework adapters also register native hooks, @Finalized ensures only the first call does work. Also move custom framework docs from README to docs/framework-setup.md. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: remove at_exit hook entirely Framework adapters (Minitest, RSpec, Cucumber) call finalize_reporters! via native hooks. at_exit was unreliable due to LIFO ordering and added complexity for a case nobody hits. Custom frameworks can call finalize_reporters! manually (documented in docs/framework-setup.md). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Add screenshot of the SnapDiff Web UI report showing side-by-side baseline vs current comparison. This was the #1 finding across 6 rounds of brutal honesty review — a visual testing gem needs visuals in its README. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Version: - Change VERSION from 1.12.0 to 1.11.0 (next release after 1.10.3.1) - Fix CHANGELOG/UPGRADING references (Before v1.10.x, not v1.11.0) README engagement improvements (from research): - Add downloads badge for social proof - Remove DeepWiki badge (noise, not social proof) - Add "5 minutes" time estimate to Quick Start header - Reframe prerequisites as confidence-building, not gatekeeping - Reframe "Why this gem?" positively (what you GET, not what you AVOID) Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Fix: ruby -rlib/... doesn't work — use -I lib -r capybara/... instead. Bump version to 1.12.0 for release. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
"Review all visual changes in one place" — explains WHY developers should use the Web UI, not just WHAT it does. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Integration tests (browser rendering with Cuprite/Selenium) are covered by the test workflow's matrix jobs. The release workflow only needs to verify gem code correctness before publishing — unit tests are sufficient and don't have flaky browser baselines. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Use setup-ruby-and-dependencies composite action (same as test workflow) instead of bare ruby/setup-ruby. This ensures identical environment: - libvips installed - Font antialiasing disabled - Ruby 4.0 (not floating 'ruby' tag) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Move Web UI screenshot to the top — first thing developers see after the tagline. Clicks through to the Web UI section for details. Visual testing gem needs a visual first impression. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Hero (top): heatmap view — shows what changed at a glance Web UI section: annotated view — shows red rectangles marking diff regions Two different views of the same tool, no duplication. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Upstream awalsh128/cache-apt-pkgs-action@latest pins actions/cache@v4 (Node 20, deprecated). Use jetthoughts fork with actions/cache@v5 (Node 24) until upstream merges the fix. Upstream PR: awalsh128/cache-apt-pkgs-action#198 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Functional Test and Test Drivers install libvips via apt — on cache miss this can take 5+ minutes. Bump from 5 to 10 minutes. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Both Functional Test and Test Report Upload jobs now write artifact links to GITHUB_STEP_SUMMARY — visible in the Actions UI job page without needing a PR comment. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
setup-ruby-and-dependencies: - Remove unconditional apt-get update (only ran when not needed) - Split vips install: cache action when cache-apt-packages=true, direct apt-get when false (no redundant double-install) release.yml: add timeout-minutes: 10 (prevent hanging release) lint.yml: add timeout-minutes: 3 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…186) * refactor: ADR-006 T3+T4 — hoist setup, consolidate assertion helpers T3: Hoist duplicated setup (base_path, new_path, driver, comparison) from 3 nested classes to parent DifferenceFinderTest. Nested classes inherit setup and only add their own (@finder). T4: Extract assert_driver_check_called(driver, check_type, times) parameterized helper. Old methods kept as one-line aliases. T6: Skipped — param order already matches Minitest convention. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: remove extra empty line in InitializationTest (standardrb) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: move PR comment + job summary into upload-screenshots action Add pr-comment input to upload-screenshots composite action. When enabled, it finds/creates a PR comment with report links automatically. Also adds job summary with artifact links. Removes ~40 lines of duplicated YAML from test.yml. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * docs: update CI integration for new composite action API - Document pr-comment input for upload-screenshots action - Document setup-ruby-and-dependencies action with inputs table - Add full example combining both actions - Move manual setup and update-baselines to collapsible sections - Update README CI snippet to show pr-comment: 'true' Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3. - [Release notes](https://github.com/softprops/action-gh-release/releases) - [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md) - [Commits](softprops/action-gh-release@v2...v3) --- updated-dependencies: - dependency-name: softprops/action-gh-release dependency-version: '3' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Add CODE_OF_CONDUCT.md, GitHub issue templates (bug/feature), architecture overview, migration guide from Percy/Chromatic/BackstopJS Expand CONTRIBUTING.md with dev workflow, PR checklist, conventions. Eliminate duplication: trim README Web UI section to reference reporters.md, replace .gitignore patterns in ci-integration.md with cross-ref, condense generated artifacts table in organization.md to one-line note.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.