-
d5afb54: ## #1129 Fix
TypeError: element.getBoundingClientRect is not a functionwhen aChainablePromiseElementis passed tocheckElementWhen
checkElement(orsaveElement) was called with aChainablePromiseElement, the lazy promise-based element reference that WebdriverIO's$()returns, the element was passed directly as an argument tobrowser.execute()without being awaited first.browser.execute()serializes its arguments for transfer to the browser context and cannot handle a pending Promise, so it arrived in the browser as a plain empty object{}instead of a WebElement reference. This causedelement.getBoundingClientRect is not a functionbecause the browser-sidescrollElementIntoViewscript received{}rather than a DOM element.- Wim Selles (@wswebcreation)
-
Updated dependencies [d5afb54]
- @wdio/[email protected]
-
994f4da: ## #857 Support ignore regions for web screenshots
Add
ignoresupport to all web screenshot methods (saveScreen/checkScreen,saveElement/checkElement,saveFullPageScreen/checkFullPageScreen) so that specified elements can be blocked out during visual comparison. This brings web parity with the native-app ignore-region support that already existed.- Ignore regions for full-page screenshots: new
determineWebFullPageIgnoreRegionsfunction that calculates ignore-region rectangles for full-page screenshots, including afullPageCropTopPaddingCSScorrection for mobile scroll-and-stitch scenarios where the address-bar shadow padding shifts element positions - Consolidated
ignoreRegionPadding: movedignoreRegionPaddingintoBaseWebScreenshotOptionsso it is inherited by all web methods instead of being duplicated per method - Fix
isAndroidNativeWebScreenshottype: ensurenativeWebScreenshotis always a boolean (was accidentally an object for LambdaTest capabilities), preventing ignore-region DPR scaling failures - Fix viewport rounding for mobile: restore
Math.round()ininjectWebviewOverlayand removeMath.minclamping ingetMobileViewPortPositionto prevent 1-pixel crop shifts during full-page stitching - Fix
scrollElementIntoViewfor scrolled pages: account forcurrentPosition(existing scroll offset) when computing the target scroll position, so elements are scrolled into view correctly when the page is already scrolled - Dismiss Chrome Start Surface on Android: when Chrome's tab-overview UI blocks the webview overlay, automatically press the Android Back button (up to 4 retries) to restore the active tab before measuring the viewport
- Add hybrid status bar blockout: on hybrid apps the statusbar was not blocked out which could result in flaky tests regarding battery and reception
- Wim Selles (@wswebcreation)
- Ignore regions for full-page screenshots: new
- Updated dependencies [994f4da]
- @wdio/[email protected]
-
0a19d78: Fix
clearRuntimeFolderclearing the actual and diff folders after each spec/feature execution instead of once before all workers start. This caused only the last spec's visual data to be present in the output when running multiple specs.- Wim Selles (@wswebcreation)
-
ed0bea6: Fix
EISDIRerror when usingresolveSnapshotPathwith the visual service. The service now usesdirname()of the resolved path as the baseline folder, preventing it from creating a directory at a path thatexpect-webdriverio's snapshot service expects to be a file. Fixes #984.- Wim Selles (@wswebcreation)
-
cbf1d22: Fix incomplete
wdio-ics:optionstype augmentation onWebdriverIO.Capabilities. The global type declaration now uses theWdioIcsOptionsinterface directly, ensuring all supported properties (logName,name) are available to TypeScript users in both standalone and multiremote configurations. Fixes #732.- Wim Selles (@wswebcreation)
-
Updated dependencies [0a19d78]
-
Updated dependencies [ce74703]
- @wdio/[email protected]
-
6ed0469: ## Fix: support
appium:optionsnested capability format andavdfallback (#1118)Appium caps need to be prefixed with
appium:, but this can feel redundant when you have a lot of caps. So you can also put them inside theappium:options-object. This was not supported by the visual module and was reported in #1118. It is now supported.The following capabilities are now correctly read from both
appium:-prefixed top-level format and the nestedappium:optionsformat:deviceNamenativeWebScreenshotavd(new, see below)
Second issue that is fixed is that for Android the
deviceNamecould be left away and theavdcould be provided. This is now also supported wheredeviceNametakes priority overavdif both are provided.- Wim Selles (@wswebcreation)
-
a3bc7a4: ## #1115 Respect
alwaysSaveActualImage: falseforcheckScreenmethodsWhen using visual matchers like
toMatchScreenSnapshot('tag', 0.9)withalwaysSaveActualImage: false, the actual image was still being saved even when the comparison passed within the threshold.The root cause was that the matcher's expected threshold was not being passed to the core comparison logic. The core used
saveAboveTolerance(defaulting to 0) to decide whether to save images, while the matcher used the user-provided threshold to determine pass/fail - these were disconnected.This fix ensures:
- When
alwaysSaveActualImage: falseandsaveAboveToleranceis not explicitly set, actual images are never saved (respecting the literal meaning of the option) - When
saveAboveToleranceis explicitly set (like matchers do internally), actual images are saved only when the mismatch exceeds that threshold
- Wim Selles (@wswebcreation)
- When
-
a3bc7a4: ## Fix:
save*methods now always save files regardless ofalwaysSaveActualImagesettingPreviously, when
alwaysSaveActualImage: falsewas set in the configuration,save*methods (saveScreen,saveElement,saveFullPageScreen,saveAppScreen,saveAppElement) were not saving files to disk, causing test failures.The
alwaysSaveActualImageoption is intended to control whether actual images are saved duringcheck*methods (comparison operations), notsave*methods. Sincesave*methods are explicitly designed to save screenshots, they should always save files regardless of this setting.This fix ensures:
save*methods always save files to disk, even whenalwaysSaveActualImage: falseis set in the configalwaysSaveActualImage: falsecontinues to work correctly forcheck*methods (as intended for issue #1115)- The behavior is now consistent:
save*= always save,check*= respectalwaysSaveActualImagesetting
Implementation details:
- The visual service overrides
alwaysSaveActualImage: truewhen callingsave*methods directly from the browser API save*methods respect whateveralwaysSaveActualImagevalue is passed to them (no special logic needed)check*methods pass through the config value (which may befalse), sosave*methods respect it when called internally- This clean separation ensures
save*methods work correctly when called directly while still respectingalwaysSaveActualImageforcheck*methods
- Wim Selles (@wswebcreation)
-
Updated dependencies [a3bc7a4]
-
Updated dependencies [a3bc7a4]
- @wdio/[email protected]
-
2a518ff: # 🐛 Bugfixes
When using visual matchers like
toMatchScreenSnapshot('tag', 0.9)withalwaysSaveActualImage: false, the actual image was still being saved even when the comparison passed within the threshold.The root cause was that the matcher's expected threshold was not being passed to the core comparison logic. The core used
saveAboveTolerance(defaulting to 0) to decide whether to save images, while the matcher used the user-provided threshold to determine pass/fail - these were disconnected.This fix ensures the matcher passes the expected threshold to the core as
saveAboveTolerance, so images are only saved when the mismatch actually exceeds the user's acceptable threshold.- Wim Selles (@wswebcreation)
-
0a2b6d0: ## #1111 Respect saveAboveTolerance when deciding to save actual images when alwaysSaveActualImage is false.
When
alwaysSaveActualImageisfalse, the actual image is no longer written to disk if the mismatch is below the configured tolerance, avoiding extra actuals when the comparison still passes.- Wim Selles (@wswebcreation)
-
Updated dependencies [0a2b6d0]
- @wdio/[email protected]
-
340fbe6: # 🐛 Bugfixes
When
autoSaveBaseline = falseandalwaysSaveActualImage = falseand a baseline image doesn't exist, the error message now provides clear guidance suggesting users setalwaysSaveActualImagetotrueif they need the actual image to create a baseline manually.- Wim Selles (@wswebcreation)
-
e4e5b5c: # 🐛 Bugfixes
When
autoSaveBaselineistrueandalwaysSaveActualImageisfalse, actual images were still saved. This patch should fix that- Wim Selles (@wswebcreation)
-
ddf68fe: # 🐛 Bugfixes
The methods should be typed as promises
- Wim Selles (@wswebcreation)
-
Updated dependencies [340fbe6]
-
Updated dependencies [e4e5b5c]
- @wdio/[email protected]
-
bde4851: This PR will implement FR #1077 which is asking not to create the actual image on success. This should create a better performance because no files are writing to the system and should make sure that there's not a lot of noise in the actual folder.
- Wim Selles (@wswebcreation)
- Updated dependencies [bde4851]
- @wdio/[email protected]
-
8ff1bc3: # 🐛 BugFix
Fix option "disableBlinkingCursor" to also work within shadowdom
- Carlo Jeske (@plusgut)
-
Updated dependencies [8ff1bc3]
- @wdio/[email protected]
-
79d2b1d: # 🐛 Bugfixes
#1073 Normalize Safari desktop screenshots by trimming macOS window corner radius and top window shadow
Safari desktop screenshots included the macOS window mask at the bottom and a shadow at the top. These artifacts caused incorrect detection of the viewable area for full page screenshots, which resulted in misaligned stitching. The viewable region is now calculated correctly by trimming these areas.
- Wim Selles (@wswebcreation)
-
782b98a: # 🐛 Bugfixes
The determination of the position of the last image in mobile fullpage webscreenshots was incorrect. This was mostly seen with iOS, but also had some impact on Android. This is now fixed
- Wim Selles (@wswebcreation)
-
2c109b3: # 🐛 Bugfixes
Ignore regions with
left: 0andright:0lead to an incorrect width which lead to an incorrect ignore area. This is now fixed- Wim Selles (@wswebcreation)
-
Updated dependencies [79d2b1d]
-
Updated dependencies [782b98a]
-
Updated dependencies [2c109b3]
- @wdio/[email protected]
-
1326e99: ## 💥 Major Release: New @wdio/image-comparison-core Package
This release introduces a completely new core architecture with the dedicated
@wdio/image-comparison-corepackage, replacing the genericwebdriver-image-comparisonmodule with a WDIO-specific solution.- The old
webdriver-image-comparisonpackage was designed for generic webdriver usage - Complex integration between generic and WDIO-specific code
- Limited test coverage (~58%) making maintenance difficult
- Mixed responsibilities between core logic and service integration
✅ New dedicated core package:
@wdio/image-comparison-core- purpose-built for WebdriverIO ✅ Cleaner architecture: Modular design with clear separation of concerns ✅ Enhanced test coverage: Improved from ~58% to ~90% across all metrics ✅ Better maintainability: Organized codebase with comprehensive TypeScript interfaces ✅ WDIO-specific dependencies: Only depends on@wdio/logger,@wdio/types, etc.- 100% branch coverage on critical decision points
- Comprehensive unit tests for all major functions
- Optimized mocks for complex scenarios
- Better test isolation and reliability
Before/After % Stmts % Branch % Funcs % Lines Previous 58.59 91.4 80.71 58.59 After refactor 90.55 96.38 93.99 90.55 The
@wdio/visual-servicenow imports from the new@wdio/image-comparison-corepackage while maintaining the same public API and functionality for users.- Modular architecture: Easier to maintain and extend
- Type safety: Comprehensive TypeScript coverage
- Clean exports: Well-defined public API
- Internal interfaces: Proper separation of concerns
✅ No breaking changes for end users ✅ Same public API maintained ✅ Existing configurations continue to work ✅ All existing functionality preserved
This refactor sets the foundation for:
- Easier addition of new features
- Better bug fixing capabilities
- Enhanced mobile and native app support
- More reliable MultiRemote functionality
- Updated most dependencies to their latest versions
- Improved security with latest package versions
- Better compatibility with current WebdriverIO ecosystem
- Enhanced performance through updated dependencies
Note: This is an architectural improvement that modernizes the codebase while maintaining full backward compatibility. All existing functionality remains unchanged for users.
- The old
-
be4272c: fix: 983 in multiremote, commands are now executed on the requested instances
-
Updated dependencies [74df53b]
-
Updated dependencies [1326e99]
- @wdio/[email protected]
- P-Courteille (@P-Courteille)
- Wim Selles (@wswebcreation)
-
d88d8dd: Optimize Mobile and Emulated device support
On some devices the layer injection, to determine the exact position of the webview, was failing. It exceeded the appium timeout and returned an error like
[1] [0-0] 2025-05-23T08:04:11.788Z INFO webdriver: COMMAND getUrl() [1] [0-0] 2025-05-23T08:04:11.789Z INFO webdriver: [GET] https://hub-cloud.browserstack.com/wd/hub/session/xxxxx/url [1] [0-0] 2025-05-23T08:04:12.036Z INFO webdriver: RESULT about:blank [1] [0-0] 2025-05-23T08:04:12.038Z INFO webdriver: COMMAND navigateTo("data:text/html;base64,CiAgICAgICAgPG .... LONG LIST OF CHARACTERS=") [1] [0-0] 2025-05-23T08:04:12.038Z INFO webdriver: [POST] https://hub-cloud.browserstack.com/wd/hub/session/xxxx/url [1] [0-0] 2025-05-23T08:04:12.038Z INFO webdriver: DATA { [1] [0-0] url: 'data:text/html;base64,CiAgICAgICAgPGh0bWw.... LONG LIST OF CHARACTERS=' [1] [0-0] } [1] [0-0] 2025-05-23T08:05:42.132Z ERROR @wdio/utils:shim: Error: WebDriverError: The operation was aborted due to timeout when running "url" with method "POST" and args "{"url":"data:text/html;base64,CiAgICAgICAgPGh0b.... LONG LIST OF CHARACTERS="}" [1] [0-0] at FetchRequest._libRequest (file:///xxxxxxx/node_modules/webdriver/build/node.js:1836:13) [1] [0-0] 2025-05-23T08:05:42.132Z DEBUG @wdio/utils:shim: Finished to run "before" hook in 91147ms [1] [0-0] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [1] [0-0] at async FetchRequest._request (file:///C:/xxxxxx/node_modules/webdriver/build/node.js:1846:20) [1] [0-0] at Browser.wrapCommandFn (c:/Projects/xxxxxx/node_modules/@wdio/utils/build/index.js:907:23) [1] [0-0] at Browser.url (c:/Projects/xxxxxxx/node_modules/webdriverio/build/node.js:5682:3) [1] [0-0] at Browser.wrapCommandFn (c:/Projects/xxxxxx/node_modules/@wdio/utils/build/index.js:907:23) [1] [0-0] at async loadBase64Html (file:///C:/Projects/xxxxxx/node_modules/webdriver-image-comparison/dist/helpers/utils.js:377:5) [1] [0-0] at async getMobileViewPortPosition (file:///C:/Projects/xxxxxx/node_modules/webdriver-image-comparison/dist/helpers/utils.js:417:9) [1] [0-0] at async getMobileInstanceData (file:///C:/Projects/xxxxxx/node_modules/@wdio/visual-service/dist/utils.js:58:28) [1] [0-0] at async getInstanceData (file:///C:/Projects/xxxxxxx/node_modules/@wdio/visual-service/dist/utils.js:189:77) [1] [0-0] 2025-05-23T08:05:42.144Z INFO @wdio/browserstack-service: Update job with sessionId xxxxxThis was caused by the
await url(data:text/html;base64,${base64Html})that injected the layer. Some browsers couldn't handle thedata:text/html;base64.We now fixed that with a different injection. It was tested on Android/iOS and on Sims/Emus/Real Devices and it worked
In a previous release we added a function to determine if a device was emulated. This resulted in incorrect screen sizes that were used for the files names for devices. This caused or failing baselines, or new files to be created because the screen sizes were not available We now improved the check and the correct screen sizes are added again to the file names and made sure that the previous generated base line could be used again.
- Wim Selles (@wswebcreation)
-
Updated dependencies [d88d8dd]
-
2f9ec42: ## 🐛 Bug-fixes
When a screenshot of an emulated device is taken, but the browser was initially started as a "desktop" session, so not with emulated caps, and the
enableLegacyScreenshotMethodproperty is set totrue, the DPR of the emulated device is taken. This resulted in incorrect crop. We now store the original dpr and use that for the crop when it's an emulated device and started as a desktop browser session.The BiDi fullpage screenshot for an emulated device is broken in the driver. We now fallback to the legacy screenshot method for BiDi and emulated devices
- Updated the multiple interfaces to use JS-Doc for better docs
- When
createJsonReportFilesis set totrueand there are a lot of differences we kept waiting. We now limited that to check a max of 5M diff-pixels or a diff threshold of 20%. If it's bigger the report will show a full coverage and extra logs are shown in the WDIO logs, something like this
[0-0] 2025-05-24T06:02:18.887Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Processing diff pixels started [0-0] 2025-05-24T06:02:18.888Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Processing 20143900 diff pixels [0-0] 2025-05-24T06:02:19.770Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Total pixels in image: 52,184,160 [0-0] 2025-05-24T06:02:19.770Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Number of diff pixels: 20,143,900 [0-0] 2025-05-24T06:02:19.770Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Diff percentage: 38.60% [0-0] 2025-05-24T06:02:19.770Z ERROR @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Too many differences detected! Diff percentage: 38.60%, Diff pixels: 20,143,900 [0-0] 2025-05-24T06:02:19.771Z ERROR @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: This likely indicates a major visual difference or an issue with the comparison. [0-0] 2025-05-24T06:02:19.771Z ERROR @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Consider checking if the baseline image is correct or if there are major UI changes.- Wim Selles (@wswebcreation)
-
Updated dependencies [2f9ec42]
-
9363467: ## 🐛 Bug-fixes
- #946: Visual Regression Changes in WDIO v9
- Fixed screen size detection in emulated mode for filenames. Previously used incorrect browser window size.
- Fixed screenshot behavior when
enableLegacyScreenshotMethod: true, now correctly captures emulated screen instead of complete screen. - Fixed emulated device handling for Chrome and Edge browsers, now properly sets device metrics based on
deviceMetricsordeviceNamecapabilities.
- Wim Selles (@wswebcreation)
- #946: Visual Regression Changes in WDIO v9
-
Updated dependencies [9363467]
-
5c6c6e2: Fix capturing element screenshots with BiDi
This release fixes #919 where an element screenshot, that was for example from an overlay, dropdown, popover, tooltip, modal, was returning an incorrect screenshot
- Wim Selles (@wswebcreation)
-
Updated dependencies [5c6c6e2]
-
bfe6aca: ## 💥 BREAKING CHANGES
Screenshots taken via WebDriver's traditional protocol often lacked precision:
- Emulated devices didn't reflect true resolutions
- Device Pixel Ratio (DPR) was often lost
- Images were cropped or downscaled
All screenshot-related methods now use the WebDriver BiDi protocol by default (if supported by the browser), enabling:
✅ Native support for emulated and high-DPR devices ✅ Better fidelity in screenshot size and clarity ✅ Faster, browser-native screenshots via
browsingContext.captureScreenshotThe following methods now use BiDi:
saveScreen/checkScreensaveElement/checkElementsaveFullPageScreen/checkFullPageScreen
⚠️ Existing baselines may no longer match. Because BiDi screenshots are sharper and match device settings more accurately, even a small difference in resolution or DPR can cause mismatches.If you rely on existing baseline images, you'll need to regenerate them to avoid false positives.
You can disable BiDi screenshots globally or per test using the
enableLegacyScreenshotMethodflag:Globally in
wdio.conf.ts:import { join } from "node:path"; export const config = { services: [ [ "visual", { baselineFolder: join(process.cwd(), "./localBaseline/"), debug: true, formatImageName: "{tag}-{logName}-{width}x{height}", screenshotPath: join(process.cwd(), ".tmp/"), enableLegacyScreenshotMethod: true, // 👈 fallback to W3C-based screenshots }, ], ], };
Or per test:
it("should compare an element successfully using legacy screenshots", async function () { await expect($(".hero__title-logo")).toMatchElementSnapshot( "legacyScreenshotLogo", { enableLegacyScreenshotMethod: true } // 👈 fallback to W3C-based screenshots ); });
- ✅ #916: Visual Testing Screenshot Behavior Changed in Emulated Devices
- Wim Selles (@wswebcreation)
- Updated dependencies [bfe6aca]
-
42956e4: ## 💥 BREAKING CHANGES
Screenshots for mobile devices were inconsistent due to platform differences. iOS captures the entire device screen (including status and address bars), while Android (using ChromeDriver) only captures the webview, unless the capability
"appium:nativeWebScreenshot": trueis used.We’ve reimplemented the logic to correctly handle both platforms by default. This fix addresses [#747](#747).
💡 Credit to Benjamin Karran (@ebekebe) for pointing us in the right direction to improve this logic!
✅ More accurate full-page and element screenshots on both Android and iOS.
⚠️ But this change may break your current baselines, especially on Android and iOS.
iOS element screenshots were previously cut from full-device screenshots, which could lead to misalignment or off-by-a-few-pixels issues.
We now use the element screenshot endpoint directly.
✅ More accurate iOS element screenshots.
⚠️ But again, this may affect your existing baselines.
The "previous" scroll-and-stitch method simulated user interaction by scrolling the page, waiting, taking a screenshot, and repeating until the entire page was captured. This works well for lazy-loaded content, but it is slow and unstable on other pages.
We now use WebDriver BiDi’s [
[browsingContext.captureScreenshot](https://webdriver.io/docs/api/webdriverBidi#browsingcontextcapturescreenshot)] to capture full-page screenshots in one go. This is the new default strategy for desktop web browsers.📌 Mobile platforms (iOS/Android) still use the scroll-and-stitch approach for now.
✅ Execution time reduced by 50%+ ✅ Logic is greatly simplified ✅ More consistent and stable results on static or non-lazy pages 📸

Still want the old scroll-and-stitch behavior or need fullpage screenshots for pages who have lazy-loading?
Use the
userBasedFullPageScreenshotoption to simulate user-like scrolling. This remains the better choice for pages with lazy-loading:// wdio.conf.ts services: [ [ "visual", { userBasedFullPageScreenshot: true, }, ], ];
Or per test:
await expect(browser).toMatchFullPageSnapshot("homepage", { userBasedFullPageScreenshot: true, });
Compare options were allowed at the root level of the service config, making them harder to group or discover.
You now get a warning if you still use root-level keys. Please move them under the
compareOptionsproperty instead.Example warning:
WARN The following root-level compare options are deprecated and should be moved under 'compareOptions': - blockOutStatusBar - ignoreColors In the next major version, these options will be removed from the root level.📘 See: [compareOptions docs](https://webdriver.io/docs/visual-testing/service-options#compare-options)
- ✅ [#747](https://github.com/your-repo/issues/747): Fixed incorrect mobile webview context data.
- 🆙 Updated dependencies
- 🧪 Improved test coverage
- 📸 Refreshed image baselines
- Wim Selles (@wswebcreation)
- Updated dependencies [42956e4]
- 7f859aa: Add
additionalSearchParamsto the Storybook Runner API - 307fbec: Add
getStoriesBaselinePathto Storybook Runner API, enabling custom file paths (e.g. files with a flat hierarchy in the baselines folder)
- 3d232d1: Fix compareOptions not passed from config to the storybook runner tests
- Updated dependencies [7f859aa]
- Updated dependencies [307fbec]
- Fábio Correia @fabioatcorreia
- alcpereira (@alcpereira)
- 77bc764: Removed unused
node-fetchdependency
- alcpereira (@ alcpereira)
- 09dbc2d: update deps
- 09dbc2d: don't check for runner
- Updated dependencies [09dbc2d]
- Christian Bromann (@christian-bromann)
- Wim Selles (@wswebcreation)
-
69d25fe: Multiple fixes:
- update deps
- fix #728: Missing Type Information in Service Configuration
- fix #745: The image comparison process stalls when utilizing the checkElement method to compare screenshots of two different elements
-
Updated dependencies [69d25fe]
- 2d033e8: Add LambdaTest support
- #691 Add option to ignore blinking cursors / carets by providing the option
disableBlinkingCursor - #692 Add support for
appium:options
- #691 Add option to ignore blinking cursors / carets by providing the option
- Updated dependencies [2d033e8]
- 4a4adf1: Fix resize dimensions for mobile
- 4a4adf1: update deps
- Updated dependencies [4a4adf1]
- Updated dependencies [4a4adf1]
- 36541dd: Fix issue 679
- cc2b0fb: Fix issue 656
- a34dd5d: Update of deps
- d751b8c: fix android app determination
-
1df5350: # Improve iPhone support
- Mobile: support iOS 18 and the iPhone 16 series for the blockouts
- Mobile: don't use the device blockouts for element screenshot
- Mobile: when the blockouts had the value
{x: 0, y: 0, width: 0, height: 0}then Resemble picked this up as a full blockout. This caused false positives for iOS
-
Updated dependencies [1df5350]
- 6e20f06: Upgrade dependencies
ea08748 Fix 495: module system issue when using with vite and storybook
This fix only allows runner to be the local one, if not, it will throw this error
pnpm test.local.desktop.storybook ─╯
> @wdio/visual-testing-monorepo@ test.local.desktop.storybook /Users/wswebcreation/Git/wdio/visual-testing
> wdio tests/configs/wdio.local.desktop.storybook.conf.ts --storybook --numShards=10 --url=https://govuk-react.github.io/govuk-react/ --skipStories="/.*(loading-box|spinner).*/"
Execution of 0 workers started at 2024-09-22T05:50:20.875Z
SevereServiceError in "onPrepare"
SevereServiceError:
A service failed in the 'onPrepare' hook
SevereServiceError:
Running `@wdio/visual-service` is only supported in `local` mode.
at VisualLauncher.onPrepare (file:///Users/wswebcreation/Git/wdio/visual-testing/packages/visual-service/dist/storybook/launcher.js:22:19)
at file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:1209:32
at Array.map (<anonymous>)
at runServiceHook (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:1206:31)
at Launcher.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:2193:13)
at async Module.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:3218:16)
Stopping runner...
at file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:1227:29
at async Launcher.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:2193:7)
at async Module.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:3218:16)
HookError [SevereServiceError]:
A service failed in the 'onPrepare' hook
SevereServiceError:
Running `@wdio/visual-service` is only supported in `local` mode.
at VisualLauncher.onPrepare (file:///Users/wswebcreation/Git/wdio/visual-testing/packages/visual-service/dist/storybook/launcher.js:22:19)
at file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:1209:32
at Array.map (<anonymous>)
at runServiceHook (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:1206:31)
at Launcher.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:2193:13)
at async Module.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:3218:16)
Stopping runner...
at file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:1227:29
at async Launcher.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:2193:7)
at async Module.run (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@[email protected]/node_modules/@wdio/cli/build/index.js:3218:16) {
origin: 'onPrepare'
}
ELIFECYCLE Command failed with exit code 1.
We've update all dependencies.
- Updated dependencies [ea08748]
- 9fdb2d2: feat: work with V9
- Updated dependencies [9fdb2d2]
- cae36ad: Update deps
-
0b01b64: ### @wdio/visual-service
Added Reporting output You now have the option to export the compare results into a JSON report file. By enabling the module option
createJsonReportFiles: true, each image that is compared will create a report stored in theactualfolder, next to eachactualimage result.The output will look like this:
{ "parent": "check methods", "test": "should fail comparing with a baseline", "tag": "examplePageFail", "instanceData": { "browser": { "name": "chrome-headless-shell", "version": "126.0.6478.183" }, "platform": { "name": "mac", "version": "not-known" } }, "commandName": "checkScreen", "boundingBoxes": { "diffBoundingBoxes": [ { "left": 1088, "top": 717, "right": 1186, "bottom": 730 } //.... ], "ignoredBoxes": [ { "left": 159, "top": 652, "right": 356, "bottom": 703 } //... ] }, "fileData": { "actualFilePath": "/Users/wswebcreation/Git/wdio/visual-testing/.tmp/actual/desktop_chrome-headless-shellexamplePageFail-local-chrome-latest-1366x768.png", "baselineFilePath": "/Users/wswebcreation/Git/wdio/visual-testing/localBaseline/desktop_chrome-headless-shellexamplePageFail-local-chrome-latest-1366x768.png", "diffFilePath": "/Users/wswebcreation/Git/wdio/visual-testing/.tmp/diff/desktop_chrome-headless-shell/examplePageFail-local-chrome-latest-1366x768png", "fileName": "examplePageFail-local-chrome-latest-1366x768.png", "size": { "actual": { "height": 768, "width": 1366 }, "baseline": { "height": 768, "width": 1366 }, "diff": { "height": 768, "width": 1366 } } }, "misMatchPercentage": "12.90", "rawMisMatchPercentage": 12.900729014153246 }When all tests are executed, a new JSON file with the collection of the comparisons will be generated and can be found in the root of your actual folder. The data is grouped by:
describefor Jasmine/Mocha orFeaturefor CucumberJSitfor Jasmine/Mocha orScenariofor CucumberJS
and then sorted by:
commandName, which are the compare method names used to compare the imagesinstanceData, browser first, then device, then platform
it will look like this
[ { "description": "check methods", "data": [ { "test": "should fail comparing with a baseline", "data": [ { "tag": "examplePageFail", "instanceData": {}, "commandName": "checkScreen", "framework": "mocha", "boundingBoxes": { "diffBoundingBoxes": [], "ignoredBoxes": [] }, "fileData": {}, "misMatchPercentage": "14.34", "rawMisMatchPercentage": 14.335403703025868 }, { "tag": "exampleElementFail", "instanceData": {}, "commandName": "checkElement", "framework": "mocha", "boundingBoxes": { "diffBoundingBoxes": [], "ignoredBoxes": [] }, "fileData": {}, "misMatchPercentage": "1.34", "rawMisMatchPercentage": 1.335403703025868 } ] } ] } ]The report data will give you the opportunity to build your own visual report without doing all the magic and data collection yourself.
- Add support to generate single JSON report files
- Refactored the CLI to use
@inquirer/promptsinstead ofinquirer
- Wim Selles (@wswebcreation)
- Updated dependencies [0b01b64]
-
26c1a17: This release contains better support for Multiremote tests on the instances and global level, including better Native App support. This is related to issue #418 which is fixed by this release
- cab1219: Update dependencies
- 2583542 / 8b1f837: add new tests
- eec29e1 / e2b2d38: update images
- 29f6f82 / 1ba9817: fix issue #418
- Wim Selles (@wswebcreation)
-
Updated dependencies [26c1a17]
-
a0e29f2: Adding storybook interaction testing
Storybook Interaction Testing allows you to interact with your component by creating custom scripts with WDIO commands to set a component into a certain state. For example, see the code snippet below:
import { browser, expect } from "@wdio/globals"; describe("Storybook Interaction", () => { it("should create screenshots for the logged in state when it logs out", async () => { const componentId = "example-page--logged-in"; await browser.waitForStorybookComponentToBeLoaded({ id: componentId, }); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-in-state` ); await $("button=Log out").click(); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-out-state` ); }); it("should create screenshots for the logged out state when it logs in", async () => { const componentId = "example-page--logged-out"; await browser.waitForStorybookComponentToBeLoaded({ id: componentId, }); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-out-state` ); await $("button=Log in").click(); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-in-state` ); }); });
Two tests on two different components are executed. Each test first sets a state and then takes a screenshot. You will also notice that a new custom command has been introduced, which can be found here.
The above spec file can be saved in a folder and added to the command line with the following command:
npm run test.local.desktop.storybook.localhost -- --spec='tests/specs/storybook-interaction/*.ts'The Storybook runner will first automatically scan your Storybook instance and then add your tests to the stories that need to be compared. If you don't want the components that you use for interaction testing to be compared twice, you can add a filter to remove the "default" stories from the scan by providing the
--skipStoriesfilter. This would look like this:npm run test.local.desktop.storybook.localhost -- --skipStories="/example-page.*/gm" --spec='tests/specs/storybook-interaction/*.ts'
A new custom command called
browser.waitForStorybookComponentToBeLoaded({ id: 'componentId' })will be added to thebrowser/driver-object that will automatically load the component and wait for it to be done, so you don't need to use thebrowser.url('url.com')method. It can be used like thisimport { browser, expect } from "@wdio/globals"; describe("Storybook Interaction", () => { it("should create screenshots for the logged in state when it logs out", async () => { const componentId = "example-page--logged-in"; await browser.waitForStorybookComponentToBeLoaded({ id: componentId, }); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-in-state` ); await $("button=Log out").click(); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-out-state` ); }); it("should create screenshots for the logged out state when it logs in", async () => { const componentId = "example-page--logged-out"; await browser.waitForStorybookComponentToBeLoaded({ id: componentId, }); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-out-state` ); await $("button=Log in").click(); await expect($("header")).toMatchElementSnapshot( `${componentId}-logged-in-state` ); }); });
The options are:
- Type:
string - Mandatory: No
- Default:
#storybook-root > :first-childfor Storybook V7 and#root > :first-child:not(script):not(style)for Storybook V6 - Example:
await browser.waitForStorybookComponentToBeLoaded({ clipSelector: "#your-selector", id: "componentId", });
This is the selector that will be used:
- to select the element to take the screenshot of
- for the element to wait to be visible before a screenshot is taken
- Type:
string - Mandatory: yes
- Example:
await browser.waitForStorybookComponentToBeLoaded({ '#your-selector', id: 'componentId' })
Use the
idof the story that can be found in the URL of the story. For example, theidin this URLhttp://localhost:6006/?path=/story/example-page--logged-outisexample-page--logged-out- Type:
number - Mandatory: No
- Default: 1100 milliseconds
- Example:
await browser.waitForStorybookComponentToBeLoaded({ id: "componentId", timeout: 20000, });
The max timeout we want to wait for a component to be visible after loading on the page
- Type:
string - Mandatory: No
- Default:
http://127.0.0.1:6006 - Example:
await browser.waitForStorybookComponentToBeLoaded({ id: "componentId", url: "https://your.url", });
The URL where your Storybook instance is hosted.
- Type:
- 169b7c5: fix(webdriver-image-comparison): export WicElement
- Updated dependencies [169b7c5]
-
66b9f11: # 💥 Breaking
This PR replaces Canvas as a dependency with Jimp. This removes the need to use system dependencies and will reduce the number of system dependency errors/issues (node-gyp/canvas and so on). This will, in the end, make the life of our end users way easier due to:
- less errors
- less complex test environments
[!note] Extensive research has been done and we have chosen to "fork" ResembleJS, adjust it by making use of Jimp instead of Canvas and break the browser API because the fork will only be used in a nodejs environment Investigation showed that creating a wrapper would even make it slower, so we went for the breaking change in the API by just replacing Canvas with Jimp
[!important] There is a performance impact where Canvas is around 70% faster than Jimp. This has been measured without using WebdriverIO and only comparing images. When the "old" implementation with WebdriverIO combined with Canvas or Jimp is compared, we hardly see a performance impact.
Update the baseline images through the command line by adding the argument
--update-visual-baseline. This will- automatically copy the actual take screenshot and put it in the baseline folder
- if there are differences it will let the test pass because the baseline has been updated
Usage:
npm run test.local.desktop --update-visual-baseline
When running logs info/debug mode you will see the following logs added
[0-0] .............. [0-0] ##################################################################################### [0-0] INFO: [0-0] Updated the actual image to [0-0] /Users/wswebcreation/Git/wdio/visual-testing/localBaseline/chromel/demo-chrome-1366x768.png [0-0] ##################################################################################### [0-0] ..........- remove Vitest fix
- add app images
- update the build
- Updated dependencies [66b9f11]
- 7713c13: Fix beforeTest incl types
- 4dcfeea: add missing dependency and update deps
- c9fab82: change console.log to wdio logger
- Updated dependencies [c9fab82]
-
f878cab: # 🚀 Feature
- Add device support for Storybook, it can be used like this
npx wdio tests/configs/wdio.local.desktop.storybook.conf.ts --storybook --devices="iPhone 14 Pro Max","Pixel 3 XL"
- Wim Selles (@wswebcreation)
-
Updated dependencies [f878cab]
- bb4ece7: Fix storybook filtering
-
b717d9a: # 💥 Breaking changes
- the new element screenshot is producing "smaller" screenshots on certain Android OS versions (not all), but it's more "accurate" so we accept this
Automatically scan local/remote storybook instances to create element screenshots of each component by adding
export const config: WebdriverIO.Config = { // ... services: ["visual"], // .... };
to your
servicesand runningnpx wdio tests/configs/wdio.local.desktop.storybook.conf.ts --storybookthrough the command line. It will automatically use Chrome. The following options can be provided through the command line-
--headless, defaults totrue -
--numShards {number}, this will be the amount of parallel instances that will be used to run the stories. This will be limited by themaxInstancesin yourwdio.conf-file. When running inheadless-mode then do not increase the number to more than 20 to prevent flakiness -
--clip {boolean}, try to take an element instead of a viewport screenshot, defaults totrue -
--clipSelector {string}, this is the selector that will be used to:- select the element to take the screenshot of
- the element to wait for to be visible before a screenshot is taken
defaults to
#storybook-root > :first-childfor V7 and#root > :first-child:not(script):not(style)for V6 -
--version, the version of storybook, defaults to 7. This is needed to know if the V6clipSelectorneeds to be used. -
--browsers {edge,chrome,safari,firefox}, defaults to Chrome -
--skipStories, this can be:- a string (
example-button--secondary,example-button--small) - or a regex (
"/.*button.*/gm") to skip certain stories
- a string (
You can also provide service options
export const config: WebdriverIO.Config = { // ... services: [ [ 'visual', { // Some default options baselineFolder: join(process.cwd(), './__snapshots__/'), debug: true, // The storybook options storybook: { clip: false, clipSelector: ''#some-id, numShards: 4, skipStories: ['example-button--secondary', 'example-button--small'], url: 'https://www.bbc.co.uk/iplayer/storybook/', version: 6, }, }, ], ], // .... }
The baseline images will be stored in the following structure:
{projectRoot} |_`__snapshots__` |_`{category}` |_`{componentName}` |_{browserName} |_`{{component-id}-element-{browser}-{resolution}-dpr-{dprValue}}.png`which will look like this
[!NOTE] Storybook 6.5 or higher is supported
hideScrollBarsis disabled by default when using the Storybook runner- By default, all element screenshots in the browser, except for iOS, will use the native method to take element screenshots. This will make taking an element screenshot more than 5% faster. If it fails it will fall back to the "viewport" screenshot and create a cropped element screenshot.
- Taking an element screenshot becomes 70% faster due to removing the fixed scroll delay of 500ms and changing the default scrolling behaviour to an instant scroll
- refactor web element screenshots and update the screenshots
- added more UTs to increase the coverage
- When the element has no height or width, we default to the viewport screen size to prevent not cropping any screenshot. An error like below will be logged in red
The element has no width or height. We defaulted to the viewport screen size of width: ${width} and height: ${height}.- There were cases where element screenshots were automatically rotated which was not intended
- Updated dependencies [b717d9a]
- 43ed502: Add font loading features:
- add
waitForFontsLoadedso the module automatically waits for all fonts to be loaded, enabled by default - add
enableLayoutTestingso all text will become transparent so- font rendering issues won't cause flakiness
- image comparison can be done on layout
- add
- Updated dependencies [43ed502]
- 14b6ae6: Support BS real device names Fix hide/remove elements
- c8fdcd3: Fix to override visibility/display value
- Updated dependencies [c8fdcd3]
- fd74a35: (feat): set default baseline folder next to test file
- e93a878: fix default snapshot path to be overwritten through method/service options
- Updated dependencies [fd74a35]
-
ef386b6: # 💥 Breaking changes:
resizeDimensionson the element can now only be an object, it has been deprecated for a while
- Next to supporting Web snapshot testing this module now also supports 💥 Native App 💥 snapshot testing. The methods
saveElement|checkElement | saveScreen | checkScreenand the matcherstoMatchElementSnapshot | toMatchScreenSnapshotare available for Native Apps
[!NOTE] This module will automatically detect the context (web | webview | native_app) and will handle all complex logic for you
The methods
saveFullPageScreen | checkFullPageScreen | saveTabbablePage|checkTabbablePagewill throw an error when they are used in the native context for native mobile apps and will look like this$ wdio tests/configs/wdio.local.android.emus.app.conf.ts Execution of 1 workers started at 2024-01-30T06:18:24.865Z [0-0] RUNNING in Android - file:///tests/specs/mobile.app.spec.ts [0-0] Error in "@wdio/visual-service mobile app.should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode" Error: The method saveFullPageScreen is not supported in native context for native mobile apps! at /wdio/visual-testing/packages/webdriver-image-comparison/src/commands/saveFullPageScreen.ts:26:15 at step (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:33:23) at Object.next (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:14:53) at /wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:8:71 at new Promise (<anonymous>) at __awaiter (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:4:12) at saveFullPageScreen (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:47:12) at Browser.<anonymous> (file:///wdio/visual-testing/packages/service/dist/service.js:101:24) [0-0] FAILED in Android - file:///tests/specs/mobile.app.spec.ts "spec" Reporter: ------------------------------------------------------------------ [/wdio/visual-testing/apps/app.apk Android #0-0] Running: /wdio/visual-testing/apps/app.apk on Android [/wdio/visual-testing/apps/app.apk Android #0-0] Session ID: c1101184-e3d5-42b5-a31f-8ebaa211f1a1 [/wdio/visual-testing/apps/app.apk Android #0-0] [/wdio/visual-testing/apps/app.apk Android #0-0] » /tests/specs/mobile.app.spec.ts [/wdio/visual-testing/apps/app.apk Android #0-0] @wdio/visual-service mobile app [/wdio/visual-testing/apps/app.apk Android #0-0] ✖ should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode [/wdio/visual-testing/apps/app.apk Android #0-0] [/wdio/visual-testing/apps/app.apk Android #0-0] 1 failing (1.5s) [/wdio/visual-testing/apps/app.apk Android #0-0] [/wdio/visual-testing/apps/app.apk Android #0-0] 1) @wdio/visual-service mobile app should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode [/wdio/visual-testing/apps/app.apk Android #0-0] The method saveFullPageScreen is not supported in native context for native mobile apps! [/wdio/visual-testing/apps/app.apk Android #0-0] Error: The method saveFullPageScreen is not supported in native context for native mobile apps! [/wdio/visual-testing/apps/app.apk Android #0-0] at /wdio/visual-testing/packages/webdriver-image-comparison/src/commands/saveFullPageScreen.ts:26:15 [/wdio/visual-testing/apps/app.apk Android #0-0] at step (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:33:23) [/wdio/visual-testing/apps/app.apk Android #0-0] at Object.next (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:14:53) [/wdio/visual-testing/apps/app.apk Android #0-0] at /wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:8:71 [/wdio/visual-testing/apps/app.apk Android #0-0] at new Promise (<anonymous>) [/wdio/visual-testing/apps/app.apk Android #0-0] at __awaiter (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:4:12) [/wdio/visual-testing/apps/app.apk Android #0-0] at saveFullPageScreen (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:47:12) [/wdio/visual-testing/apps/app.apk Android #0-0] at Browser.<anonymous> (file:///wdio/visual-testing/packages/service/dist/service.js:101:24) Spec Files: 0 passed, 1 failed, 1 total (100% completed) in 00:00:11 error Command failed with exit code 1.autoSaveBaselineis true by default, so if no baseline images are present it will automatically create a new baseline- Mobile screenshots of the complete screen now automatically exclude all native OS elements like the notification bar, home bar, address bar, and so on, the settings
blockOutSideBar | blockOutStatusBar |blockOutToolBarare now all defaulted totrue
- element screenshots could also get resized dimensions, which would cut out a bigger portion around the element. This was failing when the dimensions got out of the boundaries of the official screenshot. This has now been fixed with:
- not going outside of the screenshot
- log extra warnings
- Updated dependencies [ef386b6]
- 36d3868: Support for WebdriverIO v8
- 36d3868: (feat): add visual matcher
- Updated dependencies [36d3868]
