fix: Jasmine augmentation withContext and add Playgrounds for Jasmine, Jest & Mocha (#2010)

* Add mocha playgrounds

- expect-wdio releases breaks easily other integration since we just test with unit tests
- Adding playground will help testing further before releasing and also helps troubleshooting existing problem inside the project

* Add jasmine & jest playgrounds

* Code reviews

* Add visual snapshot to playgrounds

* Document Jasmine quicks + fix type + Add withContext problem example

* Remove soft from Jasmine since it useless

* Add eslint and no-floating-promise to playgrounds

* Fix Jasmine withContext + use project global instead of wdio one

* Use proper folder for visual test + review jasmine docs quicks

* Review

* fix temp missing ignore

* Have stable e2e

* Review doc

* More concise doc

* Remove snapshot from Jasmine, update snapshot for others

Author: dprevost-LMI

docs/API.md

@@ -63,7 +63,7 @@ expect.clearSoftFailures();
 
 ### Integration with Test Frameworks
 
-The soft assertions feature integrates with WebdriverIO's test runner automatically. By default, it will report all soft assertion failures at the end of each test (Mocha/Jasmine) or step (Cucumber).
+The soft assertions feature integrates with WebdriverIO's test runner automatically. By default, it will report all soft assertion failures at the end of each test (Mocha) or step (Cucumber).
 
 To use with WebdriverIO, add the SoftAssertionService to your services list:
 
@@ -75,7 +75,7 @@ export const config = {
   // ...
   services: [
     // ...other services
-    [SoftAssertionService]
+    [SoftAssertionService,  {}]
   ],
   // ...
 }
@@ -113,8 +113,7 @@ This is useful if you want full control over when soft assertions are verified o
 
 ### Known limitations
 
-For Jasmine, using `wdio-jasmine-framework` will give a better plug-and-play experiences, else without it, the soft assertion service and custom matchers might not work/be registered correctly.
-Moreover, if Jasmine augmentation is used, the soft assertion function are not exposed in the typing, but could still work depending of your configuration. See [this issue](https://github.com/webdriverio/expect-webdriverio/issues/1893) for more details.
+The soft assertions service is not supported under Jasmine (e.g. `@wdio/jasmine-framework`) using the global import because Jasmine is already designed to provide similar behavior out of the box.
 
 ## Default Options
 
@@ -905,7 +904,72 @@ await expect(elem).toHaveElementClass(/Container/i)
 
 ## Default Matchers
 
-In addition to the `expect-webdriverio` matchers you can use builtin Jest's [expect](https://jestjs.io/docs/expect) assertions or [expect/expectAsync](https://jasmine.github.io/api/edge/global.html#expect) for Jasmine.
+In addition to the WebdriverIO matchers, `expect-webdriverio` also provides basic matchers from Jest's [expect](https://jestjs.io/docs/expect) library.
+
+```ts
+    describe('Expect matchers', () => {
+        test('Basic matchers', async () => {
+            // Equality
+            expect(2 + 2).toBe(4);
+            expect({a: 1}).toEqual({a: 1});
+            expect([1, 2, 3]).toStrictEqual([1, 2, 3]);
+            expect(2 + 2).not.toBe(5);
+
+            // Truthiness
+            expect(null).toBeNull();
+            expect(undefined).toBeUndefined();
+            expect(0).toBeFalsy();
+            expect(1).toBeTruthy();
+            expect(NaN).toBeNaN();
+
+            // Numbers
+            expect(4).toBeGreaterThan(3);
+            expect(4).toBeGreaterThanOrEqual(4);
+            expect(4).toBeLessThan(5);
+            expect(4).toBeLessThanOrEqual(4);
+            expect(0.2 + 0.1).toBeCloseTo(0.3, 5);
+
+            // Strings
+            expect('team').toMatch(/team/);
+            expect('Christoph').toContain('stop');
+
+            // Arrays and iterables
+            expect([1, 2, 3]).toContain(2);
+            expect([{a: 1}, {b: 2}]).toContainEqual({a: 1});
+            expect([1, 2, 3]).toHaveLength(3);
+
+            // Objects
+            expect({a: 1, b: 2}).toHaveProperty('a');
+            expect({a: {b: 2}}).toHaveProperty('a.b', 2);
+
+            // Errors
+            expect(() => { throw new Error('error!') }).toThrow('error!');
+            expect(() => { throw new TypeError('wrong type') }).toThrow(TypeError);
+
+            // Asymmetric matchers
+            expect({foo: 'bar', baz: 1}).toEqual(expect.objectContaining({foo: expect.any(String)}));
+            expect([1, 2, 3]).toEqual(expect.arrayContaining([2]));
+            expect('abc').toEqual(expect.stringContaining('b'));
+            expect('abc').toEqual(expect.stringMatching(/b/));
+            expect(123).toEqual(expect.any(Number));
+
+            // Others
+            expect(new Set([1, 2, 3])).toContain(2);
+
+            // .resolves / .rejects (async)
+            await expect(Promise.resolve(42)).resolves.toBe(42);
+            await expect(Promise.reject(new Error('fail'))).rejects.toThrow('fail');
+        });
+    });
+```    
+
+### Jasmine
+
+For Jasmine, see the official documentation for [expect/expectAsync](https://jasmine.github.io/api/edge/global.html#expect), [matchers](https://jasmine.github.io/tutorials/your_first_suite#section-Matchers), and [async-matchers](https://jasmine.github.io/api/edge/async-matchers.html).
+
+**Note:**
+- With the global import in @wdio/jasmine-framework, only WebdriverIO custom matchers are registered on expectAsync (assigned to global expect), so all matchers are always async, even those that are normally synchronous.
+- Default matchers are still available if you import `expect` directly from `expect-webdriverio` instead of using the global.
 
 ## Modifiers
 
@@ -928,7 +992,7 @@ await expect(element).not.toBeDisplayed({ wait: 0 })
 await expect(browser).not.toHaveTitle('some title', { wait: 0 })
 ```
 
-Note: You can pair `.not` with asymmetric matchers, but to enable the wait-until behavior, `.not` must be used directly on the `expect()` call. 
+**Note:** You can pair `.not` with asymmetric matchers, but to enable the wait-until behavior, `.not` must be used directly on the `expect()` call. 
 
 ## Asymmetric Matchers
 
@@ -943,3 +1007,21 @@ or
 ```ts
 await expect(browser).toHaveTitle(expect.not.stringContaining('some title'))
 ```
+
+### Jasmine
+
+Even under `@wdio/jasmine-framework`, Jasmine asymmetric matchers do not work with WebdriverIO matchers. 
+
+```ts
+// DOES NOT work
+await expect(browser).toHaveTitle(jasmine.stringContaining('some title'))
+
+// Use expect
+await expect(browser).toHaveTitle(expect.stringContaining('some title'))
+```
+
+However, when using Jasmine original matchers, both works.
+```ts
+await expect(url).toEqual(jasmine.stringMatching(/^https:\/\//))
+await expect(url).toEqual(expect.stringMatching(/^https:\/\//))
+```

docs/Framework.md

@@ -146,6 +146,11 @@ See also this [documentation](https://webdriver.io/docs/assertion/#migrating-fro
 ### Jasmine
 When paired with [Jasmine](https://jasmine.github.io/), [`@wdio/jasmine-framework`](https://www.npmjs.com/package/@wdio/jasmine-framework) is also required to configure it correctly, as it needs to force `expect` to be `expectAsync` and also register the WDIO matchers with `addAsyncMatcher` since `expect-webdriverio` only supports the Jest-style `expect.extend` version.
 
+Jasmine differs from other assertion libraries in two key ways:
+1. Jasmine performs soft assertions by default, collecting failures and only failing the test at the end. Because of this, the SoftAssertion service is not needed or supported.
+2. Forcing `expectAsync` as `expect` (by `@wdio/jasmine-framework`) makes even basic matchers asynchronous. However, since Jasmine handles all promises at the end of the test, assertions appear to work properly—unlike in other frameworks, where using `await` is mandatory for correct behavior.
+ - Note: This goes against [this recommendation](https://jasmine.github.io/api/edge/async-matchers) and could cause unexpected issues.
+
 The types `expect-webdriverio/jasmine` are still offered but are subject to removal or being moved into `@wdio/jasmine-framework`. The usage of `expectAsync` is also subject to future removal.
 
 #### Jasmine `expectAsync`
@@ -173,14 +178,14 @@ Expected in `tsconfig.json`:
 ```
 
 #### Global `expectAsync` force as `expect`
-When the global ambiant is the `expect` of wdio but forced to be `expectAsync` under the hood, like when using `@wdio/jasmine-framework`, then even the basic matchers need to be awaited 
+When the global ambient `expect` is actually `expectAsync` under the hood (as with `@wdio/jasmine-framework`), it is recommended to `await` even basic matchers, even though Jasmine will handle any un-awaited assertions at the end of the test.
 
 ```ts
 describe('My tests', async () => {
     it('should verify my browser to have the expected url', async () => {
         await expect(browser).toHaveUrl('https://example.com')
 
-        // Even basic matchers requires expect since they are promises underneath
+        // Even basic matchers should have `await` since they are promises underneath
         await expect(true).toBe(true)
     })
 })     
@@ -249,4 +254,4 @@ It is recommended to build your project using this approach instead of relying o
 
 ### Cucumber
 
-More details to come. In short, when paired with `@wdio/cucumber-framework`, you can use WDIO's expect with Cucumber and even [Gherkin](https://www.npmjs.com/package/@cucumber/gherkin).
+More details to come. In short, when paired with `@wdio/cucumber-framework`, you can use WDIO's expect with Cucumber and even [Gherkin](https://www.npmjs.com/package/@cucumber/gherkin).

eslint.config.mjs

@@ -4,7 +4,8 @@ export default wdioEslint.config([
     {
         ignores: [
             'lib',
-            '**/*/dist'
+            '**/*/dist',
+            'playgrounds/**'
         ]
     },
     /**

jasmine-wdio-expect-async.d.ts

@@ -1,10 +1,37 @@
 /// <reference types="./types/expect-webdriverio.d.ts"/>
 
+declare namespace jasmine {
+
+    /**
+     * Async matchers for Jasmine to allow the typing of `expectAsync` with WebDriverIO matchers.
+     * T is the type of the actual value
+     * U is the type of the expected value
+     * Both T,U must stay named as they are to override the default `AsyncMatchers` type from Jasmine.
+     *
+     * We force Matchers to return a `Promise<void>` since Jasmine's `expectAsync` expects a promise in all cases (different from Jest)
+     */
+    // eslint-disable-next-line @typescript-eslint/no-unused-vars
+    interface AsyncMatchers<T, U> extends Omit<ExpectWebdriverIO.Matchers<Promise<void>, T>, 'toMatchSnapshot' | 'toMatchInlineSnapshot'> {
+        /**
+         * snapshot matcher
+         * @param label optional snapshot label
+         */
+        toMatchSnapshot(label?: string): Promise<void>;
+        /**
+         * inline snapshot matcher
+         * @param snapshot snapshot string (autogenerated if not specified)
+         * @param label optional snapshot label
+         */
+        toMatchInlineSnapshot(snapshot?: string, label?: string): Promise<void>;
+    }
+}
+
 /**
- * Overrides the default wdio `expect` for Jasmine case specifically since the `expect` is now completely asynchronous which is not the case under Jest or standalone.
+ * Overrides the default WDIO expect specifically for Jasmine, since `expectAsync` is forced into `expect`, making all matchers fully asynchronous. This is not the case under Jest or Mocha.
+ * Using `jasmine.AsyncMatchers` pull on WdioMatchers above but also allow to using Jasmine's built-in matchers and also `withContext` matcher.
  */
 declare namespace ExpectWebdriverIO {
-    interface Expect extends ExpectWebdriverIO.AsymmetricMatchers, ExpectLibInverse<ExpectWebdriverIO.InverseAsymmetricMatchers>, WdioExpect {
+    interface Expect {
         /**
          * The `expect` function is used every time you want to test a value.
          * You will rarely call `expect` by itself.
@@ -17,6 +44,6 @@ declare namespace ExpectWebdriverIO {
          *
          * @param actual The value to apply matchers against.
          */
-        <T = unknown>(actual: T): ExpectWebdriverIO.MatchersAndInverse<Promise<void>, T>
+        <T = unknown>(actual: T): jasmine.AsyncMatchers<T, void>
     }
-}
+}

package.json

@@ -38,6 +38,11 @@
         "types": "./jasmine.d.ts"
       }
     ],
+    "./jasmine-wdio-expect-async": [
+      {
+        "types": "./jasmine-wdio-expect-async.d.ts"
+      }
+    ],
     "./types": "./types/expect-global.d.ts",
     "./expect-global": "./types/expect-global.d.ts"
   },
@@ -65,7 +70,9 @@
     "ts:jasmine-async": "cd test-types/jasmine-async && tsc --project ./tsconfig.json",
     "checks:all": "npm run build && npm run compile && npm run tsc:root-types && npm run test && npm run ts",
     "watch": "npm run compile -- --watch",
-    "prepare": "husky install"
+    "prepare": "husky install",
+    "playgrounds:setup": "for dir in playgrounds/*/; do cd \"$dir\" && npm install && cd ../..; done",
+    "playgrounds:checks:all": "for dir in playgrounds/*/; do cd \"$dir\" && npm run checks:all && cd ../..; done"
   },
   "dependencies": {
     "@vitest/snapshot": "^4.0.16",

playgrounds/jasmine/.gitignore

@@ -0,0 +1 @@
+node_modules/

playgrounds/jasmine/eslint.config.mjs

@@ -0,0 +1,28 @@
+import tsEslintPlugin from '@typescript-eslint/eslint-plugin';
+import tsParser from '@typescript-eslint/parser';
+
+export default {
+  files: ['**/*.ts', '**/*.js'],
+  languageOptions: {
+    parser: tsParser,
+    parserOptions: {
+      project: './tsconfig.json',
+      sourceType: 'module',
+      ecmaVersion: 2021,
+    },
+    globals: {
+      NodeJS: true,
+      require: true,
+      module: true,
+      __dirname: true,
+      process: true,
+    },
+  },
+  plugins: {
+    '@typescript-eslint': tsEslintPlugin,
+  },
+  rules: {
+    ...tsEslintPlugin.configs['recommended'].rules,
+    '@typescript-eslint/no-floating-promises': 'error',
+  },
+};