Enhanced expect wdio typing

[dprevost-LMI]

TL;DR
Fix typing to:

1. Return promises only when needed (no await for strings, allow correct usage of no-floating-promises)
2. Show TS errors when using element matchers on non-element types

Summary

1. Wrong return types

Have promise used only when required so that no-floating-promises can be used and detect when await is missing throughout the code, including the wdio expect cases.

In short, even though it was more complicated, instead of having both void | Promise<void> returned by expect/matchers, we are now returning Promise<void> only for Wdio Matchers and for the default ones from the expect Lib, it is void.

// Today this is flagged as requiring `await` since ExpectWebDriverIO forces the return of a `Promise<void>`
expect('text').toBe('text')

2. TS Error when using element/browser matcher with the wrong type

Have a Typescript error when using the wrong actual type
Using an approach where we define a function based on the typing of the actual T, we can make TS use never to error cases where the type is undesired

interface WdioElementOrArrayMatchers<R, ActualT = unknown> {
  toHaveText: ActualT extends ChainableElement ? (
      text: string | RegExp | ExpectWebdriverIO.PartialMatcher<string> 
      options?: ExpectWebdriverIO.StringOptions
  ) => Promise<R> : never
}

// Works
await expect(chainableElement).toHaveText('text')
// @ts-expect-error
await expect(browser).toHaveText('text')

⚠️ However, one limitation is function overloading, which is now unusable since we cannot declare twice the same variable.

For toHaveHeight, we had to combine the two signatures in one! This point could need a debate!

Now extending expect

Inspired by this comment, we aim to align our approach more closely with the expect library, potentially reducing some inconsistencies.

Pros:

1. The asymmetrical matcher definition is lighter since we were already reusing the exact definition
2. We now expose for free closeTo and arrayOf working out-of-the-box with jest-matcher-utils
3. The not is now handled automatically, as it was previously overwritten.

Cons:

1. Might be more subject to breaks if expect library changes, but less likely to happen since it is a solid base shared in multiple places.

Jasmine

Jasmine is special because expectAsync always returns a Promise as the final result, unlike other frameworks. Moreover, in expect-webriverio we augment the Jasmine namespace to have wdio matchers on expectAsync; however, in wdio-jasmine-framework, we force the expect to be expectAsync.

This raises several questions and inconsistencies that are outside the scope of this PR. This issue was raised to document what needs to happen to help clarify Jasmine support later.

In the context of this PR, we have documented what is supported and some limitations, and ensured, through tests, that these are still supported.

Jest

Even though we support @types/jest and @jest/global, the latter is recommended since Jest's maintainer does not support/recommend @types/jest, and it may even be behind and therefore causing problems (subject to be dropped later)

The matchers toMatchSnapshot and toMatchInlineSnapshot conflict with those in Jest's library. Best effort was made to support both wdio and Jest definitions.

We cannot only use @jest/global and augment it; we need @types/jest. This Jest's issue might changes that one day

Other:

• For custom matchers, some concerns had already been previously raised in Separate Matchers between browser, element and mock matchers #1408; however, this PR proposes an alternative solution that addresses them.
• wdio/await-expect does not interfere with this PR's changes. Moreover, this plugin could be reviewed to utilize no-floating-promises and perform even better once this PR is merged.
• The soft assertion has two limitations that are raised in this issue
  1. It does not support basic matchers like toBe or toEqual, making it less interesting. This code is the problem; supporting only custom matchers
  2. Currently, it forces the use of await here, creating another problem if point 1 is resolved. With basic matchers, no await should be required

BREAKING

• Minor: standalone.d.ts is deleted since the default ExpectWebDriverIO namespace is now correctly working and covering the standalone case
• Minor: jest-global.d.ts is renamed to expect-global.d.ts since it is unrelated to Jest; we provide expect globally for the WDIO's expect.
• Minor: Before sample was on ExpectWebdriverIO.PartialMatcher, now it is on WdioAsymmetricMatcher
• Minor: expect.not.anything()/any(X) typing is no longer supported since in Jest it was not even supported
• Medium: Under Jasmine, with expect as WDIO global, typing was returning void | Promise<void>, making it "working" with the forced expectAsync needs to be awaited, but now it is void
  • To "stay" backward now, people using Jasmine will need to add "expect-webdriverio/expect-wdio-jasmine-async" in their tsconfig.josn#types
• Minor: expect.not now has a different type for AsymetricsMatcher vs InverseAsymmetricMatchers since 'any' and 'anything' are not supported in inverse, as the expect and Jest library does.

Testing

Jest:

• This boilerplate project tests Jest with only @jest/global + no-floating-promise
• This boilerplate project tests Jest with @jest/global + @types/jest + no-floating-promise

Jasmine:

• This boilerplat project test Jasmine with @types/Jasmine + @wdio/jasmine-framework + no-floating-promise
• This boilerplate project tests Jasmine as global with expect-webdriverio/jasmine and expectAsync

Tasks

• Finalize Jasmine case (see also fix(@wdio/jasmine-framework): become independant from expect-webdriverio webdriverio#14592)
  • Review limited asymmetric matcher list...
• Verify usability in a real runnable case
  • Jest
  • Jasmine
  • standalone
• Document potential breaking changes and/or try to minimize them
• Document or verify if the linter wdio/await-expect could counter changes from this PR
• Verify global import from main webdriverio project
• Integrate in webdriverio with yalc yalc is not working
• Finalize TODOs
  • 3 TODOs are remaining, before completing them, I wanted to know if extengin expect is the right approach
• Enter issue for Jest & Jasmine augmentation limitation. See entered issues
• Verify if we can support Promise<void> for basic matcher under Jasmine because of the forced expectAsync while not breaking Jest and standalone!
• Restest Jasmine one last time
• Retry integrating the latest expect-webdriverIO with yalc in the wdio project to validate if it is still "working" correctly
  • Understand why we need to expect.not to register asymmetrics matchers here since it is no longer exposed - Still not sure, but InverseAsymmetricMatchers type is now provided to better support different AsymmetricMatchers and Inverse AsymmetricMatchers keyof for webdriver project

[christian-bromann]

Amazing work 🙏 ❤️ some minor nits

[dprevost-LMI]

@christian-bromann I would be ready for a release + integration in WebdriverIO

Two "important" points:

1. Jasmine users might see a breaking typing change where await is not "required" by the typing since by default the lib returns void. For those cases, they need to pull the new type expect-webdriverio/expect-wdio-jasmine-async
2. To integrate the new expect-webdriverio in the webdriverio project, the following will be required. I can do it

In webdriverio/packages/wdio-runner/src/utils.ts

        const matcherKey = SUPPORTED_ASYMMETRIC_MATCHER[arg.$$typeof as keyof typeof SUPPORTED_ASYMMETRIC_MATCHER] as keyof AsymmetricMatchers
        const inverseMatcherKey = SUPPORTED_ASYMMETRIC_MATCHER[arg.$$typeof as keyof typeof SUPPORTED_ASYMMETRIC_MATCHER] as keyof InverseAsymmetricMatchers
        const matcher = ('inverse' in arg && arg.inverse ? expect.not[inverseMatcherKey] : expect[matcherKey]) as unknown as (sample: string) => unknown

[christian-bromann]

LGTM 👍 exceptional work!

(some minor formatting which I will go ahead and merge)

[christian-bromann]

I will merge and update WebdriverIO core tomorrow.

[dprevost-LMI]

Temporary alternative:
If you are not using the softAssertion service and neither Jasmine, you can use the patch below to have the correct typing and also be able to use @typescript-eslint/no-floating-promises in your project even on the wdio custom matchers

expect-webdriverio+5.4.1.patch

[dprevost-LMI]

@christian-bromann @erwinheitzman I thought this PR was ready to merge, but I feel there is still work to be done to get your whole agreement.
Can you enumerate your required criteria to get this one ready? Thanks!

[erwinheitzman]

@dprevost-LMI hi I think we are good to go unless Christian is waiting for something (referring to the merging of the core)