Skip to content

feat: Improve QR scanning reliability for dark borderless codes#759

Open
AntonioVentilii wants to merge 12 commits intomainfrom
feat/Improve-QR-scanning-reliability-for-dark-borderless-codes
Open

feat: Improve QR scanning reliability for dark borderless codes#759
AntonioVentilii wants to merge 12 commits intomainfrom
feat/Improve-QR-scanning-reliability-for-dark-borderless-codes

Conversation

@AntonioVentilii
Copy link
Copy Markdown
Collaborator

@AntonioVentilii AntonioVentilii commented Mar 23, 2026

Motivation

The QR code reader was using html5-qrcode, which has a known issue with inverted (dark-themed, borderless) QR codes on mobile — they simply cannot be scanned. The library is also unmaintained.

Known upstream issues:

A first attempt (#757) tried to fix this by increasing the scan region tolerance within the same library, but that approach still failed on mobile (see test results in that PR).

Since the root cause is a limitation of html5-qrcode itself (no support for inverted QR codes), and the library is no longer maintained, this PR replaces it with barcode-detector — a Barcode Detection API polyfill powered by ZXing C++ WebAssembly under the hood. The underlying engine natively supports inverted code detection (tryInvert is enabled by default). We tested it manually on a mobile device and dark borderless QR codes are now successfully scanned.

Thanks to @sea-snake for the tip!

Changes

  • Removed the html5-qrcode dependency entirely in component QRCodeReader.
  • In component QRCodeReader:
    • Camera access is now handled directly via the browser navigator.mediaDevices.getUserMedia API, requesting the rear-facing camera (facingMode: "environment") at up to 1920×1080 resolution.
    • The video feed is rendered in a native <video> element.
    • QR decoding uses the standard BarcodeDetector API (via barcode-detector/ponyfill). A detector instance is created with formats: ["qr_code"] and detector.detect(videoElement) is called at ~10 fps via setInterval(scan, 100). The polyfill accepts the <video> element directly — no intermediate <canvas> needed.
    • A frame-processing guard (isProcessingFrame) prevents overlapping decode calls.
    • Cleanup on destroy properly clears the scan interval and stops all media tracks.
    • The scan overlay is now implemented with a .scan-overlay container with a .scan-region box using box-shadow to mask the surrounding area and a border styled with the primary color. This replaces the old :global(#qr-shaded-region) overrides that targeted html5-qrcode's internal DOM.
  • Updated the library reference in the QR code reader docs page from jsQR to barcode-detector.
  • Added barcode-detector@^3.1.1 to dependencies and removed html5-qrcode.

Tests

Manual testing on mobile: dark borderless QR codes are now successfully scanned (this was the main validation, as the core issue only reproduced on real mobile devices).

IMG_5315 IMG_5314

Screenshots

Updated E2E snapshots.

AntonioVentilii and others added 10 commits March 19, 2026 15:45
Use the BarcodeDetector API polyfill (backed by the same ZXing C++ WASM
engine) instead of zxing-wasm directly. This simplifies the component by
removing the hidden canvas — BarcodeDetector.detect() accepts the video
element directly.

Made-with: Cursor
@AntonioVentilii AntonioVentilii marked this pull request as ready for review March 24, 2026 17:17
@AntonioVentilii AntonioVentilii requested review from a team as code owners March 24, 2026 17:17
github-merge-queue bot pushed a commit to dfinity/oisy-wallet that referenced this pull request Mar 25, 2026
…des (#12162)

# Note

This PR is a fork of dfinity/gix-components#759.

Awaiting that review, we simply copy the components in our repo and add
it to the next release, since it is a time-sensitive issue.

# Motivation

The QR code reader was using `html5-qrcode`, which has a known issue
with inverted (dark-themed, borderless) QR codes on mobile — they simply
cannot be scanned. The library is also unmaintained.

Known upstream issues:
- mebjas/html5-qrcode#766
- mebjas/html5-qrcode#468
- mebjas/html5-qrcode#94

Since the root cause is a limitation of `html5-qrcode` itself (no
support for inverted QR codes), and the library is no longer maintained,
this PR replaces it with
[`barcode-detector`](https://www.npmjs.com/package/barcode-detector) — a
[Barcode Detection
API](https://developer.mozilla.org/en-US/docs/Web/API/BarcodeDetector)
polyfill powered by ZXing C++ WebAssembly under the hood. The underlying
engine natively supports inverted code detection (`tryInvert` is enabled
by default). We tested it manually on a mobile device and dark
borderless QR codes are now successfully scanned.

Thanks to @sea-snake for the tip!

# Changes (copied from gix-components's `QRCodeReader`)

- Removed the `html5-qrcode` dependency entirely in component
`QRCodeReader`.
- In component `QRCodeReader`:
- Camera access is now handled directly via the browser
`navigator.mediaDevices.getUserMedia` API, requesting the rear-facing
camera (`facingMode: "environment"`) at up to 1920×1080 resolution.
  - The video feed is rendered in a native `<video>` element.
- QR decoding uses the standard `BarcodeDetector` API (via
`barcode-detector/ponyfill`). A detector instance is created with
`formats: ["qr_code"]` and `detector.detect(videoElement)` is called at
~10 fps via `setInterval(scan, 100)`. The polyfill accepts the `<video>`
element directly — no intermediate `<canvas>` needed.
- A frame-processing guard (`isProcessingFrame`) prevents overlapping
decode calls.
- Cleanup on destroy properly clears the scan interval and stops all
media tracks.
- The scan overlay is now implemented with a `.scan-overlay` container
with a `.scan-region` box using `box-shadow` to mask the surrounding
area and a border styled with the primary color. This replaces the old
`:global(#qr-shaded-region)` overrides that targeted `html5-qrcode`'s
internal DOM.
- Updated the library reference in the QR code reader docs page from
`jsQR` to `barcode-detector`.
- Added `barcode-detector@^3.1.1` to dependencies and removed
`html5-qrcode`.

# Tests

Manual testing on mobile: dark borderless QR codes are now successfully
scanned (this was the main validation, as the core issue only reproduced
on real mobile devices).

<img width="1071" height="1428" alt="IMG_5315"
src="https://github.com/user-attachments/assets/f25cbe60-3d92-472c-917e-ffdfb5f1b343"
/>
<img width="1179" height="2556" alt="IMG_5314"
src="https://github.com/user-attachments/assets/6fd3d30d-2080-4464-8263-80ce3d544698"
/>
Copy link
Copy Markdown
Contributor

@yhabib yhabib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some small questions/nits.

  1. Can you add the CSP comment?
  2. Do you know how this new dep. impacts the bundle size?

.scan-region {
width: min(90cqw, 90cqh);
aspect-ratio: 1;
box-shadow: 0 0 0 9999px white;
Copy link
Copy Markdown
Contributor

@yhabib yhabib Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] have you tested this with a dark theme? I don't think it adapts well 🤔

const [qrResult] = results;

if (nonNullish(qrResult)) {
dispatch("nnsQRCode", qrResult.rawValue);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] Is it intended to keep pushing events even after successful one? should it not stop? do you know how it worked with the previous library?

// Make sure the camera is stopped if the component is destroyed before
// scanning is starte.
await html5QrCode?.stop();
if (nonNullish(videoElement)) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] I guess this check is purely TS, no? Otherwise why not stop the stream in the else statement?

}
};

scanInterval = setInterval(scan, 100);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] setInterval(scan, 100) fires every 100ms regardless of how long detect() takes. isProcessingFrame guard prevents overlap, but frames will be skipped if detection is slow. Eg

 Time 0ms:    setInterval fires → scan() starts → detect() begins (takes ~150ms)
  Time 100ms:  setInterval fires → scan() starts → isProcessingFrame is true → early return
  Time 200ms:  setInterval fires → scan() starts → detect() just finished at ~150ms, so isProcessingFrame is false → detect() begins again
  Time 300ms:  setInterval fires → scan() starts → isProcessingFrame is true → early return

What about sth like:

  const scan = async () => {
    if (isDestroyed || isNullish(videoElement) ||
        videoElement.readyState < HTMLMediaElement.HAVE_CURRENT_DATA) {
      return;
    }

    try {
      const results = await detector.detect(videoElement);
      const [qrResult] = results;
      if (nonNullish(qrResult)) {
        dispatch("nnsQRCode", qrResult.rawValue);
      }
    } catch {
      // No QR visible
    }

    if (!isDestroyed) scanInterval = setTimeout(scan, 100);
  };

  scanInterval = setTimeout(scan, 100);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants