Updated review of Web Neural Network API

[anssiko]

Hello TAG!

I'm requesting an updated TAG review of WebNN API - previous TAG review: #933

• Explainer¹: https://github.com/webmachinelearning/webnn/blob/main/explainer.md
• Specification: https://www.w3.org/TR/webnn/
• WPT Tests: https://github.com/web-platform-tests/wpt/tree/master/webnn
• User research: N/A
• Security and Privacy self-review²: https://github.com/webmachinelearning/webnn/blob/main/security-privacy.md
• GitHub repo: https://github.com/webmachinelearning/webnn/
• Primary contacts:
  • Anssi Kostiainen, @anssiko, Intel (chair)
  • Dominique Hazael-Massieux, @dontcallmedom, W3C (staff contact)
  • Ningxin Hu, @huningxin, Intel (editor)
  • Dwayne Robinson, @fdwr, Microsoft (editor)
• Organization/project driving the specification: W3C Web Machine Learning Working Group
• Multi-stakeholder support³: Participation by all major browser vendors. Implementation experience across major OSes, platform APIs and key JavaScript ML frameworks.
  • Chromium comments: High-level feedback addressed. Informed by this feedback, the group has initiated work on core operator set.
  • Mozilla comments: Web Neural Network API mozilla/standards-positions#1215
  • WebKit comments: Web Neural Network API WebKit/standards-positions#486
• Status/issue trackers for implementations⁴: Chrome Platform Status, Edge WebNN Developer Preview

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Previous early design review, if any: 2000, 2022, 2024
• Relevant time constraints or deadlines: We plan to publish a new CRS during Q2 2025
• The group where the work on this specification is currently being done: W3C Web Machine Learning Working Group
• Major unresolved issues with or opposition to this specification: none
• This work is being funded by:

You should also know that...

Below is a summary of substantive changes since the previous TAG review in 2024 to assist you in this delta review.

Significant new features:

• Add 12 new operators ("Wave 3") to support well-known transformer models based on an analysis of top 20 downloaded Hugging Face models. The models comprise of a diverse set of tasks, including Small Language Models and Large Language Models, object detection, sentence similarity, feature extraction, speech recognition, image segmentation, text classification, and text generation. See issue, PR, presentation for details on these operators and motivation.
• Add MLTensor interface that represents a tensor, an input or output to an MLGraph. Update MLContext interface with new createTensor(), readTensor(), writeTensor() methods, change compute() to dispatch() for scheduling the workload, add new destroy() method for MLContext, MLGraph, MLTensor. These changes allow buffer reuse, chained inference, better resource management with explicit destruction, and the opportunity to share memory with WebGPU, minimizing copies. See MLTensor Explainer for details.

Other substantial changes:

• Remove MLDeviceType abstraction based on implementation feedback, see Device Selection Explainer for details
• Expose the API to shared workers and service workers to allow shared logic in a worker and use in web extensions
• Add MLContext.lost attribute to signal context lost error at the earliest opportunity for improved debuggability
• Add opSupportLimits() to allow checking whether operators/types are supported before creating a graph
• Enable Float16Array by default, note Uint16Array workaround informatively
• Convert MLOperand methods into readonly attributes (dataType, shape) per Web Platform Design Principles
• Add optional MLOperatorOptions.label for more diagnosable error messages
• Add MLNumber for specifying numeric inputs of any type
• Add resource contention and denial of service considerations to security considerations

The group would like to seek TAG's explicit advise on the following:

• Operator naming 2D vs 2d Operator naming 2D vs 2d webmachinelearning/webnn#821

[matatk]

Hi @anssiko, and thank you for your review request. We have reviewed and discussed the spec, and have the following thoughts...

On operator naming, "2D" is an initialism (for "2 Dimensional" or similar), and so the guidance in https://w3ctag.github.io/design-principles/#casing-rules says that it should be "All caps, except when the first word in a method or property".

We would appreciate if the WG would evaluate the likely impacts on sustainability from introducing this API, perhaps in collaboration with the Sustainable Web IG. There are several competing likely effects, including the comparative energy efficiency of personal devices vs datacenters, the greater efficiency of WebNN over WebGPU for the same workload, increased use of neural nets as they get easier to access, faster device obsolescence if older devices can't effectively run the workloads this API encourages, and likely other considerations. Any sustainability impacts might be balanced by increased utility and privacy, but it would be good to know what we're signing up for.

It would be good to have official standards positions from Mozilla and Webkit. Simply participating in a WG isn't a reliable signal of the browser engines' support for a feature.

We endorse the concerns that Chrome expressed in webmachinelearning/webnn#453, that WebNN is a proposal that browsers implement a compilation and optimization engine for yet another high level language, on top of the Javascript, Wasm, and WebGPU engines that we've already signed up for. WebNN might be worth doing, but it's a relatively high cost, and we'd like some evidence that it's going to pay off.

Inkeeping with the above questions and concerns, we are closing this review as satisfied with concerns, and look forward to hearing in due course with regards to sustainability, and the other issues raised above, as you continue to develop this work.

Finally, we also want to acknowledge the great and valuable work that your group has done in creating the Ethical Principles for Web Machine Learning - thank you!

[anssiko]

Thank you @matatk & TAG crew for your review feedback! Below is an update on our progress:

On webmachinelearning/webnn#821, "2D" is an initialism (for "2 Dimensional" or similar), and so the guidance in https://w3ctag.github.io/design-principles/#casing-rules says that it should be "All caps, except when the first word in a method or property".

We are proceeding per your guidance in webmachinelearning/webnn#821

We would appreciate if the WG would evaluate the likely impacts on sustainability from introducing this API, perhaps in collaboration with the Sustainable Web IG. There are several competing likely effects, including the comparative energy efficiency of personal devices vs datacenters, the greater efficiency of WebNN over WebGPU for the same workload, increased use of neural nets as they get easier to access, faster device obsolescence if older devices can't effectively run the workloads this API encourages, and likely other considerations. Any sustainability impacts might be balanced by increased utility and privacy, but it would be good to know what we're signing up for.

The group has initiated this work, tracked in webmachinelearning/webnn#861

It would be good to have official standards positions from Mozilla and Webkit. Simply participating in a WG isn't a reliable signal of the browser engines' support for a feature.

This review request has been updated with links to the respective standards positions.

We endorse the concerns that Chrome expressed in webmachinelearning/webnn#453, that WebNN is a proposal that browsers implement a compilation and optimization engine for yet another high level language, on top of the Javascript, Wasm, and WebGPU engines that we've already signed up for. WebNN might be worth doing, but it's a relatively high cost, and we'd like some evidence that it's going to pay off.

The group discussed this and notes WebNN API is not a new language, but a new JS API that unlocks new devices that cannot be reached from Wasm and WebGPU.

To demonstrate the value of this API, the group works closely with implementers and uses mechanisms such as Developer Previews and Origin Trials to test the API with real developers and workloads.

Inkeeping with the above questions and concerns, we are closing this review as satisfied with concerns, and look forward to hearing in due course with regards to sustainability, and the other issues raised above, as you continue to develop this work.

Thank you again, we will continue in the dedicated issues and will share updates as appropriate.

Finally, we also want to acknowledge the great and valuable work that your group has done in creating the Ethical Principles for Web Machine Learning - thank you!

Much appreciated, I have relayed your thank you to the group.

[tarekziade]

It would be good to have official standards positions from Mozilla and Webkit. Simply participating in a WG isn't a reliable signal of the browser engines' support for a feature.

for ref mozilla/standards-positions#1215 (comment)