Delta review (to CR) of Web Neural Network API

[anssiko]

Hi TAG!

I'm requesting a delta TAG review of the Web Neural Network API.

More details about this review request

The Web Neural Network API (or WebNN API in short) is a specification for constructing and executing computational graphs of neural networks. It provides web applications with the ability to create, compile, and run machine learning networks on the web browsers. The WebNN API may be implemented in web browsers using the available native operating system machine learning APIs for the best performance and reliability of results.

• Explainer: https://github.com/webmachinelearning/webnn/blob/master/explainer.md
• Specification URL: https://www.w3.org/TR/webnn/
• Tests: mocha tests, migrating to wpt/webnn
• Security and Privacy self-review: completed, see wide review tracker
• GitHub repo: https://github.com/webmachinelearning/webnn
• Primary contacts:
  • Ningxin Hu (@huningxin), Intel, Editor
  • Chai Chaoweeraprasit (@wchao1115), Microsoft, Editor
  • Anssi Kostiainen (@anssiko), Intel, Chair
• Organization(s)/project(s) driving the specification: participants of the Web Machine Learning Working Group
• Key pieces of existing multi-stakeholder review or discussion of this specification: Web and Machine Learning workshop report and spec GH issues
• External status/issue trackers for this specification:

Further details:

• I have reviewed the TAG's API Design Principles
• Relevant time constraints or deadlines: CR publication slated Q4 2022
• The group where the work on this specification is currently being done: Web Machine Learning Working Group
• The group where standardization of this work is intended to be done:
• Major unresolved issues with or opposition to this specification: N/A
• This work is being funded by: N/A

For the full review template, please unfold the above section ⤴️

The initial TAG review completed Oct 2021. This delta request focuses your attention on the following architectural changes and issues since the previous review:

• Naming of the sync and async methods: createContext, build and compute. The WG has considered two API naming conventions (x() + xSync() or xAsync() + x()) but was unable to reach consensus and resolved to seek TAG recommendation. See Support asynchronous context creation webmachinelearning/webnn#272

• Related to the naming issue, the WG decided to restrict the sync API to worker context only. This API complements the async API. The key use case for the sync API is to support Wasm code generators. The async API is the recommended path for mainstream use cases. We would like to hear the TAG perspective on this API split. We are aware that the worker-only sync API design is a rare exception on the web platform.

• The WG resolved to drop support for WebGL and focused on WebGPU interoperability.

The CR publication is slated for Q4 2022 so your feedback is the most impactful if it arrives by the end Oct 2022 latest.

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify @anssiko

For context, these are the related issues in the WebNN GH repo:

• Support asynchronous context creation webmachinelearning/webnn#272
• Should restrict the sync APIs to only exist in Workers? webmachinelearning/webnn#229
• RfC: drop WebGL interoperability requirements webmachinelearning/webnn#268

[anssiko]

@cynthia, the first bullet "naming of the sync and async methods" is the most time critical for the WG. This is to avoid breaking changes to the mainstream parts of the API as we approach Dev Trial.

To ensure you use your review time effectively, I want to clarify we are not asking for a full (re-)review of this spec, but a recommendation on this one question. The latter two bullets are mainly for your information and to provide supporting information and context. Thank you!

[torgo]

Hi @anssiko just to let you know we're working on getting you feedback here this week.

[anssiko]

@torgo thank you for keeping us updated. The WG is slated to discuss this on 2022-11-03.

[littledan]

I'm wondering, what's the motivation for the sync API? I think async APIs should be friendly to WebAssembly in the near future, given that WebAssembly JavaScript Promise Integration reached Phase 3 in the Wasm CG.

[anssiko]

@littledan this was also brought up by @domenic in webmachinelearning/webnn#272 (comment), more context there.

Do we have an estimate when this integration might ship in multiple engines?

[littledan]

Sorry, I don't have any information about timelines here; maybe @fgmccabe does

[fgmccabe]

Probably the next engine to implement JSPI will be Firefox. However, that will likely not be before sometime next year; they have a lot of other things to work on.

[cynthia]

Apologies this took so long, especially given the small change. Given the implementation support ot do this nicely is currently not ready, we discussed this at length and came to a conclusion (albeit somewhat distant from unanimous) that the following tradeoff would probably be acceptable (while not ideal).

• x() + xSync() as conventions - we will add a design principle on this.
• Assuming xSync() would block the main thread for longer than we would likely be comfortable with, keeping it limited to worker should at least prevent inappropriate use degraging user experience.

That said, we would like to see a migration path forward once WASM Promise integration lands on multiple implementations. Thank you for your patience, and we're excited to see this work move forward.

[anssiko]

Thank you @cynthia & TAG on behalf of the WG. Both the spec and the WIP implementation now follow this convention.

[anssiko]

Hi again TAG!

NB: I'm piggypacking on this issue to retain context, but please let me know if I should file a new issue instead. On behalf of the WG I hope the TAG is happy to see these changes and look forward to your comments.

We're looking to publish a new CR Snapshot of the Web Neural Network API in Q1'24 and wanted to give you a heads up with the following high-level summary of changes for your information and review:

Since the initial Candidate Recommendation Snapshot the Working Group has gathered further implementation experience and added new operations and data types needed for well-known transformers to support generative AI use cases. In addition, the group has removed select features informed by this implementation experience: higher-level operations that can be expressed in terms of lower-level primitives in a performant manner, and support for synchronous execution. The group has also updated the specification to use modern authoring conventions to improve interoperability and precision of normative definitions and is developing a new feature, a backend-agnostic storage type, to improve performance and interoperability between the WebNN, WebGPU APIs and purpose-built hardware for ML.

You are probably happy to see we're removing support for synchronous execution per your guidance (removal discussed in webmachinelearning/webnn#531, we expect to land this change ahead this publication) and moving toward JSPI that is coming finally.

Edit: fix link to webmachinelearning/webnn#531