webmachinelearning / webnn-polyfill Goto Github PK

View Code? Open in Web Editor NEW

100.0 10.0 18.0 612.04 MB

🧠⚙️ Web Neural Network API polyfill based on TensorFlow.js

Home Page: https://www.npmjs.com/package/@webmachinelearning/webnn-polyfill

License: Apache License 2.0

JavaScript 77.95% TypeScript 11.41% HTML 0.81% Shell 0.03% Python 9.80%

webnn-polyfill's Introduction

WebNN Polyfill

A JavaScript implementation of the Web Neural Network API.

Backends

The implementation of this webnn-polyfill is based on TensorFlow.js that supports the following 3 backends:

TensorFlow.js CPU Backend, pure-JS backend for Node.js and the browser.
TensorFlow.js WebGL Backend, WebGL backend for the browser.
TensorFlow.js WASM Backend, WebAssembly backend for the browser.

Notes

CPU backend is the only supported backend for Node.js.
WASM backend does not support all the ops and some test failures are thus expected.

Usage

Import the packages

Via NPM

import '@webmachinelearning/webnn-polyfill';

Via a script tag

<script src="https://cdn.jsdelivr.net/npm/@webmachinelearning/webnn-polyfill/dist/webnn-polyfill.js"></script>

Set backend

WebNN Polyfill requires setting backend to enable TensorFlow.js.

When running in Node.js, recommend using CPU backend for its higher numerical precision.

    const backend = 'cpu';
    const context = await navigator.ml.createContext();
    const tf = context.tf;
    await tf.setBackend(backend);
    await tf.ready();

When running in browsers, recommend using WebGL backend for better performance.

    const backend = 'webgl'; // 'cpu' or 'wasm'
    const context = await navigator.ml.createContext();
    const tf = context.tf;
    await tf.setBackend(backend);
    await tf.ready();

When running in browsers with WASM backend.

    const backend = 'wasm';
    const context = await navigator.ml.createContext();
    const wasm = context.wasm;

    // 1- Enforce use Wasm SIMD binary
    wasm.setWasmPath(`${path}/tfjs-backend-wasm-simd.wasm`);

    // 2- Use Wasm SIMD + Threads bianry if supported both SIMD and Threads
    // 2.1- Configure by the path to the directory where the WASM binaries are located
    //        wasm.setWasmPaths(`https://unpkg.com/@tensorflow/tfjs-backend-wasm@${tf.version_core}/dist/`);
    //      or mapping from names of WASM binaries to custom full paths specifying the locations of those binaries
    //        wasm.setWasmPaths({
    //          'tfjs-backend-wasm.wasm': 'renamed.wasm',
    //          'tfjs-backend-wasm-simd.wasm': 'renamed-simd.wasm',
    //          'tfjs-backend-wasm-threaded-simd.wasm': 'renamed-threaded-simd.wasm'
    //        });
    wasm.setWasmPaths(${prefixOrFileMap}); 
    // 2.2- Configure threads number manually, or it will use the number of logical CPU cores as the threads count by default
    wasm.setThreadsCount(n); // n can be 1, 2, ...

    const tf = context.tf;
    await tf.setBackend(backend);
    await tf.ready();

Please refer to the setPolyfillBackend() usage in tests for concrete examples on how to best implement backend switching for your project.

Samples

Web Machine Learning Community Group provides various Samples (GitHub repo) that make use of the WebNN API. These samples fall back to the webnn-polyfill if the browser does not have a native implementation of the WebNN API available by default.

Build and Test

Setup

> git clone --recurse-submodules https://github.com/webmachinelearning/webnn-polyfill
> cd webnn-polyfill & npm install

Build

Development build

> npm run build

Production build

> npm run build-production

Test

Run tests in node.js.

> npm test

Run tests in web browser.

> npm start

Open the web browser and navigate to http://localhost:8080/test

Default backend is CPU backend, you could change to use WebGL backend by http://localhost:8080/test?backend=webgl,
or use Wasm backend by http://localhost:8080/test?backend=wasm

Run only CTS tests in node.js.

> npm run test-cts

Run only CTS tests in web browser.

> npm start

Open the web browser and navigate to http://localhost:8080/test/cts.html

Other scripts

Build docs

> npm run build-docs

Lint

> npm run lint

Format

> npm run format

Start dev server

> npm run dev

Watch files

> npm run watch

License

This project is licensed under the Apache License Version 2.0.

webnn-polyfill's People

Contributors

Stargazers

Watchers

Forkers

huningxin christywl brucedai fujunwei mingmingtasd lcsouzamenezes lionkunonly wangli69087 markghx dibenedetto o2alexanderfedin sigmadeltacom limkokholefork peggy2015 qpc-github quantum-platinum-cloud reillyeon

webnn-polyfill's Issues

Implement the ModelBuilder and update the execution API

Implement the changes of webmachinelearning/webnn#94.

Reuse mocha tests for web platform tests

Raised by @anssiko in 3 September 2020 WebML CG meeting,

anssik: interested in figuring out if we can reuse mocha tests for w-p-t tests

https://‌web-platform-tests.org/

Reduce ops should allow `0` in `axes` option

    if (options.axes !== undefined) {
      utils.assert(
          utils.isIntegerArray(options.axes) &&
              options.axes.every(v => v > 0 || v === -1),
          'The starts parameter is invalid.');

https://github.com/webmachinelearning/webnn-polyfill/blob/master/src/nn/ops/reduce.ts#L20

Add model test for MobileNetV2

MobileNetV2 is one of the first-wave models.

Enable the CI

Support conv2d op with 'ihwo' filterLayout.

Link webmachinelearning/webnn#161

Support ReduceL1 and ReduceL2 and add test cases

CI failed to execute "npm run build-production --if-present"

Detail error message:

ERROR in /home/runner/work/webnn-polyfill/webnn-polyfill/src/nn/utils.ts
./src/nn/utils.ts 2:30-71
[tsl] ERROR in /home/runner/work/webnn-polyfill/webnn-polyfill/src/nn/utils.ts(2,31)
      TS2307: Cannot find module '@tensorflow/tfjs-core/src/ops/conv_util' or its corresponding type declarations.

ERROR in /home/runner/work/webnn-polyfill/webnn-polyfill/src/nn/ops/conv2d.ts
[tsl] ERROR in /home/runner/work/webnn-polyfill/webnn-polyfill/src/nn/ops/conv2d.ts(2,31)
      TS2307: Cannot find module '@tensorflow/tfjs-core/src/ops/conv_util' or its corresponding type declarations.

ERROR in /home/runner/work/webnn-polyfill/webnn-polyfill/src/nn/ops/pool2d.ts
[tsl] ERROR in /home/runner/work/webnn-polyfill/webnn-polyfill/src/nn/ops/pool2d.ts(2,31)
      TS2307: Cannot find module '@tensorflow/tfjs-core/src/ops/conv_util' or its corresponding type declarations.

Implement the sync API and require pre-allocated output buffers

Implement the changes introduced by webmachinelearning/webnn#174

The changes include:

Change MLGraphBuilder.build and MLGraph.compute to sync API.

Change MLGraph.compute to require pre-allocated output buffers.

Simplify the the input / output resources binding by MLResource. Leave MLInput only for dynamic input shape case.

Implement activation functions as fused operators to BatchNorm, Conv2d, gru and gruCell

Implement the spec change introduced by webmachinelearning/webnn#188

Implement the missing element-wise unary operations

Spec: https://webmachinelearning.github.io/webnn/#api-neuralnetworkcontext-unary

Publish a npm package

Raised by @pyu10055 when reviewing lenet example webmachinelearning/webnn-samples#1 (comment)

[Test] add TinyYoloV2 into the model test

TinyYoloV2 is one of the first wave models and is supported by webnn-samples, please add it into the model test. Thanks.

Need update test-data submodule with latest commit id.

Implement concat

Spec: https://webmachinelearning.github.io/webnn/#api-neuralnetworkcontext-concat

Implement the optional dictionaries for operator's optional parameters

Implement the spec change introduced by webmachinelearning/webnn#112

Change min and max values of clamp to float

Align with webmachinelearning/webnn#206:
ClampOptions {
float minValue;
float maxValue;
};

[Test] Add tests for 'conv2d' with 'dilations' option

While developing DeepLab V3 MobileNet V2 example, I captured a bug when using 'conv2d' with 'dilations' option and identified it is a tfjs-webgl-backend issue, see details at tensorflow/tfjs#5293.

So we may need to add relevant tests to cover the gap, besides above checkpoint, tests from https://github.com/tensorflow/tfjs/blob/master/tfjs-core/src/ops/depthwise_conv2d_test.ts maybe good examples for making good test coverage.

Where I capture the issue?

I covert the frozen graph from http://download.tensorflow.org/models/deeplabv3_mnv2_pascal_trainval_2018_01_29.tar.gz to a tflite model, and found the issue at node of MobilenetV2/expanded_conv_14/depthwise/depthwise.

Code snippet to reproduce

    const input = tf.fill([1, 65, 65, 960], 0, 'float32');
    const weights = tf.fill([3, 3, 960, 1], 1, 'float32');
    const result = tf.depthwiseConv2d(input, weights, 1, 'same', 'NHWC', 4);

Support rounding type of pool2d operations

Implement the changes introduced by webmachinelearning/webnn#208

Update CTS tests with MLContext, MLGraphBuilder and MLGraph.

Implement reduction operations

Spec: https://webmachinelearning.github.io/webnn/#api-neuralnetworkcontext-reduce

Implement missing element-wise binary operations

Spec: https://webmachinelearning.github.io/webnn/#api-neuralnetworkcontext-binary

The missing binary ops include sub, div, max and min.

Implement l2Pool2d

Spec: https://webmachinelearning.github.io/webnn/#api-neuralnetworkcontext-pool2d

Add some more concat test cases with two inputs.

Host api docs and unit tests

@anssiko mentioned that

I suggest you link the docs and unit tests (both LGTM) from README.md. They’d be hosted at webmachinelearning.github.io when this PR lands as follows:

https://webmachinelearning.github.io/webnn-polyfill/docs/

https://webmachinelearning.github.io/webnn-polyfill/test/

[test] Failed to test mobilenetv2 batchnorm nchw with test_data_set_0.

Base on MobileNet v2-1.0 model and test data of mobilenet/README.md, we're adding mobilenetv2 batchnorm model tests of pr #49, with input and output data of test_data_set_0, we got this AssertionError error:

AssertionError: expected false to be true
    at Module.checkValue (utils.js:37:12)

, the delta of expected data of ouput[97] 0.015684127807617188, with actual data output[97] 0.015800103545188904 is 0.00011597573757171631, it isn't satisfied with the criteria by atol 1e-5 as and rtol as 1e-3.

delta <= atol + rtol * expected

For CI pass, I skipped to test mobilenetv2 batchnorm model tests with this test_data_set_0.

@huningxin Any suggestion, thanks.

The implement of 'same-upper' autoPad isn't align with the definition in Spec.

According to the definition in Spec

With the "same-upper" option, the padding values are automatically computed such that the additional ending padding
of the spatial input dimensions would allow all of the input values in the corresponding dimension to be filtered.

the implement code as below isn't align with the definition in Spec,

          // https://github.com/webmachinelearning/webnn-polyfill/blob/master/src/nn/ops/conv2d.ts#L136
          // Calculate the explicit paddings for 'same-upper'
          if (this.autoPad_ === MLAutoPad['same-upper']) {
            padding = [[0, 0], [0, 0], [0, 0], [0, 0]];
            const outputSizes = [0, 0];
            for (let i = 0; i < 2; ++i) {
              outputSizes[i] = Math.ceil(input.shape[1 + i] / this.strides_[i]);
            }
            const totalPadding: [number, number] = [0, 0];
            for (let i = 0; i < 2; ++i) {
              totalPadding[i] = this.strides_[i] * (outputSizes[i] - 1) +
                  this.outputPadding_[i] +
                  ((filter.shape[i] - 1) * this.dilations_[i] + 1) -
                  input.shape[1 + i];
            }
            for (let i = 0; i < 2; ++i) {
              padding[i + 1][0] =
                  totalPadding[i] - Math.floor(totalPadding[i] / 2);
              padding[i + 1][1] = Math.floor(totalPadding[i] / 2);
            }

while the above code actually matches 'same-lower' definition :

The "same-lower" option is similar but padding is applied to the beginning padding of the spatial input dimensions instead
of the ending one.

Add a link to open a new issue into the API docs

@anssiko mentioned in webmachinelearning/webnn-samples#4 (comment)

aybe we should consider adding a feedback mechanism as well in the API docs (e.g. simply add a link to open a new issue in the webnn-polyfill repo) to capture web developer feedback early. I suspect API docs is much more familiar interface to learn about the API than the web spec repo for web developers, and we might capture more such feedback that way.

Use TFJS as a dependency injection and not bake it in polyfill distribution.

Linked webmachinelearning/model-loader#5 (comment)

"FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory" happened randomly on CI

I tried to upgrade Node version to 14 for CI environment, and added --max_old_space_size=10240 parameter for npm command, it seems to have no effect.

Fix CTS failures due to API change

There are 50 failures of CTS due to API change of #36 .

@BruceDai , please help fix them. Thanks!

[Test] add the ResNet model test

ResNet is one of the first-wave models and is supported by webnn-samples. Please add it into the model test. Thanks.

The error message in console is

worker.js onmessage() captured an uncaught exception: ReferenceError: o is not defined

@Honry , could you please take a look?

Extend the conv2d operation to support transposed convolution, an essential upsample tool for encoder-decoder models.

Add instanceNormalization in addition to batch-normalization. Instance-normalization is a fused operation for a normalization subgraph that computes the mean and variance values per-feature instance on the fly.

Replace the sqrt unary operation with a more generic pow binary operation used by the normalization process.

Add pad operation that supports all 4 padding modes found in various frameworks.

Add resample operation to support both upsampling and downsampling of feature instances. This operation is used in the ONNX version of the style-transfer models.

Implement 'same-lower' autoPad option for conv2d op.

There's implement on https://github.com/webmachinelearning/webnn-polyfill/blob/master/src/nn/ops/conv2d.ts#L136

          // Calculate the explicit paddings for 'same-lower'
          if (this.autoPad_ === MLAutoPad['same-lower']) {
            padding = [[0, 0], [0, 0], [0, 0], [0, 0]];
            const outputSizes = [0, 0];
            for (let i = 0; i < 2; ++i) {
              outputSizes[i] = Math.ceil(input.shape[1 + i] / this.strides_[i]);
            }
            const totalPadding: [number, number] = [0, 0];
            for (let i = 0; i < 2; ++i) {
              totalPadding[i] = this.strides_[i] * (outputSizes[i] - 1) +
                  this.outputPadding_[i] +
                  ((filter.shape[i] - 1) * this.dilations_[i] + 1) -
                  input.shape[1 + i];
            }
            for (let i = 0; i < 2; ++i) {
              padding[i + 1][0] =
                  totalPadding[i] - Math.floor(totalPadding[i] / 2);
              padding[i + 1][1] = Math.floor(totalPadding[i] / 2);
            }
          }

while below two blocks don't implement to support 'same-lower':

https://github.com/webmachinelearning/webnn-polyfill/blob/master/src/nn/ops/conv2d.ts#L171

          if (this.autoPad_ === MLAutoPad['same-upper']) {
            padding = 'same';
          } else {
            throw new Error(
                'tf.depthwiseConv2d only supports the same-upper auto pad.');
          }

https://github.com/webmachinelearning/webnn-polyfill/blob/master/src/nn/ops/conv2d.ts#L201

        if (this.autoPad_ === MLAutoPad['same-upper']) {
          padding = 'same';
          this.outputSizes_ = [
            input.shape[1] * this.strides_[0],
            input.shape[2] * this.strides_[1],
          ];
        } else {
          throw new Error(
              'tf.conv2dTranspose only supports the same-upper auto pad.');
        }

I think these throw error blocks also need to implement 'same-lower' likes https://github.com/webmachinelearning/webnn-polyfill/blob/master/src/nn/ops/conv2d.ts#L136.

@huningxin What's your opinion, thanks