Coder Social home page Coder Social logo

petamoriken / float16 Goto Github PK

View Code? Open in Web Editor NEW
89.0 7.0 5.0 8.52 MB

Stage 3 IEEE 754 half-precision floating-point ponyfill

Home Page: https://github.com/tc39/proposal-float16array

License: MIT License

JavaScript 97.96% HTML 1.59% TypeScript 0.45%
javascript half-precision float16array float16 binary16 fp16 ieee754 typescript

float16's Introduction

IEEE 754 half-precision floating-point for JavaScript
See the archive of the ES Discuss Float16Array topic for details

npm downloads jsDelivr hits npm version deno version
dependencies license codecov

Sauce Labs browser matrix

Install

Node.js

npm install @petamoriken/float16
yarn add @petamoriken/float16

Bun

bun add @petamoriken/float16

Import

Node.js, Bun or Bundler (webpack, rollup.js, esbuild, etc)

// ES Modules
import {
  Float16Array, isFloat16Array, isTypedArray,
  getFloat16, setFloat16,
  f16round,
} from "@petamoriken/float16";
// CommonJS
const {
  Float16Array, isFloat16Array, isTypedArray,
  getFloat16, setFloat16,
  f16round,
} = require("@petamoriken/float16");

Deno

You can get modules from the deno.land/x hosting service.

import {
  Float16Array, isFloat16Array, isTypedArray,
  getFloat16, setFloat16,
  f16round,
} from "https://deno.land/x/float16/mod.ts";

Browser

Deliver a browser/float16.mjs or browser/float16.js file in the npm package from your Web server with the JavaScript Content-Type HTTP header.

<!-- Module Scripts -->
<script type="module">
  import {
    Float16Array, isFloat16Array, isTypedArray,
    getFloat16, setFloat16,
    f16round,
  } from "DEST/TO/float16.mjs";
</script>
<!-- Classic Scripts -->
<script src="DEST/TO/float16.js"></script>
<script>
  const {
    Float16Array, isFloat16Array, isTypedArray,
    getFloat16, setFloat16,
    f16round,
  } = float16;
</script>
Or, you can use jsDelivr CDN.
<!-- Module Scripts -->
<script type="module">
  import {
    Float16Array, isFloat16Array, isTypedArray,
    getFloat16, setFloat16,
    f16round,
  } from "https://cdn.jsdelivr.net/npm/@petamoriken/float16/+esm";
</script>
<!-- Classic Scripts -->
<script src="https://cdn.jsdelivr.net/npm/@petamoriken/float16/browser/float16.min.js"></script>
<script>
  const {
    Float16Array, isFloat16Array, isTypedArray,
    getFloat16, setFloat16,
    f16round,
  } = float16;
</script>

Support engines

This package only requires ES2015 features and does not use environment-dependent features (except for inspect/), so you can use it without any problems. It works fine with the current officially supported versions of Node.js.

Float16Array implemented by Proxy and Reflect, so IE11 is never supported even if you use polyfills.

Pre-transpiled JavaScript files (CommonJS, IIFE)

lib/ and browser/ directories in the npm package have JavaScript files already transpiled, and they have been tested automatically in the following environments:

  • Node.js: Active LTS
  • Firefox: last 2 versions and ESR
  • Chrome: last 2 versions
  • Edge: last 2 versions
  • Safari: last 2 versions

API

Float16Array

Float16Array is similar to TypedArray such as Float32Array (MDN).

const array = new Float16Array([1.0, 1.1, 1.2, 1.3]);
for (const value of array) {
  // 1, 1.099609375, 1.2001953125, 1.2998046875
  console.log(value);
}

// Float16Array(4) [ 2, 2.19921875, 2.3984375, 2.599609375 ]
array.map((value) => value * 2);

isFloat16Array

isFloat16Array is a utility function to check whether the value given as an argument is an instance of Float16Array or not.

const buffer = new ArrayBuffer(256);

// true
isFloat16Array(new Float16Array(buffer));

// false
isFloat16Array(new Float32Array(buffer));
isFloat16Array(new Uint16Array(buffer));
isFloat16Array(new DataView(buffer));

isTypedArray

isTypedArray is a utility function to check whether the value given as an argument is an instance of a type of TypedArray or not. Unlike util.types.isTypedArray in Node.js, this returns true for Float16Array.

const buffer = new ArrayBuffer(256);

// true
isTypedArray(new Float16Array(buffer));
isTypedArray(new Float32Array(buffer));
isTypedArray(new Uint16Array(buffer));

// false
isTypedArray(new DataView(buffer));

getFloat16, setFloat16

getFloat16 and setFloat16 are similar to DataView methods such as DataView#getFloat32 (MDN) and DataView#setFloat32 (MDN).

declare function getFloat16(view: DataView, byteOffset: number, littleEndian?: boolean): number;
declare function setFloat16(view: DataView, byteOffset: number, value: number, littleEndian?: boolean): void;
const buffer = new ArrayBuffer(256);
const view = new DataView(buffer);

view.setUint16(0, 0x1234);
getFloat16(view, 0); // 0.0007572174072265625

// You can append methods to DataView instance
view.getFloat16 = (...args) => getFloat16(view, ...args);
view.setFloat16 = (...args) => setFloat16(view, ...args);

view.getFloat16(0); // 0.0007572174072265625

view.setFloat16(0, Math.PI, true);
view.getFloat16(0, true); // 3.140625

f16round (alias: hfround)

f16round is similar to Math.fround (MDN). This function returns nearest half-precision float representation of a number.

declare function f16round(x: number): number;
Math.fround(1.337); // 1.3370000123977661
f16round(1.337); // 1.3369140625

Float16Array limitations (edge cases)

Float16Array has some limitations, because it is impossible to completely reproduce the behavior of TypedArray. Be careful when checking if it is a TypedArray or not by using ArrayBuffer.isView, and when using Web standards such as structuredClone and WebGL.

Built-in functions

Built-in TypedArray objects use "internal slots" for built-in methods. Some limitations exist because the Proxy object can't trap internal slots (explanation).

This package isn't polyfill, in other words, it doesn't change native global functions and static/prototype methods.

E.g. ArrayBuffer.isView is the butlt-in method that checks if it has the [[ViewedArrayBuffer]] internal slot. It returns false for Proxy object such as Float16Array instance.

ArrayBuffer.isView(new Float32Array(10)); // true
ArrayBuffer.isView(new Float16Array(10)); // false

The structured clone algorithm (Web Workers, IndexedDB, etc)

The structured clone algorithm copies complex JavaScript objects. It is used internally when invoking structuredClone(), to transfer data between Web Workers via postMessage(), storing objects with IndexedDB, or copying objects for other APIs (MDN).

It can't clone Proxy object such as Float16Array instance, you need to convert it to Uint16Array or deal with ArrayBuffer directly.

const array = new Float16Array([1.0, 1.1, 1.2]);
const cloned = structuredClone({ buffer: array.buffer });

WebGL

WebGL requires Uint16Array for buffer or texture data whose types are gl.HALF_FLOAT (WebGL 2) or ext.HALF_FLOAT_OES (WebGL 1 extension). Do not apply the Float16Array object directly to gl.bufferData or gl.texImage2D etc.

// WebGL 2 example
const vertices = new Float16Array([
  -0.5, -0.5,  0,
   0.5, -0.5,  0,
   0.5,  0.5,  0,
]);

const buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);

// wrap in Uint16Array
gl.bufferData(gl.ARRAY_BUFFER, new Uint16Array(vertices.buffer), gl.STATIC_DRAW);
gl.vertexAttribPointer(location, 3, gl.HALF_FLOAT, false, 0, 0);

gl.bindBuffer(gl.ARRAY_BUFFER, null);
gl.enableVertexAttribArray(location);

Others

See JSDoc comments in src/Float16Array.mjs for details. If you don't write hacky code, you shouldn't have any problems.

Float16Array custom inspection

Provides custom inspection for Node.js and Deno, which makes the results of console.log more readable.

Node.js

// ES Modules
import { Float16Array } from "@petamoriken/float16";
import { customInspect } from "@petamoriken/float16/inspect";

Float16Array.prototype[Symbol.for("nodejs.util.inspect.custom")] = customInspect;
// CommonJS
const { Float16Array } = require("@petamoriken/float16");
const { customInspect } = require("@petamoriken/float16/inspect");

Float16Array.prototype[Symbol.for("nodejs.util.inspect.custom")] = customInspect;

Deno

import { Float16Array } from "https://deno.land/x/float16/mod.ts";
import { customInspect } from "https://deno.land/x/float16/inspect.ts";

// deno-lint-ignore no-explicit-any
(Float16Array.prototype as any)[Symbol.for("Deno.customInspect")] = customInspect;

Development

Manual build and test

Manual build

This repository uses corepack for package manager manager. You may have to activate yarn in corepack.

corepack enable yarn

Download devDependencies.

yarn

Build lib/, browser/ files.

yarn run build

Build docs/ files (for browser test).

yarn run docs

Test

This repository uses corepack for package manager manager. You may have to activate yarn in corepack.

corepack enable yarn

Download devDependencies.

yarn

Node.js test

NODE_ENV=test yarn build:lib
yarn test

Browser test

NODE_ENV=test yarn build:browser
yarn docs

Access docs/test/index.html with browsers.

You can access current test page (power-assert version) in master branch.

License

MIT License

This software contains productions that are distributed under the Apache 2.0 License. Specifically, index.d.ts is modified from the original TypeScript lib files.

float16's People

Contributors

deepkolos avatar dependabot-preview[bot] avatar dependabot[bot] avatar greenkeeper[bot] avatar nickedelenbos avatar petamoriken avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

float16's Issues

Conversion fails in some cases

0.499994 will be converted to 0.25

  <script type="module">
    import {
      Float16Array, isFloat16Array, isTypedArray,
      getFloat16, setFloat16,
      f16round,
    } from "https://cdn.jsdelivr.net/npm/@petamoriken/float16/+esm";

    const array = new Float16Array([0.499994]);
    console.log(array[0])
  </script>

image

Using float16 in WebGL2?

Hi. Thanks for making this library. I have a WebGL2 project that requires potentially thousands of floating point texture reads per draw call. Cache and bandwidth performance could be improved by using a fp16 texture instead of fp32, but trying to naively use float16.Float16Array as a replacement for Float32Array causes the error WebGL: INVALID_OPERATION: texImage2D: no bound PIXEL_UNPACK_BUFFER in chrome.
My original code is:
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB32F, res[0], res[1], 0, gl.RGB, gl.FLOAT, new Float32Array(bvhBuffer));
My new/broken code is:
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB16F, res[0], res[1], 0, gl.RGB, gl.HALF_FLOAT, new float16.Float16Array(bvhBuffer));

Any guidance would be appreciated.

conversions to float16 are incorrect for very small numbers

I've been looking into tests now that the proposal is stage 3, and in the process I noticed there's at least one bug in how this library handles rounding of very small values.

Specifically:

The smallest float16 value (call it F16_MIN) is 5.9604644775390625e-8.

Half of that is 2.98023223876953125e-8, which is precisely representable as a double, i.e. a JS number.

The next precisely representable double above F16_MIN/2 is 2.980232238769531911744490042422139897126953655970282852649688720703125e-8. Call that TEST.

TEST should round to F16_MIN: TEST is closer to F16_MIN than to 0. (F16_MIN - TEST < TEST)

But console.log(f16round(2.980232238769531911744490042422139897126953655970282852649688720703125e-8)) gives 0. Per above, that's wrong.

Undefined function when importing float16 from another library in CRA which uses module bundler

Scenario:
My library contains your package as a dependency, this library is imported from create-react-app (CRA) project.

Library code:

const { getFloat16 } = require("@petamoriken/float16");

function conversion(value) {
  var buffer = new ArrayBuffer(4);
  var view = new DataView(buffer);
  view.setUint16(0, value);
  return getFloat16(view, 0, false);
}

The issue:

Uncaught TypeError: getFloat16 is not a function
    at myLibrary(conversion.js:7:1)

Why would the import inside the CRA project be undefined when the library has it as dependency?

I've checked for circular dependencies using:
npx madge --circular --extensions ts,js ./

float16.js: unknown Statement of type "ForOfStatement"

Uncaught Error: Module build failed (from ./node_modules/babel-loader/lib/index.js): Error: ~project/node_modules/@petamoriken/float16/browser/float16.js: unknown Statement of type "ForOfStatement"

Since this hasn't been reported before, I suspect the issue is elsewhere.

@petamoriken/float16 which is a dependency of geotiff which is a dependency of OpenLayers

Complete error from Chrome

olPersonMap.js:487 Uncaught Error: Module build failed (from ./node_modules/babel-loader/lib/index.js):
Error: /myApp/node_modules/@petamoriken/float16/browser/float16.js: unknown Statement of type "ForOfStatement"
at Ep.explodeStatement (:3000/myApp/node_modules/regenerator-transform/lib/emit.js:541:13)
at :3000/myApp/node_modules/regenerator-transform/lib/emit.js:305:12
at Array.forEach ()
at Ep.explodeStatement (:3000/myApp/node_modules/regenerator-transform/lib/emit.js:304:22)
at Ep.explode (:3000/myApp/node_modules/regenerator-transform/lib/emit.js:268:40)
at PluginPass. (:3000/myApp/node_modules/regenerator-transform/lib/visit.js:98:17)
at PluginPass. (:3000/myApp/node_modules/regenerator-transform/lib/util.js:25:17)
at newFn (:3000/myApp/node_modules/@babel/traverse/lib/visitors.js:159:21)
at NodePath._call (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:46:20)
at NodePath.call (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:36:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:92:8)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitSingle (:3000/myApp/node_modules/@babel/traverse/lib/context.js:72:19)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:121:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitMultiple (:3000/myApp/node_modules/@babel/traverse/lib/context.js:67:17)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:119:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitSingle (:3000/myApp/node_modules/@babel/traverse/lib/context.js:72:19)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:121:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitMultiple (:3000/myApp/node_modules/@babel/traverse/lib/context.js:67:17)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:119:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitSingle (:3000/myApp/node_modules/@babel/traverse/lib/context.js:72:19)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:121:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitMultiple (:3000/myApp/node_modules/@babel/traverse/lib/context.js:67:17)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:119:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitSingle (:3000/myApp/node_modules/@babel/traverse/lib/context.js:72:19)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:121:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at NodePath.visit (:3000/myApp/node_modules/@babel/traverse/lib/path/context.js:90:52)
at TraversalContext.visitQueue (:3000/myApp/node_modules/@babel/traverse/lib/context.js:96:16)
at TraversalContext.visitMultiple (:3000/myApp/node_modules/@babel/traverse/lib/context.js:67:17)
at TraversalContext.visit (:3000/myApp/node_modules/@babel/traverse/lib/context.js:119:19)
at traverseNode (:3000/myApp/node_modules/@babel/traverse/lib/traverse-node.js:18:17)
at ./node_modules/@petamoriken/float16/browser/float16.js (olPersonMap.js:487:1)
at webpack_require (bootstrap:63:1)
at ./node_modules/geotiff/dist-module/geotiffimage.js (geotiffimage.js:1:1)
at webpack_require (bootstrap:63:1)
at ./node_modules/geotiff/dist-module/geotiff.js (geotiff.js:1:1)
at webpack_require (bootstrap:63:1)
at ./node_modules/ol/source/GeoTIFF.js (GeoTIFF.js:1:1)
at webpack_require (bootstrap:63:1)
at ./node_modules/ol/source.js (source.js:1:1)
at webpack_require (bootstrap:63:1)

Example does not work, map returns Proxy Object

Thanks for this fantastic library!

from the example, the expected output of the following map should be:

    const array = new Float16Array([1.0, 1.1, 1.2, 1.3]);
    const hey = array.map((value) => value);
    console.log(">>>", hey)

Expected Output

>>> [ 2, 2.19921875, 2.3984375, 2.599609375 ]

Actual Output (wrong)

>>> Proxy(Float16Array) {0: 32256, 1: 32256, 2: 32256, 3: 32256, buffer: ArrayBuffer(8), byteLength: 8, byteOffset: 0, length: 4}

This, however, works:

    const array = new Float16Array([1.0, 1.1, 1.2, 1.3]);
   
    const hey = array.map((value) => console.log(">>>", value ));

Outputs: (correct)

=>>> 2
>>> 2.19921875
>>> 2.400390625
>>> 2.599609375

Float16Array constructor doesn't work with SharedArrayBuffers

Float16Array constructor doesn't work with SharedArrayBuffers. Normally it should also work with those buffers just like the other typed arrays.

  console.log(
    new Float16Array(new ArrayBuffer(8)), // Float16Array(4) [ 0, 0, 0, 0 ]
    new Float16Array(new SharedArrayBuffer(8)) // Float16Array(0) []
  );
  console.log(
    new Uint16Array(new ArrayBuffer(8)), // Uint16Array(4) [ 0, 0, 0, 0 ]
    new Uint16Array(new SharedArrayBuffer(8)) // Uint16Array(4) [ 0, 0, 0, 0 ]
  );

Uncaught Error: Module build failed (from ./node_modules/babel-loader/lib/index.js): Error: ~project/node_modules/@petamoriken/float16/browser/float16.js: unknown Statement of type "ForOfStatement"

Uncaught Error: Module build failed (from ./node_modules/babel-loader/lib/index.js): Error: ~project/node_modules/@petamoriken/float16/browser/float16.js: unknown Statement of type "ForOfStatement"

Since this hasn't been reported before, I suspect the issue is elsewhere.

@petamoriken/float16 which is a dependency of geotiff which is a dependency of OpenLayers

Float16Array lookups appear to be much less performant compared to Float32Array lookups

Hi,
I am using your library in hopes to cut down on memory consumption in my application. I have an array of floats with ~15 million entries, and I have found array lookups to be much slower since converting my Float32Array data type into a Float16Array.

I am trying to figure out if I am doing something incorrectly or if a performance hit is expected on lookups of an array of such a large size. Float16.js get() method is taking lots of time on my performance scans.

float_16_arr[i]; // much slower
float_32_arr[i]; // much faster

Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.