Coder Social home page Coder Social logo

mrg7 / human Goto Github PK

View Code? Open in Web Editor NEW

This project forked from vladmandic/human

0.0 0.0 0.0 324.22 MB

Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition

Home Page: https://vladmandic.github.io/human/demo/index.html

License: MIT License

JavaScript 6.62% TypeScript 53.90% CSS 8.86% HTML 30.61%

human's Introduction

Git Version NPM Version Last Commit License GitHub Status Checks Vulnerabilities

Human Library

AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition,
Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis,
Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition, Body Segmentation


JavaScript module using TensorFlow/JS Machine Learning library

  • Browser:
    Compatible with both desktop and mobile platforms
    Compatible with CPU, WebGL, WASM backends
    Compatible with WebWorker execution
  • NodeJS:
    Compatible with both software tfjs-node and
    GPU accelerated backends tfjs-node-gpu using CUDA libraries

Check out Live Demo app for processing of live WebCam video or static images

  • To start video detection, simply press Play
  • To process images, simply drag & drop in your Browser window
  • Note: For optimal performance, select only models you'd like to use
  • Note: If you have modern GPU, WebGL (default) backend is preferred, otherwise select WASM backend

Demos

Project pages

Wiki pages

Additional notes


See issues and discussions for list of known limitations and planned enhancements

Suggestions are welcome!



Options

All options as presented in the demo application...

demo/index.html

Options visible in demo


Examples


Face Close-up:
Face


Face under a high angle:
Angle


Full Person Details:
Pose


Pose Detection:
Pose


Body Segmentation and Background Replacement:
Pose


Large Group:
Group


Face Similarity Matching:
Extracts all faces from provided input images,
sorts them by similarity to selected face
and optionally matches detected face with database of known people to guess their names

demo/facematch

Face Matching


Face3D OpenGL Rendering:

demo/face3d

Face Matching


468-Point Face Mesh Defails:
(view in full resolution to see keypoints)

FaceMesh




Quick Start

Simply load Human (IIFE version) directly from a cloud CDN in your HTML file:
(pick one: jsdelirv, unpkg or cdnjs)

<script src="https://cdn.jsdelivr.net/npm/@vladmandic/human/dist/human.js"></script>
<script src="https://unpkg.dev/@vladmandic/human/dist/human.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/human/1.4.1/human.js"></script>

For details, including how to use Browser ESM version or NodeJS version of Human, see Installation


Inputs

Human library can process all known input types:

  • Image, ImageData, ImageBitmap, Canvas, OffscreenCanvas, Tensor,
  • HTMLImageElement, HTMLCanvasElement, HTMLVideoElement, HTMLMediaElement

Additionally, HTMLVideoElement, HTMLMediaElement can be a standard <video> tag that links to:

  • WebCam on user's system
  • Any supported video type
    For example: .mp4, .avi, etc.
  • Additional video types supported via HTML5 Media Source Extensions
    Live streaming examples:
    • HLS (HTTP Live Streaming) using hls.js
    • DASH (Dynamic Adaptive Streaming over HTTP) using dash.js
  • WebRTC media track using built-in support

Example

Example simple app that uses Human to process video input and
draw output on screen using internal draw helper functions

// create instance of human with simple configuration using default values
const config = { backend: 'webgl' };
const human = new Human(config);

function detectVideo() {
  // select input HTMLVideoElement and output HTMLCanvasElement from page
  const inputVideo = document.getElementById('video-id');
  const outputCanvas = document.getElementById('canvas-id');
  // perform processing using default configuration
  human.detect(inputVideo).then((result) => {
    // result object will contain detected details
    // as well as the processed canvas itself
    // so lets first draw processed frame on canvas
    human.draw.canvas(result.canvas, outputCanvas);
    // then draw results on the same canvas
    human.draw.face(outputCanvas, result.face);
    human.draw.body(outputCanvas, result.body);
    human.draw.hand(outputCanvas, result.hand);
    human.draw.gesture(outputCanvas, result.gesture);
    // and loop immediate to the next frame
    requestAnimationFrame(detectVideo);
  });
}

detectVideo();

or using async/await:

// create instance of human with simple configuration using default values
const config = { backend: 'webgl' };
const human = new Human(config); // create instance of Human

async function detectVideo() {
  const inputVideo = document.getElementById('video-id');
  const outputCanvas = document.getElementById('canvas-id');
  const result = await human.detect(inputVideo); // run detection
  human.draw.all(outputCanvas, result); // draw all results
  requestAnimationFrame(detectVideo); // run loop
}

detectVideo(); // start loop

or using interpolated results for smooth video processing by separating detection and drawing loops:

const human = new Human(); // create instance of Human
const inputVideo = document.getElementById('video-id');
const outputCanvas = document.getElementById('canvas-id');
let result;

async function detectVideo() {
  result = await human.detect(inputVideo); // run detection
  requestAnimationFrame(detectVideo); // run detect loop
}

async function drawVideo() {
  if (result) { // check if result is available
    const interpolated = human.next(result); // calculate next interpolated frame
    human.draw.all(outputCanvas, interpolated); // draw the frame
  }
  requestAnimationFrame(drawVideo); // run draw loop
}

detectVideo(); // start detection loop
drawVideo(); // start draw loop

And for even better results, you can run detection in a separate web worker thread




Default models

Default models in Human library are:

  • Face Detection: MediaPipe BlazeFace - Back variation
  • Face Mesh: MediaPipe FaceMesh
  • Face Iris Analysis: MediaPipe Iris
  • Face Description: HSE FaceRes
  • Emotion Detection: Oarriaga Emotion
  • Body Analysis: MoveNet - Lightning variation
  • Hand Analysis: MediaPipe Hands
  • Body Segmentation: Google Selfie
  • Object Detection: CenterNet
  • Body Segmentation: Google Selfie

Note that alternative models are provided and can be enabled via configuration
For example, PoseNet model can be switched for BlazePose, EfficientPose or MoveNet model depending on the use case

For more info, see Configuration Details and List of Models




Human library is written in TypeScript 4.3
Conforming to JavaScript ECMAScript version 2020 standard
Build target is JavaScript EMCAScript version 2018


For details see Wiki Pages
and API Specification


Stars Forks Code Size CDN
Downloads Downloads Downloads

human's People

Contributors

butzyung avatar ghostnumber7 avatar meeki007 avatar vladmandic avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.