Coder Social home page Coder Social logo

pose-animator-peer's Introduction

Notes

This is a sloppy copy of https://github.com/yemount/pose-animator. It is edited to run with the webrtc server and client from https://github.com/lisajamhoury/WebRTC-Simple-Peer-Examples

Run express-server from the command line from https://github.com/lisajamhoury/WebRTC-Simple-Peer-Examples (No need to open localhost:3000)

Run yarn watch in this folder, this project will connect to your socket server (express_server) on port 80.

The webrtc client is included in utils.

All changes to the demo need to be made in camera.js because it's being built before it is included in the camera.html. See the import line at camera.js line 38. If you need additional functions from the client you need to import them there.

See lines 212 - 223 in camera.js for get and send data functionality.

Pose Animator

Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh. It borrows the idea of skeleton-based animation from computer graphics and applies it to vector characters.

This is running in the browser in realtime using TensorFlow.js. Check out more cool TF.js demos here.

This is not an officially supported Google product.

cameraDemo

cameraDemo

In skeletal animation a character is represented in two parts:

  1. a surface used to draw the character, and
  2. a hierarchical set of interconnected bones used to animate the surface.

In Pose Animator, the surface is defined by the 2D vector paths in the input SVG files. For the bone structure, Pose Animator provides a predefined rig (bone hierarchy) representation, designed based on the keypoints from PoseNet and FaceMesh. This bone structure’s initial pose is specified in the input SVG file, along with the character illustration, while the real time bone positions are updated by the recognition result from ML models.

cameraDemo

// TODO: Add blog post link. For more details on its technical design please check out this blog post.

Demo 1: Camera feed

The camera demo animates a 2D avatar in real-time from a webcam video stream.

Demo 2: Static image

The static image demo shows the avatar positioned from a single image.

Build And Run

Install dependencies and prepare the build directory:

yarn

To watch files for changes, and launch a dev server:

yarn watch

Platform support

Demos are supported on Desktop Chrome and iOS Safari.

It should also run on Chrome on Android and potentially more Android mobile browsers though support has not been tested yet.

Animate your own design

  1. Download the sample skeleton SVG here.
  2. Create a new file in your vector graphics editor of choice. Copy the group named ‘skeleton’ from the above file into your working file. Note:
    • Do not add, remove or rename the joints (circles) in this group. Pose Animator relies on these named paths to read the skeleton’s initial position. Missing joints will cause errors.
    • However you can move the joints around to embed them into your illustration. See step 4.
  3. Create a new group and name it ‘illustration’, next to the ‘skeleton’ group. This is the group where you can put all the paths for your illustration.
    • Flatten all subgroups so that ‘illustration’ only contains path elements.
    • Composite paths are not supported at the moment.
    • The working file structure should look like this:
        [Layer 1]
        |---- skeleton
        |---- illustration
              |---- path 1
              |---- path 2
              |---- path 3
    
  4. Embed the sample skeleton in ‘skeleton’ group into your illustration by moving the joints around.
  5. Export the file as an SVG file.
  6. Open Pose Animator camera demo. Once everything loads, drop your SVG file into the browser tab. You should be able to see it come to life :D

pose-animator-peer's People

Contributors

lisajamhoury avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.