benfoxall / jumbotron

NOTE - this is in no way complete

Currently it gives a few helper functions for interacting with js-aruco output.

---

jumbotron

libraries

• https://github.com/jcmellado/js-aruco
• http://sylvester.jcoglan.com/

(not used, but could be useful)

• https://github.com/calvinfo/socket-ntp
• https://help.pubnub.com/entries/24122123-How-do-I-Synchronize-Multiple-Devices-

usage

steps

1. master creates room
2. clients register as being part of the room, display markers
3. master observes clients
4. master closes room, notify clients of world

// mid-level api - recognition

// potentially call several times to observe different angles + new markers
jumbotron.observe(imagedata)

// calculate the world dimensions, and positions of each marker
jumbotron.construct();
// -> {width: 2000, height: 3000, markers: [{id: 32, transform: […]}]}

// get going
jumbotron.start();

---

Development

This should only matter if you are hacking on the internals and fixing stuff (which would be very appreciated).

Setup

Dependencies are managed by bower, install them with:

bower install

Then you need to serve the files through a web server. SimpleHTTPServer is a pretty easy way to do that

python -m SimpleHTTPServer

Then visit localhost:8000 - most interesting stuff is in the /hacks folder.

recognition approach

The marker recognition is performed by js-aruco. The pose estimation component requires the physical size of the marker, which we don't have. Instead, we use the constraint that all markers lie on the same plane to work out their positions relative to each other. Also, by approaching it this way - it allows us to combine several observations cover larger areas and also handle observations with missing markers.

// the structure of marker output from js-aruco
[
	{id:a, corners:[/*pixel coords*/]},
	{id:b, corners:[/*  ...  */]}
]

The first step is to work out the relative positions from the context of each marker (where the other markers have been transformed to make the marker a 1×1 square positioned at 0,0).

From this, we build a 'relative vector grid', which stores observations between markers.

{
	'a-b':[[2,2]],
	'b-a':[[-2,0]]
}

We then take this to build a representation of the markers in an arbitraty space. The current approach is to cycle through observations:

• first item becomes the base for the rest of the space
• any linked markers are translated based from that marker
• any existing marker is scaled, rotated based on it's own view of other markers

Which should end up with something like:

{
	a: [[1,0,0],[0,1,0],[0,0,1]], // identity
	b: [/* rotate(-45) translate(2,2) scale(2) */],
}

---

resolves benfoxall/benfoxall.github.com#16

jumbotron's Issues

Constraint solving approach

I just watched https://www.youtube.com/watch?v=72sWgwaAoyk - constraint solving with javascript.

I'm pretty sure an approach like this would kick ass. I'm just not 100% about how the problem could/would be laid out as a linear constraint problem (@chris-hughes?)

cassowary seems pretty cool - interesting (possibly relevant) demo from a while back

the plan

@chris-hughes, what do you think?

target representation format

Need to work out what kind of output we want from this

alternate markers

snaptag - could look pretty cool
pinch - simple, no camera device needed