Add a Unit Testing Framework and Write Unit Tests about webgazer

[RobinReborn]

I've started working on testing. I'm using Mocha and Puppeteer.

I'm testing on an instance of webgazerExtractor.js because I can access everything there and eventually adapt the testing framework into something that compares accuracies of different models and regressions.

It's not ready for a PR request but you can see the basic code here:

https://github.com/RobinReborn/WebGazer/blob/testing/test/puppeteer.js

Am open for recommendations of what to test

[jsimonson2013]

@Skylion007 Do you have a particular framework in mind?

[Skylion007]

I'm open to suggestions. :)

[jsimonson2013]

I started setting up karma + mocha over on this fork.

It looks like @TristanVALCKE has done work already with Karma + Jasmine in #52 which seems like a good route, too.

[Skylion007]

So end2end the demos on the website and the calibration page should be tested and make sure it doesn't error.

Also a lot of low hanging fruit for unit testing the utilities and math functions, but we will likely rewrite those soon. Testing all the regressions modes would be good too.

Since this JavaScript, just testing to make sure the the websites don't error out would be a great improvement.

[RobinReborn]

Is there a way of generating input for the calibration demo?

[Skylion007]

@RobinReborn Isn't that exactly what the dataset extractor does? ;)

[jeffhuang]

There's some glue needed to train the separate model and then make it work with the UI

[RobinReborn]

It looks like there are some differences between the current calibration and the study_dot-test.webm videos (based on the .mov file).

Aside from the differences in the size of and graphics of the dots, the dots are presented in an order for the study_dot-test experiment. Current calibration test let's the user pick them in any order except the center one which must be picked last.

So I'll have to change the current calibration test to use the existing dataset to test callibration.

[Skylion007]

Oh yeah, I forgot we updated the calibration page. I mean you could record inputs in a similar way and play it back. Does puppeeter not support some similar functionality?

[RobinReborn]

I'm not sure how I would capture mouse input - and I've discovered that it's difficult to give two programs access to the webcam simultaneously in Ubuntu.

[Skylion007]

Puppeter recorder or something similar could work: https://www.checklyhq.com/docs/puppeteer-recorder/

[RobinReborn]

That doesn't store mouse position.

Could we modify the existing calibration so that the center point can be clicked at any time? We could hide the point from the user.

[Skylion007]

So perhaps we should start with a simpler tasks and just have some Unit Testing / Benchmarking for the Regression components? For instance, an easy way to evaluate they don't error / get an accuracy given a video and corresponding data.

[RobinReborn]

I am working on getting basic tests but the issue is that webgazer has so many dependencies it's not easy to test it at a basic level.

It needs webgl and canvas as well as a DOM.

I'm using puppeteer, and aside from examining the DOM I can also execute javascript on the page. So I can test anything but it will be executed on the videos from webgazerExtract. And anything puppeteer gets from a page is returned as a serializable object so there are some limits to how much I can easily inspect, but I can ensure that the functions work.

[Skylion007]

Another task is probably to factorize the dependencies out as much as possible. LIke we should limit how much of the API actually has to interact with the DOM / WebGL. It's possible to set TensorflowJS to only use the CPU by the way by changing the backend if that helps. That should remove the WebGL dependency.

…

On Mon, Aug 17, 2020 at 5:29 PM Robin Gane-McCalla ***@***.***> wrote: I am working on getting basic tests but the issue is that webgazer has so many dependencies it's not easy to test it at a basic level. It needs webgl and canvas as well as a DOM. I'm using puppeteer, and aside from examining the DOM I can also execute javascript on the page. So I can test anything but it will be executed on the videos from webgazerExtract. And anything puppeteer gets from a page is returned as a serializable object so there are some limits to how much I can easily inspect, but I can ensure that the functions work. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPVMX3KP2LU37GLEPXKRWLSBGOMPANCNFSM4CGIO6AA> .

[RobinReborn]

Right now I have puppeteer working and have managed to send video files as input to the webcam.

I'm using both webgazerExtractClient.html and calibration.html to test webgazer functions. I'm mainly using those pages to execute javascript within an appropriate context, I'm not testing the full functionality of those pages just yet. Benchmarking tests in this context isn't perfect because any results will include the amount of time it takes for the website to respond.

I don't want this PR to be too big so I won't try to get full code coverage. It will include top level webgazer functions (the ones in index.mjs) and some regression tests.

I look forward to any feedback on the testing structure, I did spend some time looking for alternate ways of testing before settling on puppeteer but it's possible that the format I'm writing my tests in isn't the best.

After we agree on the best way of writing the tests it won't be too hard to create new tests, though it will take a lot of work to get 100% code coverage.