danielpaz / text20 Goto Github PK

It appears that if no recalibration is being performed the gaze pos is approx. 
correct (set aside the tracker inaccuracies); however when a local recal. is 
performed the gaze data often wanders away quickly, very often to screen 
positions rather far way (~200px). 

Check implementation alternatives...

Either as a single threaded waiting solution or a multithreaded solution.

We should check how long it takes an eye tracker measurement to travel 

1) from the device to the tracking server
2) from the tracking server to the plugin
3) from the plugin to the web frontend 

Clicking on a chart link opens another history chart JFrame regardless if one 
is already displayed or not.

~> ServerPanel.java  ~> CommandListener ~> showHistoryFrame(...)

Otherwise external plugins won't be able to supersede internal functionality. 

1) test XStream converters (said to be faster)
2) find a faster PNG encoding / decoding library

Sometimes, even when using the tracking device mouse, the message "data rate 
low" is emitted and the framework stops, without any notice. 

n/t

When an application is being reloaded in most (if not all) cases, the 
application/plugin will stop working. Java console ouput looks like this:

java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at de.dfki.km.text20.services.trackingdevices.eyes.impl.mouse.MouseTrackingDeviceProviderImpl$MouseTrackingDevice$1.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:680)
java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at de.dfki.km.text20.browserplugin.services.sessionrecorder.impl.xstream.SessionStreamer$WriterThread.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:680)
java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at de.dfki.km.text20.browserplugin.browser.browserplugin.impl.BrowserPluginImpl$4.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:680)
Exception in thread "LogFileWriter.serializer" 
java.lang.IllegalMonitorStateException
    at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:127)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1239)
    at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:431)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:406)
    at net.xeoh.plugins.diagnosis.local.impl.serialization.java.LogFileWriter$1.run(LogFileWriter.java:75)
    at java.lang.Thread.run(Thread.java:680)
Exception in thread "pool-1-thread-1" java.lang.IllegalMonitorStateException
    at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:127)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1239)
    at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:431)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:406)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:680)

JavaScript console is empty.

What steps will reproduce the problem?
1. Open 'Simple Test.html' in FF 3.6.6.
2. Open Firebug console.
3. Keep refreshing the page till the error appears.

What is the expected output? What do you see instead?

Expected: No errors. What i see: 1 error.

Please use labels and text to provide additional information.

See screenshot.

There are a number of implementations for the 'EyeTrackingEvent' interface, not 
all of them seem to  handle the areValid() function correctly. 

Also, the return values should be better documented and 'normalized' in a way, 
that similar situations return similar values (e.g., no eyes -> always return 
null, not sometimes -1, sometimes null) 

The draw methods take alot of cpu cycles... unnecessarily.

It should report something like this:

"An error occurred in the tracking server. Please check the file diagnosis.text 
in the tracking server's directory"

The error should not be reported several times when the application is started 
again. However, when the error was resolved, a message like 

"Normal operation" 

should emitted. We can also think of verbalizing the most frequent causes, as 
long as it is not annoying. 

The file diagnosis.text should contain a human readable diagnosis info. 

We should have "getting started" guide (PDF) with lots of pictures and examples 
that 

1) Gives an overview what the text is and what it is capable of
2) How to take off rapidly (explain, for example, how to create a 
2.1) HTML and a 
2.2) Java Hello World
3) Provide pointers to the Documentation and explain the internal structure of 
3.1) The architecture
3.2) The JavaScript layer 
3.3) The Java layer
3.4) The Tracking Server

TODO:

- Add PEEP package to Framework archive
- Provide README
- Adjust version number (everything becomes 1.2 for next release).
- Adjust liks from PEEP 

Especially Diagnosis and Plugin appear to have a higher than necessary startup 
time, maybe due to network lookups. Check why they take so long to initialize 
and fix that. 

1) Some Diagnosis functionality should be added to the plugin to see if it 
works in applet mode. If it does:
2) Most (if not all) calls to logging.xxx should be replaced with calls to some 
diagnosis channel. 

.

n/t

Use useObjectTag-Flag for debug purposes only.

Should be removed in future.

Right now a plain XML stream is written, but that gets big pretty soon. We 
should investigate if stream-compression is suitable and robust (browsers quit 
quite fast). 

java.lang.IllegalStateException: Socket to remote was closed.
    at net.sf.lipermi.handler.CallProxy.invoke(CallProxy.java:87)
    at $Proxy14.newTrackingEvent(Unknown Source)
    at de.dfki.km.text20.trackingserver.common.remote.impl.CommonServerRegistry$1.run(CommonServerRegistry.java:152)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)

For example, when starting the tracking server at 1280, then starting the 
eyeReader (changes resolution to 1024), the gaze events will be off ... 

And it should be checked if the code can, at all, recover. 

(applies to LowLevelAdapter (and maybe others?))

- When the tracking server comes up it should be able to transmit diagnosis 
information to the Diagnosis to enhance debugging (like driver problems, ...)

- When the diagnosis starts and does not find the tracking server it should 
start anyway and provide sensible help

text2.0 with eye tracking and emotiv support may be a great tool to annotate 
mediawiki content. It can provide valuable feedback to the authors of articles 
from the reader. 

If there are positions in the article many readers stuck, the author gets 
feedback to explain those things in more easy terms. On the same time, the 
article might hide less relevant complex information. Also the suggestion to 
creating links on terms users tend to not know about can be helpful for the 
author. 

The definition of unknown words can get imported from the proper articles using 
a popup interface as the instaview[1] script does.

The author might provide special tags like <important> and <optional> to 
annotate what is needed to understand the topic and what is only for deepening 
the understanding. A colored highlighting in editing mode can give hints where 
the readers stuck. 

[1] http://en.wikipedia.org/w/index.php?title=User:Pilaf/instaview.js 

See QSExpCtrl ...

Currently the diagnosis crashes if not both devices are found.

For high-speed tracking devices (like the TX300) we need to improve the network 
communication, as it currently generates more overhead than necessary.  

Right now, the sessions still contain entries like:

<content>au<DF>en</content>
<content><FC>ber</content>
...

Upon calling text20.core.deregister(spans) the following error is thrown to 
Safari console:
TypeError: Result of expression 'connector.transmitElementRemoved' [undefined] 
is not a function.

And at Chrome:
Uncaught TypeError: Object #<an Object> has no method 'transmitElementRemoved'

Code fragment:

if (data.allSpans) {
    text20.core.deregister(data.allSpans);
    data.allSpans = null;
}

On registration of a new image URL the plugin should download / obtain the 
image and store it in the session stream as well, so that the replay can look 
it up and display it. 

Image URLs supported should be file:// and http:// URLs.

Startup the app and quit it within very little time (1-2s). The window will 
stay there forever until the launcher app is force-terminated.

- Listen to the brain tracker and get a channel list
- Show the channels in an own tab or panel with checkboxes
- Turning them on/off influences the channels drawn in the brain history chart

... better for debugging ...

1) Use complete area (not only upper left)
2) Center circle drawing

What steps will reproduce the problem?
1. Open Bug.html in FF or Chrome (No error in Safari).
2. Open a javascript console.
3. Remove % character and repeat.

What is the expected output? What do you see instead?

Expected: No errors. Result: 1 error.

Please use labels and text to provide additional information.

See attachment.

Many minor code issues, e.g. missing javadocs, never used local variables, 
etc...

No error message is printed, and no output generated either. 

Right now images are ignored. 

Currently it needs quite some hacking to 'fake' or convert data to our replay 
format. There should be an easy API to output a replay-file.

See LowLevelAdapter.java

Again, for debugging, there should be a transparent fullscreen overlay to 
visualize arriving data. The overlay should be accessible though the Java *and* 
JS API. 

Right now the generated code contains many compile warnings with our current 
warning settings. 

Possible solutions: 

1) Either change generation options (if possible)

2) Generate _to JAR_ and include that one.

1) Check that each file contains a LGPL header
2) Check that the names are correct (the one who started the file is printed in 
header, major contributors are added as well)

Starting the demos from Opera crashes the browser. No idea why.

Due to a exe4j/com4j bug the normal launcher we use leaks memory and cannot be 
used.  The current launcher was built with launch4j but lacks the ability to 
log sysout and syserr to files, which makes debugging the tracking server 
extremely hard.

Possible solutions:

1) Either logging has to be built directly in the tracking server.

2) Launch4j has to configured to use logging (is that possible?) 

3) The exe4j launcher has to stop leaking (I already spent 3-4 days on that and 
didn't succeed)


I guess that 1) will be the most efficient solution. 

I.e. change 1200 -> 30s. 

See this stack trace:

2010-jul-07 18:32:03 net.xeoh.plugins.base.impl.loader.FileLoader loadFrom
VARNING: Supplied path does not exist. Unable to add plugins from there.
2010-jul-07 18:32:04 
de.dfki.km.text20.trackingserver.common.remote.impl.CommonServerRegistry init
INFO: Listening on URL 
lipe://7.32.8.253:2256/de.dfki.km.text20.trackingserver.eyes.remote.TrackingServ
erRegistry
2010-jul-07 18:32:04 
de.dfki.km.text20.trackingserver.eyes.adapter.impl.tobii.TobiiGazeAdapter setup
INFO: Initializing Eye-Tracker connection
java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
 at java.lang.reflect.Method.invoke(Unknown Source)
 at net.xeoh.plugins.base.impl.PluginManagerImpl.processPluginLoadedAnnotationForThisPlugin(PluginManagerImpl.java:450)
 at net.xeoh.plugins.base.impl.PluginManagerImpl.hookPlugin(PluginManagerImpl.java:427)
 at net.xeoh.plugins.base.impl.loader.AbstractLoader.processPending(AbstractLoader.java:264)
 at net.xeoh.plugins.base.impl.loader.AbstractLoader.tryToLoadClassAsPlugin(AbstractLoader.java:207)
 at net.xeoh.plugins.base.impl.loader.InternalClasspathLoader.loadAllClasspathPluginClasses(InternalClasspathLoader.java:102)
 at net.xeoh.plugins.base.impl.loader.InternalClasspathLoader.loadFrom(InternalClasspathLoader.java:70)
 at net.xeoh.plugins.base.impl.PluginManagerImpl.doAddPluginsFrom(PluginManagerImpl.java:226)
 at net.xeoh.plugins.base.impl.PluginManagerImpl.addPluginsFrom(PluginManagerImpl.java:208)
 at de.dfki.km.text20.trackingserver.Launcher.main(Launcher.java:68)
Caused by: com4j.ExecutionException: com4j.ComException: 80040154 
CoCreateInstance failed : Class not registered : .\com4j.cpp:137
 at com4j.ComThread.execute(ComThread.java:205)
 at com4j.Task.execute(Task.java:23)
 at com4j.COM4J.createInstance(COM4J.java:95)
 at com4j.COM4J.createInstance(COM4J.java:73)
 at de.dfki.eyetracker.tetapi.ClassFactory.createTetCalibProc(Unknown Source)
 at de.dfki.km.text20.trackingserver.eyes.adapter.impl.tobii.TobiiCalibrator.buildUp(TobiiCalibrator.java:83)
 at de.dfki.km.text20.trackingserver.eyes.adapter.impl.tobii.TobiiGazeAdapter.setup(TobiiGazeAdapter.java:160)
 at de.dfki.km.text20.trackingserver.common.remote.impl.CommonServerRegistry.setupAdapter(CommonServerRegistry.java:175)
 at de.dfki.km.text20.trackingserver.common.remote.impl.CommonServerRegistry.pluginAdded(CommonServerRegistry.java:169)
 ... 13 more
Caused by: com4j.ComException: 80040154 CoCreateInstance failed : Class not 
registered : .\com4j.cpp:137
 at com4j.Native.createInstance(Native Method)
 at com4j.COM4J$CreateInstanceTask.call(COM4J.java:111)
 at com4j.COM4J$CreateInstanceTask.call(COM4J.java:1)
 at com4j.Task.invoke(Task.java:44)
 at com4j.ComThread.run0(ComThread.java:164)