fabiocarrara / meye Goto Github PK

It would be very useful to choose camera source to online version. I need to select a non built-in camera for processing.

Thank you

When the ROI is on the command panel there is no way to interact with video buttons and slider

On both Firefox and Chrome, nothing happens if I press:
T and Y or ALT+T and ALT+Y

I also tried all possible keyboard shortcut using ALT

I think that this feature should be True on startup.
In particularly when webcam is enabled.

In the wiki, the trigger section only shows 2 triggers while in the web app there are 4.
Also keyboard shortcuts are outdated.

Please add a link to the LICENCE file into the LINKS section

framerate seems to drop dramatically during the time. If the live plot is checked showing the entire trace, fps constantly drop to 4 or 5 fps (or even more) in both firefox and chrome. Without a live plot, the effects are milder even though constant.

Default exported filename with the same name as the input

Hello! Thank you very much for developing such a helpful tool.

I've been trying to train a network from my labeled frames, but I'm running into an error:

Traceback (most recent call last): File "C:\Users\systemses\downloads\pupillometry\meye-master\train.py", line 161, in <module> main(args) File "C:\Users\systemses\downloads\pupillometry\meye-master\train.py", line 38, in main print(exp) File "C:\Users\systemses\downloads\pupillometry\meye-master\expman\experiment.py", line 162, in __str__ with pd.option_context('display.width', None), pd.option_context('max_columns', None): File "C:\Users\systemses\anaconda3\envs\tensorflow\lib\site-packages\pandas\_config\config.py", line 434, in __enter__ self.undo = [(pat, _get_option(pat, silent=True)) for pat, val in self.ops] File "C:\Users\systemses\anaconda3\envs\tensorflow\lib\site-packages\pandas\_config\config.py", line 434, in <listcomp> self.undo = [(pat, _get_option(pat, silent=True)) for pat, val in self.ops] File "C:\Users\systemses\anaconda3\envs\tensorflow\lib\site-packages\pandas\_config\config.py", line 128, in _get_option key = _get_single_key(pat, silent) File "C:\Users\systemses\anaconda3\envs\tensorflow\lib\site-packages\pandas\_config\config.py", line 116, in _get_single_key raise OptionError("Pattern matched multiple keys") pandas._config.config.OptionError: 'Pattern matched multiple keys'

Problems with data visualization. The plot became strange if I press pause and then resume after a while.
One possible solution is to reset the plot every time I push pause and then resume.

I think we can consider pause/resume as different recording sessions. So the old data can be flushed.

could be done by stepping time on the video html element

There is still a URL coming soon label in the README file, but no link for the model is currently present

Clear on Resume sometimes leaves the last point sample in the plots list

Problems with data visualization. The plot became strange if I press pause and then resume after a while.
One possible solution is to reset the plot every time I push pause and then resume.

I think we can consider pause/resume as different recording sessions. So the old data can be flushed.

I think that a profile mechanism would be advantageous.
Maybe as a cookie or as some (down)loadable file.

Every time I access MEYE, I need to set up everything again; it would be beneficial if this is accomplished by using cookies or in any other way. It would also be more simple to retrieve the acquisition parameters.

Add a button to the wiki in the GUI.

Describe the bug
A clear and concise description of what the bug is.

When loading either a color or grayscale video (mp4 (mp4v) or avi (XVID)), I get the following traceback:

Traceback (most recent call last):
  File "/git/meye/predict.py", line 81, in <module>
    main(args)
  File "/git/meye/predict.py", line 53, in main
    (pupil_y, pupil_x), pupil_area = compute_metrics(pupil_map, thr=args.thr, nms=True)
  File "/git/meye/utils.py", line 27, in compute_metrics
    p = nms_on_area(p, s)
  File "/git/meye/utils.py", line 9, in nms_on_area
    labels, num_labels = label(x, structure=s)  # find connected components
  File "/lib/python3.10/site-packages/scipy/ndimage/_measurements.py", line 184, in label
    raise RuntimeError('structure and input must have equal rank')
RuntimeError: structure and input must have equal rank

Also oddly, tqdm shows 1/1. Not sure if that should be number of frames.

To Reproduce

Steps to reproduce the behavior:

1. I am running python predict.py ~/Downloads/meye-segmentation_i128_s4_c1_f16_g1_a-relu.hdf5 ~/Desktop/output.avi using a video I created and the v1 model file from Github

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: macOS 12.6
• Browser: n/a (Python)
• Version: Python 3.10

Additional context
Using meye-2022-01-24.h5 model works.

Adding a print of the shape of x and in nms_on_area(x, s) when using meye-segmentation_i128_s4_c1_f16_g1_a-relu.hdf5:

(128, 128, 2) (3, 3)

as opposed to for meye-2022-01-24.h5:

(128, 128) (3, 3)

We can bind triggers to 8 buttons [e.g. 1 to 8] and just code multiple button presses as a 8 bit integer.:
00000000 is 0 no button presses
00000001 is 1 button 1 has been pressed
00000010 is 2 button 2 has been pressed
01010101 is 85 button 1,3,5,7 ave been pressed together

This would expand the potential capabilities of trigger acceptance... Do you think it would be feasible?

• find xy pupil center
• filter smaller connected components

Issues

Noted some small issues with use (easily fixed by users) that I think could be noted in the wiki to help users.

1. If you switch tabs within the same browser in which meye is running, the video continues to play as if it is processing, but data collection halts.

2. If you select "auto update" for either the graphical display or the data table:

   a) data output is significantly slower (on some sample files an average of less than 10 reads/s with "auto update selected" vs ~40-50 reads/s)

   b) with "auto update" for graphical display and/or data table selected, data output continually slows as processing progresses, resulting in more reads for earlier parts of the recording than later (for example in a ~5 minute video during the first 60s I was getting >10 reads/s, 250-310s in was ~6 reads/s, and the last 60s <5 reads/s) . Could pose downstream issues depending on analysis as data could be weighted towards earlier video segments. Output is much more stable when "auto update" is not selected

Suggestions

Mostly, I think it would be nice just to have this documented in the wiki so that users are aware. I especially did not notice the 2b issue until I had run all of my videos and started analyzing the output.

It may be advisable to disable the "auto update" (Issue 2) considering the fewer reads and decreased stability of output reads, but users should be aware of the issue.

Issue 1 is easily fixed by the user - you cannot switch between tabs in a single window but may open a new browser window (or bring other applications to the front). This allows meye to work in the background uninterrupted so long as it is the "active" tab in that window.

Additional context

I have been running meye on google chrome (Version 91.0.4472.77 and 90.0.4430.93) on an iMac (Catalina 10.15.7), have not tested other browsers to see if these are global or specific to chrome

maybe with button and on seek, without adding samples to the table

Since HTML5 does not support avi format, I suggest to add a link or redirect to this site:
https://www.freeconvert.com/avi-to-mp4
or any other website that allows to change movie format

In the jupyter notebook the function

predict()

needs to be modified to

model()

A great thing would be to add shortcuts for start/stop the acquisition through keystrokes!

In the Matlab version of Meye, I tried to replicate Example 3 from example.m but on an image of mine.
When using
ROI = [330,70,130,100];
I get the following error:

Unable to perform assignment because the size of the left side is 130-by-100 and the size of the right
side is 100-by-130.

Error in Meye/predictImage (line 79)
                pupilMask(roiPos(2):roiPos(2)+roiPos(3)-1,...

I think that lines 79 and 80 of Meye.m contain an error with the indices of roiPos and should be changed to:

pupilMask(roiPos(2):roiPos(2)+roiPos(4)-1,...
      roiPos(1):roiPos(1)+roiPos(3)-1) = imresize(rawMask, [roiPos(4), roiPos(3)],"bilinear");

Now it runs but I think there is still something wrong.

maybe buttons loading a category in input fieldset

I'm working on a GUI to help people expand the training dataset.
For now, the software is MATLAB based but the idea would be to also compile it (should be easy for windows).

It works to me, but for sure it will have bugs and fixes to work on

It could be nice if a link to it could be added in meye's README

Here's the tool:
https://github.com/LeonardoLupori/pLabeler

We need to add this to the code
https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html

There is no way to edit the ROI on smartphone

Hey,
The model is having some difficulties predicting the pupil on a small subset of 2P mouse eyes.
I'm collecting images from those movies to improve the training dataset, but in the meantime, I found it to be extremely effective to simply invert the images that I feed to the net with:

import cv2 as cv
invertedFrame = cv.bitwise_not(frame)

It may be useful to add a toggle button to the webapp to invert images in case one is having problems.