Coder Social home page Coder Social logo

codeschmiedehgw / blitz Goto Github PK

View Code? Open in Web Editor NEW
1.0 3.0 0.0 62.95 MB

Bulk Loading and Interactive Time series Zonal analysis

License: GNU General Public License v3.0

Python 100.00%
bulk-loader image-processing image-viewer multiprocessing pyqt5 pyqtgraph python time-series-analysis

blitz's Issues

Normalization

We should change the normalization toolbox from pyqtgraph to our own version.
Some notes:

  • Their normalization always copies all the currently loaded (maybe masked) images before manipulating.
  • Something doesn't work with Subtraction and Divide.
  • Because only the shown images are manipulated, masking, flipping, transposing and all other operations cannot be combined with the current normalization.

Before implementing anything, what kind of normalization do we really need?

For Version X

PREFACE:
This section gives an overview about interesting features. They are not directly to be implemented, but rather be discussed first.

LUT:

  • autoscale feature (ie a button next to the LUT)
  • when scaling LUT manually -> deactivate "Auto-fit"
  • using normalization: HOR/ VERT are not changing, still showing the now normalized version.
  • Should stay constant when changing between min /mx and such

General (these are features to be discussed):

  • Use of Ini file or such to enable preconfig when loading
  • ROI could be user modifiable (like the measurement tool.)
  • Link to GITHUB and INP in Menubar
  • show current Version in Titlebar
  • "OPEN" should refer to the current Path if Images / Video are already loaded
  • Timeline "cropping", so we can "only" show images from /to (i.e. 77 - 332 in a Set of 1000)
  • Save settings per folder: create a small (hidden) file if a folder (or file) is open for the first time. Here we can save dataset specifics, like crop region / Mask, LUT, for that specific dataset
  • Some kind of masking? Like loading a binary of same size?
  • saving program status as preset (like Max. RAM)
  • Link as DOI ur such [https://docs.github.com/de/repositories/archiving-a-github-repository/referencing-and-citing-content]
  • Tooltips
  • Small Manual explaining functionality (+ video)
  • Add the compiler command in Readme (for quick compiling) OR we generate a second MD file "for developers?" (is that a good practice):
pyinstaller --onefile --windowed --name BLITZ --icon=./resources/icon/blitz.ico blitz_main.py
  • small (red, or color of point) dot showing the intersection of the crosshair in both h,v line plots
  • Infos like: - Area and - Circum for the measurement tool. (Maybe even a bounding rect and the corresp. length, width?)

GUI:

  • scroll through images when on timeline instead of zoom; Maybe enable CTRL or SHIFT or such for navigating

Long-term:

  • PCA Implementation (This needs Discussion. Maybe needs own Programm ("Donner")
  • Image Subtraction and Divide are giving problems and are slow. Create our own Implementation with:
    • blur
    • opac / alpha
    • time shift ( so a mean range can "lag" some frames behind)
      i.e: Range of background is "3" and t_shift is "2" -> so image 27 is subtracted by the mean of images 23:25 (=27- 2 - (2:0)
    • different paths (See #18 )

TODO for v1.0.0

BUGS:

  • Loading images with resize leads to image loading failure. -> FIX
  • When using Timeline Operation: horz / vert Profile is showing the original image Profile. not the timeline operation

Visualz / Handling:

  • make two containers (splitted vertical) on the right hand side. LUT should be visible all the time.
  • Manipulation "View" should be moved to "File". Mask probably as well.
  • Timeline Operation and LUT should be visible at the same time.

Features:

  • When clicking in timeline, keyboard arrows do not work! -> Make them!
  • Option: Load Color Images as grayscale

Performance:

  • Flip / Transpose takes time (Is Operation on full matrix -> only on the shown image; later on the matrix)
  • Active "Update ROI only on drop" per default, if images are bigger than 2 MPx
  • STD Calc is very heavy on MEM (Maybe not important)

Add Webserver communication

Brainstorming:

Data:

  • We have a Web Server / Program (i.e. dash). Here you can activate some function "push to blitz" / "expose to port"
  • We have a Database
    in both cases: we expose the images / npy / data

Loading:

  • We have BLITZ "listening" to a URL (or such) for these images. When arriving, we display them.

Features:

  • Maybe autoload, so we listen all the time and refresh when data is changing

Performance

  • Update on Drop should be activated by some other criteria. not matrix size alone, but rather ROI Size * Images < threshold
  • Do we recalculated mean / STD and such all the time? For big data this shouldbe stored.

Drag & Drop of different files leads sometimes to errors. Restarting BLITZ solves it

Traceback (most recent call last):
File "C:\INP\Projekte\Python\BLITZ\blitz\layout\main.py", line 543, in load_images
self.image_viewer.load_data(
File "C:\INP\Projekte\Python\BLITZ\blitz\layout\viewer.py", line 94, in load_data
self.setImage(self.data.image)
File "C:\Users\Mattern\AppData\Local\Programs\Python\Python311\Lib\site-packages\pyqtgraph\imageview\ImageView.py", line 322, in setImage
raise TypeError("Image must be NumPy array or any object "
TypeError: Image must be NumPy array or any object that provides compatible attributes/methods:
['dtype', 'max', 'min', 'ndim', 'shape', 'size']

Web Connection Keep Listening

Once a web connection is established, keep it open until manually closed by the user (e.g. at button press).
This way, not only one image can be sent by the server.

Comments for v1.0.1

Feature:

  • "Busy" Signal (Something to show the user, we are still working) When Masking or ROI or other stuff takes a bit longer

Performance:
-Update on Drop should be activated by some other criteria. not matrix size alone, but rather ROI Size * Images < threshold

Haptics / Looks:

  • Current Path should be set to the folder the images were loaded from.
  • LUT and Options should be 50/50 % in height when starting. Also: switch places

Minor:

  • if ROI is deactivated, mouse(wheel) cant be used to navigate in timeline

Timeline / normalization Issue

With subtract mean"range" in "selection" mode (the "aqua" checkbox):
(Checked with v1.1.0)

  • when moving the blue box, mean is not updated. Have to uncheck / check the "aqua"checkbox, then its updated.
  • When blue box is active: left and right click in timeline do nothing
  • entering digits in the second range box alters numbers in the first

GUI / Haptics / Looks

  • Current Path should be set to the folder the images were loaded from.
  • LUT and Options should be 70 (LUT)/30 % in height when starting. Also: switch places
  • Mask should be above View
  • Handles of Mask and Polygon should be bigger. They are hard to get via mouse.

For V1.0.2

  • Pop up window when doing cost intensiv stuff like: Loading and calculating
  • LUT / Options dialog 60 /40 or such
  • when using Normalization, H/V are not updated
  • ROI handles should be bigger as well
  • Mask has two handles in the lower right corner
  • Show "Calculating" Window only when its actually calculating. not when saved is loaded

General Performance

Observations:

  • STD can take long. Also consumes a lot of RAM
  • even small Videofiles (~200MB) Can take GBs of RAM. Calculation of Backgrund subtraction blows it :-)
  • Can we make (more) use of GPU?

Web Connectivity

Sometimes im running into

[NET] No such file available, aborting

issues, from out of nowhere.
Docker logs for wolke shows no error.
Maybe more info on client side "Filename" and such to debug properly

1.0.3 issues

  • beta is not dynamically updated (value change should be directly be seen)
  • invert mousewheel in timeline
  • when dropping new file (or folder) "timeline Operation" should be back to "All Images", because this is shown in the plot

Minor Bugs / Improvements

Bugs:

  • x,y,value is not updating when hovering with mouse over a pixel and then using arrows to scroll through the timeline
  • Subtract Mean (and other manipulations) should get a waiting signal, to show the system is still responsive
  • Measure Tool: Pixels can be any int (atm its only up to 99); [mm] can have more than two digits.
  • File extensions seem to be case sensetive

Feature:

  • Max. Ram Settings should be saved in ini
  • add third Parameter for the Crosshair ("Time"). So we can "average" over time as well

Logging

Small Issue:

  • Grayscale is the norm. So when loading it, dont give a warning in yellow, just an info in green.
    DAU will cry...

PS:
Next meeting explain dim =4 :-)

Linux installation - Command not found after poetry install

Hi,

I'm trying to install BLITZ on Linux following the install instructions from the README.md file.
After a successful¹ installation BLITZ fails to run with a 'command not found error', as it can be seen in the code below.

¹

  • I'm using python 3.9.18 here (in a virtual environment), but the same was verified on another system running python 3.11
  • I had to change the version of PyQt5 to 5.15.2 in the pyproject.toml file because this looks to be the only version that poetry can install on Linux

(blitz) fellype@plasma:BLITZ$ poetry install
Updating dependencies
Resolving dependencies... (1.3s)

Package operations: 11 installs, 1 update, 0 removals

  • Updating packaging (23.2 /home/fellype/progs_nao_instalaveis/blitz/lib/python3.9/site-packages -> 23.2)
  • Installing llvmlite (0.40.1)
  • Installing numpy (1.24.4)
  • Installing pyqt5-sip (12.13.0)
  • Installing qtpy (2.4.1)
  • Installing natsort (8.4.0)
  • Installing numba (0.57.1)
  • Installing opencv-python (4.8.1.78)
  • Installing psutil (5.9.6)
  • Installing pyqt5 (5.15.2)
  • Installing pyqtgraph (0.13.3)
  • Installing qdarkstyle (3.2.3)

Writing lock file

Installing the current project: BLITZ (1.0.1)
(blitz) fellype@plasma:BLITZ$ poetry run BLITZ
Command not found: BLITZ
(blitz) fellype@plasma:BLITZ$ 

Bugs from 1.1.0

  • Switching back from "Max" to "-" does not show loading
  • Switching between "reduction" Modes centers Crosshair (should NOT happen)
  • "Filename" in the Statusbar should NOT be seperated (Or better: Configurable in INIT)
  • Font (i.e X,Y,V in Statusbar) : Bigger (maybbe also Init)
  • If a folder has subfolders (or other files), although 99% are images, we cant load the folder
  • we should "Show" somewhere the loading options (in red or such). So the user sees, that a REDUCED or A SUBSET was loaded

Performance / GPU

These can be easily done:
pg.setConfigOptions(useNumba=True)
pg.setConfigOptions(useOpenGL=True)

this is a bit more complex:
(Maybe as an option to set in the ini File)
User needs to install cupy manually beforehands.
pg.setConfigOptions(useCupy=True) # Requires CUDA and CuPy ... Bad for small laptops

"Better" Export Function?

Das muß nicht unbedingt. Wenn man weiss wie es geht, (also unten über Export Button) kann man bereits die "Optimierten" Bilder exportieren.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.