- Interactive tools
- Brush
- Knife
- Bucket
- Change the working image size
- Plugin system
- Region computations - plugin features that compute regions
- Property computations - plugin features that compute properties of regions
- deadline - Jan 4 2022
- Tools - plugin specific tools
- deadline - TBD
- The UI is still WIP, so when you click buttons when no folder is opened yet, the App may throw Exceptions
- The label modifying logic (e.g. Brush painting...) is done in the main Thread, therefore the interactive tools are still a bit unresponsive. I'll be optimizing this and moving critical stuff into separate process soon.
Download and install miniconda
from: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
NOTE: On Windows, run steps 3 - 7 in the Anaconda Prompt
- Clone this repository
- Go to the root directory of the cloned repository
- Create a new virtual environment by running
conda create -n arth python=3.8
in terminal. - Activate the virtual environment:
conda activate arth
- Install the requirements:
python -m pip install -r requirements.txt
- Install the app:
python -m pip install -e .
- In the repo's root directory activate the virtual environment (if not active):
conda activate arth
- Run:
python arthropod_describer/app.py
- Import a folder with photos by clicking
File -> Open folder
.- The app will restructure the folder as follows:
- 4 folders will be created:
images
: all the images from the opened folder will be moved here.masks
: binary images marking bugs.sections
: integer label images marking sections of bug bodiesreflections
: binary images marking reflection spots on bugs
- NOTE: as the app modifies the folder structure, maybe it's better to work with a copy of the data.
- On the left a photo can be selected to view and modify its label images.
- At top of photo view, active label image can be selected:
Bug
,Segments
,Reflections
Bug
serves as a mask forSegments
andReflections
- On the right, a colormap can be selected; for a selected colormap the active label can be selected.
- The selected label will be used by tools, which can be selected below the colormap selection widget.
- On the
Plugin
tab you can browse the installed plugins and theComputations
that they provide. - In photo view:
Mouse wheel
: zooms in/outMouse drag
+ holdingMouse wheel
: view panningLeft click
: active tool actionRight click
: nothing yet- Twice
SHIFT
press - quick label search
- Currently the label images are saved to disk when the user changes switches to another photo. This should be discussed, as in the old ArthDescriber version, the label images had to be explicitly "Approved" to be saved.
This implementation of Arthropod Describer is essentially just a viewer for photos and their label images with three interactive tools; the main functionality comes from plugins.
Plugins (each in its own separate directory) must be placed into {root}/arthropod_describer/plugins/
.
An example of plugin directory structure:
.
├── __init__.py
├── plugin.py
├── properties
│ ├── __init__.py
│ └── average_intensity.py
├── tools
│ ├── __init__.py
│ └── some_cool_tool.py
├── plugin.py
└── regions
├── __init__.py
├── body.py
├── eraser.py
└── legs.py
A plugin consists of RegionComputation
s, PropertyComputation
s and Tool
s.
RegionComputation
s must be placed in regions
dir, and similarly for PropertyComputation
s and Tool
s
To create a plugin, follow the above example directory structure, placed into the directory plugins/
Plugin is essentially just an aggregation of Computation
s and Tool
s, so start with creating them first.
You must provide a name and description for every Computation
that your plugin provides.
You do that by providing a class docstring in the required format.
Example:
class BodyComp(RegionComputation):
"""
NAME: Primitive bug body finder.
DESC: Labels the whole body of a bug based on thresholding the blue channel.
"""
For Computations
that require user input, you can provide the user the option to tweak parameters.
Again, you define the user parameters in the docstring:
class BodyComp(RegionComputation):
"""
NAME: Primitive bug body finder.
DESC: Labels the whole body of a bug based on thresholding the blue channel.
USER_PARAMS:
NAME: Blue threshold
KEY: threshold
PARAM_TYPE: INT
VALUE: 200
DEFAULT_VALUE: 200
MIN_VALUE: 0
MAX_VALUE: 255
NAME: Filter size for small components
KEY: filter_size
PARAM_TYPE: INT
VALUE: 25
DEFAULT_VALUE: 25
MIN_VALUE: 1
MAX_VALUE: 55
"""
NAME
- this will be displayed in the AppKEY
- for internal identification, must be unique within the definingComputation
PARAM_TYPE
must be from{INT, STR, BOOL}
VALUE
- current valueDEFAULT_VALUE
- self-explanatoryMIN_VALUE
,MAX_VALUE
- used only whenPARAM_TYPE
: INT, also self-explanatory
Computation
s that can operate on individual regions, they can be declared as REGION_RESTRICTED
, as follows:
class RegionEraser(RegionComputation):
"""
NAME: Region eraser
DESC: Erases regions with the selected labels.
REGION_RESTRICTED
"""
Basically, put REGION_RESTRICTED
anywhere in the docstring.
When the user will activate a REGION_RESTRICTED
Computation
, besides the Computation's user parameters, the user will be
also provided the option to select individual region labels on which the Computation
will operate.
Subclass the class RegionComputation
located in the arthropod_describer.common.plugin
module and override the methods/properties:
@property
def requires(self) -> Set[LabelType]:
pass
@property
def computes(self) -> Set[LabelType]:
pass
def __call__(self, photo: Photo, labels: Optional[Set[int]] = None) -> Set[LabelType]:
pass
The requires
and computes
property indicates which label types (LabelType.BUG, LabelType.REGIONS, LabelType.REFLECTION
) it
needs for computation or produces as a results respectively.
The most important is the __call__
method, where the actual computation takes place. It takes Photo
as a parameter, and
you have an access to these images in the numpy.ndarray
format:
image
- the input RGB imagebug_mask
- mask indicating the whole insect (currently the value 1000 indicates insect, 0 everything else)regions_image
- label image indicating various parts of the insect (legs, thorax...)reflection_mask
- mask indicating spots on the insect that should be excluded from computations of properties. The value 1000 indicates that a particular pixel should be excluded.
The result of the __call__
method is a Python set of LabelType
indicating which label images were modified.
Example __call__
implementation:
def __call__(self, photo: Photo, labels: Optional[Set[int]] = None) -> Set[LabelType]:
cop = photo.regions_image.label_img.copy()
for lab in labels:
cop = np.where(cop == lab, 0, cop)
photo.regions_image = cop
return {LabelType.REGIONS}
When finished with implementing Computation
, assembly them into Plugin
.
Subclass the class Plugin
from arthropod_describer.common.plugin
.
Example:
from arthropod_describer.common.plugin import Plugin, PropertyComputation, RegionComputation
from arthropod_describer.common.common import Info
from arthropod_describer.common.tool import Tool
class TestPlugin(Plugin):
"""
NAME: Test
DESCRIPTION: This is a test plugin for the arthropod describer tool
"""
def __init__(self, info: Optional[Info] = None):
super().__init__(info)