Coder Social home page Coder Social logo

ome / omero-parade Goto Github PK

View Code? Open in Web Editor NEW
1.0 7.0 11.0 780 KB

OMERO.web plugin for displaying Dataset thumbnails or Plates in webclient center panel

License: GNU Affero General Public License v3.0

HTML 1.40% JavaScript 61.56% CSS 5.78% Python 30.88% Shell 0.25% Dockerfile 0.13%
omero webapp

omero-parade's Introduction

OMERO.parade

An OMERO.web app for filtering Data in OMERO.web centre panel.

For full details see SUPPORT.md.

Requirements

  • OMERO 5.6.0 or newer
  • Python 3.6 or newer

Installing from PyPI

This section assumes that an OMERO.web is already installed.

Install the app using pip:

$ pip install -U omero-parade

Add parade custom app to your installed web apps:

$ bin/omero config append omero.web.apps '"omero_parade"'

Display parade in the centre of the webclient:

$ bin/omero config append omero.web.ui.center_plugins \
    '["Parade", "omero_parade/init.js.html", "omero_parade"]'

Now restart OMERO.web as normal.

Build

In order to build you need:

  • npm version equal or greater to 3.0! npm version equal or greater than 5.2 is recommended!
$ npm install

To build an uncompressed version and automatically rebuild when source files change, run:

$ npm run watch

To build an uncompressed version, run:

$ npm run build-dev

To build a compressed, minified version for production, run:

$ npm run build

Custom Filtering

Users can customize the filtering options available by adding their own python modules to the setting:

omero.web.parade.filters

The current default setting lists the omero_parade app itself and two other modules that are in the same directory and are therefore expected to be on the PYTHONPATH when the app is installed.

'["omero_parade", "omero_parade.annotation_filters", "omero_parade.table_filters"]'

Each of these modules contains an omero_filters.py which is expected to implement 2 methods: get_filters and get_script.

The get_filters method is used to compile the list of filters returned by the URL /omero_parade/filters/.

Some examples of get_filters

# Return a list of filter names.
def get_filters(request, conn):
    return ["Rating", "Comment", "Tag"]

The request may include plate or dataset ID if we only want to support the filter for certain data types. In this example we could even check whether an OMERO.table exists on the plate.

def get_filters(request, conn):
    if request.GET.get('plate', None) is not None:
        return ["Table"]
    return []

The get_script function for a named filter should return a JsonResponse that includes a list of parameters for the user input to the filter and a JavaScript filter function.

The JavaScript function will be called for each image to filter and will also be passed in a params object with the user input.

# Return a JS function to filter images by various params.
def get_script(request, script_name, conn):

    dataset_id = request.GET.get('dataset')
    // OR...
    plate_id = request.GET.get('plate')

    if script_name == "Rating":
        # Load rating data for images in Dataset or Wells in Plate...
        # ...
        # var ratings = {imageId: rating} for all images
        var js_object_attr = 'id';  # or 'wellId' if filtering Wells

        # Return a JS function that will be passed an object
        # e.g. {id: 1} for Image or {id: 1, wellId:2} for Image in Well.
        # and should return true or false
        f = """(function filter(data, params) {
            var ratings = %s;
            var match = ratings[data.%s] == params.rating;
            return (params.rating === '-' || match);
        })
        """ % (json.dumps(ratings), js_object_attr)

        filter_params = [{'name': 'rating',
                        'type': 'text',
                        'values': ['-', '1', '2', '3', '4', '5'],
                        'default': '-',
                        }]
        return JsonResponse(
            {
                'f': f,
                'params': filter_params,
            })

Custom Data Providers

Custom data providers return numerical data for Images that can be shown in a table for sorting, or plotted in a graph. NB: Even if data applies to Wells, you need to map this to Image ID, since that is the common denominator that is used to identify images in the various list, grid or plot layouts.

Using the same setup as for filtering above, each module listed in the omero.web.parade.filters setting can also contain a data_providers.py file that implements two methods get_dataproviders and get_data.

Examples for omero_parade/data_providers.py

def get_dataproviders(request, conn):
    return ["ROI_count"]


def get_data(request, data_name, conn):
    """Return data for images in a Dataset or Plate."""
    dataset_id = request.GET.get('dataset')
    plate_id = request.GET.get('plate')
    field_id = request.GET.get('field')

    # ... get img_ids for container, then...

    if data_name == "ROI_count":
        # Want to get ROI count for images
        params = ParametersI()
        params.addIds(img_ids)
        query = "select roi.image.id, count(roi.id) from Roi roi "\
                "where roi.image.id in (:ids) group by roi.image"
        p = query_service.projection(query, params, conn.SERVICE_OPTS)
        roi_counts = {}
        for i in p:
            roi_counts[i[0].val] = i[1].val
        return roi_counts

Release process

This repository uses bump2version to manage version numbers. To tag a release run:

$ bumpversion release

This will remove the .dev0 suffix from the current version, commit, and tag the release.

To switch back to a development version run:

$ bumpversion --no-tag [major|minor|patch]

specifying major, minor or patch depending on whether the development branch will be a major, minor or patch release. This will also add the .dev0 suffix.

Remember to git push all commits and tags.

License

This project, similar to many Open Microscopy Environment (OME) projects, is licensed under the terms of the GNU General Public License (GPL) v2 or later.

Copyright

2019-2020, The Open Microscopy Environment

omero-parade's People

Contributors

chris-allan avatar dependabot[bot] avatar emilroz avatar jburel avatar joshmoore avatar manics avatar sbesson avatar will-moore avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

omero-parade's Issues

Click on dataset

Click on an orphaned dataset that is not expanded.
The images are loaded in the tree but not in parade
It works if the dataset is in a project

Bug: javascript not packaged in devspace

see: #64 (comment)

Uncaught TypeError: omero_parade.omero_parade is not a function
    at jQuery.fn.init.load_plugin_content ((index):3024)
    at proxy (jquery-1.11.1.js:547)
    at update_plugin_content (ome.popup.js?_5.6.dev5:722)
    at HTMLBodyElement.<anonymous> (ome.popup.js?_5.6.dev5:759)
    at HTMLBodyElement.dispatch (jquery-1.11.1.js:4641)
    at HTMLBodyElement.elemData.handle (jquery-1.11.1.js:4309)
    at Object.trigger (jquery-1.11.1.js:4550)
    at HTMLBodyElement.<anonymous> (jquery-1.11.1.js:5260)
    at Function.each (jquery-1.11.1.js:383)
    at jQuery.fn.init.each (jquery-1.11.1.js:136)

@sbesson suspects this is due to the difference between virtualenv and pip -mvenv

Filter with OR and NOT

Feedback from workshops via @pwalczysko:

Want to be able to exclude images - e.g. with rating 1 or Tag "bad".
Also select using OR e.g. Dataset 1 OR Dataset 2.

Bug: table_cols List Index out of range

From https://www.openmicroscopy.org/qa2/qa/feedback/31187/

Traceback (most recent call last):

File "/opt/omero/venv_web/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/omeroweb/decorators.py", line 538, in wrapped
retval = f(request, *args, **kwargs)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/omero_parade/views.py", line 128, in filter_script
return module.get_script(request, filter_name, conn)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py", line 194, in get_script
'default': table_cols[0],

IndexError: list index out of range

Load Tables from Datasets within a Project

See https://forum.image.sc/t/using-omero-parade-on-dataset-level-and-image-level-omero-tables/50180/7

When loading data for filtering, don't just check for csv or OMERO.tables on the parent Project.
Also check on the child Datasets.

The returned JSON would have to indicate which Datasets the filters were coming from. E.g.

/parade/filters/?project=152

"data": [
"ROI_count",
"Rating",
"Comment",
"Tag",
"Key_Value",
"Table",
"idr0021_merged_results.csv",
"Batch_ROI_Export.csv",
"roi_intensities.csv"
"Dataset 123: dataset_values.csv"
]

These would have to be shown in a list that indicates which Dataset they're coming from.
And when we load a filter, the URL would also need to indicate which Dataset the table came from (as it would if the Dataset were an orphan)

Multiple OMERO.tables picked randomly

See https://forum.image.sc/t/multiple-omero-tables-crash-omero-parade/72477

Currently we do this in table_filters/data_providers.py, which picks a "random" table since _bulk_file_annotations() returns unsorted results from HQL query:

def get_table(conn, objtype, objid):
    data = _bulk_file_annotations(None, objtype, objid, conn=conn).get(
        'data', [])
    if len(data) < 1:
        return None
    # Just use the first Table we find
    # TODO: handle multiple tables!?
    data = data[0]

Simply need to sort by ID, then pick the highest ID (most recent).

NB: Supporting multiple OMERO.tables (as we do for csv) is a much bigger task

Parade for histo cytometry application

I was wondering if parade was able to process quite easily histo cytometry application (with dataset for different condition/activation on Omero) if the segmentation results is given as mask (to extract results) ? Scatterplot are interactive in parade with the image isn't-it so it seems quite powerful for that ?

Data provider min() error if no images

For example, select an empty Orphaned Dataset and "Add Table Data..." > "ROI_count"

Since the img_ids list is empty, we get empty values list in
numpy.amin(values) which fails.

Nightshade error reported with URL /parade/data/Uk9JX2NvdW50/ (ROI_count) with no query string for e.g. ?dataset=1, which gives us an empty img_ids list. Don't know how this occurred in the UI.

Internal Server Error: /parade/data/Uk9JX2NvdW50/
Traceback (most recent call last):
 File "/opt/omero/web/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
   response = wrapped_callback(request, *callback_args, **callback_kwargs)
 File "/opt/omero/web/OMERO.web/lib/python/omeroweb/decorators.py", line 488, in wrapped
   retval = f(request, *args, **kwargs)
 File "/opt/omero/web/venv/lib/python2.7/site-packages/omero_parade/views.py", line 180, in get_data
   'min': numpy.amin(values).item(),
 File "/usr/lib64/python2.7/site-packages/numpy/core/fromnumeric.py", line 1987, in amin
   out=out, keepdims=keepdims)
 File "/usr/lib64/python2.7/site-packages/numpy/core/_methods.py", line 14, in _amin
   out=out, keepdims=keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity

Map Annotations filter with duplicate Keys

See https://forum.image.sc/t/anomalies-in-searching-kvpairs-in-parade-and-main-omero-search-tool/29124

If an Image has Map annotations Key: val_A and Key: val_B then one of these will be ignored since we return a "Dict of {'key': {iid: 'value', iid: 'value'}}". E.g. for 2 images:

{"Key": {123: val_B, 456: 'foo'}

and val_A is ignored.

# Dict of {'key': {iid: 'value', iid: 'value'}}

Simplest "fix" is to concatenate values e.g.

    if map_values[kv.name][iid]:
        map_values[kv.name][iid] += kv.value
    else:
        map_values[kv.name][iid] = kv.value

filter_list getObjects(None)

Reported at https://www.openmicroscopy.org/qa2/qa/feedback/29402/

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero_parade/views.py", line 114, in filter_list
filters.extend(module.get_filters(request, conn))

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py", line 53, in get_filters
csv_anns = get_csv_annotations(conn, obj_type, obj_id)

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero_parade/csv_filters/data_providers.py", line 34, in get_csv_annotations
obj = conn.getObject(obj_type, obj_id)

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero/gateway/__init__.py", line 3268, in getObject
obj_type, oids, params, attributes, opts)

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero/gateway/__init__.py", line 3349, in buildQuery
"'Image' not %r" % obj_type)

AttributeError: getObjects uses a string to define obj_type, E.g. 'Image' not None

Add table data combobox

  • log in as user-1 on web-dev-merge
  • select archiveddv
  • In parade:Select in combobox "Add table Data" the ROI count option
Traceback (most recent call last):

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/opt/hudson/workspace/WEB-DEV-merge-deploy/OMERO.py/lib/python/omeroweb/decorators.py", line 483, in wrapped
    retval = f(request, *args, **kwargs)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/omero_parade/views.py", line 181, in get_data
    'histogram': list(histogram)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/http/response.py", line 535, in __init__
    data = json.dumps(data, cls=encoder)

  File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)

  File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)

  File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/core/serializers/json.py", line 115, in default
    return super(DjangoJSONEncoder, self).default(o)

  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")

TypeError: 6 is not JSON serializable


<WSGIRequest
path:/omero_parade/data/Uk9JX2NvdW50/,
GET:<QueryDict: {u'_': [u'1524141990109'], u'dataset': [u'1457']}>,
POST:<QueryDict: {}>,
COOKIES:{'_ga': 'GA1.2.74430613.1492428942',
 '_gid': 'GA1.2.1403166767.1524053537',
 'csrftoken': 'FojPGWicIBZJrzOBiUS03HPIRxsM73I2',
 'phpbb3_a9mfv_k': '',
 'phpbb3_a9mfv_sid': 'a419d989a95ebbcf51915de7ed5a1532',
 'phpbb3_a9mfv_u': '1',
 'sessionid': '5x7euiyxys7eoel6p62qapoa126myv19',
 'style_cookie': 'null'},
META:{u'CSRF_COOKIE': u'FojPGWicIBZJrzOBiUS03HPIRxsM73I2',
 'HTTP_ACCEPT': 'application/json, text/javascript, */*; q=0.01',
 'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.9,fr;q=0.8',
 'HTTP_CONNECTION': 'close',
 'HTTP_COOKIE': '_ga=GA1.2.74430613.1492428942; csrftoken=FojPGWicIBZJrzOBiUS03HPIRxsM73I2; phpbb3_a9mfv_k=; style_cookie=null; phpbb3_a9mfv_u=1; phpbb3_a9mfv_sid=a419d989a95ebbcf51915de7ed5a1532; sessionid=5x7euiyxys7eoel6p62qapoa126myv19; _gid=GA1.2.1403166767.1524053537',
 'HTTP_HOST': 'web-dev-merge.openmicroscopy.org',
 'HTTP_REFERER': 'http://web-dev-merge.openmicroscopy.org/webclient/',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36',
 'HTTP_X_FORWARDED_FOR': '134.36.162.209',
 'HTTP_X_FORWARDED_PROTO': 'http',
 'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest',
 'PATH_INFO': u'/omero_parade/data/Uk9JX2NvdW50/',
 'QUERY_STRING': 'dataset=1457&_=1524141990109',
 'RAW_URI': '/omero_parade/data/Uk9JX2NvdW50/?dataset=1457&_=1524141990109',
 'REMOTE_ADDR': '127.0.0.1',
 'REMOTE_PORT': '49318',
 'REQUEST_METHOD': 'GET',
 'SCRIPT_NAME': u'',
 'SERVER_NAME': '127.0.0.1',
 'SERVER_PORT': '4080',
 'SERVER_PROTOCOL': 'HTTP/1.0',
 'SERVER_SOFTWARE': 'gunicorn/19.7.1',
 'gunicorn.socket': <socket._socketobject object at 0x725b980>,
 'wsgi.errors': <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x690f890>,
 'wsgi.file_wrapper': <class 'gunicorn.http.wsgi.FileWrapper'>,
 'wsgi.input': <gunicorn.http.body.Body object at 0x690fb10>,
 'wsgi.multiprocess': True,
 'wsgi.multithread': False,
 'wsgi.run_once': False,
 'wsgi.url_scheme': 'http',
 'wsgi.version': (1, 0)}>

cc @will-moore @emilroz @chris-allan

Support OMERO.table StringColumn

From https://www.openmicroscopy.org/qa2/qa/feedback/21658/ and https://www.openmicroscopy.org/qa2/qa/feedback/21733/

If a user has an OMERO.table with StringColumns they get listed in the "Add Table data..." menu but when chosen, the loading of data fails because histogram can't be created (error below).

Seems that when loading filters (includes ALL columns from a table) we only try to get numpy min/max/histogram for DoubleColumn and LongColumn (https://github.com/ome/omero-parade/blob/master/omero_parade/table_filters/omero_filters.py#L74)

However, for loading Table data, we don't test column type, so numpy.histogram, numpy.amin, numpy.amax are tried for e.g. StringColumn which fails.
If I comment-out these lines in views.py get_data() then it returns just the values dict which is displayed in the OMERO.parade table OK.
Not sure why we need to calculate histogram, min and max since they're not used by the UI in any way?

For StringColumns that actually contain Long/Float values, I wrote a script to convert columns to LongColumn or DoubleColumn in a new Table:
https://gitlab.com/openmicroscopy/incubator/python-scripts/blob/master/omero_table_stringcol_to_doublecol.py
but this doesn't help if the values are actually Strings.

File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/omero_parade/views.py", line 177, in get_data
histogram, bin_edges = numpy.histogram(values, bins=bins)
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/lib/histograms.py", line 676, in histogram
bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/lib/histograms.py", line 299, in _get_bin_edges
first_edge, last_edge = _get_outer_edges(a, range)
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/lib/histograms.py", line 250, in _get_outer_edges
first_edge, last_edge = a.min(), a.max()
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/core/_methods.py", line 32, in _amin
return umr_minimum(a, axis, None, out, keepdims, initial)
TypeError: cannot perform reduce with flexible type
path:/parade/data/VGFibGVfU3BvdCBDZWxsIENvdW50/,

Removing filters re-loads other filters

From #27 (comment)
Each time a filter is removed the key_value pairs are reloaded and UI regenerated
Example:

  • Add ROI_count
  • Add Key_value. wait for the UI to be built
  • Remove ROI_count
  • The Key_value component will be rebuilt.

Filter Table outlier

Just documenting this scenario for reference / future thought...

In preparing the OME meeting workshop, I generated a Table from ROIs and wanted to filter by area ("max_points"). However, 1 image had no 'blobs' on it so the ImageJ "Analyse Particles" polygon was the whole plane.
This single outlier meant that the min/max range was huge and the histogram failed to render since it had 1950 bins!

screen shot 2018-05-05 at 06 33 11

Also the first movement of the slider instantly filtered out ALL images except the outlier.
I deleted the outlier image, but this didn't help as the Table data is loaded without checking whether images exist for each row (and looking at the code it seems this would be quite hard to do without some lengthy parsing of the data).

However, my only solution was to re-build the Table from ROIs after deleting the image, so it would be nicer if we can think of any improvement to this workaround.

Multiple fields per well

Following up on the discussion started on: https://forum.image.sc/t/omero-parade-accepting-stringcolumns-from-omero-metadata-and-loading-multiple-plates/27767/16 on how to deal with multiple fields/images per well, I'd like to offer some suggestions:

  1. on @will-moore 's concern how to how handle the Plate view display, I think the current approach of just showing the first field in the well is adequate since the Plate view is likely only used in the filtering step when Well level experimental metadata is used to filter down to the conditions of interest. Maybe put up a warning that only field 1 per well is shown. To make it more "cute", it would be fun to use the "Field positions in well" panel to show per field heatmap detail when the user selects a single well while in heatmap mode.
  2. Most of the measured values that one would want to display are typically reported per field and not per well. So indeed, the Table/List view and Scatter Plot view should show the fields/images. And consequently the bulkannotations tables should be attached to Field/Image rather than to Well as currently. To make that convenient, it would be nice to provide an update to Populate_Metadata.py and its underlying code to easily do so.
  3. The above will also resolve an issue with displaying whatever image data is selected using the Scatter Plot or Table/List: currently such a selection is a list of wells which iviewer does NOT accept; if such a selection is a list of images, then iviewer can accept that as a multi-selected set and provide great visualization of that set.

Thanks for considering,
Damir

Tests still disabled

While cleaning up directories from the Python 3 migration, I ran across 8f38710:

commit 8f38710a90de8f38ab716c758b8dc575d9e0fa27 (HEAD, will/django_1.9)
Author: jmoore <[email protected]>
Date:   Fri Nov 15 14:12:45 2019 +0000

    Disable pytest for this repo

    For some reason, on travis, three "tests" are failing:

    ```
    data_providers (unittest.loader._FailedTest) ... ERROR
    omero_filters (unittest.loader._FailedTest) ... ERROR
    data_providers (unittest.loader._FailedTest) ... ERROR
    ```

    but not locally. For the moment, disabling pytest. When/if
    a `test/` directory is created, the script will fail.

diff --git a/.omeroci/app-build b/.omeroci/app-build
new file mode 100755
index 0000000..3992193
--- /dev/null
+++ b/.omeroci/app-build
@@ -0,0 +1,19 @@
+#!/bin/bash
+
+source /infra/utils
+
+set -e
+set -u
+set -x
+
+cd $TARGET
+DIR=$(setup_dir)
+
+export DJANGO_SETTINGS_MODULE=omeroweb.settings
+export OMERODIR=/opt/omero/web/OMERO.web
+export ICE_CONFIG=${OMERO_DIST}/etc/ice.config
+
+if [ -e test ]; then
+   echo Tests currently disabled
+   exit 1
+fi

Ignore #header row for csv tables

Just got a bug report from a user trying to use OMERO.parade on a Dataset at https://demo.openmicroscopy.org/webclient/?show=dataset-6308
This bug report was actually due to the user trying to filter using a csv that had the #header for populate script, so that the column names are not read correctly and therefore the first value added at

column_data[name].append(value.strip())

is the actual column name, so that all columns contain at least 1 string. So this fails:

            # try to convert to number (if not empty string)
            values = [float(v) if len(v) > 0 else v for v in values]

and we get this exception:

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/views.py" in filter_script
  128.                 return module.get_script(request, filter_name, conn)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py" in get_script
  194.                       'default': table_cols[0],

Exception Type: IndexError at /parade/filters/script/metadata.csv/
Exception Value: list index out of range

Character encoding

It seems that when table columns have a "ยต" in their name, OMERO.parade is not able to use them for graphs.
These tables are come from a Fiji Jython script where the results are saved as tables, like this one in Groovy. Maybe there is something to do there I guess? (Replacing u'\xb5' by "u" before saving the table works for example.)

The following error appears:

Traceback (most recent call last):

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/omeroweb/decorators.py", line 538, in wrapped
    retval = f(request, *args, **kwargs)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/omero_parade/views.py", line 172, in get_data
    data_name = data_name.decode('utf-8')

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 23: invalid start byte

Re-design with crossfilter?

Having had a look at the crossfilter docs, some thoughts on what adoption of crossfilter might give us:

Current issues:

  • Would be nice to get rid of eval() for filter functions
  • Loading a whole table of data: only use 1 column at a time. Need to load table again to filter by another column.
  • Have to load the table data for filter AND again if we want to show in table.
  • Don't handle multiple tables attached to parent Project / Screen.
  • Don't handle text columns for OMERO.tables
  • Don't handle numerical values for Map annotations

Crossfilter provides lots of nice features, but how would it fit into parade?

  • Assume we don't need current 'parade filter' eval() functionality, and we simply use 'dataprovider' to load a table (all, or selected columns), which is what crossfilter expects.
  • Init crossfilter with a table of data. Then allow the user to choose dimensions to filter by. Less-than, greater-than, equal or range for numerical columns, equal/contains for string columns.
  • May want to group-by Dataset, Tags or various table columns.
  • After crossfilter init (and create dimension etc) is it possible to load additional data for filtering or display? Not as far as I can see. Might need to re-init crossfilter and re-create dimensions and filters.
  • Might want to plot data with https://dc-js.github.io/dc.js/ which seems to give a lot of nice features out of the box. How does user choose what to plot?
  • Don't know how well dc.js works with React.

cc @chris-allan @emilroz

Right hand panel issue with multi-select images

Selecting multiple images using results in a different right hand panel being displayed.

To reproduce:
Whilst holding ctl (or cmd) select one image, then another

screenshot 2018-12-04 at 11 12 49

Now select a third image

screenshot 2018-12-04 at 11 17 55

Finally, unselect the third image to return to two images

screenshot 2018-12-04 at 11 18 08

Webclient controls while in parade

Sorry, I cannot find the original issue here, neither in Open nor in Closed.
So, opening a new issue.

  • have a parade graph on display in central pane
  • go to right-hand pane, click on some control, e.g. addition of Tag (the plus sign)
  • observe that the popup of the Tag window happens under the graph of parade in central pane, and the control inside the Tag window are not possible to click

Note: this happens also if the left-hand pane (LHP)
screen shot 2018-08-25 at 20 25 52
tree controls are deployed, such as Rendering settings on Dataset from context menu on LHP

Initial TODOs

Feedback & TODOs from initial presentation to team today:
See notes at https://docs.google.com/document/d/1bh9OnP00BRA_yCbuWo5uuFk7qtexz8nWZZR6_DgAZro/

  • Jason: find limit of image counts
  • JM: don't load thumbnails by default - heatmap
  • Chris: eventually replace webclient centre panel
  • JM: Export plot
  • Petr: export filtered view of data
  • Kenny: Save / bookmark filtered state
  • Chris: need to move to OME repo, pip installable, travis etc.
  • Jason: likes OMERO.parade name!
  • Heatmap on 'grid' thumbnails

Graph in parade y-axis Firefox

In Parade, click onto the Graph box icon.
The graph appears as expected.
The graph has a selection widget on x-axis as expected.
But in FF on Mac, there is no seletction widget on y-axis, neither is the y-axis labelled with numbers.
The same graph has the missing y-axis widget when you switch to Chrome

Firefox
screen shot 2018-06-06 at 12 22 42

Chrome

screen shot 2018-06-06 at 12 26 03

cc @will-moore @chris-allan

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.