Coder Social home page Coder Social logo

ome / omero-parade Goto Github PK

View Code? Open in Web Editor NEW
1.0 7.0 12.0 673 KB

OMERO.web plugin for displaying Dataset thumbnails or Plates in webclient center panel

License: GNU Affero General Public License v3.0

HTML 1.39% JavaScript 61.25% CSS 6.68% Python 30.31% Shell 0.25% Dockerfile 0.13%
omero webapp

omero-parade's Issues

Map Annotations filter with duplicate Keys

See https://forum.image.sc/t/anomalies-in-searching-kvpairs-in-parade-and-main-omero-search-tool/29124

If an Image has Map annotations Key: val_A and Key: val_B then one of these will be ignored since we return a "Dict of {'key': {iid: 'value', iid: 'value'}}". E.g. for 2 images:

{"Key": {123: val_B, 456: 'foo'}

and val_A is ignored.

# Dict of {'key': {iid: 'value', iid: 'value'}}

Simplest "fix" is to concatenate values e.g.

    if map_values[kv.name][iid]:
        map_values[kv.name][iid] += kv.value
    else:
        map_values[kv.name][iid] = kv.value

Multiple OMERO.tables picked randomly

See https://forum.image.sc/t/multiple-omero-tables-crash-omero-parade/72477

Currently we do this in table_filters/data_providers.py, which picks a "random" table since _bulk_file_annotations() returns unsorted results from HQL query:

def get_table(conn, objtype, objid):
    data = _bulk_file_annotations(None, objtype, objid, conn=conn).get(
        'data', [])
    if len(data) < 1:
        return None
    # Just use the first Table we find
    # TODO: handle multiple tables!?
    data = data[0]

Simply need to sort by ID, then pick the highest ID (most recent).

NB: Supporting multiple OMERO.tables (as we do for csv) is a much bigger task

Add table data combobox

  • log in as user-1 on web-dev-merge
  • select archiveddv
  • In parade:Select in combobox "Add table Data" the ROI count option
Traceback (most recent call last):

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/opt/hudson/workspace/WEB-DEV-merge-deploy/OMERO.py/lib/python/omeroweb/decorators.py", line 483, in wrapped
    retval = f(request, *args, **kwargs)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/omero_parade/views.py", line 181, in get_data
    'histogram': list(histogram)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/http/response.py", line 535, in __init__
    data = json.dumps(data, cls=encoder)

  File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)

  File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)

  File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/core/serializers/json.py", line 115, in default
    return super(DjangoJSONEncoder, self).default(o)

  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")

TypeError: 6 is not JSON serializable


<WSGIRequest
path:/omero_parade/data/Uk9JX2NvdW50/,
GET:<QueryDict: {u'_': [u'1524141990109'], u'dataset': [u'1457']}>,
POST:<QueryDict: {}>,
COOKIES:{'_ga': 'GA1.2.74430613.1492428942',
 '_gid': 'GA1.2.1403166767.1524053537',
 'csrftoken': 'FojPGWicIBZJrzOBiUS03HPIRxsM73I2',
 'phpbb3_a9mfv_k': '',
 'phpbb3_a9mfv_sid': 'a419d989a95ebbcf51915de7ed5a1532',
 'phpbb3_a9mfv_u': '1',
 'sessionid': '5x7euiyxys7eoel6p62qapoa126myv19',
 'style_cookie': 'null'},
META:{u'CSRF_COOKIE': u'FojPGWicIBZJrzOBiUS03HPIRxsM73I2',
 'HTTP_ACCEPT': 'application/json, text/javascript, */*; q=0.01',
 'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.9,fr;q=0.8',
 'HTTP_CONNECTION': 'close',
 'HTTP_COOKIE': '_ga=GA1.2.74430613.1492428942; csrftoken=FojPGWicIBZJrzOBiUS03HPIRxsM73I2; phpbb3_a9mfv_k=; style_cookie=null; phpbb3_a9mfv_u=1; phpbb3_a9mfv_sid=a419d989a95ebbcf51915de7ed5a1532; sessionid=5x7euiyxys7eoel6p62qapoa126myv19; _gid=GA1.2.1403166767.1524053537',
 'HTTP_HOST': 'web-dev-merge.openmicroscopy.org',
 'HTTP_REFERER': 'http://web-dev-merge.openmicroscopy.org/webclient/',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36',
 'HTTP_X_FORWARDED_FOR': '134.36.162.209',
 'HTTP_X_FORWARDED_PROTO': 'http',
 'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest',
 'PATH_INFO': u'/omero_parade/data/Uk9JX2NvdW50/',
 'QUERY_STRING': 'dataset=1457&_=1524141990109',
 'RAW_URI': '/omero_parade/data/Uk9JX2NvdW50/?dataset=1457&_=1524141990109',
 'REMOTE_ADDR': '127.0.0.1',
 'REMOTE_PORT': '49318',
 'REQUEST_METHOD': 'GET',
 'SCRIPT_NAME': u'',
 'SERVER_NAME': '127.0.0.1',
 'SERVER_PORT': '4080',
 'SERVER_PROTOCOL': 'HTTP/1.0',
 'SERVER_SOFTWARE': 'gunicorn/19.7.1',
 'gunicorn.socket': <socket._socketobject object at 0x725b980>,
 'wsgi.errors': <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x690f890>,
 'wsgi.file_wrapper': <class 'gunicorn.http.wsgi.FileWrapper'>,
 'wsgi.input': <gunicorn.http.body.Body object at 0x690fb10>,
 'wsgi.multiprocess': True,
 'wsgi.multithread': False,
 'wsgi.run_once': False,
 'wsgi.url_scheme': 'http',
 'wsgi.version': (1, 0)}>

cc @will-moore @emilroz @chris-allan

Re-design with crossfilter?

Having had a look at the crossfilter docs, some thoughts on what adoption of crossfilter might give us:

Current issues:

  • Would be nice to get rid of eval() for filter functions
  • Loading a whole table of data: only use 1 column at a time. Need to load table again to filter by another column.
  • Have to load the table data for filter AND again if we want to show in table.
  • Don't handle multiple tables attached to parent Project / Screen.
  • Don't handle text columns for OMERO.tables
  • Don't handle numerical values for Map annotations

Crossfilter provides lots of nice features, but how would it fit into parade?

  • Assume we don't need current 'parade filter' eval() functionality, and we simply use 'dataprovider' to load a table (all, or selected columns), which is what crossfilter expects.
  • Init crossfilter with a table of data. Then allow the user to choose dimensions to filter by. Less-than, greater-than, equal or range for numerical columns, equal/contains for string columns.
  • May want to group-by Dataset, Tags or various table columns.
  • After crossfilter init (and create dimension etc) is it possible to load additional data for filtering or display? Not as far as I can see. Might need to re-init crossfilter and re-create dimensions and filters.
  • Might want to plot data with https://dc-js.github.io/dc.js/ which seems to give a lot of nice features out of the box. How does user choose what to plot?
  • Don't know how well dc.js works with React.

cc @chris-allan @emilroz

Parade for histo cytometry application

I was wondering if parade was able to process quite easily histo cytometry application (with dataset for different condition/activation on Omero) if the segmentation results is given as mask (to extract results) ? Scatterplot are interactive in parade with the image isn't-it so it seems quite powerful for that ?

ValueError histogram with Nan values

https://www.openmicroscopy.org/qa2/qa/feedback/33035/

Traceback (most recent call last):

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/decorators.py", line 538, in wrapped
retval = f(request, *args, **kwargs)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/views.py", line 195, in get_data
bins=bins)

File "<__array_function__ internals>", line 6, in histogram

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/numpy/lib/histograms.py", line 792, in histogram
bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/numpy/lib/histograms.py", line 396, in _get_bin_edges
first_edge, last_edge = _get_outer_edges(a, range)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/numpy/lib/histograms.py", line 324, in _get_outer_edges
"autodetected range of [{}, {}] is not finite".format(first_edge, last_edge))

ValueError: autodetected range of [nan, nan] is not finite

Graph in parade y-axis Firefox

In Parade, click onto the Graph box icon.
The graph appears as expected.
The graph has a selection widget on x-axis as expected.
But in FF on Mac, there is no seletction widget on y-axis, neither is the y-axis labelled with numbers.
The same graph has the missing y-axis widget when you switch to Chrome

Firefox
screen shot 2018-06-06 at 12 22 42

Chrome

screen shot 2018-06-06 at 12 26 03

cc @will-moore @chris-allan

builds failing with ERR_OSSL_EVP_UNSUPPORTED

All builds are failing now, since upgrading from node 16 with:

$ npm run build

> [email protected] build
> webpack --progress --config webpack.prod.js

10% building 0/1 modules 1 active ...modules/babel-loader/lib/index.js!/Users/wmoore/Desktop/PARADE/omero-parade/src/js/index.jsnode:internal/crypto/hash:71
  this[kHandle] = new _Hash(algorithm, xofLen);
                  ^

Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:71:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (/Users/wmoore/Desktop/PARADE/omero-parade/node_modules/webpack/lib/util/createHash.js:135:53)
    at NormalModule._initBuildHash (/Users/wmoore/Desktop/PARADE/omero-parade/node_modules/webpack/lib/NormalModule.js:417:16)
    at /Users/wmoore/Desktop/PARADE/omero-parade/node_modules/webpack/lib/NormalModule.js:452:10
    at /Users/wmoore/Desktop/PARADE/omero-parade/node_modules/webpack/lib/NormalModule.js:323:13
    at /Users/wmoore/Desktop/PARADE/omero-parade/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at /Users/wmoore/Desktop/PARADE/omero-parade/node_modules/loader-runner/lib/LoaderRunner.js:233:18
    at context.callback (/Users/wmoore/Desktop/PARADE/omero-parade/node_modules/loader-runner/lib/LoaderRunner.js:111:13)
    at /Users/wmoore/Desktop/PARADE/omero-parade/node_modules/babel-loader/lib/index.js:41:103 {
  opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
  library: 'digital envelope routines',
  reason: 'unsupported',
  code: 'ERR_OSSL_EVP_UNSUPPORTED'
}

Lots of places e.g. https://stackoverflow.com/questions/69665222/node-js-17-0-1-gatsby-error-digital-envelope-routinesunsupported-err-os suggest using --openssl-legacy-provider in the npm scripts args, but this doesn't seem to help.

I have looked at starting a clean project using vite.js as a bundler instead of webpack, but I can't work out what the equivalent of the output.library is in vite that would build a library we can include with a <script> tag. In fact, this is only documented for webpack 3 (even though we are using webpack 4):
https://webpack-v3.jsx.app/configuration/output/#output-library

Also tried outpub.hashFunction (see https://stackoverflow.com/questions/69692842/error-message-error0308010cdigital-envelope-routinesunsupported) with various values to no effect.

Tried npm audit fix --force too, but that didn't work.

It says at https://stackoverflow.com/questions/69692842/error-message-error0308010cdigital-envelope-routinesunsupported/71334532#71334532 to use react-scripts 5.0.0 since it uses webpack 5.0.

We don't use react-scripts but I could try to upgrade webpack... https://webpack.js.org/migrate/5/

cc @jburel

Filter Table outlier

Just documenting this scenario for reference / future thought...

In preparing the OME meeting workshop, I generated a Table from ROIs and wanted to filter by area ("max_points"). However, 1 image had no 'blobs' on it so the ImageJ "Analyse Particles" polygon was the whole plane.
This single outlier meant that the min/max range was huge and the histogram failed to render since it had 1950 bins!

screen shot 2018-05-05 at 06 33 11

Also the first movement of the slider instantly filtered out ALL images except the outlier.
I deleted the outlier image, but this didn't help as the Table data is loaded without checking whether images exist for each row (and looking at the code it seems this would be quite hard to do without some lengthy parsing of the data).

However, my only solution was to re-build the Table from ROIs after deleting the image, so it would be nicer if we can think of any improvement to this workaround.

numpy.histogram ArrayMemoryError

https://www.openmicroscopy.org/qa2/qa/feedback/32955/

Traceback (most recent call last):

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/decorators.py", line 538, in wrapped
retval = f(request, *args, **kwargs)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/views.py", line 128, in filter_script
return module.get_script(request, filter_name, conn)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/table_filters/omero_filters.py", line 84, in get_script
histogram, bin_edges = numpy.histogram(values, bins=bins)

File "<__array_function__ internals>", line 6, in histogram

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/numpy/lib/histograms.py", line 792, in histogram
bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/numpy/lib/histograms.py", line 448, in _get_bin_edges
endpoint=True, dtype=bin_type)

File "<__array_function__ internals>", line 6, in linspace

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/numpy/core/function_base.py", line 128, in linspace
y = _nx.arange(0, num, dtype=dt).reshape((-1,) + (1,) * ndim(delta))

numpy.core._exceptions._ArrayMemoryError: Unable to allocate 2.19 TiB for an array with shape (300822149856,) and data type float64

Initial TODOs

Feedback & TODOs from initial presentation to team today:
See notes at https://docs.google.com/document/d/1bh9OnP00BRA_yCbuWo5uuFk7qtexz8nWZZR6_DgAZro/

  • Jason: find limit of image counts
  • JM: don't load thumbnails by default - heatmap
  • Chris: eventually replace webclient centre panel
  • JM: Export plot
  • Petr: export filtered view of data
  • Kenny: Save / bookmark filtered state
  • Chris: need to move to OME repo, pip installable, travis etc.
  • Jason: likes OMERO.parade name!
  • Heatmap on 'grid' thumbnails

Data provider min() error if no images

For example, select an empty Orphaned Dataset and "Add Table Data..." > "ROI_count"

Since the img_ids list is empty, we get empty values list in
numpy.amin(values) which fails.

Nightshade error reported with URL /parade/data/Uk9JX2NvdW50/ (ROI_count) with no query string for e.g. ?dataset=1, which gives us an empty img_ids list. Don't know how this occurred in the UI.

Internal Server Error: /parade/data/Uk9JX2NvdW50/
Traceback (most recent call last):
 File "/opt/omero/web/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
   response = wrapped_callback(request, *callback_args, **callback_kwargs)
 File "/opt/omero/web/OMERO.web/lib/python/omeroweb/decorators.py", line 488, in wrapped
   retval = f(request, *args, **kwargs)
 File "/opt/omero/web/venv/lib/python2.7/site-packages/omero_parade/views.py", line 180, in get_data
   'min': numpy.amin(values).item(),
 File "/usr/lib64/python2.7/site-packages/numpy/core/fromnumeric.py", line 1987, in amin
   out=out, keepdims=keepdims)
 File "/usr/lib64/python2.7/site-packages/numpy/core/_methods.py", line 14, in _amin
   out=out, keepdims=keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity

Multiple fields per well

Following up on the discussion started on: https://forum.image.sc/t/omero-parade-accepting-stringcolumns-from-omero-metadata-and-loading-multiple-plates/27767/16 on how to deal with multiple fields/images per well, I'd like to offer some suggestions:

  1. on @will-moore 's concern how to how handle the Plate view display, I think the current approach of just showing the first field in the well is adequate since the Plate view is likely only used in the filtering step when Well level experimental metadata is used to filter down to the conditions of interest. Maybe put up a warning that only field 1 per well is shown. To make it more "cute", it would be fun to use the "Field positions in well" panel to show per field heatmap detail when the user selects a single well while in heatmap mode.
  2. Most of the measured values that one would want to display are typically reported per field and not per well. So indeed, the Table/List view and Scatter Plot view should show the fields/images. And consequently the bulkannotations tables should be attached to Field/Image rather than to Well as currently. To make that convenient, it would be nice to provide an update to Populate_Metadata.py and its underlying code to easily do so.
  3. The above will also resolve an issue with displaying whatever image data is selected using the Scatter Plot or Table/List: currently such a selection is a list of wells which iviewer does NOT accept; if such a selection is a list of images, then iviewer can accept that as a multi-selected set and provide great visualization of that set.

Thanks for considering,
Damir

Support OMERO.table StringColumn

From https://www.openmicroscopy.org/qa2/qa/feedback/21658/ and https://www.openmicroscopy.org/qa2/qa/feedback/21733/

If a user has an OMERO.table with StringColumns they get listed in the "Add Table data..." menu but when chosen, the loading of data fails because histogram can't be created (error below).

Seems that when loading filters (includes ALL columns from a table) we only try to get numpy min/max/histogram for DoubleColumn and LongColumn (https://github.com/ome/omero-parade/blob/master/omero_parade/table_filters/omero_filters.py#L74)

However, for loading Table data, we don't test column type, so numpy.histogram, numpy.amin, numpy.amax are tried for e.g. StringColumn which fails.
If I comment-out these lines in views.py get_data() then it returns just the values dict which is displayed in the OMERO.parade table OK.
Not sure why we need to calculate histogram, min and max since they're not used by the UI in any way?

For StringColumns that actually contain Long/Float values, I wrote a script to convert columns to LongColumn or DoubleColumn in a new Table:
https://gitlab.com/openmicroscopy/incubator/python-scripts/blob/master/omero_table_stringcol_to_doublecol.py
but this doesn't help if the values are actually Strings.

File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/omero_parade/views.py", line 177, in get_data
histogram, bin_edges = numpy.histogram(values, bins=bins)
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/lib/histograms.py", line 676, in histogram
bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/lib/histograms.py", line 299, in _get_bin_edges
first_edge, last_edge = _get_outer_edges(a, range)
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/lib/histograms.py", line 250, in _get_outer_edges
first_edge, last_edge = a.min(), a.max()
File "/home/omero_user/omeroenv2/local/lib/python2.7/site-packages/numpy/core/_methods.py", line 32, in _amin
return umr_minimum(a, axis, None, out, keepdims, initial)
TypeError: cannot perform reduce with flexible type
path:/parade/data/VGFibGVfU3BvdCBDZWxsIENvdW50/,

Webclient controls while in parade

Sorry, I cannot find the original issue here, neither in Open nor in Closed.
So, opening a new issue.

  • have a parade graph on display in central pane
  • go to right-hand pane, click on some control, e.g. addition of Tag (the plus sign)
  • observe that the popup of the Tag window happens under the graph of parade in central pane, and the control inside the Tag window are not possible to click

Note: this happens also if the left-hand pane (LHP)
screen shot 2018-08-25 at 20 25 52
tree controls are deployed, such as Rendering settings on Dataset from context menu on LHP

Removing filters re-loads other filters

From #27 (comment)
Each time a filter is removed the key_value pairs are reloaded and UI regenerated
Example:

  • Add ROI_count
  • Add Key_value. wait for the UI to be built
  • Remove ROI_count
  • The Key_value component will be rebuilt.

Character encoding

It seems that when table columns have a "ยต" in their name, OMERO.parade is not able to use them for graphs.
These tables are come from a Fiji Jython script where the results are saved as tables, like this one in Groovy. Maybe there is something to do there I guess? (Replacing u'\xb5' by "u" before saving the table works for example.)

The following error appears:

Traceback (most recent call last):

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/omeroweb/decorators.py", line 538, in wrapped
    retval = f(request, *args, **kwargs)

  File "/omero/.virtualenvs/web/lib/python3.7/site-packages/omero_parade/views.py", line 172, in get_data
    data_name = data_name.decode('utf-8')

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 23: invalid start byte

Ignore #header row for csv tables

Just got a bug report from a user trying to use OMERO.parade on a Dataset at https://demo.openmicroscopy.org/webclient/?show=dataset-6308
This bug report was actually due to the user trying to filter using a csv that had the #header for populate script, so that the column names are not read correctly and therefore the first value added at

column_data[name].append(value.strip())

is the actual column name, so that all columns contain at least 1 string. So this fails:

            # try to convert to number (if not empty string)
            values = [float(v) if len(v) > 0 else v for v in values]

and we get this exception:

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/views.py" in filter_script
  128.                 return module.get_script(request, filter_name, conn)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py" in get_script
  194.                       'default': table_cols[0],

Exception Type: IndexError at /parade/filters/script/metadata.csv/
Exception Value: list index out of range

Load Tables from Datasets within a Project

See https://forum.image.sc/t/using-omero-parade-on-dataset-level-and-image-level-omero-tables/50180/7

When loading data for filtering, don't just check for csv or OMERO.tables on the parent Project.
Also check on the child Datasets.

The returned JSON would have to indicate which Datasets the filters were coming from. E.g.

/parade/filters/?project=152

"data": [
"ROI_count",
"Rating",
"Comment",
"Tag",
"Key_Value",
"Table",
"idr0021_merged_results.csv",
"Batch_ROI_Export.csv",
"roi_intensities.csv"
"Dataset 123: dataset_values.csv"
]

These would have to be shown in a list that indicates which Dataset they're coming from.
And when we load a filter, the URL would also need to indicate which Dataset the table came from (as it would if the Dataset were an orphan)

Right hand panel issue with multi-select images

Selecting multiple images using results in a different right hand panel being displayed.

To reproduce:
Whilst holding ctl (or cmd) select one image, then another

screenshot 2018-12-04 at 11 12 49

Now select a third image

screenshot 2018-12-04 at 11 17 55

Finally, unselect the third image to return to two images

screenshot 2018-12-04 at 11 18 08

Bug: javascript not packaged in devspace

see: #64 (comment)

Uncaught TypeError: omero_parade.omero_parade is not a function
    at jQuery.fn.init.load_plugin_content ((index):3024)
    at proxy (jquery-1.11.1.js:547)
    at update_plugin_content (ome.popup.js?_5.6.dev5:722)
    at HTMLBodyElement.<anonymous> (ome.popup.js?_5.6.dev5:759)
    at HTMLBodyElement.dispatch (jquery-1.11.1.js:4641)
    at HTMLBodyElement.elemData.handle (jquery-1.11.1.js:4309)
    at Object.trigger (jquery-1.11.1.js:4550)
    at HTMLBodyElement.<anonymous> (jquery-1.11.1.js:5260)
    at Function.each (jquery-1.11.1.js:383)
    at jQuery.fn.init.each (jquery-1.11.1.js:136)

@sbesson suspects this is due to the difference between virtualenv and pip -mvenv

Click on dataset

Click on an orphaned dataset that is not expanded.
The images are loaded in the tree but not in parade
It works if the dataset is in a project

filter_list getObjects(None)

Reported at https://www.openmicroscopy.org/qa2/qa/feedback/29402/

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero_parade/views.py", line 114, in filter_list
filters.extend(module.get_filters(request, conn))

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py", line 53, in get_filters
csv_anns = get_csv_annotations(conn, obj_type, obj_id)

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero_parade/csv_filters/data_providers.py", line 34, in get_csv_annotations
obj = conn.getObject(obj_type, obj_id)

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero/gateway/__init__.py", line 3268, in getObject
obj_type, oids, params, attributes, opts)

File "/opt/omero/web/venv3/lib/python3.6/site-packages/omero/gateway/__init__.py", line 3349, in buildQuery
"'Image' not %r" % obj_type)

AttributeError: getObjects uses a string to define obj_type, E.g. 'Image' not None

Tests still disabled

While cleaning up directories from the Python 3 migration, I ran across 8f38710:

commit 8f38710a90de8f38ab716c758b8dc575d9e0fa27 (HEAD, will/django_1.9)
Author: jmoore <[email protected]>
Date:   Fri Nov 15 14:12:45 2019 +0000

    Disable pytest for this repo

    For some reason, on travis, three "tests" are failing:

    ```
    data_providers (unittest.loader._FailedTest) ... ERROR
    omero_filters (unittest.loader._FailedTest) ... ERROR
    data_providers (unittest.loader._FailedTest) ... ERROR
    ```

    but not locally. For the moment, disabling pytest. When/if
    a `test/` directory is created, the script will fail.

diff --git a/.omeroci/app-build b/.omeroci/app-build
new file mode 100755
index 0000000..3992193
--- /dev/null
+++ b/.omeroci/app-build
@@ -0,0 +1,19 @@
+#!/bin/bash
+
+source /infra/utils
+
+set -e
+set -u
+set -x
+
+cd $TARGET
+DIR=$(setup_dir)
+
+export DJANGO_SETTINGS_MODULE=omeroweb.settings
+export OMERODIR=/opt/omero/web/OMERO.web
+export ICE_CONFIG=${OMERO_DIST}/etc/ice.config
+
+if [ -e test ]; then
+   echo Tests currently disabled
+   exit 1
+fi

csv_filters IndexError: list index out of range for table_cols

Haven't looked into this but looks like no table_cols are found in a csv, giving an IndexError...
Reported at https://www.openmicroscopy.org/qa2/qa/feedback/40337/

Traceback (most recent call last):

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/decorators.py", line 538, in wrapped
retval = f(request, *args, **kwargs)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/views.py", line 128, in filter_script
return module.get_script(request, filter_name, conn)

File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py", line 194, in get_script
'default': table_cols[0],

IndexError: list index out of range

Bug: table_cols List Index out of range

From https://www.openmicroscopy.org/qa2/qa/feedback/31187/

Traceback (most recent call last):

File "/opt/omero/venv_web/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/omeroweb/decorators.py", line 538, in wrapped
retval = f(request, *args, **kwargs)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/omero_parade/views.py", line 128, in filter_script
return module.get_script(request, filter_name, conn)

File "/opt/omero/venv_web/lib64/python3.6/site-packages/omero_parade/csv_filters/omero_filters.py", line 194, in get_script
'default': table_cols[0],

IndexError: list index out of range

Filter with OR and NOT

Feedback from workshops via @pwalczysko:

Want to be able to exclude images - e.g. with rating 1 or Tag "bad".
Also select using OR e.g. Dataset 1 OR Dataset 2.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.