Unreliable ggplot support about panel

[philippjfr]

Thanks for finding that, BytesIO is what I tried first but couldn't get it to work. That approach looks much better.

[philippjfr]

Just tried it, kernel still hangs or restarts 😞

[philippjfr]

If you display the panel and then update the parameter a bunch of times until it freezes you also get to see this:

Exception ignored in: 'garbage collection'
KeyError: Trying to release object ID 140526695556776 while not preserved

Not sure if that's related.

[philippjfr]

From the console:

Warning: the rpy2 object we are trying to release has no associated capsule.
Fatal Python error: unexpected exception during garbage collection

Thread 0x000070000cd53000 (most recent call first):
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/zmq/utils/garbage.py", line 47 in run
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x000070000c438000 (most recent call first):
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/parentpoller.py", line 39 in run
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x000070000bf35000 (most recent call first):
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 295 in wait
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 551 in wait
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/IPython/core/history.py", line 829 in run
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/IPython/core/history.py", line 58 in needs_sqlite
  File "<decorator-gen-24>", line 2 in run
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x000070000b92c000 (most recent call first):
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/heartbeat.py", line 61 in run
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x000070000b429000 (most recent call first):
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/tornado/platform/kqueue.py", line 66 in poll
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/tornado/ioloop.py", line 863 in start
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/iostream.py", line 72 in _thread_main
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 864 in run
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/threading.py", line 884 in _bootstrap

Current thread 0x00007fff91a7a380 (most recent call first):
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/six.py", line 587 in iteritems
  File "/Users/philippjfr/development/bokeh/bokeh/core/properties.py", line 1385 in validate
  File "/Users/philippjfr/development/bokeh/bokeh/core/property/bases.py", line 283 in prepare_value
  File "/Users/philippjfr/development/bokeh/bokeh/core/property/bases.py", line 148 in themed_default
  File "/Users/philippjfr/development/bokeh/bokeh/core/property/descriptors.py", line 545 in instance_default
  File "/Users/philippjfr/development/bokeh/bokeh/core/property/descriptors.py", line 670 in _get_default
  File "/Users/philippjfr/development/bokeh/bokeh/core/property/descriptors.py", line 653 in _get
  File "/Users/philippjfr/development/bokeh/bokeh/core/property/descriptors.py", line 457 in __get__
  File "/Users/philippjfr/development/bokeh/bokeh/model.py", line 676 in _visit_immediate_value_references
  File "/Users/philippjfr/development/bokeh/bokeh/model.py", line 58 in collect_models
  File "/Users/philippjfr/development/bokeh/bokeh/document/events.py", line 237 in generate
  File "/Users/philippjfr/development/bokeh/bokeh/protocol/messages/patch_doc.py", line 91 in process_document_events
  File "/Users/philippjfr/development/bokeh/bokeh/protocol/messages/patch_doc.py", line 55 in create
  File "/Users/philippjfr/development/bokeh/bokeh/protocol/__init__.py", line 46 in create
  File "/Users/philippjfr/development/pyviz_panels/panel/util.py", line 103 in diff
  File "/Users/philippjfr/development/pyviz_panels/panel/util.py", line 112 in push
  File "/Users/philippjfr/development/pyviz_panels/panel/panels.py", line 218 in update_panel
  File "/Users/philippjfr/topographica/external/param/param/parameterized.py", line 584 in __set__
  File "/Users/philippjfr/topographica/external/param/param/__init__.py", line 443 in __set__
  File "/Users/philippjfr/topographica/external/param/param/__init__.py", line 612 in __set__
  File "<ipython-input-3-c433360bdda8>", line 2 in <module>
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963 in run_code
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2903 in run_ast_nodes
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2785 in _run_cell
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2662 in run_cell
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533 in run_cell
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196 in do_execute
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399 in execute_request
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235 in dispatch_shell
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283 in dispatcher
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/tornado/stack_context.py", line 277 in null_wrapper
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 432 in _run_callback
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 480 in _handle_recv
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 450 in _handle_events
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/tornado/stack_context.py", line 277 in null_wrapper
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/tornado/ioloop.py", line 888 in start
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477 in start
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/traitlets/config/application.py", line 658 in launch_instance
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/site-packages/ipykernel_launcher.py", line 16 in <module>
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/runpy.py", line 85 in _run_code
  File "/Users/philippjfr/miniconda/envs/anacondaviz/lib/python3.6/runpy.py", line 193 in _run_module_as_main

[jbednar]

Wonder if it's an error in rpy2's garbage collection, then, not our code. Hmm. Much easier to fix issues in our own code!

BTW, I thought that bypassing the GGPlotPanel code altogether and returning the result of image_png in the view method in user code would be sufficient for it to work with HTMLPanel, as in the code below (i.e. work as well as the existing example does, anyway). But I've seen a variety of errors when I try that, for cases that I think should be the same. (Sometimes I get JS errors TypeError: Cannot read property 'comm_manager' of undefined, sometimes HTMLPanel complains it doesn't know what to do with objects of the given type (despite it being something that has a _repr_html_), and sometimes it has no obvious errors but no output displayed.) I've gotten too confused to capture what's going on here, but thought it might be interesting, as I thought the result from output_png() (an IPython Image) should already be displayable by HTML.

from rpy2.robjects.packages import importr, data
import rpy2.robjects.lib.ggplot2 as g2
from IPython.display import Image
from rpy2.ipython.ggplot import image_png

import param
from panel import Panel, Column
from panel.panels import HTMLPanel

datasets = importr('datasets')
mtcars = data(datasets).fetch('mtcars')['mtcars']

class GGPanel(param.Parameterized):
    point_size = param.Number(default=1, bounds=(1, 10))

    @param.depends('point_size')
    def view(self):
        gp = g2.ggplot(mtcars)
        pp = (gp + g2.aes_string(x='wt', y='mpg')
                 + g2.geom_point(g2.aes_string(colour='qsec'), size=self.point_size)
                 + g2.scale_colour_gradient(low="yellow", high="red") 
                 + g2.geom_smooth(method='loess') 
                 + g2.labs(title="mtcars", x='wt', y='mpg'))
        return HTMLPanel(image_png(pp))
    
panel = GGPanel()
Column(panel, panel.view)

[philippjfr]

Same problem as before, IPython's Image._repr_html_(), which is what image_png returns, does not return anything.

[philippjfr]

Turned out to be down to lacking cleanup of param watch callbacks which presumably prevented the R objects from being garbage collected. I've now pushed those changes to master and the ggplot example now works without fail for me.

[jbednar]

Great! Works fine for me now.