Fix canvas widget sizing issue about ipyvtklink

[akaszynski]

:S It might actually be difficult to fix this. This is what I do in ipycanvas: https://github.com/martinRenou/ipycanvas/blob/master/src/widget.ts#L404, we compute the mouse coordinates depending on the view size and the canvas size.

Turns out this isn't too bad. The event gives you the current size of the canvas as displayed within jupyterlab. You can simply scale the relative mouse offset with:

if 'offsetX' in event:
    scale_x = self.width/event['boundingRectWidth']
    event['offsetX'] = round(event['offsetX']*scale_x)
    scale_y = self.height/event['boundingRectHeight']
    event['offsetY'] = round(event['offsetY']*scale_y)

[martinRenou]

I did not know there were so many information in the mouse event with ipyevents :)

[martinRenou]

For sure, I did not think too much about the actual fix yet.

You can find dev install instructions here: https://ipycanvas.readthedocs.io/en/latest/installation.html#development-installation. Note that because you will touch the TypeScript code you will need to rerun npm run build every time you change something, and refresh your browser page to see the update.

There are only two files in ipycanvas that matters here, the Python file and the TypeScript one.

One thing to know is that the CanvasModel contains the "main" canvas, the one on which we draw everything, and each CanvasView has a canvas that is actually displayed on the page, and it is kind-of a copy of the model canvas.

So I guess the responsive workflow would be to listen to container size changes in views. This can be done by waiting for the resize event from Phosphor, implementing processPhosphorMessage in CanvasView, like done here for example. And we could try to fit the biggest canvas we can in the available size (can be computed by getBoundingClientRect, as done here). Note that resizing a Canvas will clear it, so you would need to redraw the content on it.

[martinRenou]

Note that there is a difference between the size of the Canvas in pixels, and the actual size of the Canvas on the screen. For now I force them to be the same in ipycanvas, but I guess that is what we want to change. We still want to expose the first one to the Python interface, but not the second one, as it would be per-view.

[martinRenou]

Don't hesitate to open a draft PR when you have something (even if it does not do anything yet), so that I can quickly guide you

[martinRenou]

I might have time to work on this this week-end. Please let me know if you started something, I don't want to discard your work :)

[banesullivan]

Can you publish a release for ipycanvas to let us use this?

I tried it out from the master branch without noticing any change... I may not have installed it correctly. Or perhaps we need to do the same thing for the ipywidgets.Image inside the canvas?

[martinRenou]

I just released ipycanvas 0.5.0, it's available on pip, soon available on conda.

I tried ipyvtk_simple with 0.5.0 locally and setting canvas.layout.width = '100%' seems to do what you want! :)

[martinRenou]

Hi! I see you are using ipycanvas in this package, this is great!

Maybe that should be a feature to add in ipycanvas? It might make sense to add a window_resize event to the canvas, which would allow the Python dev to resize the canvas to the new container size if wanted.

[banesullivan]

👋 @martinRenou

I'm all for this being handled in ipycanvas! Is this something you all could easily add there? Or should I try to implement a solution here then maybe backport it to ipycanvas?

[martinRenou]

Is this something you all could easily add there?

Definitely :) And I think it makes total sense as it's something other people might want as well.

[banesullivan]

Awesome! Please let me know when you all can add this - it'd be great to have before I push this out to PyPI

[martinRenou]

Mmmh, thinking about it, there is something that annoys me a bit.

Like other widgets, the Canvas widget can be displayed multiple times on the page (which we call views), and we cannot force each view to have the same container size. Some views might be in big containers, other views in small containers.

That being said, I am not sure it's a good idea to provide an API like a container resize event. It might be better to provide a "responsive" mode where each Canvas view tries its best to fit in its container, respecting the current ratio of the Canvas. What do you think?

[banesullivan]

I think that'll work well as long as the ratio of the canvas is maintained

[banesullivan]

@martinRenou, do you think it'd be possible/easy to land such a feature in a soon-ish time frame? It'd be awesome to have that "responsive" mode for the canvas so we can count on this widget looking good and being usable across browser widths

[banesullivan]

I could possibly add some resize event handlers in the meantime that keep an eye on the canvas' width as it is resized with the browser and then resize the underlying VTK render window, but that can greatly affect the resolution/quality of the rendered image.

[martinRenou]

It might not be too much work, although I do this work on my free time so it will of course be slower. I don't have much free week-ends coming.
But if you can come up with a PR I am happy to review it quickly and merge it.

[banesullivan]

It might not be too much work, although I do this work on my free time so it will of course be slower. I don't have much free week-ends coming.

Totally fair - I appreciate all the help you're able to give!

But if you can come up with a PR I am happy to review it quickly and merge it.

Being totally unfamiliar with creating widgets and the ipycanvas source, could you point me to a few places to send me in the right direction?

[banesullivan]

Awesome, thanks so much for these details! It sounds like everything I'd need to get in the source and start trying things out. I will see if I can find time to try working on this and report back (it may be a little while for me also)

[banesullivan]

I haven't had the time to start yet. Thanks for your willingness to help on this!

[martinRenou]

I think I might have found an elegant solution that does not require any API changes in ipycanvas. Check it out! https://github.com/martinRenou/ipycanvas/pull/111

[martinRenou]

This has been implemented in https://github.com/martinRenou/ipycanvas/pull/111 and https://github.com/martinRenou/ipycanvas/pull/113.

You will soon be able to do:

canvas.layout.width = '100%'
canvas.layout.height = 'auto'

to get a "responsive" canvas which takes as much space as available while respecting the aspect ratio.

[martinRenou]

Can you publish a release for ipycanvas to let us use this?

Will do.

I tried it out from the master branch without noticing any change...

Did you execute the snippet of code I provided? How did you install from master?

Or perhaps we need to do the same thing for the ipywidgets.Image inside the canvas?

Not sure I understand what you are saying. There is no use of ipywidgets.Image in ipycanvas. Do you use a image widget in ipyvtk_simple?

[banesullivan]

Did you execute the snippet of code I provided?

Yep, since out ViewInteractiveWidget inherits from Canvas, I added the following to the init:

# Set Canvas to autofill the space
self.layout.width = '100%'
self.layout.height = 'auto'

How did you install from master?

Just a simple pip install from the master branch: pip install git+git://github.com/martinRenou/ipycanvas@master

(within a fresh Docker image)

Do you use a image widget in ipyvtk_simple?

We use an ipywidgets.Image to show the screenshot of the VTK scene within the canvas.

https://github.com/Kitware/ipyvtk-simple/blob/master/ipyvtk_simple/viewer.py#L57-L61

Do we need to scale the image to fill the canvas dynamically? Is the hard setting of the image size here likely the culprit?

[martinRenou]

Just a simple pip install from the master branch: pip install git+git://github.com/martinRenou/ipycanvas@master

If you are using Jupyter Notebook that should be enough.

We use an ipywidgets.Image to show the screenshot of the VTK scene within the canvas.
https://github.com/Kitware/ipyvtk-simple/blob/master/ipyvtk_simple/viewer.py#L57-L61
Do we need to scale the image to fill the canvas dynamically? Is the hard setting of the image size here likely the culprit?

I see. This should be fine. It should work IMO.

Also which browser do you use?

I'll give ipyvtk_simple a shot on my side before releasing. There might be something I missed.

[martinRenou]

Actually you might need to reinstall the Notebook extension:

jupyter nbextension install --py --sys-prefix ipycanvas
jupyter nbextension enable --py --sys-prefix ipycanvas

Not entirely sure this is needed when pip is ran without -e

[banesullivan]

This works wonderfully! One issue arises if using a width other than 100%: the mouse events registers from ipyevents do not register in the correct place. At the moment, I'm not sure how to work around this

[martinRenou]

the mouse events registers from ipyevents do not register in the correct place. At the moment, I'm not sure how to work around this

:S It might actually be difficult to fix this. This is what I do in ipycanvas: https://github.com/martinRenou/ipycanvas/blob/master/src/widget.ts#L404, we compute the mouse coordinates depending on the view size and the canvas size.

Which mouse events are you using? Are those not already supported in ipycanvas? Maybe we should support the ones you are using in ipycanvas.