digirati-co-uk / canvas-panel Goto Github PK

On the patchwork example, the scroll bar is not appearing in the pop-up annotations on Chrome.

Example annotation:

{
  "@id": "https://ra-exhibition-pmc.dlc.services/pmc/annotationlist/c9495f6c815358306e84dd121bc16ae7/6",
  "@type": "oa:Annotation",
  "motivation": "oa:linking",
  "on": "https://presley-pmc.dlc.services/iiif/raa/Vol10/canvas/c3#xywh=887.0800000000002,446.95288888888933,1377.0555555555554,165.24666666666656",
  "resource": {
    "@id": "https://ra-exhibition-pmc.dlc.services/index/exhibitors/B#Francesco%20Bartolozzi+%28Engraver%29",
    "label": "Bartolozzi, Francesco (Engraver) Exhib. 1769-1799"
  }
}

Following links/displaying enlarged images within the annotation pop-up will be dealt with by the surrounding page through an event model (to be agreed during development)

The fibers that are currently making up the prototype are not locked into any sort of event model, so its open to any event bus or model thats required.

In non-fullscreen view popup annotations that go outside the viewing area are cropped with no ability to scroll down the text. Seems to be like this across all browsers on desktop.

• Known Viewer requirements from multiple Digirati (and other) clients
  • Complex ranges
  • Text reader
  • Minimal viewers
  • Maximal viewers
• Annotation Studio
• Manifest Editor
• Range Editor (incl segmentation-assisted range editor)

according to the latest React docs here: https://reactjs.org/docs/react-component.html,
componentWillReceiveProps should be avoided and will be renamed in v.17.

It is used frequently in core, and AFAICT could be easily be replaced with componentDidUpdate

Painting the text into an invisible but selectable layer on top of the canvas (as you can do in a PDF), so that users can magically cut and paste text out of a IIIF resource.

CanvasPanel can support multiple modes of rendering textual content - somewhere else, in cooperation with the containing app, or directly, with various degrees of visibility.

Related to #176

Related to #108

Explore button is too prominent on the page, making it look like a call to action, instead of breaking out of the space. Needs to better show that it unlocks the scrolling behaviour and allows free roaming of the image.

Another example of where CanvasPanel needs to collaborate with the containing application.

CP has done the scene evaluation and knows what content requires an auth interaction, but it isn't responsible for the UI for initiating that auth interaction. But it can still orchestrate it - or collaborate with an auth flow component to orchestrate it.

Annotation pop-ups must be able to house rich media: HTML text and images for this iteration, with the capacity for audio and video in future iterations.

Questions around formatting of this type of annotation

• Single or multiple bodies
• Should annotation body be HTML, or some other structure
• Are the images IIIF images or just static links to images

There should be 3 available gestures on the mobile fullscreen mode:

• swipe left and right to navigate next and previous
• pinch to zoom
• tap to view detail on the current slide

the outline of the peg is slightly off when viewed in firefox for variation 3

Fullscreen viewer should be responsive on desktop, not just on devices.

To be refined: Mobile experience for scroll plugin example.

Remove or put behind configuration, the drop in transparency when freely-exploring in the scroll plugin.

Not hugely important but would it be more meaningful to rename the patchwork plugin to something like "annotated zoom" i.e. the functionality it provides rather than the first object. Will be rather strange to keep on calling it patchwork plugin for completely different objects!

From the build logs:

** WARNING **
Using custom ruby version 2.2.3, this will slow down the build.
To ensure fast builds, set the RUBY_VERSION environment variable, or .ruby-version > file, to an included ruby version.
Included versions: 2.2.9 2.4.3 2.3.6

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Preset name not found within published preset config (monorepo:angularcli). Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

PBI: CH-41

Browsers include:

• Last 2 versions (Edge, FF, Chrome)
• IE 10
• IE 11
• Mobile Safari (iOS 9.3+)
• Chrome (android)

There are some errors when viewing the documentation on mobile safari, related to the fullscreen API. cannot call on of undefined or something similar. Also potentially related, the V&A example does not load at all (probably caused by this error) and on safari desktop its very unstable, sometimes causing crashes. Lots of RAF requests (~60k per frame) might be related to OSD.

Stylable through CSS with customisable markers

This should happen intrinsically with the way we are building. Will integrate BEM-js with customisable class names, to cover the multiple instances on a single page requirement.

Also applies to other requirements for CSS

CSS styling for the above

Can the editor have an intuitive way to control the initial point of interest coordinates for the image. This will control the initial zoom level to help avoid slideshow layouts with black bars outside of the image area.

Suggestion is a preview window for the editor which has the pan and zoom controls. The editor can then move the image to the desired position and then a save button can store those coordinates in the manifest.

Happy to chat if this isn't clear.

Canvas panel as a component to drive reading - for people who are going to spend a long time reading a text. I need to write up my notes into a narrative then see how that fits in with Canvas Panel features (could be Canvas Panel inside the UV, or in any ad-hoc bookreader UI).

Then do more work on the developer narrative.

While the behaviour of CanvasPanel for many scenes is completely deterministic, some content requires, or would benefit from, user interaction.

Examples:

When there is a Choice (oa:Choice), CanvasPanel needs to supply that information somehow to the containing app, so that the containing app can generate the UI to offer the choice to the user; when the user chooses, that containing app needs to notify CP of the choice that was made. It's not CanvasPanel's job to provide that UI, but it is CP's job to make that UI and associated code trivially easy for someone knocking up a quick viewer.

More generally, with many linked annotation lists / otherContent, the containing app needs to offer UI to the user to control the visibility by list, and/or by motivation, and/or by language.

And then, there is all the anno content that is available but not desirable to render directly on the canvas. As a developer who wants to make innovative user interfaces quickly, I'm relying on CanvasPanel to do all the heavy lifting for display - but I am prepared to handle some of that display in my app, because not all the anno content should be painted onto the canvas. I don't know in advance what content the canvas bears, CP helps me process it. So it has features that are not just about it drawing pixels, but about processing content.

Example - for supplementing content in P3 (or text transcriptions in a linked annolist with otherContent), I might want to display the text off to the side, where it is selectable.

So, I need to be notified by CanvasPanel of the availability of this textual content, or query CP for the availability of the content; I want to get the content, so that I can render it.

But maybe I want to invoke a special CP mode that draws the text on top of the canvas, fitted to the anno boxes, so that it is selectable (a cool feature: #180)

All these things involve my containing app knowing about what canvaspanel has for a given canvas, sometimes obtaining that content through API on CP (rather than having to process the canvas itself again from scratch), sometimes taking care of the display externally and other times telling CP how to display it.

Example - I'd want to use CP to create something as simple as this:

https://github.com/tomcrane/wellcome-today/blob/gh-pages/viewer-min.html
(http://tomcrane.github.io/wellcome-today/viewer.html) and add support for displaying the text.

When rendering annotations, they should double check that they should be rendered (another check of the canvas target). May be related to React 16 async? To replicate, quickly page through annotation lists, you will see annotations appearing that shouldn't. They disappear after a re-render.

After clicking on a pin if click within vicinity of the marker on the zoomed in view then the view exits out to original size.

Confusing if I have not clicked on the actual close PIN to exit and expect it to zoom in

Behaviour seen in both Chrome and Firefox

The viewport will move and zoom in a certain amount when an annotation is selected
Configurable zoom speed on transition between zoomed out view and close-up

Since we are framing annotations (see diagram in #1 ) in a bounding box, we will zoom into fit the whole box into the viewport. There will be configuration for:

• Padding around bounds of image
• Speed of transition when zooming in
• Speed of transition when zooming out

So far the layers on the canvas have been relative to the canvas itself, but we may need layers that are relative to the viewer or viewport when it makes sense (such as box selector layer for example). In this case we want a projection painted onto the canvas where the real element is, and a way to relay back the position of the canvas to the this layer.

Top Layer - viewport
Cube - projection layer, moves with viewport
Bottom layer - canvas, relays projected co-ordinates to projection

Annotations will be placed as markers on an image

In order to maintain a responsive design, we've decided to represent the regions of the image using annotation boxes with a height, width, X and Y. This will be displayed to the user as a pin.

1. X co-ordinate of the annotation, from the target
2. Y co-ordinate of the annotation, from the target
3. Width of the annotation, from the target
4. Height of the annotation, from the target
5. X position of pin, driven by configuration (50%)
6. Offset at the top, driven by configuration (3.5% of canvas height)

• Dotted line won't be visible, driven by configuration
• Area in dotted line might be click-able to view annotation (?)

Designed responsively: on desktop, the annotation body should appear in the top left when a marker is selected. In mobile view, annotations should appear at the bottom, to avoid obscuring the image. Annotation may take up entire canvas in mobile view - (If this is possible through configurable CSS, this functionality is not required)

A lot of this will be configurable with CSS, but we will include basic positioning examples.

Full-screen option. Mobile view will require configurable prompt to enter full-screen mode before zoom/scroll functionality is enabled (to avoid competing event listeners).

This can use some of the tech from Galway viewer to initially show an high quality image and load the OSD when zoom has started, with the addition of a prompt for mobile view.

• Identify transcribing annotations
• Load text annos
• Read them aloud with https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API

An icon with a label indicating that to start you scroll down:

Replacing the current explore button.

PBI - TBC

Currently annotations can't be placed on the static image viewport. It's also diverged form the OSD viewport, so they are not swappable anymore. This should be updated to stub out the missing functionality or make functional in a way that makes sense for the static image viewport (cropping image instead of zooming to for example.)

Looking at canvaspanel demo on android in firefox – nothing visible at all

There should be a fullscreen mode for the slideshow viewer on mobile, to launch a slideshow-like experience.

I think Safari is really struggling with the scaling up and scaling down in a single operation. Need to look into a different approach for fixed sized zooming (like resizing elements individually).

For the backlog...

OpenSeadragon is probably not going to be the best renderer for a canvas that contains audio and video. Especially an audio only, duration only Canvas! And what about mixed scenes?

Maybe some Canvases include an image that has an image service, but OSD (or rather, a deep zoom interaction) is still not the best choice to render it.

At some point CanvasPanel should be a general purpose renderer, invoking whatever components are required for the scene. This is hard. In the UV for example, an AV scene may require the dash.js library for mpeg-dash, or the hls.js library for HLS; it may need to play an mp4 and render an image; it needs to draw text. I don't know what the best answer to this is, but CP is a good place to explore it.

Typical developer scenarios for people making viewers quickly and simply.

They want to pass canvases for CP to render.

They might want to support image choice in their UIs.

They might want to render the text content of the canvas.

What would a really simple viewer look like?

Starting point: http://tomcrane.github.io/wellcome-today/viewer-no-dep.html

This tiny viewer has no dependencies. Can we turn it into a deep zoom viewer that can display image choice UI, render complex scenes, display the text alongside, by referencing CP and writing a bit more (simple) code?

For plugins should create multiple bundles:

• React + ReactDOM + OpenSeadragon
• Just React + ReactDOM (bring your own seadragon)

These should be part of the build and available in the UMD folder.

We need to establish a clear stance on CP's architecture, so that its relationship to other components is well understood.

Other components may be inner components of CanvasPanel, like OpenSeadragon
Other components may be outer components that use CanvasPanel, like a viewer, or Annotation Studio

It would be rare to use CP on its own. It's not a viewer, except in the very simple sense that an img tag is a viewer for images and OpenSeadragon is a viewer for TilesSources.

Given a manifest, you need some notion of a containing app to see anything, because CP has no idea what a manifest is. But the point of CP is, that containing app could be very very simple.

For things like the V&A viewers, it's not clear where CP ends and the viewer begins.

We will require multiple instances of the canvas panel on one webpage.

This should be another intrinsic property of the build.

PBI: CH-41
Demo: http://canvas-panel.netlify.com/#/examples/fullpage

This demo / proof of concept will continue development and become a plugin that is configurable. It will be tested against supported browsers and configuration documented.

PBI: CH-41

Documentation for scrolling plugin:

• Installation
• Available configuration
• Browser support

Currently, manifests are cached with 'force-cache' as fetch option. Making this configurable would allow for more dynamic applications that for example add annotations to the manifest in real time.

If select a annotation and then full screen

The annotation box remains visible but the screen goes blank, when click on blank screen the canvas appears in the bottom left corner

Need to press esc to get out of full screen

the following link does not load - https://canvas-panel.netlify.com/#/examples/patchwork

Error message:
SCRIPT438: Object doesn't support property or method 'endsWith'