Image Blackboard (was Image Processing refactoring) about architecture

[balloob]

I remember that we had a similar discussion when image processing was introduced. I think that one of our thoughts was that we don't want to become a full image detection pipeline. By keeping it simple, we can enable some use cases and still keep it fairly simple.

Right now we can enable a motion detector to trigger reading a camera and seeing who is there for example.

For continuous screen detection I would expect that to happen outside HA and we only receive events of things that are discovered.

[dgomes]

OK, we don't want feature creep, lets scratch image_processing pipelines off.

I'm not talking about continuous screen detection, I'm just talking about getting the image that was processed by a current image_processing platform:

• Face is detected
• We get the image where the face was detected (not a new one)

Does extending image_processing to supply an Image make sense ?

[dgomes]

Would a PR to add SINK entity to image_processing be accepted ?

SINK would be a camera entity (would also issue a PR for a sink_camera that can receive images)

[dgomes]

My previous proposal would create a circular dependency between image_processing and camera.

• Image Processing should be a proper service, without any dependency on camera component. Ideally you would have a queue to receive images, and a queue with processed images.

But I also see a second opportunity:

• Notifications that include images, have the images passed around through the filesystem (very bad when you are running on an RPi3). It would be interesting to also share images with Notification System.

• Passing images through the event bus will just through hundreds of kbytes through the bus (requiring conversion of binary images to base64 for json serialization).

To tackle all these issues, my proposal would be for the development of a blackboard for images. The blackboard could live in hass.data.

I have no problem diving into coding this solution, just would like some feedback beforehand.

[Kane610]

I don't know much about the architecture here but do you really need to pass along the image? Can't you extend camera component to keep its own queue instead? And image processing would consume event and use image? Possibly with a lock to not clean image during processing. Alternatively some other place that can consume and keep images decoupled from both camera and processing

[dgomes]

@Kane610 your first proposal would still create a dependency between components (instead of moving to a decoupled solution)

"other place" is my proposal :)

[Kane610]

Allright, maybe I misunderstood :)