Investigate GPU filters about picamera

[waveform80]

This thread seems to suggest that there may soon be an MMAL path to get textures to OpenGL which may obviate the need to dig into OpenMAX

[waveform80]

Digging into OpenGL/ES at the moment. The pyopengl project looks like the best bet for ctypes-based header conversions for the GL libraries. On the plus side, GL/ES headers now seem to be converted (looking at the Launchpad code). On the downside, they're not in the Raspbian packaged version (haven't checked PyPI yet) so either we need to incorporate a hack to account for that in our Raspbian packaged version, or we make it an optional dependency and tell users to install the PyPI version for now until Raspbian catches up (I'm tempted by the latter).

On API design: I suspect we may be able to get away with a single new parameter for the various capture/record/preview methods - something like gl_scene which accepts an object to use as the GL scene. When unspecified or None, everything operates as before. When specified, use a GL specific rendering path.

For cooked output this should be relatively easy to incorporate - the GL stuff goes before the encoders in the pipeline so it's just like the resizer - shouldn't be too difficult to incorporate. However, raw captures are another matter - can we even get YUV/RGB/etc. output from a GL scene? Even if we can it'll probably look very different at the call level to the existing raw capture stuff. Depending on difficult, I may have to leave raw capture from GL scenes as a future effort.

[fedtf]

Any news about this? I don't know much about OpenGL and Raspberry internals, but I'm trying to investigate into the topic. (By the way, thanks so much for the great documentation!)

It looks like this kind of functionality is available in raspistill with --gl* arguments. Concerning the implementation, I guess one place to look at is here, but, searching through shared libraries in /opt/vc/lib, I found neither functions from RaspiTex.c nor some VCOS functions used there. Namely, I failed to find vcos_semaphore_post and vcos_semaphore_wait, which is weird since functions for creating and deleting semaphores are available in libvcos.so. Not sure whether the same can be done with MMAL.

[byronformwalt]

You might look into the latest version of raspicam which you need to compile from source. I've tested it out, and it works. 100% MMAL and OpenGL (no OpenMAX). It demonstrates how to use OpenGLESv2 for realtime capture and rendering to textures. On my system (RPi3 Model B, Camera V1) I can do 1024x1024 from 720p at approximately 60 fps. Given this info, it is surely possible to use Python ctypes to build and register custom MMAL components that facilitate building an OpenGLESv2-based image processing pipeline. The alternative I see would be to add GPU acceleration for OpenCV to the Pi. The last I checked, GPU acceleration via OpenCV was still not supported for this platform.

[fedtf]

This is exactly what I'm looking into. The link in my comment goes to one of the files in the raspicam source responsible for passing textures to OpenGL. I'm just confused with the absence of some VCOS functions in the shared libraries, which will be needed in order to port this to python.

[byronformwalt]

Ok, I see what you mean, now. You threw me off when you referenced raspistill instead of raspicam, and I didn't open your link the first time. Sorry about that. ...Indeed neither vcos_semaphore_post nor vcos_semaphore_wait is defined in the namespace of libvcos.so or any other shared library.

Instead, you will find vcos_semaphore_wait defined in /opt/vc/interface/vcos/pthreads/vcos_platform.h beginning on line 251:

/*
 * Counted Semaphores
 */
VCOS_INLINE_IMPL
VCOS_STATUS_T vcos_semaphore_wait(VCOS_SEMAPHORE_T *sem) {
   int ret;
   /* gdb causes sem_wait() to EINTR when a breakpoint is hit, retry here */
   while ((ret = sem_wait(sem)) == -1 && errno == EINTR)
      continue;
   vcos_assert(ret==0);
   return VCOS_SUCCESS;
}

... and vcos_semaphore_post is defined in the same file beginning on line 341:

VCOS_INLINE_IMPL
VCOS_STATUS_T vcos_semaphore_post(VCOS_SEMAPHORE_T *sem) {
   int rc = sem_post(sem);
   vcos_assert(rc == 0);
   (void)rc;
   return VCOS_SUCCESS;
}

sem_post and sem_wait are the underlying function calls being made into the shared library called libpthread.so . That is where several other basic semaphore operations are implemented:

pi@raspirox:/usr/lib/arm-linux-gnueabihf $ lddump libpthread.so |grep sem_
    sem_close()
    sem_destroy()
    sem_getvalue()
    sem_init()
    sem_open()
    sem_post()
    sem_timedwait()
    sem_trywait()
    sem_unlink()
    sem_wait()

It would be nice if there were a great set of documentation for this platform? It seems none of us ever has the time to make one. I hope this helps answer you're concern.

[byronformwalt]

I updated my response to correct the library for set_*() functions. They are in libpthread.so, not libasan.so.

…

On Jun 24, 2017, at 3:57 AM, Byron Formwalt ***@***.***> wrote: Ok, I see what you mean, now. You through me off when you referenced raspistill instead of raspicam, and I didn't open your link the first time. Sorry about that. ...Indeed neither vcos_semaphore_post nor vcos_semaphore_wait is defined in the namespace of libvcos.so or any other shared library. Instead, you will find vcos_semaphore_wait defined in /opt/vc/interface/vcos/pthreads/vcos_platform.h beginning on line 251: /* * Counted Semaphores */ VCOS_INLINE_IMPL VCOS_STATUS_T vcos_semaphore_wait(VCOS_SEMAPHORE_T *sem) { int ret; /* gdb causes sem_wait() to EINTR when a breakpoint is hit, retry here */ while ((ret = sem_wait(sem)) == -1 && errno == EINTR) continue; vcos_assert(ret==0); return VCOS_SUCCESS; } ... and vcos_semaphore_post is defined in the same file beginning on line 341: VCOS_INLINE_IMPL VCOS_STATUS_T vcos_semaphore_post(VCOS_SEMAPHORE_T *sem) { int rc = sem_post(sem); vcos_assert(rc == 0); (void)rc; return VCOS_SUCCESS; } sem_post and sem_wait are the underlying function calls being made into the shared library at libasan.so. That is where several other basic semaphore operations are implemented: ***@***.***:/usr/lib $ lddump asan |grep sem sem_destroy() sem_getvalue() sem_init() sem_post() sem_timedwait() sem_trywait() sem_wait() It would be nice if there were a great set of documentation for this platform? It seems none of us ever has the time to make one. I hope this helps answer you're concern. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#17 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA-opJMD2MkwPiUEVSjwjHOo4A17Jadvks5sHOuFgaJpZM4BJmYB>.