Comments (6)
This gets into some topics I had hoped to postpone, but I do have some thoughts. Allocating the surfaces is just the first step towards solving the set of problems I listed in the XDC presentation. After that, you need to figure out how to use them. Part of that is beyond the scope of an allocation API that doesn't do any sort of rendering on its own, but part of it relies on the allocation API to be the arbiter of certain properties. Ultimately, I think the below will be split into multiple separate issues, but since they're all relevant to your questions, listing them here for now:
- How do we access these images in GL/Vulkan/EGL - In the simplest sense, I believe a single allocation should be bound to a single Vulkan image or EGLImage. What people making window systems are going to want to do is assemble swapchains (in Vulkan terms) out of those images. In Vulkan, that's easy: Swapchains already are collections of specially allocated images. However, one use case I like to keep in mind is the vendor-agnostic EGL platform driver mechanism Miguel presented on at XDC. This relies on having a lower-level EGLSurface you can wrap with another EGLSurface exposed to the user. When the user calls CreatePlatformSurface(myNativePlatform), the vendor-agnostic platform layer creates one of these lower-level EGLSurface objects instead, backing it with some surfaces it allocates. In our initial sample code, we used EGLStream + Stream producer surfaces. I'd propose introducing a simpler surface type: EGLImageSurface. This would be an EGLSurface made up of previously allocated EGLImage objects. Basically, a vehicle to bind an EGLImage to framebuffer 0. Calling SwapBuffers on these surfaces would simply pump some buffers within the GL/GLES drivers. The idea is basically borrowed from GBM surfaces, but moves the construct into EGL, since EGL is the only API that really needs the concept.
- How are surface state transitions managed - Once you start using images with non-trivial capabilities, you need to make sure they're in the right state for the engine you're using. On some hardware, the same allocation can be used as a render target, a texture, and a display surface, and a video encode source surface, but not at the same time. Since SwapBuffers() may not be used, and doesn't know which usages it is transitioning between, a separate mechanism is needed. This roughly corresponds to the issue Vulkan image layout transitions solve. My proposal for a while has been something along the lines of an "eglFlushImage()" call. The questions is what parameters such a call takes. I would propose that the allocator library exposes a set of states and their corresponding usage bits. The states would be opaque, but the application could pick one based on the desired usage. For example, if the application requested display, render target, and texture usage, then queried for the available states, it might get back two states, one with the display usage associated with it, and one with the render target and texture usage associated with it.
- How are surface accesses synchronized? - This is arguably out of the allocator's scope entirely. The app could choose various methods to perform synchronization. For example, if all the usage occurs in EGL, EGLSync objects would be sufficient. The obvious choice here would be android/linux fence fds. However, those are only available on Linux, and only on very modern kernels from my understanding.
Coming back around to the broader question, given the above, my view is that GBM could be implemented as a wrapper on top of the allocator library. However, I don't see current GBM sitting side-by-side with the allocator library, and I don't view the current GBM<->EGL paths as correct solutions. Even with a GBM implementation sitting on top of the allocator API defined here, there wouldn't be sufficient information available to implement an EGL window surface on top of a GBM surface, nor to import a foreign allocator surface into GBM and begin displaying it, for the reasons outlined above in the state transitions bullet.
from allocator.
This does end up sounding more like a "buffer manager" than simply an allocator. Although possibly the only thing really needed that isn't already covered elsewhere (gbm, fence-fd's and the related EGL extensions, etc) is surface state transition.. I guess we have half of that in the form of usage_t
. We just need an API to do something like: int buffer_transition_to(alloc_bo_t *bo, alloc_dev_t *dev, usage_t *new_state)
.. plus maybe some api to get/set fence fd's??
from allocator.
Not sure if you were implying this, but I don't think the allocator library shouldn't actually perform the transitions. That requires a graphics command buffer in most cases, and I don't want the allocator to have to get in the business of setting one of those up and handling all the synchronization with other command buffers it entails. It should just describe the transitions in a format other APIs can consume. For example, Vulkan already has extensive infrastructure for doing state transitions. The allocator could just export some data that can be passed into a Vulkan pipeline barrier as an extension.
from allocator.
I was assuming that allocator backend was just a shim that calls into driver to do whatever. But yeah, I guess if that was exposed as a "liballoc" API then we'd need some sort of context object that maps to an EGLContext/etc, and that probably isn't a direction we want to go..
Originally was thinking this way because something like a v4l camera/decoder/etc doesn't have some userspace API.. but I guess practically it is only the gpu (gl/vk/cl) drivers that would need state transitions. So I guess if we only care about this when the buffer transitions between a gpu driver and something else (other gpu driver, or something that isn't a gpu driver) then handling it within the gpu api makes sense.
from allocator.
Yes, currently we manage to get away with only the GPU APIs performing transitions. That means compositors tends to require a GPU API if they want to re-purpose buffers behind the client's back (E.g., take a buffer intended for scanout and start using it as a video encode source), but that hasn't been an onerous requirement for anyone thus far. For us, the GPU is the only engine that can perform such a transition fast enough to be useful anyway.
I'd be interested to hear whether this is a problematic assumption for other vendors.
from allocator.
Agree that we want to keep layout transitions out of the liballoc API.
A more generic wording for shunting this to other parts of the stack is to say the native API of the buffer producer should provide the means to do the layout transitions. If the producer is advanced enough to require layout transitions it should provide a way to do them. Currently this matches only GPUs, but may extend to video DSPs or whatever in the future.
from allocator.
Related Issues (8)
- What does the length_in_words field in capabilities/usage mean? HOT 1
- Add max width/max height constraints HOT 5
- Is this dead project?
- How do we want to specify formats? HOT 20
- Can we actually describe everything with positive capabilities and a few set constraints? HOT 5
- How [de]centralized do we want to be? HOT 22
- How does the allocator know what back-ends to load? HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from allocator.