szeged / webrender Goto Github PK
View Code? Open in Web Editor NEWThis project forked from servo/webrender
A GPU-based renderer for the web
Home Page: https://doc.servo.org/webrender/
License: Mozilla Public License 2.0
This project forked from servo/webrender
A GPU-based renderer for the web
Home Page: https://doc.servo.org/webrender/
License: Mozilla Public License 2.0
The pixel values we receive for textures (color0, cache_rgba8, data16, data32 ...) are displayed still pale-coloured compared to the expected output. Changing the color format to Srgba8 from Rgba8 in case of color0
, color1
, color2
and cache_rgba8
made the images display with the correct colors, but for YUV images this causes problems.
Other Textures like data16
, data32
... have Rgba32F formats, changing these to Srgba8 is not a valid option.
Is there a way to make gfx_rs compute the correct values, instead of touching the shader code or recomputing the pixel values?
Add the depth testing/writing for images and depth capability for pipelines/renderpasses.
Vulkan requires spir-v format, which we can compile at build time.
Depends on: #2
This is a meta issue for gfx-hal
realated work.
Currently we don't handle window resize, the RenderTargetView needs to be updated upon resize.
The original clear_target
uses scissors to only clear a part of an image.
Our clear_target
mimic the same calls, but it won't work, because the set_scissor
sets a pipeline state and we clearing without a pipeline.
We need to clear outside a pipeline, and afaik https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#clears-outside is the only way, and it is not take any rect params.
(The clear_attachments
requires a pipeline but uses rects.)
As a workaround we could use clear rect shaders like last year, or maybe the FillBuffer
.
Or an empty shader/pipeline with clear_attachments
.
We need this to compare our implementation with the original webrender.
Add support for the following shaders:
Primitive Shaders
Cache Shaders
Chache Clip Shaders
*
Not tested.
Our current problem with the cs_*
shaders, that they usually draw to a texture, then other shaders load those from cache texture sampler.
When this happens, we should change the pso's RVT
to the texture, and when the Option<Texture>
is None
, change it back to the default RTV
.
The main goal is to adopt Gfx_rs in WebRender which can be used to draw on the screen with different gfx driver.
In order to check our progress, we use the wrench's reftests.
Current status: N/A
Add support for other shader types. Currently the only primitive shaders are supported.
The main difference at this point the input vertices.
The brush shaders can use the PrimitiveInstance structure as well. (which used by the primitive shader)
Running with dx12 fails with "Combined image samplers have no default expression representation.", when running SPIRV_Cross. This points here:
https://github.com/KhronosGroup/SPIRV-Cross/blob/ab3907e4b59b243e7391a71f5e962c1bb5ab8451/spirv_glsl.cpp#L2299
Now that most of the major components are in place, we should pay more attention to the validation layer.
We can collect the issues here and identify the corresponding code, then fix it.
Define the correct pipeline, vertex, etc. definitions.
They depends on the shaders and there are 3 type of shaders: primitive(simple&transform), cache, chache_clip.
Our current approach is to implement one of them (ps_rectangle), then implement the rest.
Running the border image related reftests (border-image.yaml and border-image-fill.yaml) display the expected results , but the tests are failing, because the pixel data we get with readpixels is different from what we see in the window.
Here is a screenshot from the running border-image test:
The expected result:
And the debug image result from wrench:
Update the current version of webrender and gfx in order to stay relevant/sync with the code.
For Vulkan, we need to update the current shader code to add Location
information.
Replace the current implementation with with gfx's textures (surface, sampler, view).
gfx_texture from PistonDevelopers is a good starting point.
Some of the shaders require different blending modes.
Further reading: https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#framebuffer-blending
In build.rs, when we create the shaders, we should add the location information for in/out and uniforms programmatically. Next, we store these information and use it in the pipeline creation.
#67 fixed the segfault on windows, but causes a weird behavior when drawing images. The blob example on windows looks like the following:
The expected result:
The following steps are required
Rebase to the current upstream WebRender, for the shader updates. (I cherry-picked them for now)
Change array return types in shaders to out parameters (I have a patch for this on my working branch):
zakorgy@ee7cbf2
D2Array is not supported in gfx-rs yet: https://github.com/gfx-rs/gfx/blob/8d0ef5ba96bb4a2ecc5671975677323f11773dc5/src/backend/dx12/src/device.rs#L609,
we fallback to D2 texture for now: zakorgy@c494908
Find out why the copy_buffer_to_image
fails on the assert here: https://github.com/gfx-rs/gfx/blob/8d0ef5ba96bb4a2ecc5671975677323f11773dc5/src/backend/dx12/src/command.rs#L1289
When runing the basic_hal
example the third copy_buffer_to_image
call fails with values: buffer_width = 240, width = 1020
because buffer_width
should be greater or equal than width
.
The other strange thing is, that the previous two copy_buffer_to_image
also generates errors with the debug layer turned on. Here is a DebugView capture for this:
00000001 0.00000000 [9276] D3D12 ERROR: ID3D12CommandList::CopyTextureRegion: The region specified by D3D12_TEXTURE_COPY_LOCATION:PlacedFootprint extends past the end of the buffer it is placed on. The size required by PlacedFootprint is 536641536, as the fields of PlacedFootprint::Placement are as follows: RowPitch is 262144, Height is 2048, and Format is R32G32B32A32_FLOAT. PlacedFootprint::Offset is 0, which requires the buffer to have 536641536 bytes; but the buffer only has 67108864 bytes. [ RESOURCE_MANIPULATION ERROR #869: COPYTEXTUREREGION_INVALIDSRCPLACEMENT]
00000002 0.00087751 [9276] D3D12 ERROR: ID3D12CommandList::CopyTextureRegion: The region specified by D3D12_TEXTURE_COPY_LOCATION:PlacedFootprint extends past the end of the buffer it is placed on. The size required by PlacedFootprint is 536641536, as the fields of PlacedFootprint::Placement are as follows: RowPitch is 262144, Height is 2048, and Format is R32G32B32A32_FLOAT. PlacedFootprint::Offset is 16384, which requires the buffer to have 536657920 bytes; but the buffer only has 67108864 bytes. [ RESOURCE_MANIPULATION ERROR #869: COPYTEXTUREREGION_INVALIDSRCPLACEMENT]
00000003 0.03093249 [9276] D3D12 WARNING: Process is terminating. Using simple reporting. Please call ReportLiveObjects() at runtime for standard reporting. [ STATE_CREATION WARNING #0: UNKNOWN]
With a newer gfx version and the above mentioned shader patches, the issues in #53 seem to be solved.
NOTE: My working branch is not up to date with szeged/master at the moment, because we still have an issue(#68) on windows, which came in with #63.
Some draw calls require different blending state.
Change our current Program/pso sturcture to handle these cases.
Current idea: use RawRenderTarget
in the pipeline definitions, create multiple pso-s with different blending state and use the proper one in draw calls.
WebRender: https://github.com/szeged/webrender/commits/gfx-wr-4ab9b9b
Servo: https://github.com/szeged/servo/commits/gfx-5d1e027-05-18
Known issues:
gfx_glutin
vs servo_glutin
BGRA
Srgba
for RGBAF32
Yuv
Converting Sampler structures to raw data
Our previous solution was to convert everything manually. Now we do with slice casting (except from gradient, because there is something off (most probably with the length)).
gfx_glutin
vs servo_glutin
The latest version of gfx-rs
uses glutin 0.8.0
and this version doesn't contains WindowProxy
anymore. But servo-glutin
still uses WindowProxy
in the compositor.
Handle Resize (window and texture)
We don't support any resize yet.
BGRA
Gradient data comes in bgra format, but the B8_G8_R8_A8
is not implemented on OpenGL
backend (There are TODO
-s for placeholder), so we swap the red and blue bytes before updating the texture. Is there a specific blocker for implementing B8_G8_R8_A8
? We found a minimal workaround for the use case which we need, but this isn't in the current code yet: zakorgy/gfx@364c47b
Srgba
for RGBAF32
Our colors still off because we use RGBAF32
for our "non-texture" samplers (e.g. Data16
, Layer
).
The color values are stored in the data* samplers (at least seems that way) and now the gamma correlation.
Yuv
The Yuv
support is not working correctly, we know what seems to be the problem and working on it.
I noticed that webrender/src/Cargo.toml
has Vulkan dependency. This is not expected. I believe this crate should be totally backend-agnostic, since it doesn't create any context.
wrench
should depend on backends, and servo
should, but not the main WR crate.
We facing a strange behavior, which looks like some kind of (hard?) limit.
It can be reproduced with the following test.
We draw 20 x 20 green rect (every fifth row is yellow), the we draw 20x20 red rect next (400.0 offset) to our first rects.
Original result:
With our code it looks like this:
Some observation:
So it looks like to me there is a u8::max (0..255) hard limit somewhere.
Or (more likely?) our texture width size (4*256=1024) has something to do with it.
Storing too much data can overflow. In our case we use for a (primitive
) shader (which can be either simple
or transform
) 4 psos for blending with 2 variant for depth write
.
ps_rectangle
= type x blend x depth= 2 x 4 x 2 = 16 psos
And we use 22 primitive shaders (352 psos) and 6 other shaders (9 psos).
On a low/medium-end PC this can cause an overflow.
On a high-end it can be reproduced with this patch: dati91@374b360
The error:
Program received signal SIGSEGV, Segmentation fault.
0x0000555555a28581 in webrender::renderer::Renderer::new (window=<error reading variable: Cannot access memory at address 0x7fffff375cd0>,
options=<error reading variable: Cannot access memory at address 0x7fffff375ce0>,
initial_window_size=<error reading variable: Cannot access memory at address 0x7fffff375d58>) at src/renderer.rs:705
705 pub fn new(window: Rc<window::Window>,
The current implementation already uses an image for storing the vertex data.
Refactor this to a separate structure and extend to support the textures for the shaders.
A structure should contain an Image (with memory).
And depending on the usage:
We tried the following test:
https://github.com/szeged/webrender/blob/master/wrench/reftests/border/border-clamp-corner-radius.yaml
Which has a reference image: (we use it flipped for now)
https://github.com/szeged/webrender/blob/master/wrench/reftests/border/border-clamp-corner-radius-ref.png
The result of the test: https://gist.github.com/zakorgy/cb95b2b9e7158ce557f67ec71e9fc8fa
image comparison, max difference: 1, number of differing pixels: 29
Usually we use the following site to compare images quickly: https://huddle.github.io/Resemble.js/
A screenshot about the compare (the purple dots represent the differences, 29 as the result above indicate):
cc @kvark
I started look into the remaining failing tests and our first issue is that we use our instance buffers totally wrong.
Currently we bind the instances and call draw, but if we use the same program again, we will overwrite the current buffer.
Should be an easy fix. ๐ค
The program structure represents shader programs, pipelines, different blendmodes etc.
In webrender, there are 5 types of shaders with different modes:
All of them share common parts, the main difference is the different inputs and samplers.
In the long term, we should create pipelines, render passes, etc. reusable as possible with no duplication.
Both WR and gfx-rs are heavily tested by CI (Travis + Appveyor).
No reason we can't have some coverage here as well.
The current master(b04d40a) causes a segfault when running some of the wrench reftests, e.g. border-suite-2.yaml
or reftests\aa\rounded-rects.yaml
platform: windows
Running the reftests returns with the following errors:
error[E0432]: unresolved import `webrender::renderer::CpuProfile`
--> src/wrench.rs:24:27
|
24 | use webrender::renderer::{CpuProfile, GpuProfile};
| ^^^^^^^^^^ no `CpuProfile` in `renderer`
error[E0432]: unresolved import `webrender::renderer::GpuProfile`
--> src/wrench.rs:24:39
|
24 | use webrender::renderer::{CpuProfile, GpuProfile};
| ^^^^^^^^^^ no `GpuProfile` in `renderer`
error: no method named `set_profiler_enabled` found for type `webrender::Renderer` in the current scope
--> src/main.rs:515:41
|
515 | wrench.renderer.set_profiler_enabled(profiler);
| ^^^^^^^^^^^^^^^^^^^^
error: no method named `gl` found for type `webrender::Renderer` in the current scope
--> src/png.rs:58:32
|
58 | let data = wrench.renderer.gl().read_pixels(0,
| ^^
error[E0277]: the trait bound `[u8]: std::marker::Sized` is not satisfied
--> src/png.rs:58:9
|
58 | let data = wrench.renderer.gl().read_pixels(0,
| ^^^^ the trait `std::marker::Sized` is not implemented for `[u8]`
|
= note: `[u8]` does not have a constant size known at compile-time
= note: all local variables must have a statically known size
error[E0308]: mismatched types
--> src/wrench.rs:180:69
|
180 | let (renderer, sender) = webrender::renderer::Renderer::new(window.clone_gl(), opts, size).unwrap();
| ^^^^^^^^^^^^^^^^^ expected reference, found struct `std::rc::Rc`
|
= note: expected type `&glutin::Window`
found type `std::rc::Rc<gleam::gl::Gl + 'static>`
error: no method named `gl` found for type `webrender::Renderer` in the current scope
--> src/wrench.rs:194:35
|
194 | let gl_version = renderer.gl().get_string(gl::VERSION);
| ^^
error: no method named `gl` found for type `webrender::Renderer` in the current scope
--> src/wrench.rs:195:36
|
195 | let gl_renderer = renderer.gl().get_string(gl::RENDERER);
| ^^
error: no method named `gl` found for type `webrender::Renderer` in the current scope
--> src/wrench.rs:357:27
|
357 | self.renderer.gl().viewport(0, 0, dim.width as i32, dim.height as i32);
| ^^
error: no method named `get_frame_profiles` found for type `webrender::Renderer` in the current scope
--> src/wrench.rs:385:23
|
385 | self.renderer.get_frame_profiles()
| ^^^^^^^^^^^^^^^^^^
error: no method named `debug_renderer` found for type `webrender::Renderer` in the current scope
--> src/wrench.rs:407:32
|
407 | let dr = self.renderer.debug_renderer();
| ^^^^^^^^^^^^^^
The following steps required:
renderer.gl()
related callsIn almost every case we receive the bytes for textures in BGRA order, but for Yuv images we receive these in RGBA order. Until now we have used Rgba format for color0
but using Bgra instead made our code less hacky (we don't need to reverse the red and blue bytes in update_texture_data
). This modification made our Yuv image colors are displayed wrong. I have the following ideas as a solution:
TextureSampler
for color0
. I tried this, but this approach displayed some random memory content to the screen.Any advice is welcome.
A texture can be used for multiple purpose: data (storing numbers for different structures), image (storing an actual image)
In webrender, there are 3 types of texture used:
The vertex data is stores values for all the shaders.
The cache images are results of previous render passes.
The image(s) (mostly the color0) used to render actual images.
In the long term the images should be re-sizable for reuse.
The goal is to make gfx backend work on Android
The dither is a special image which used by some shader.
It is optional and currently it is disabled.
Depends on: #47
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.