szeged / webrender Goto Github PK

The pixel values we receive for textures (color0, cache_rgba8, data16, data32 ...) are displayed still pale-coloured compared to the expected output. Changing the color format to Srgba8 from Rgba8 in case of color0, color1, color2 and cache_rgba8 made the images display with the correct colors, but for YUV images this causes problems.
Other Textures like data16, data32 ... have Rgba32F formats, changing these to Srgba8 is not a valid option.
Is there a way to make gfx_rs compute the correct values, instead of touching the shader code or recomputing the pixel values?

Add the depth testing/writing for images and depth capability for pipelines/renderpasses.

Vulkan requires spir-v format, which we can compile at build time.

Depends on: #2

This is a meta issue for gfx-hal realated work.

Currently we don't handle window resize, the RenderTargetView needs to be updated upon resize.

The original clear_target uses scissors to only clear a part of an image.

Our clear_target mimic the same calls, but it won't work, because the set_scissor sets a pipeline state and we clearing without a pipeline.

We need to clear outside a pipeline, and afaik https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#clears-outside is the only way, and it is not take any rect params.
(The clear_attachments requires a pipeline but uses rects.)

As a workaround we could use clear rect shaders like last year, or maybe the FillBuffer.
Or an empty shader/pipeline with clear_attachments.

We need this to compare our implementation with the original webrender.

• Enable Profiler
• Enable DebugRenderer
  • Enable CPU profiling
  • Enable GPU profiling

Add support for the following shaders:

Primitive Shaders

• ps_rectangle
• ps_rectangle_clip
• ps_text_run
• ps_text_run_subpixel
• ps_image
• ps_yuv_image
• ps_border_corner
• ps_border_edge
• ps_gradient
• ps_angle_gradient
• ps_radial_gradient
• ps_box_shadow*
• ps_cache_image*
• ps_blend*
• ps_hw_composite*
• ps_split_composite*
• ps_composite*

Cache Shaders

• cs_box_shadow
• cs_text_run
• cs_blur

Chache Clip Shaders

• cs_clip_rectangle
• cs_clip_image
• cs_clip_border

* Not tested.

• Examine how the primitive shaders work
• Concatenate primitive shaders to have everything in build time
• Compare The Validator of Servo Webrender if it can be used for the concatenation

Our current problem with the cs_* shaders, that they usually draw to a texture, then other shaders load those from cache texture sampler.

When this happens, we should change the pso's RVT to the texture, and when the Option<Texture> is None, change it back to the default RTV.

The main goal is to adopt Gfx_rs in WebRender which can be used to draw on the screen with different gfx driver.

In order to check our progress, we use the wrench's reftests.

Current status: N/A

Add support for other shader types. Currently the only primitive shaders are supported.

The main difference at this point the input vertices.
The brush shaders can use the PrimitiveInstance structure as well. (which used by the primitive shader)

Running with dx12 fails with "Combined image samplers have no default expression representation.", when running SPIRV_Cross. This points here:
https://github.com/KhronosGroup/SPIRV-Cross/blob/ab3907e4b59b243e7391a71f5e962c1bb5ab8451/spirv_glsl.cpp#L2299

Now that most of the major components are in place, we should pay more attention to the validation layer.

We can collect the issues here and identify the corresponding code, then fix it.

Define the correct pipeline, vertex, etc. definitions.

They depends on the shaders and there are 3 type of shaders: primitive(simple&transform), cache, chache_clip.

Our current approach is to implement one of them (ps_rectangle), then implement the rest.

border-suite wrench test shows differing color values at the connecting edges.
This could be a result of wrong blend mode settings.
Here is an image created with Resemble.js showing the differences well:

Running the border image related reftests (border-image.yaml and border-image-fill.yaml) display the expected results , but the tests are failing, because the pixel data we get with readpixels is different from what we see in the window.
Here is a screenshot from the running border-image test:

The expected result:

And the debug image result from wrench:

• Check if it is possible to render WebGL with the current implementation to offscreen and draw with Gfx_rs on the screen
• Adopt a preliminary implementation in WebRender

Update the current version of webrender and gfx in order to stay relevant/sync with the code.

• Update webrender
• Update gfx

For Vulkan, we need to update the current shader code to add Location information.

Replace the current implementation with with gfx's textures (surface, sampler, view).
gfx_texture from PistonDevelopers is a good starting point.

Some of the shaders require different blending modes.

Further reading: https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#framebuffer-blending

In build.rs, when we create the shaders, we should add the location information for in/out and uniforms programmatically. Next, we store these information and use it in the pipeline creation.

• Add the location info programmatically
• Save these info in a map-like structure
• Generate pipelines from these

#67 fixed the segfault on windows, but causes a weird behavior when drawing images. The blob example on windows looks like the following:

The expected result:

The following steps are required

• Rebase to the current upstream WebRender, for the shader updates. (I cherry-picked them for now)

• Change array return types in shaders to out parameters (I have a patch for this on my working branch):
  zakorgy@ee7cbf2

• D2Array is not supported in gfx-rs yet: https://github.com/gfx-rs/gfx/blob/8d0ef5ba96bb4a2ecc5671975677323f11773dc5/src/backend/dx12/src/device.rs#L609,
  we fallback to D2 texture for now: zakorgy@c494908

• Find out why the copy_buffer_to_image fails on the assert here: https://github.com/gfx-rs/gfx/blob/8d0ef5ba96bb4a2ecc5671975677323f11773dc5/src/backend/dx12/src/command.rs#L1289
  When runing the basic_hal example the third copy_buffer_to_image call fails with values: buffer_width = 240, width = 1020 because buffer_width should be greater or equal than width.
  The other strange thing is, that the previous two copy_buffer_to_image also generates errors with the debug layer turned on. Here is a DebugView capture for this:

00000001	0.00000000	[9276] D3D12 ERROR: ID3D12CommandList::CopyTextureRegion: The region specified by D3D12_TEXTURE_COPY_LOCATION:PlacedFootprint extends past the end of the buffer it is placed on. The size required by PlacedFootprint is 536641536, as the fields of PlacedFootprint::Placement are as follows: RowPitch is 262144, Height is 2048, and Format is R32G32B32A32_FLOAT. PlacedFootprint::Offset is 0, which requires the buffer to have 536641536 bytes; but the buffer only has 67108864 bytes. [ RESOURCE_MANIPULATION ERROR #869: COPYTEXTUREREGION_INVALIDSRCPLACEMENT]	
00000002	0.00087751	[9276] D3D12 ERROR: ID3D12CommandList::CopyTextureRegion: The region specified by D3D12_TEXTURE_COPY_LOCATION:PlacedFootprint extends past the end of the buffer it is placed on. The size required by PlacedFootprint is 536641536, as the fields of PlacedFootprint::Placement are as follows: RowPitch is 262144, Height is 2048, and Format is R32G32B32A32_FLOAT. PlacedFootprint::Offset is 16384, which requires the buffer to have 536657920 bytes; but the buffer only has 67108864 bytes. [ RESOURCE_MANIPULATION ERROR #869: COPYTEXTUREREGION_INVALIDSRCPLACEMENT]	
00000003	0.03093249	[9276] D3D12 WARNING: Process is terminating. Using simple reporting. Please call ReportLiveObjects() at runtime for standard reporting. [ STATE_CREATION WARNING #0: UNKNOWN]	

With a newer gfx version and the above mentioned shader patches, the issues in #53 seem to be solved.

NOTE: My working branch is not up to date with szeged/master at the moment, because we still have an issue(#68) on windows, which came in with #63.

Some draw calls require different blending state.
Change our current Program/pso sturcture to handle these cases.

Current idea: use RawRenderTarget in the pipeline definitions, create multiple pso-s with different blending state and use the proper one in draw calls.

WebRender: https://github.com/szeged/webrender/commits/gfx-wr-4ab9b9b
Servo: https://github.com/szeged/servo/commits/gfx-5d1e027-05-18

Known issues:

• Converting Sampler structures to raw data
• gfx_glutin vs servo_glutin
• Handle Resize (window and texture)
• BGRA
• Srgba for RGBAF32
• Yuv

1. Converting Sampler structures to raw data
   Our previous solution was to convert everything manually. Now we do with slice casting (except from gradient, because there is something off (most probably with the length)).

2. gfx_glutin vs servo_glutin
   The latest version of gfx-rs uses glutin 0.8.0 and this version doesn't contains WindowProxy anymore. But servo-glutin still uses WindowProxy in the compositor.

3. Handle Resize (window and texture)
   We don't support any resize yet.

4. BGRA
   Gradient data comes in bgra format, but the B8_G8_R8_A8 is not implemented on OpenGL backend (There are TODO-s for placeholder), so we swap the red and blue bytes before updating the texture. Is there a specific blocker for implementing B8_G8_R8_A8? We found a minimal workaround for the use case which we need, but this isn't in the current code yet: zakorgy/gfx@364c47b

5. Srgba for RGBAF32
   Our colors still off because we use RGBAF32 for our "non-texture" samplers (e.g. Data16, Layer).
   The color values are stored in the data* samplers (at least seems that way) and now the gamma correlation.

6. Yuv
   The Yuv support is not working correctly, we know what seems to be the problem and working on it.

I noticed that webrender/src/Cargo.toml has Vulkan dependency. This is not expected. I believe this crate should be totally backend-agnostic, since it doesn't create any context.

wrench should depend on backends, and servo should, but not the main WR crate.

We facing a strange behavior, which looks like some kind of (hard?) limit.
It can be reproduced with the following test.
We draw 20 x 20 green rect (every fifth row is yellow), the we draw 20x20 red rect next (400.0 offset) to our first rects.
Original result:

With our code it looks like this:

Some observation:

• It only draws the first 256 rects (a strange coincidence?)
  • and without drawing the red rects it still just 256
• If we draw the red rects it starts from the where we finished our green/yellow rects
  • it should start from the top (with x + 400 offset)
  • and it draws 256 red rects

So it looks like to me there is a u8::max (0..255) hard limit somewhere.
Or (more likely?) our texture width size (4*256=1024) has something to do with it.

Storing too much data can overflow. In our case we use for a (primitive) shader (which can be either simple or transform) 4 psos for blending with 2 variant for depth write.

ps_rectangle = type x blend x depth= 2 x 4 x 2 = 16 psos
And we use 22 primitive shaders (352 psos) and 6 other shaders (9 psos).

On a low/medium-end PC this can cause an overflow.
On a high-end it can be reproduced with this patch: dati91@374b360

The error:

Program received signal SIGSEGV, Segmentation fault.
0x0000555555a28581 in webrender::renderer::Renderer::new (window=<error reading variable: Cannot access memory at address 0x7fffff375cd0>,
    options=<error reading variable: Cannot access memory at address 0x7fffff375ce0>,
    initial_window_size=<error reading variable: Cannot access memory at address 0x7fffff375d58>) at src/renderer.rs:705
705         pub fn new(window: Rc<window::Window>,

The current implementation already uses an image for storing the vertex data.
Refactor this to a separate structure and extend to support the textures for the shaders.

A structure should contain an Image (with memory).
And depending on the usage:

• srv: if the image is an input of the shader
• rtv: if is a result of a render pass
• dsv: if the image needs depth

We tried the following test:
https://github.com/szeged/webrender/blob/master/wrench/reftests/border/border-clamp-corner-radius.yaml
Which has a reference image: (we use it flipped for now)
https://github.com/szeged/webrender/blob/master/wrench/reftests/border/border-clamp-corner-radius-ref.png

The result of the test: https://gist.github.com/zakorgy/cb95b2b9e7158ce557f67ec71e9fc8fa
image comparison, max difference: 1, number of differing pixels: 29

Usually we use the following site to compare images quickly: https://huddle.github.io/Resemble.js/
A screenshot about the compare (the purple dots represent the differences, 29 as the result above indicate):

cc @kvark

• OpenGL (Default)
• DirectX11 #25
• Vulkan #26
• Metal #27

I started look into the remaining failing tests and our first issue is that we use our instance buffers totally wrong.

Currently we bind the instances and call draw, but if we use the same program again, we will overwrite the current buffer.

Should be an easy fix. 🤞

The program structure represents shader programs, pipelines, different blendmodes etc.

In webrender, there are 5 types of shaders with different modes:

• Primitive Shader
  • Simple
  • Transform
• Brush
  • Opaque
  • Alpha
• Text Shader
  • Simple
  • Transform
  • Glyph Transform
• Cache
  • Alpha
  • Color
• ClipCache

All of them share common parts, the main difference is the different inputs and samplers.

In the long term, we should create pipelines, render passes, etc. reusable as possible with no duplication.

Both WR and gfx-rs are heavily tested by CI (Travis + Appveyor).
No reason we can't have some coverage here as well.

The current master(b04d40a) causes a segfault when running some of the wrench reftests, e.g. border-suite-2.yaml or reftests\aa\rounded-rects.yaml
platform: windows

The composite is a special case, when we copy between images.
This will fix a lot of test cases ;)

Depends on: #47

Running the reftests returns with the following errors:

error[E0432]: unresolved import `webrender::renderer::CpuProfile`
  --> src/wrench.rs:24:27
   |
24 | use webrender::renderer::{CpuProfile, GpuProfile};
   |                           ^^^^^^^^^^ no `CpuProfile` in `renderer`

error[E0432]: unresolved import `webrender::renderer::GpuProfile`
  --> src/wrench.rs:24:39
   |
24 | use webrender::renderer::{CpuProfile, GpuProfile};
   |                                       ^^^^^^^^^^ no `GpuProfile` in `renderer`

error: no method named `set_profiler_enabled` found for type `webrender::Renderer` in the current scope
   --> src/main.rs:515:41
    |
515 |                         wrench.renderer.set_profiler_enabled(profiler);
    |                                         ^^^^^^^^^^^^^^^^^^^^

error: no method named `gl` found for type `webrender::Renderer` in the current scope
  --> src/png.rs:58:32
   |
58 |     let data = wrench.renderer.gl().read_pixels(0,
   |                                ^^

error[E0277]: the trait bound `[u8]: std::marker::Sized` is not satisfied
  --> src/png.rs:58:9
   |
58 |     let data = wrench.renderer.gl().read_pixels(0,
   |         ^^^^ the trait `std::marker::Sized` is not implemented for `[u8]`
   |
   = note: `[u8]` does not have a constant size known at compile-time
   = note: all local variables must have a statically known size

error[E0308]: mismatched types
   --> src/wrench.rs:180:69
    |
180 |         let (renderer, sender) = webrender::renderer::Renderer::new(window.clone_gl(), opts, size).unwrap();
    |                                                                     ^^^^^^^^^^^^^^^^^ expected reference, found struct `std::rc::Rc`
    |
    = note: expected type `&glutin::Window`
               found type `std::rc::Rc<gleam::gl::Gl + 'static>`

error: no method named `gl` found for type `webrender::Renderer` in the current scope
   --> src/wrench.rs:194:35
    |
194 |         let gl_version = renderer.gl().get_string(gl::VERSION);
    |                                   ^^

error: no method named `gl` found for type `webrender::Renderer` in the current scope
   --> src/wrench.rs:195:36
    |
195 |         let gl_renderer = renderer.gl().get_string(gl::RENDERER);
    |                                    ^^

error: no method named `gl` found for type `webrender::Renderer` in the current scope
   --> src/wrench.rs:357:27
    |
357 |             self.renderer.gl().viewport(0, 0, dim.width as i32, dim.height as i32);
    |                           ^^

error: no method named `get_frame_profiles` found for type `webrender::Renderer` in the current scope
   --> src/wrench.rs:385:23
    |
385 |         self.renderer.get_frame_profiles()
    |                       ^^^^^^^^^^^^^^^^^^

error: no method named `debug_renderer` found for type `webrender::Renderer` in the current scope
   --> src/wrench.rs:407:32
    |
407 |         let dr = self.renderer.debug_renderer();
    |                                ^^^^^^^^^^^^^^

The following steps required:

• Reenable the CpuProfile and GpuProfile if necessary.
• Replace renderer.gl() related calls

In almost every case we receive the bytes for textures in BGRA order, but for Yuv images we receive these in RGBA order. Until now we have used Rgba format for color0 but using Bgra instead made our code less hacky (we don't need to reverse the red and blue bytes in update_texture_data). This modification made our Yuv image colors are displayed wrong. I have the following ideas as a solution:

1. Manually change the red and blue bytes, so move the hacky part to Yuv images.
2. Enable swizzling on OpenGL backend, so we can switch the red and blue channels when creating Textures (This is unimplemented in gfx_rs, and it seems like OpenGL 3.3 feature).
3. Make a different pipeline for Yuv images with different color formats.
4. Extend the existing pipeline with additional TextureSampler for color0. I tried this, but this approach displayed some random memory content to the screen.

Any advice is welcome.

A texture can be used for multiple purpose: data (storing numbers for different structures), image (storing an actual image)

In webrender, there are 3 types of texture used:

• Vertex Data
  • Resource Cache
  • Clip Scroll Nodes
  • Local Clip Rects
  • Render Tasks
• Cache Images
  • Cache{RGBA8|A8}
  • SharedA8
• Images
  • Color{0..3}

The vertex data is stores values for all the shaders.
The cache images are results of previous render passes.
The image(s) (mostly the color0) used to render actual images.

In the long term the images should be re-sizable for reuse.

The goal is to make gfx backend work on Android

The dither is a special image which used by some shader.
It is optional and currently it is disabled.

Depends on: #47