alelievr / hdrp-ui-camera-stacking

Optimized implementation of camera stacking for UI only in HDRP.

HDRP UI Camera Stacking

The HDRP UI Camera Stacking package allows you to stack multiple camera rendering UI only at a fraction of the cost of a standard camera.

This is achieved by taking advantage of the customRender feature to render only the GUI elements and nothing else, with the only downside to not be able to render Lit objects which are generally not found for UI.

UI.mp4

This implementation also provides all the benefits of standard camera stacking:

• No post process bleeding on UI (motion vectors, ect.)
• No clipping in the geometry

Installation

The UI Camera Stacking package supports the following versions:

Unity Version	HDRP Version	Compatible
2019.4.x	7.x	❌
2020.1.x	8.x	❌
2020.2.x and 2020.3.x	10.x	✔️
2021.1.x	11.x	✔️
2021.2.x	12.x	✔️
2022.1.x	13.x	✔️
2022.2.x	14.x	✔️
2022.3.x	14.x	✔️
2023.1.x	15.x	✔️

Instructions

HDRP UI Camera stacking is available on the OpenUPM package registry, to install it in your project, follow the instructions below.

1. Open the Project Settings and go to the Package Manager tab.
2. In the Scoped Registry section, click on the small + icon to add a new scoped registry and fill the following information:

Name:     Open UPM
URL:      https://package.openupm.com
Scope(s): com.alelievr

3. Next, open the Package Manager window, select My Registries in the top left corner and you should be able to see the HDRP UI Camera Stacking package.
4. Click the Install button and you can start using the package :)

⚠️ If you don't see My Registries in the dropdown for some reason, click on the + icon in the top left corner of the package manager window and select Add package from Git URL, then paste com.alelievr.hdrp-ui-camera-stacking and click Add.

Note that sometimes, the package manager can be slow to update the list of available packages. In that case, you can force it by clicking the circular arrow button at the bottom of the package list.

Tutorial

First, create a UI Camera gameobject with the menu UI > Camera (HDRP)

In this new UI Camera there is a component called HD Camera UI, this is the component that will do the rendering of the UI. It's important to correctly configure this component because the camera depth and culling mask are ignored when this component is added.

In the UI Camera gameobject a canvas was also created and correctly configured with the "Screen Space - Camera" mode. You can add your UI in this canvas.

Property	Description
Ui Layer Mask	Layer mask of your UI objects, by default it's set to UI.
Priority	Used to define a draw order between the UI cameras, a high priority means rendered in front of the other cameras.
Compositing Mode	Select how you want the compositing to happen. By default the Automatic will blend the UI render texture with the camera color after the camera rendering. The Custom mode allows you to replace the compositing material by your own fullscreen shader. The Manual mode disables the compositing and you have to do your own one with the renderTexture field in HDCameraUI.
Target Camera	Specifies which camera will receive the UI. By default the Main value only adds the UI to the main camera. The All value outputs the UI to every camera in the scene. The Layer value allows you to filter cameras by layer. The Specific mode allows you to only output the UI to a specific camera.
Graphics Format	Format of the color buffer used to render the UI, the default is 16 bit RGBA to avoid banding and keep the alpha channel.
Render In Camera Buffer	If true, the UI will also be rendered in the attached camera color buffer.

Performances

In HDRP using more than one camera have a very high performance cost. While you can avoid most of the performance issue on the GPU side with correct culling settings and disabling almost every in the frame settings, you won't be able to escape the CPU cost.

The scenes used for the performance test are available in the Benchmark folder of the project. For HDRP, the Graphics Compositor was used to perform the UI camera stacking. The UI camera had custom frame settings optimized to render GUI (transparent unlit objects).

All performance metrics were captured at a resolution of 1920x1080 on Windows 10 in the Unity Editor.

Setup	CPU Time (ms)	GPU Time (ms)
HDRP camera stacking	~1.35	0.45
Custom UI camera stacking	~0.10	0.18

Without much surprise, we can see a big difference on CPU side, mostly because we're skipping all the work of a standard HDRP camera. On the GPU side things are pretty even though the camera is still a bit slower because of the overhead of compute shader and fullscreen passes that can't be disabled in the frame settings.

System Information

• System: Windows 10 Pro, DirectX 11
• CPU: i9-10980k 3.00GHz 18 cores
• GPU: RTX 3090

Limitations

Rendering Lit objects is not supported. Currently the UI rendering happen before the rendering of the main camera, thus before any lighting structure is built so it's not possible to access the lighting data when rendering the UI for camera stacking.

Future Improvements

• Fullscreen effect applied after rendering the UI
• Add an injection point before post processes
• Support post processing volumes for UI Cameras.

hdrp-ui-camera-stacking's Issues

UI Blur with HDRP-UI-Camera-Stacking

Hi.
Is it possible to an UI Blur against the 3D background with the HDRP-UI-Camera-Stacking similar to the the HDRP-Custom-Passes#screenspace-camera-ui-blur?

If yes, would it be possible to add an example in the HDRP-UI-Camera-Stacking repository?

Post-processing update?

Did you ever get post-processing to work with this?
It would be great since then I could 'curve my UI' using a post process.

There is a sort of work around to do this but it's overcomplicated with render-to-textures and it doesn't ever look correct.

VR Support?

It's great to see this referenced from the official Unity blog!

While I understand this is absolutely no guarantee for support here would still be great if this would indeed support VR where performance is pretty critical (and goes bad with a separate UI camera right now).

As far as I can see that would mostly mean supporting the right RT layout, and supporting stereo instancing in the compositing shader...

MainCamera is white in build, UI is drawn on top - in Version 1.4

I got Version 1.4 from UPM. It worked fine in Editor, but in build the image was white, only the UI layer was drawn on top.
I created a Dev Build and connected the Frame Debugger, it seems to be very similar to this issue:
#18
But adding the Shaders to "always include" didn't fix it for me.

Have a look at: UI Transparency Background Init.shader
There is MainTex defined as "2D" but sampled as array:

5 _MainTex("Main Texture", 2D) = "white" {}
13 TEXTURE2D(_MainTex);
52 float4 s = SAMPLE_TEXTURE2D_X_LOD(_MainTex, s_linear_clamp_sampler, uv, 0);

Unity throws an "error" (yellow warning icon though) if I select this shader file. Something about implicit truncation of vector type.
If you change SAMPLE_TEXTURE2D_X_LOD to SAMPLE_TEXTURE2D_LOD the error is gone. But the problem is not fixed.

I also tried returning "0.5" for the alpha - that results in my scene being half-visible (but overlayed with 50% white - as expected)
But neither 1 nor 0 work perfectly - also expected. 0.01 looks ok for colors, but stuff on UI Camera show a trace over multiple frames... - this is clearly not the right way to fix it. But it helped me pinpoint the problem.

So I changed everything to 2DArray (like it's done in HDRP UI Compositing.shader):

_MainTex("Main Texture", 2DArray) = "white" {}
TEXTURE2D(_MainTex);
SAMPLE_TEXTURE2D_X_LOD(_MainTex, s_linear_clamp_sampler, uv, 0);

That "worked" but the scene was still too bright, didn't look like it's simply overlayed with a white semi-transparent image. Instead it looked like it's gamma instead of linear? So maybe we are close to a fix here: Using "2DArray" (etc.) in the "UI Transparency Background Init.shader" is step one, and step 2 could have to do with linear/gamma color space. But I'm stuck here. Maybe it helps you to reproduce and find the error?
I'm using linear color space and I haven't tested with gamma yet.

But I saw the LinearToGammaSpace function in the shader, altering this didn't have an effect for me.
Also the Composite function had a commented line where you probably worked on the alpha parts:

    return s;
    // return float4((s.rgb), s.a);

Which may be a hint that I'm not completely on the wrong track here.

For now I "fixed" it now by replacing the 1.4 with 1.2 from here: https://github.com/alelievr/HDRP-UI-Camera-Stacking/releases

Can VFX can shown with UI ?

Hi, thank for your work

So we wanted to show VFX in our UI but we have this effect :

I seem that it's because transparancy isn't manage but is this a way to make this work ?

Each UI have two drawcall?

I only create one image, but there are two drawcall

Feature Request: Render UI to RenderTexture attached to Camera

It would be great to be able to use the performance benefits of single-pass rendering when using a render texture output on the camera as well.

UI Mask not working.

UI Mask component seems to be ignored by the custom renderer.
with standard camera

with HD Camera UI

Unity 2020.3.20f HDRP 10.7.0

Out of memory when toggling cameras on and off repeatedly

We were getting out of memory issues on xbox when objects with HDCameraUI on them were toggled on and off frequently (on opening and closing our pause menu for example). This doesn't happen in editor seemingly, but maybe I just had more memory available.

I presume the issue is that internalRenderTexture gets newed OnEnable every time, so the old texture is orphaned and wasn't getting cleaned up automatically (at least on xbox). Moving as much of the OnEnable functionality as possible into Start seems to have fixed the issue, however it also doesn't seem ideal that once you turn a HDCameraUI component on then it holds on to a screen sized buffer for the entirety of its lifetime (and then possibly leaks it once it's destroyed if that is what was happening before), so I also made changes to release the RenderTexture on Disable/Destroy.

void Start()
{
    data = GetComponent<HDAdditionalCameraData>();
    attachedCamera = GetComponent<Camera>();

    if (data == null)
        return;

    data.customRender -= StoreHDCamera;
    data.customRender += StoreHDCamera;

    // TODO: Add VR support
    internalRenderTexture = new RenderTexture(1, 1, 0, graphicsFormat, 1);
    internalRenderTexture.dimension = TextureDimension.Tex2DArray;
    internalRenderTexture.volumeDepth = 1;
    internalRenderTexture.depth = 24;
    internalRenderTexture.name = "HDCameraUI Output Target";

    cullingSampler = new ProfilingSampler("UI Culling");
    renderingSampler = new ProfilingSampler("UI Rendering");
    uiCameraStackingSampler = new ProfilingSampler("Render UI Camera Stacking");
    copyToCameraTargetSampler = new ProfilingSampler("Copy To Camera Target");
    initTransparentUIBackgroundSampler = new ProfilingSampler("Init Transparent UI Background");

    if (blitWithBlending == null)
        blitWithBlending = Shader.Find("Hidden/HDRP/UI_Compositing");
    if (blitInitBackground == null)
        blitInitBackground = Shader.Find("Hidden/HDRP/InitTransparentUIBackground");

    CameraStackingCompositing.uiList.Add(this);
}

void OnEnable()
{
    if (data == null)
        return;

    data.customRender -= StoreHDCamera;
    data.customRender += StoreHDCamera;
    
    CameraStackingCompositing.uiList.Add(this);
}

void OnDisable()
{
    if (data == null)
        return;

    data.customRender -= StoreHDCamera;
    CameraStackingCompositing.uiList.Remove(this);
    if(internalRenderTexture.IsCreated()) internalRenderTexture.Release();
    internalRenderTexture.width = 1;
    internalRenderTexture.height = 1;
}

private void OnDestroy()
{
    OnDisable();
}

Changing aspect ratio?

Hi

Thanks for the great package!
But there is one problem I've encountered, the UI do not display on the same view rect as main camera.
If I change the view rect at runtime, the UI still displays on the old one and becomes stretched.
Is there anyway to force HDRP UI Camera to keep the same view rect as main camera?

Custom Passes ignored

Hello,

We are using 3 Custom Passes for the UI Blur (explained here: https://www.turiyaware.com/blog/unity-ui-blur-in-hdrp).

Here is an example:

1. UI_01 (Before Pre Refraction): All the UI elements, that can get blurred, such as backgrounds, etc.
2. UI_02 (Before Transparent): Blurring element, blurring UI_01.
3. UI_03 (Before Post Process): The UI elements that never get blurred such as texts over the blurring elements.

It is not possible for us to directly blur the background image as the blurring UI element over it is dynamic.

The custom passes are working in a normal camera. But when using the HD Camera UI, all the custom passes are ignored and the UI_02 is not blurring UI_01 anymore, but only the 3D scene beneath it.

Is it possible for the HD Camera UI to take the custom passes into account?

Thank you!

Feature request - Render To Texture

Hi, I've been using a separate camera to render my minimap.
It's basically a camera above the player that renders icons on each object with a minimap script on them.

Normal camera works fine using this technique (render to texture) but I imagine I'm using more power than I should for this basic effect.

When I use this camera to do the same effect, it doesn't seem to render to texture and puts right across the screen.
With the other camera technique, it renders fine in the bottom left of the screen.

Thanks

[Question] What is the benefit of using this technique compared to a Custom Pass?

Hi,

What is the benefit of using this technique compared to a Custom Pass when I just need to stack screen space UI that skips any post-processing over my main camera?

My Custom Pass config looks like this:

It works and performs well, but I'm always trying to understand every available solution out there :)

Is this repository's solution even faster? Or does it produce better visual quality? Or is it intended for some more complex UI scenarios?

Regards,
Stepan

im getting this error help

'Display' does not contain a definition for 'activeEditorGameViewTarget'

Rendering to only one specified camera

I want my UI Camera to only render to my my main camera. Currently, it renders to all my cameras, with no simple solution to disable this!

[Question] Is Lit support even possible in the future?

Hi,

I'm making a stealth game. For AI vision I had to solve light level measurement (so that NPCs don't see the player and/or other NPCs when they're hiding in the dark).

I tried reproducing GPU lighting on CPU (for just one point in space). I managed to reproduce attenuation from HDRP source code, but that still doesn't produce complete results. I wasn't able to fully account for the final light falloff like this. Maybe I missed something, I thought Attenuation functions are everything I need for direct real-time lighting.

So I abandoned that solution and added second camera that only renders octahedron placed inside the player chest and/or other NPCs.
For example on frame 1 I render octahedron (A) from the top facing downward, on frame 2 I then render the same octahedron from the bottom facing upward. Then I do the same on frames 3 and 4 for octahedron (B) etc.
This ends up in a render texture that I then sample.

But dayum... the performance is terrible, even when the second camera has insanely small frustum.
The technique presented in this repository seems amazing, but doesn't support lit geometry. Plus I will still need a second camera anyway, right?

Sorry if this question is a bit unrelated, but I'm a bit desperate here.

Any better ideas how to handle what I'm doing? Any tips? Please <3

The translucent effect of a sprite with translucency mixed into the scene is different from that of overlay mode

Dependency error with Unity 2020.3.14f1

The package caused a dependency issue with Unity 2020.3.14f1 because it requires a minimum HDRP version of 11.0.0.
The maximum HDRP package version that can be installed in this Unity version is 10.5.1.
This leads to a dependency error and in my case it uninstalled the HDRP package completely.

Does not operate normally in build mode

Unity : 2021.3.0f1

There is no problem in editor mode.
However, there is a problem with the runtime after the build. (exe)

Nothing is drawn on the screen.
The screen is black.

To be exact, the screen visible through the camera is not rendered.
The UI, which is ScreenSpace Overlay, is rendered.

Disabling the UI Camera Object at runtime will draw the screen normally.
When reactivated, the screen turns black.
There is no particular error message.

In order to accurately communicate the cause of this problem,
I tried to create a similar environment in a clean project,
but it was not reproduced. (It works normally even at runtime)

Variables not included in the Clean Project are:

• Addressable
• Multi-Scene Structure
• Shader and Art Resources etc
  I didn't check including these,
  but I thought these were not the cause of the problem.
  (Can these affect?)

Which part should I check?

No split screen/screen rect support?

I am making a split screen game using cinemachine, and all seems well, except that the HDRP UI Camera doesnt seem to account for the camera that its rendering Screen rect (as this has been halfed for each of my 2 cameras).

I've looked at the code, I assume I need to change the internal texture size (as Im using the automatic option) to something other then camera.pixelWidth and camera.pixelHeight? or is there another way?

RenderTexture not working

I'm... trying, but it's not working. )

Feature request / help needed

Hey, I'm trying to use this package alongside volumes for post processing on the UI, is this possible?

RenderTexture' does not contain a definition for 'depthStencilFormat'

It seems RenderTexture.depthStencilFormat is not found in Unity2020.3.
Simply commenting-out line 132 in HDCameraUI.cs looks working.

Package errors in Unity 2020.3.41

The package doesn't compile in the latest 2020.3 LTS version of Unity, throwing the following errors:

CameraStackingCompositing.cs(86,53): error CS0173: Type of conditional expression cannot be determined because there is no implicit conversion between 'UnityEngine.RenderTexture' and 'UnityEngine.Rendering.BuiltinRenderTextureType'

and

HDCameraUI.cs(248,146): error CS0117: 'ClearFlag' does not contain a definition for 'DepthStencil'

Both are odds and I don't know how to fix them.