hecomi / uwindowcapture

This allows you to use Windows Graphics Capture / PrintWindow / BitBlt in Windows to capture multiple windows individually and easily use them as Texture2D in Unity.

uWindowCapture

uWindowCapture is an Unity asset to capture windows and make them available in Unity as Texture2D.

Screenshots

Install

• Unity Package
  • Download the latest .unitypackage from Release page.
• Git URL (UPM)
  • Add https://github.com/hecomi/uWindowCapture.git#upm to Package Manager.
• Scoped Registry (UPM)
  • Add a scoped registry to your project.
    • URL: https://registry.npmjs.com
    • Scope: com.hecomi
  • Install uWindowCapture in Package Manager.

How to use

Check out the following article (written in Japanese, so please use a translation service).

• https://tips.hecomi.com/entry/2018/08/26/231618

uwindowcapture's Issues

Window messages aren't sorted, resulting in errors when a window is only active for a single frame

When using screenshot tools like ShareX, there's weird window stuff happening: a temporary screenshot window appears for a single frame and is removed again immediately.

This seems to cause issues with the UwcManager and the window messages – sometimes the "added" messages appear after the "removed" messages, and sometimes there is no "removed" message at all.

Here's a screenshot with the events being logged - the first number is the frame index:

To reproduce:

• in UwcWindowList.cs, add "listItem.Enable();" as last line in OnWindowAdded to add every found window automatically
• download and install https://getsharex.com/
• make a screenshot with ShareX while Uwc is running
• note you'll get a broken white window (no texture)

Possible to grab partial desktop?

Testing latest and it works great!

Is there a setting/possibility/instructions to support capturing Desktop(window title area only)

Capturing window works ok, desktop works ok....but i'd like to capture the window area from desktop (have some overlapped windows in that area)... and don't want to always grab/crop entire desktop.

thanks.

Application crash with WGC

Hi Hecomi,

Sorry for yet another bug report! I'm unsure of the exact cause of this one but it happens upon capturing a window.

Player.log doesn't show any crash, it looks like the process was killed.

uWindowCapture.log shows the following:
[uWC::Log][2021-05-02 21:50:51] Start [uWC::Log][2021-05-02 21:59:26] WindowsGraphicsCapture::DestroyPoolAndSession() throws an exception: 0x800706be [uWC::Log][2021-05-02 21:59:28] WindowsGraphicsCapture::DestroyPoolAndSession() throws an exception: 0x800706ba

(If I find more information about this I'll add it here)

onWindowChangedが毎フレーム発生してしまう

uWindowCaptureバージョン 0.5.0

お世話になります。
uWindowCaptureのUwcWindowCaptureをAddComponent()及びonWindowChangedをAddListenerすると、onWindowChangedイベントが毎フレーム発生してしまいます。
desktopIndexプロパティの更新などを行うと毎フレームの発生は止まります。
できれば、AddCmponent()直後においても毎フレームのイベントが発生しないようにしていただければと思います。

よろしくお願いします。

再現プロジェクト
https://drive.google.com/file/d/1iYfgUPHH2U4zX-w3FyN_X939RqqOuHJH/view?usp=sharing

Window captures potentially leak vram

Steps to repro:

• Capture a window
• Monitor Dedicated GPU Memory in the "Details" tab of Task Manager.

Trying to learn more about compiling DLLs

Is the source for compiling the DLL file included in here or is that a sperate project not included?
Can I technically modify source and compile out a new dll?

EDIT: Realized it comes with the .proj file so I can open that and work from there. Issue solved.

(Feature suggestion) Multi-core window capture

Hi there!

Is it possible to capture windows async?
I researched it myself briefly but wasn't sure if PrintWindow could be used async.

If so, this would be an amazing additional as generally capturing just one window at 30 FPS can be enough to max out a single core on a lot of CPUs.

Thanks! :)

カーソル描画の ON/OFF が動作しない

下記コードを試しましたがカーソルが常に描画（キャプチャ）されてしまいます。

UwcWindow.cursorDraw = false;
UwcWindowTexture.drawCursor = false;

・Exsamples/Single Window → 現象は発生しません。
・Exsamples/Window List → 現象は発生します。
・WindowsGraphicsCapture モードで発生します。
　Windows10 20H2 で Cursor support の判定は true になっています。

複数のウインドウを対象とした場合に発生しているように思えます。

下記のように「変化判定」のif文をコメントアウトするとカーソル描画は OFF に出来ました。

WindowsGraphicsCapture.cpp

line 297
void WindowsGraphicsCapture::EnableCursorCapture(bool enabled)
{
//if (isCursorCaptureEnabled_ == enabled) return;

カーソルを OFF にする場合の注意事項などありますでしょうか？

RaycastHit.Texturecoord returning Vector2.zero in the actual Build, but not in the Editor

The RaycastHit is working in the Editor and i get the correct Vector2 values, but as soon as i build the Unity Application and run it outside the Editor RaycastHit.Texturecoord is returning Vector2.zero. The Window Transform has a MeshCollider, Convex is turned off. Doublechecked if the Texture is readable, which it is in the Editor as well as in the build. Any idea why this is happening? I am using Unity version 2019.2.13f1

Unity 2019.2.x DX12 crash

• Editor crashes when entering play mode on all example scenes in Unity 2019.2.x and Unity 2019.3.0f6 in DX12

large memory leak.

'uWindowCapture' could not be found (are you missing a using directive or an assembly reference?)

Hello, I added uWindowCapture package via UPM git url. I'm wanting to reference the classes from the uWindowCapture namespace in my own scripts but after adding the #using uWindowCapture; I'm getting the following error:

error CS0246: The type or namespace name 'uWindowCapture' could not be found (are you missing a using directive or an assembly reference?)

Some child windows have extra buffer area.

Actual

Captured

Random Crash because of Access Violation (0xc0000005)

First of all, congratulations for your project. It is awesome and works great.

Now to my issue:
I have been faced with an error that sometimes when I run my built game it crashes as soon as it starts. This happens rarely and randomly, around 1/10 of the time.

While checking the Unity crash logs I found that:

uWindowCapture.dll caused an Access Violation (0xc0000005)
  in module uWindowCapture.dll at 0033:f76b5372.

and also:
Read from location 0000000000000000 caused an access violation.

From the tests I have run, the Crash logs always point to this same issues.

I believe this happens because uWC is trying to access memory in use by another process. But I have not tested that.

I'm currently working with Unity 2019.3.5f1 on a Windows 10 Build 19041

If I missed something please tell me to update my Issue.
(Also: uLipSync looks super COOL. I'll check it out as soon as I can)

DPI scaling issue when moving window between 2 monitors with separate scaling

There is a DPI scaling issue when moving window between 2 monitors with separate scaling.
To reproduce:
Start capturing a window on one monitor, then, move the window to a monitor with different scaling. (150% to 100% in my testing)

Example:
https://i.gyazo.com/ab4c4529bd14c3428a4b74c2d3290f72.png

hope this helps! :)

About screen share

Hello, I know this is an asset to capture windows, but is it possible for sharing live screen networks?

thanks.

Maximized windows are cut off on the bottom

When capturing a maximized window, a few pixels at the bottom of the window are cut off.

I believe this is an issue with Windows returning an incorrect position for the window as using GetWindowRect returns the same size and coordinates uWindowCapture is using.

If you're able to find a solution to this, that would be amazing. :)

UwcGetScreenHeight (and Width) build crash

I'm unsure of the specific cause, but in some cases when capturing a window using PrintWindow the following crash occurs using Unity 2019.3.0b7:

Read from location 0000000000000000 caused an access violation.

Stack Trace of Crashed Thread 1732:
0x00007FFD5C2F5372 (uWindowCapture) UwcGetScreenHeight
0x00007FFD5C2EBB2E (uWindowCapture) UwcUpdate
0x00007FFD5C2EFDAB (uWindowCapture) UwcGetScreenHeight
0x00007FFD5C2F00A7 (uWindowCapture) UwcGetScreenHeight
0x00007FFD5C31D3D8 (uWindowCapture) UwcGetScreenHeight
0x00007FFD72037BD4 (KERNEL32) BaseThreadInitThunk
0x00007FFD739ACED1 (ntdll) RtlUserThreadStart

Crash file can be found here:
https://pastebin.com/raw/YCfxs8ij

Voicemeter Potato can't be captured with Windows.Graphics.Capture

Voicemeter Potato can't be captured with Windows.Graphics.Capture, latest version v3.0.1.8. (32 or 64-bit version)

The window shows a white texture, cursor input is inverted. Top is bottom.

Works as intended with PrintWindow, and, Voicemeter is able to be captured by OBS using their Windows.Graphics.Capture method.

UWP apps are displayed with wrong sizes.

UwcIsWindowsGraphicsCaptureCursorCaptureEnabledApiSupported crash

Hi Hecomi,

Loving the new update, Windows.Graphics.Capture is great!

Though I have found a crash:
0x00007FFF29546122 (uWindowCapture) UwcIsWindowsGraphicsCaptureCursorCaptureEnabledApiSupported ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FFF29539287) 0x00007FFF29539287 (uWindowCapture) (function-name not available) 0x00007FFF2953D8F7 (uWindowCapture) UwcIsWindowsGraphicsCaptureCursorCaptureEnabledApiSupported 0x00007FFF2953DC17 (uWindowCapture) UwcIsWindowsGraphicsCaptureCursorCaptureEnabledApiSupported 0x00007FFF29570210 (uWindowCapture) UwcIsWindowsGraphicsCaptureCursorCaptureEnabledApiSupported 0x00007FFF479053FE (KERNEL32) BaseThreadInitThunk 0x00007FFF496A590B (ntdll) RtlUserThreadStart

This same system also returned 'Unsupported' for all displays by uDesktopDuplication, so I assume all their displays are off their main GPU.

They are on Windows 10 version 2004 or later, Windows.Graphics.Capture is supported by the OS version at least.

bad performance with window.GetPixels

Hello, I want to get the rawdata of the texture. So I use Texture2D.GetRawTextureData which give me an empty texture. I searched and found maybe dx and opengl makes it impossible to get the right rawdata.
So I use window.GetPixels and copy the Color32[] to a new Texture and it worked. But when I use window.GetPixels the fps was changed to 15 from 60.
I want to know if I have a better way to do this , or an enchancement to the function

?

Hung up when stopping in editor

Windows touch and/or mouse injection.

Would it be possible to interact on the received texture with the mouse.
The mouse movement is ececuted on the sender window?

Is there any way to not show the yellow frame?

Some windows block the window capture from changing

It seems that some windows (Such as Spotify for Windows 10) will block the window capture from changing to another window. (Changing the window title variable)

Maybe this is an issue with all UWP apps? I didn't test others yet. :)

unity2020 and win8

Hi @hecomi, I'm getting errors about not being able to load the .dll - is this because I need a newer version of windows than win8?

Mouse cursor capture only updates position if cursor is on the main display

I.E. You have two monitor captures, you can see the mouse cursor just fine on the main display, however, the secondary display will not show the cursor if it is on it, and will instead show a mirror of the cursor on the primary display.

onWindowChangedイベントでwindow.widthやwindow.heightなどが変更前の値になる

uWindowCaptureバージョン 0.5.0

お世話になります。
WindowChangedイベントで、window.width/window.height等のプロパティを参照すると、イベント名過去形のイベントですので変更後のモニターのサイズが取得されるのが期待する動作だと思いますが、
変更前のモニターのサイズ(type = WindowTextureType.Desktopにしています)が取得されてしまいます。
変更後のモニターのサイズが取得されるように修正を希望します。

よろしくお願いします。

再現プロジェクト
https://drive.google.com/file/d/1iYfgUPHH2U4zX-w3FyN_X939RqqOuHJH/view?usp=sharing

linker errors when building with IL2CPP

Good day! I am trying to build the uwindowcapture plugin with IL2CPP but im getting the following error when building:

Msvc Install Version: 15.0
Msvc Install SDK Directory: C:\Program Files (x86)\Windows Kits\10
Msvc Linker Path: C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.exe
Msvc Compiler Path: C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe

il2cpp.exe didn't catch exception: Unity.IL2CPP.Building.BuilderFailedException: WindowTexture.cpp
\Temp\StagingArea\Data\il2cppOutput\WindowTexture.cpp(172): error C3861: 'max': identifier not found
..along with errors for other references to the max function.

is this caused by assembly stripping? what could I do to fix this?

Texture becomes white when attaching component by AddComponent

https://twitter.com/gtk2k/status/1086273281529831432

Some right-click context menus flicker a lot when Window Capture is running

To reproduce this issue:

1. Open the DesktopLayouter example scene or any scene that is capturing all the windows, and hit play.
2. Open another instance of Unity, with any project, so that the second instance of the editor will be captured (it won't work by just using the same instance of Unity, because it will stop capturing when you try to do the next step).
3. In the second instance, right-click on an object in the hierarchy.

What you will probably see is that both the actual context menu, and the in-scene child window menu in the first instance of Unity, flicker a lot. This seems to happen both in-editor, and also in a build.

uWindowCapture fails to work on Unity scene change.

Hello, I've never written one of these GitHub issues, so hopefully what I'm doing is correct.

uWindowCapture fails its validation if I simply change Unity's scene while in Play. No matter what I do in the settings for the Uwc Window Texture, it does not change anything.

Here is a quick recording showcasing this.
Here is the only code used, which just changes the scene:

void Update()
    {
        if (Input.GetKeyDown(KeyCode.Alpha1))
        {
            SceneManager.LoadScene("WindowTest1");
        }
        if (Input.GetKeyDown(KeyCode.Alpha2))
        {
            SceneManager.LoadScene("WindowTest2");
        }
    }

As you can see, once a scene change has occurred, uWindowCapture simply stops working.

Let me know what can be done, I appreciate your time in reading this.

Using Unity version: 2021.1.4f1.2345.4

Is it possible to use Direct X 12?

Right now I'm using Direct X 12 for raytracing, and I would love to use this in my environment, the problem is that is not compatible with DX12, my question is, is it possible to create a new version that would be DX12 compatible? I am not a experienced coder, I would love to do it myself but I don't know where to start.
Thanks!

How to hide yellow selection box when capturing window?

I am using uWindowCapture, and I noticed that when capturing windows, there is a yellow selection box surrounding the chosen window. I would like to request a feature or information on how to remove this yellow selection box when select a window, as it can be visually distracting in some use cases.

Thank you for your attention and support. I look forward to hearing any possible solutions or updates regarding this issue.

Windows Crash from CaptureService.dll eating up pagefile

I have a beefy computer, 64GB ram, 3090 24gb, 3950x. I have a pagefile of 160gb that fills up if I leave a unity build with uWindowCapture running overnight. The strange thing, if I watch it during the day in the editor, it doesn't seem to be eating up the pagefile, but I mostly go in and out of play mode during the day. Today was the first time I was able to verify that the pagefile was linked to the unity project build as when windows crashed, it just killed the capture but the unity app stayed running, I had meminfo up with the whole pagefile filled, and it didn't empty down to 22gb until I killed the unity built app. Its using the windows game capture methodology.

DPI scaling issues with child windows

Hi Hecomi,

There appears to be some issues with DPI scaling and the placement of child windows.

In legacy applications that are not DPI aware, the child window placement and scale may be incorrect.

Please try testing with something like 'Audacity' as I can reproduce the issue in this with Windows set to 150% scaling! 👍
Other applications that are DPI aware have their child windows in the correct location and scaled correctly.

In these older applications like Audacity, it also seems the cursor becomes slightly offset from the location of the raycast.

uWindowCapture.logファイルの出力制御について

http://tips.hecomi.com/entry/2018/08/26/231618
このページの説明において
"UwcManager.instance.debugMode を DebugMode.None に設定してください。"
と書かれており、一応、コード上で
UwcManager.instance.debugMode = DebugMode.None
と記述しているのですが、uWindowCapture.logファイルが出力されてしまいます。
どういったタイミングで設定すれば出力しないようにできるのでしょうか？

Linux support

Are there any current or future plans to add Linux support?
Proton doesn't currently support the Windows Graphics Capture api.

Windows cannot display until I focus this program.

Some windows idle for a while.
When I change the texture to this window, it cannot display.
But it still in the UwcManager.windows
I cannot find the difference from other windows which display smoothly.
(I have check the isValid, IsAlive, IsVisible, but all of them is same)
Can I check the windows variable to remove it from my dropdown or request a message to restart the window to display?
Thank you very much.

(Feature suggestion) Ability to flip texture

Hi,

I'm using this asset to display a window capture texture onto a SteamVR Overlay, however, the texture is flipped and as far as I'm aware there's no way to flip it back.

Is it possible to implement an option to flip the texture itself? I'm well aware this could be done through a material within Unity, but in my case the texture is going directly into a SteamVR Overlay, so no material options are available.

Loving the asset so far though, good job. :)

Support win7？

Hello, does this plugin consider supporting win7?

Some windows cannot be captured

Description of Issue
I tried capturing different windows on my system, but some of them seem to not work.
Capturing the complete desktop works and the content of all applications is displayed. Only capturing the windows on it's own results in a complete black or white window.

For example Opera while writing this issue

Tested and not working
Chrome, Firefox, Edge, Opera, TeamSpeak, Task-Manager, VisualStudio Code

Tested and working
Telegram, Windows Explorer, Google Play Desktop App, Thunderbird, Keepass, VisualStudio, OBS

System Information
Unity 2019.3.6f1
Windows 10 Pro Insider Preview Build 19577.rs_prerelease.200228-1439

Other Information
I have tried the different properties for capturing in different combinations, but all have the same result. I also only tried your examples

Recent windows update (KB4541335) causes some windows to flicker.

Recent windows update (KB4541335) causes some windows to flicker. The flickering briefly causes the Windows 7 style 'Aero' border to appear on windows. I believe this is a Windows bug as it has only started with that specific update.

A friend of mine said it's caused by the window re-drawing when being captured too frequently, however, this same issue is happening with the capture framerate set to 30 FPS.

Can reproduce using Spotify's desktop app, or I believe Steam.

Resizing a captured window causes black window in Unity (WGC)

Resizing a captured window causes black window in Unity when using Windows.Graphics.Capture.

About mouse cursor status

Hello, I set the mouse cursor to false through the code, but it cannot take effect. It can only take effect by clicking Draw Cursor in the Inspector in Unity.
Is there a problem with my writing? I set it through windowTex.drawCursor = false

CaptureService crashes aren't handled correctly

Hi Hecomi!

Crashes of the 'CaptureService' aren't handled well and cause the application to exit.

You can crash the CaptureService by killing 'explorer.exe' in Task Manager to reproduce this issue. (Explorer randomly crashes on some users systems and it brought this to my attention, Windows is 'flawless'!)

Let me know if you need any more details! Thanks for your good work! :)

LNK1104 for uWindowCapture.def

I believe this is me not knowing how to properly configure my project, but I am receiving this error when trying to compile from main branch. Can anyone help me?

Project path doesn't contain spaces.
uWindowCapture.def is on uWindowsCapture project root structure.

VR support

Hello , it would be nice to support the oculus and steam vr

Desktop情報取得のタイミング

現在、uDesktopDuplicationと併用して使用しています。
uDesktopDuplicationが使えない環境においてuWindowCaptureを使用する形をとっています。
uDesktopDuplicationにおいてはuDesktopDuplication.Manager.instanceにアクセスすることでマネージャーのインスタンス化が行われすぐにmonitorCountなどの情報が取得できますが、uWindowCaptureの場合、uWindowCapture.UwcManager.Instanceにアクセスしてマネージャーのインスタンス化を行ってもすぐにはdesktopCountなどの情報が初期化されませんでした。
ですので、uWindowCaptureにおいてですトップ情報の初期化が行われたというイベント等が欲しいと思いIssueにあげさせていただきました。
どうかよろしくお願いします。