kirides / screencapture

This repository has been moved to https://github.com/kirides/go-d3d

Screencapture

This repository has been moved to https://github.com/kirides/go-d3d

Motivation

I wanted to learn more about COM interop with Go and create a somewhat usable screen sharing tool

Libaries used

This application uses D3D11 IDXGIOutputDuplication to create a somewhat realtime desktop presentation

• github.com/mattn/go-mjpeg for mjpeg streaming
• github.com/kbinani/screenshot for comparison with GDI BitBlt (slightly modified source, to support re-using image.RGBA)
• golang.org/x/exp/shiny/driver/internal/swizzle for faster BGRA -> RGBA conversion (see shiny LICENSE)
• github.com/pixiv/go-libjpeg/jpeg for fast jpeg encoding
  • enable with go build -tag jpegturbo

Demo

Just build the application and run it. It should serve all your monitors on an URL like http://127.0.0.1:8023/watch?screen=N
where screen=N <- N is the monitor index, starting at zero (0).

By changing the lines in cmd/example/main.go regarding the streaming, you can switch between GDI BitBlt or IDXGIOutputDuplication

// ...
framerate := 10
for i := 0; i < n; i++ {
    // ...
    // streamDisplay(ctx, i, framerate, stream)  // <= USE GDI BitBlt
    streamDisplayDXGI(ctx, i, framerate, stream) // <= USE IDXGIOutputDuplication
    // captureScreenTranscode(ctx, i, desiredFps)
    http.HandleFunc(fmt.Sprintf("/mjpeg%d", i), stream.ServeHTTP)
}
// ...

screen recording with ffmpeg

The code contains the function captureScreenTranscode which allows you to record the selected screen directly into ffmpeg and transcode it to h264 in an mp4 container.

Performance

Performance is not optimized to 100%, there are still thing that could be improved.

• only copying the dirty-rectangles (less GPU<->CPU communication)
• faster swizzle implementation using AVX/2 (less time for converting the BGRA texture)
• profile ... profile ... profile

Overall the current implementation is about 2-5x faster than GDI BitBlt (depending on the resolution, the higher the bigger the difference) and uses a lot less resources for cases where there arent any changes to the screen.

app.manifest

To make use of IDXGIOutput5::DuplicateOutput1, an application has to provide support for PerMonitorV2 DPI-Awareness (Windows 10 1703+) This is usually done by providing an my-executable.exe.manifest file either next to the executable, or as an embedded resource.

In this application there are calls to IsValidDpiAwarenessContext and SetThreadDpiAwarenessContext which circumvent the requirement.

screencapture's Issues

Build issue missing Library

Modularity/Public packages

Hi! I have used your code to facilitate screen capturing for an app I am making, but I had to copy the repo locally and replace some imports. Would you be interested in making the code more friendly for other users that want to import parts of it (e.g. d3d, swizzle)? I can also create a PR that adds that if you don't want to invest the time to do it.

Thanks in advance!

Stream with dxgi no output.

I had ran the code on windows10 platform, but could not get any output on localhost:8023/watch after i build the code.
However i change the function of acquiring stream by using old Bltbit, then i got the video.
are there some bugs in calling interface of dxgi?

screencapture example passes wrong display index

The screencapture example once uses a param called n for getting the screen bounds screenshot.GetScreenBounds, which takes the displays index displayIndex, and then uses the same parameter for d3d.NewIDXGIOutputDuplication, which takes the displays number output.

I believe this is an off-by-one error because I had to do this for my records not to get messed up:

bounds := screenshot.GetDisplayBounds(displayNum - 1)
//                                                 ^ if not subtracting one from the parameter passed
//                                                   into NewIDXGIOutputDupilcation, this is wrong

Consider splitting up into multiple modules

First off, thank you so much for building screencapture!!

Would you consider splitting it up into multiple modules? As it stands, I think this is the only Golang module that implements IDXGI.

Other Windows API wrappers don't even support it, but it's clearly needed: lxn/win#122

You could also consider setting up GitHub Sponsors :)

Many thanks from Stuttgart