Preview handlers fail to handle 64-bit handles about powertoys

[Aaron-Junker]

So hey, I'd like to work on this, thus as mandated in #28769, here's my request of an attempt to fix it.

Good catch. Yes, absolutely, do implement it! Thank you!

[github-actions]

Hi I'm an AI powered bot that finds similar issues based off the issue title.

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it. Thank you!

Closed similar issues:

• Monaco Preview handler hangs/crashes (#30385), similarity score: 0.79
• Source code files (Monaco) crashes with Unhandled Exception (#23474), similarity score: 0.78
• Explorer preview pane add-on crashes the .NET (#23415), similarity score: 0.78
• MSEdgeWebView2 crashes when trying to preview a file through Monaco in file explorer (#30370), similarity score: 0.77
• "Catastrophic failure" on file previews (#23475), similarity score: 0.76

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

[z4pf1sh]

I did some debugging myself, and the faulty line turns out to be:

PowerToys/src/modules/previewpane/MonacoPreviewHandler/Program.cs

Line 33 in fd6de9a

int hwnd = Convert.ToInt32(args[1], 16);

... where it assumes that the passed window handle is a signed 32-bit number.

This should not be how it's implemented. This time, Windows passed a 64-bit handle to the process, with args[1] = FFFF FFFF A290 0D2C, as shown in my Immediate Window:

And the process promptly crashed, with an exception of "Value was either too large or too small for a UInt32."

The confusing thing for me is this line below, where it's explicitly converted to IntPtr, when the hwnd is actually used:

PowerToys/src/modules/previewpane/MonacoPreviewHandler/Program.cs

Line 42 in fd6de9a

_previewHandlerControl.SetWindow((IntPtr)hwnd, s);

With reference from Microsoft Learn on the IntPtr struct:

The IntPtr type is designed to be an integer whose size is the same as a pointer. That is, an instance of this type is expected to be 32 bits in a 32-bit process and 64 bits in a 64-bit process.

Here's my thinking: why don't just convert the input string to a IntPtr at the very beginning, instead of the flow of: A Potentially 64-bit Hexadecimal String -> A Signed 32-bit Integer -> A Platform-dependent Handle Type, when you can parse it at the first shot? I think this should be a relatively easy fix, and I will try to fix it with a PR once approved.

[z4pf1sh]

So hey, I'd like to work on this, thus as mandated in #28769, here's my request of an attempt to fix it.

[z4pf1sh]

According to comments in the PR #32826 (comment), this affects all other preview handlers as well. I will work on applying the same fix to all other affected handlers as well once I got the approval.