Video Encode API: Crash in CreateComponent with AMD Hybrid/Switchable GPU systems about amf

[MikhailAMD]

The dump in the rar file doesn't match the text file in the report on obsproject.com forum. Dump doesn't have AMF on the stack of the main thread while the text file does .

1. I would need a dump file with AMF on the stack to prevent the crash.
2. To make it working VCE plug-in should enumerate DX devices and find the correct AMD one. By default AMF creates the first available DX device ( first adapter) which happens to be Intel and cannot use it.

[Xaymar]

1. AMF is on the Call Stack in the Main Thread, just doubleclick on the entry to go back to it. OBS's crash handler caught it to ensure a safe-ish shutdown of OBS.
2. Shouldn't the expected behaviour be to return AMF_NO_DEVICE or AMF_NOT_SUPPORTED (no device that supports this type of encoding)?

I've asked the user to also provide a log and crash log for the crash now.

[Xaymar]

• Log: 2016-11-15 17-35-08.txt
• Crash Log: Crash 2016-11-15 17-35-17.txt

[MikhailAMD]

Yes, I can see the stack if I get VS2015, Originally I tried VS2013. The crash happens in undocumented use of HEVC encoder. I cannot debug or comment on this. Does it happen with AVC? Also please ensure that you are using AMD DX device. AMF can get wrong default one. All device selection should be done by the client code.

[Xaymar]

Oh it happens in HEVC, that explains a lot.

[Xaymar]

I've removed HEVC testing from the plugin for now, waiting for the user to confirm that it no longer crashes.

[Xaymar]

No longer crashes.