Crashing when exiting the game Rayman 3: Hoodlum Havoc about d3d8to9

[CookiePLMonster]

Are d3d8, d3d9, d3dx9_xx loaded at that time still or already released?

[elishacloud]

All three dlls (d3d8, d3d9, d3dx9_xx) are still loaded. None of them have been released yet.

[elishacloud]

Ok, I did a bit more troubleshooting on this. The issue is not related the check-ins that were done in the last few days. My testing was flawed there. I was doing most of my tests inside my larger project dxwrapper. If I pull all the changes into my project then the latest code runs fine, even with all of the three latest check-ins.

However when I run the code standalone, after just compiling the d3d8to9 project (by itself) the game will crash on exit (for this particular game). Even if I revert back to the February 0c5e710 check-in, it still crashes on exit for this game. I am not quite sure what the difference is between the way I am using this code in my larger project and how the code is running standalone.

[elishacloud]

If I compile the d3d8to9 project for Release then everything works fine. It is only when I compile this for Debug that it crashes. This could just be my issue. I am going to close this ticket.

[CookiePLMonster]

I wouldn't close it but rather rename and keep the issue open - a Debug only issue like this indicates something is trying to use resources after they've been already freed (0xDDDDDD looks like memory which has been invalidated after being freed). While unrelated to the latest changes, it may indicate either a bug in d3d8to9 or in the game itself.

If it's a game bug, should we guard against it?

[elishacloud]

Ok, reopening based on CookiePLMonster's comments.

[CookiePLMonster]

I tried the same with GTA III and enabled all heap debugging features I could think of - no issues. So if this is caused by d3d8to9, it's probably not an "universal" issue.

Have you tried identifying what's being written to this address? (0x5F5E34 looks like a pointer to a structure, maybe some D3D8 resource and this is a Release call?)

May you upload your game EXE somewhere?

[CookiePLMonster]

Alternately, a quick way to verify what's going on would be adding a custom assertion to all ::Release() calls and then compiling in Release so you don't crash.

Could try adding something like this on top of all ::Release() functions:

if ( _ref == 0 )
   __debugbreak();

then see what happens when you quit the game.

[elishacloud]

First of all I tried with all of the following Direct3D 8 games and only Rayman 3: Hoodlum Havoc has this issue:

• Tom Clancy's Splinter Cell
• Star Wars Starfighter
• Moto Racer 3
• Hitman 2: Silent Assassin
• Haegemonia: Legions of Iron

I tried adding a break into the Release() function, but did not catch anything. But I did open this up in OllyDbg and it appears that Raymond 3 is calling the EndScene() and then calling the Release() function. As far as I can understand it the line that it actually crashes on (address: 0046FD9C) is suppose to be calling the Release() function. It would seem that for some reason Raymond 3 lost the address for the Release() function and is instead trying to call 0xDDDDDD.

[elishacloud]

I figured out exactly where it is crashing. It is crashing on two lines:

1. d3d8to9_device.cpp line 79:

if (myRef == 0)
{
	delete this;		// <---- Crashing on this line
}

2. d3d8to9_surface.cpp line 41:

if (myRef == 0)
{
	delete this;		// <---- Crashing on this line
}

This is clearly an issue with the d3d8to9 wrapper. I am not quite sure how to fix this. I tested using __try and __except and that did not fix it. I also tried adding an if statement before the delete this; to
"delete" only if "this" had a value. But neither one worked for me.

If I remark out these lines then (other than a memory leak) everything works fine.

[CookiePLMonster]

This is puzzling... since you entered this code path then this pointer must've been valid (since it read _ref without issues) - what's the value of this at the moment delete gets called?

[elishacloud]

Ok, I just realized that crashing because a line exists is not the same thing as crashing on a line. It seems to be that it will crash if delete this; exists but I don't think it is actually crashing on this line.

It is actually crashing when Raymond3.exe tries to call Release():

It seems to keep looping and calling Release() until it crashes. Sometimes it crashes on the first loop and sometimes on the second loop.

[crosire]

It will crash if that line exists because d3d8to9 will delete the device there, which with the CRT debug runtime active will overwrite the entire memory region with 0xD. The game however attempts to use the device afterwards (which should not happen) and access' the device object at offset 8 (0xE5 - 0xDD). This can have two reasons:

A) The game uses the D3D8 device after releasing all its references to it.
B) The game does still hold a reference to the device, but d3d8to9 miscalculates the reference count and therefore destroys the device too early.

I'm guessing it's B, but in that case d3d8to9 should at least print a warning about a reference count mismatch to the log before crashing later.

[CookiePLMonster]

It might not necessarily leave a message in log file if it hasn't been flushed at the time of crashing - try adding LOG.flush() below ich logging call and then check it!

And yes I'm guessing B too - maybe it would be useful to assert if real_ref == wrapper_ref in all AddRef and Release calls? While it makes no sense for the other wrapper types (since your real d3d9 object might have far more references than the wrapper and it's all legal), it should probably work fine for device.

[elishacloud]

Crosire described exactly the issue. You are both right it is B. However, it is crashing in Direct3DSurface8::Release() first and there is no log in this Release() function so that is why we are not seeing the log. But if I rem out the delete this; line in the Direct3DSurface8::Release() function and add a LOG.flush() to the Direct3DDevice8::Release() function then you can see this in the log file: "Reference count for 'IDirect3DDevice8' object 0B37DAC8 (1) is inconsistent."

The inconsistent reference is also mentioned here. Furthermore I have described the reference count issues with this wrapper here.

I took a queue from CookiePLMonster and added code to make real_ref == wrapper_ref in these two specific Release() calls. If I add the following lines into these functions then no crash or memory leak:

InterlockedExchange(&_ref, ref);
myRef = ref;

[CookiePLMonster]

hmm, though surface wrapper should be fairly safe to release independently from the underlying d3d9 object unless the wrapper is not increasing reference properly somewhere - which potentially could release the d3d9 surface before all its wrappers are destroyed. Device reference count issue should be investigated first though - it is not unlikely those two are related.

[CookiePLMonster]

The thing is, current refcounting is busted indeed. Since wrappers' refcounts are meant to be transparent and they return 'real' refcounts to the user code, loops like this:

while ( d3d8_resource->Release() > 0 );

are going to fail since d3d8to9 wrapper refcount is always equal or less to real d3d8 resource refcount, thus if we are returning resource refcount instead of our own, the code is trying to release our object too many times. This is exactly what Rayman seems to be doing.

I branched the code and tweaked this so underlying resources don't have their refcounts touched (only released from wrapper destructors) and instead wrapper refcounts are returned to the client. This should resolve the issue, hopefully not resulting in any resource leaks (shouldn't cause any, though). Try compiling code from this branch in Debug mode and then observe the results:

https://github.com/CookiePLMonster/d3d8to9/tree/alternate-refcounting

[elishacloud]

Yes, that update works. No crash or other issues. I also tried it with a few other games and seems to work fine with them also.