RDB outdated. about project64

[Lithium64]

I posted two years ago, info about recently dumped roms.
http://forum.pj64-emu.com/showthread.php?t=3695

[theboy181]

Alec64 Roms and 64dd too

[Fanatic-64]

I plan on dedicating to the RDB once a few issues are resolved. I'm going to add all the new dumped ROMs and configure all games optimally (or at least that's the goal).

I'm not sure about keeping Public Domain ROMs though, they usually don't work and are often just trivial tests, which I doubt people will keep on the ROM explorer anyway.

[cxd4]

Public domain ROMs usually work the easiest, with less issues than commercial ones. It's just that some of them are designed to break emulators or defy standard practice. In most cases, if a homebrew ROM doesn't work, it is worth filing an issue report on the GitHub.

As for keeping them in the RDB...maybe not since a lot of times they're compatible with any combination of settings. It helps to not have "Bad ROM?" spammed all over the Project64 window though, and that's what happens if your ROM directory is full of PD ROMs that aren't found in the RDB.

[project64]

It is better to have the public domain roms in there for the point of the BAD ROM message.

there is lots of different clean up needed for the rdb like compatibility and comments.

like should we have different comments based off different plugins, should we also have md5 of the roms in the file.

[cxd4]

In a perfect world, I don't think any RDB file should need to exist at all. The concept of an RDB file should boil down to which recompiler tactics to employ if using the recompiler core, or whether to use the recompiler instead of the interpreter, and maybe some core optimizations outside of recompilers like not emulating TLB, assume 32-bit etc..

Some comments about third-party plugins maybe could use some updating for when people complain that the first-party ones have issues without knowing about today's plugins, though most of the plugin information written by Smiff is still actually kind of accurate, since said plugins have all still been dead since before his disappearance.

[cxd4]

I would avoid the MD5 thing personally since that makes Project64 look less like a N64 hardware emulator, more like a gaming application (which maybe it is). IIRC pj64 already compares the ROM header CRC32's to the calculated one to make sure it's up-to-date or something; I thought that was already secure enough? Besides, "bad ROMs" or "hacked ROMs" don't really exist in a genuine sense. As far as the N64 hardware's concerned, modifying Zelda or Mario ROMs to permanently give you infinite health or whatever, is like an official change made by the developers themselves, not a "hack" or a corrupt change. pj64 should be capable of supporting such trivial changes, no matter how infinitely many, without complaining about something like a MD5 mismatch.

If anything, the RDB should probably be identifying games less strictly, rather than more strictly, like using the 2-letter cartridge ID code instead of a full CRC. Then doing a CRC comparison to the header to optionally complain if it's not a known good dump later on.

[Fanatic-64]

The current CRC method is not perfect, since there is this entry for example:

[2F700DCD-176CC5C9-C:50]
Good Name=Turok - Dinosaur Hunter (E) (V1.1)/(V1.2)
Internal Name=TUROK_DINOSAUR_HUNTE

Both the (V1.1) and (V1.2) ROMs compute the same CRC, but are different and compute the different MD5s.

And while we're at it, does PJ64 even read the Internal Name= parameter of the RDB?

[project64]

it does not read Internal Name, just writes it .. it was more for when you edited a rom and looked at the file you knew what it belonged to.

[project64]

@Fanatic-64 you do not have to wait for everything to be perfect to start editing the rdb, you can start making small changes and just do pull requests on those changes

[Fanatic-64]

I know, I just wanted to sort out the issues related to settings before starting.

[project64]

what issues in particular?

[Fanatic-64]

Default settings like VI, AI, Fixed/Synced Audio, etc. Also I was planning to undo this commit: dd4a3b3 Since Counter Factor=0 works well for a few games (TWINE and Perfect Dark are 2 I know of).

[project64]

mostly the problem with CF=0 is there no UI for it .. also while it works it really should not be a valid option .. maybe I should allow it, it just fells wrong, but that should not necessarily mean it should not be in.

[project64]

is there any changes you would like to see for defaults?

[Fanatic-64]

Advanced Block Linking OFF would be the only one left.

[oddMLan]

I agree, most games work better without it (in DK64 makes loading areas slower, for example).

[project64]

is there any game where you would have advanced block linking on ?

[LegendOfDragoon]

In my experience, advanced block linking is a good option. I enable it for most games I play, because I see a noticeable increase in performance with it enabled. I certainly would rather keep it on by default and disable it for games with known issues.

Super Smash Bros is better with it on. Just tried Super Mario 64 and also saw an increase in performance with enabling advanced block linking.

[project64]

advanced block linking should always be faster in general, just compiling is slower .. so it is a trade off how much compiling it is doing, so when something is starting like levels/areas there will always be more compiling there.

[Fanatic-64]

Advanced Block Linking causes games to be choppier (framerate is not as steady as with it off), increases loading/transition times, and is prone to freeze/crash at some point in most games (needs extensive testing to ensure it is stable throughout the game).

Most computers should be able to run games at full speed without it (if you can't run Super Mario 64 at 60 VI/s without it then your PC is seriously underpowered). Maybe it could be made a global option so it can be enabled universally if you are desperate for performance. It could also be tested for the most demanding games (GoldenEye, etc) and enabled if it doesn't cause instability. It would be unfeasible to test it for every game since that would require completing the entire N64 catalog and then some, plus for the majority of games the only practical effect would be the above drawbacks.

[cxd4]

I would just default to interpreter for accuracy/fundamental reasons and not re-compiler settings but I understand that's not the intended focus of the RDB and likely nobody would agree with me there. It's to get people to notice and complain about bugs in the recompiler so we can see the limits of how far it can be optimized by recomp. (I think the opposite effect could happen...maybe by defaulting to recompiler most people are already happy with the current amount of speed they're getting, if it defaulted to interpreter for all/unidentified games they would be more encouraged to fix speed issues.)

So I guess if ABL=on breaks most games than ABL=off does, it would be temporarily wisest choice to default to ABL=on to encourage breaking as many games as possible to encourage more development in the RDB optimizations. But in an accurate emulator it should probably defualt to ABL=off (like for demos not tested in goodn64/rdb), but you can't have it both ways I guess.

[LegendOfDragoon]

Oh ok. I never noticed any issues with load times, maybe it's just the particular games I play.

Maybe I'll try profiling sometime and see if there's any way to improve advanced block linking.

[Fanatic-64]

Heck, maybe we need to have 2 RDBs: One for low-end PCs configured for speed (at the cost of smoothness, stability, etc) and another for high-end PCs configured for maxium stability, smoothness, etc.

[project64]

ABL on on a slow computer and off on a fast one ?

what other difference would there be between the two different rdb's ?

[Fanatic-64]

Meh, scratch that idea. But I still vote for ABL as a global setting and disabled by default. Also, we should set a target system specifications for which the settings in the RDB will work at full speed. My current computer where I test has a Intel Core i5-4210U 1.7 GHz (up to 2.7 GHz in single core), Intel HD Graphics 4400, Nvidia GeForce 840M (though only works with DX9+ and OGL) and 8 GB DDR3 RAM.

[project64]

I have no idea what it would be, need others input on that .. I just try to make it as fast and stable as possible .. I do not play enough with it it to know what the system specs should be.

[Predator82Germany]

May the names of No-Intro should taken for new RDB?
Dolphin make a settings-file for each game, usefull here?

[Fanatic-64]

Separate setting files for over 2000 ROMs is a terrible idea.

The name system could be changed through...

[DerekTurtleRoe]

Could updating the RDB and removing the GameFAQ be the same issue? Mostly cosmetic changes?

#108

[project64]

what still needs to be done here, or can this be closed ?

[DerekTurtleRoe]

Update the RDB, I guess. lol

[project64]

what in particular needs updating tho ?

[death-droid]

I reckon this issue can be closed, DSX updated the RDB quite heftly so should be more just fine tuning individual games rather than one huge issue on it all.

[DerekTurtleRoe]

Oh, sorry @project64.

@death-droid I think he wants to move to an online database, and have the offline database in some sort of open format, so it is easier to work with. This is more about what we want to do with the RDB going forward.

My recommendation would be to look at the MESS list of ROMs and see what is missing. You might also check the no-intro and DAT files from various other preservation websites. That shouldn't be hard with something like WinMerge, diff, or something similar. We could also just keep digging on the internet for prototypes, betas, and stuff on places like Unseen64.net, Assemblergames, GBATemp, etc.

[LegendOfDragoon]

@AmbientMalice are there any roms that are still missing, that you're aware of?

[AmbientMalice]

@LegendOfDragoon A few Rareware prototype roms. Conker and Perfect Dark, namely. The problem is that these roms exist in two forms - original and hacked. Only the hacked versions work on emulators.

Also some fan translation stuff for Virtual Pro Wrestling 2 - but I'm not really keen on littering the RDB with WIP fan translations.

[DerekTurtleRoe]

@AmbientMalice Do you happen to know what they modified to make it work? Were they just patching out DRM, or was it patched to run on retail systems because they were developer builds or something?

[AmbientMalice]

@vgturtle127 Rare's dev carts required the use of an authentication dongle. The hacked roms have this check removed. (There may have been headers added, too, but I can't recall.)

PJ64 should probably have a dedicated section in the RDB for notable, non-translation rom hacks, too. There is some big Banjo Kazooie hack I saw Grand Kirkhope RTing stuff from on twitter.

[oddMLan]

The hacked Conker Proto (ECTS) and Debug build are already in the RDB.

[DerekTurtleRoe]

A dongle? Huh, I didn't know that. Cool! Thanks.

Can you check for headers with GoodN64?

[theboy181]

Tool64 I think.

It has ROM info, and the tools is great to have if you have to byte swap your ROM's

http://www.theisozone.com/downloads/other-consoles/n64/tool64v111zip/

[DerekTurtleRoe]

Oh yeah! Made by the guy who created the Super Mario 64 tools, right?

[theboy181]

Not too sure actually. I found it looking for a tool to byte-swap ROM's to patch the ALECK64 games. I needed them to create the RDB entries. @LegendOfDragoon was nice enough to add them for me.

[DerekTurtleRoe]

I found his website. http://qubedstudios.rustedlogic.net/index.html

He has a lot of N64 tools and other applications. Pretty cool stuff.