thpatch / thcrap

Touhou Community Reliant Automatic Patcher

Touhou Community Reliant Automatic Patcher

Description

Basically, this is an almost-generic, easily expandable and customizable framework to patch Windows applications in memory, specifically tailored towards the translation of Japanese games.

It is mainly developed to facilitate self-updating, multilingual translation of the Touhou Project games on Touhou Patch Center, but can theoretically be used for just about any other patch for these games, without going through that site.

Main features of the base engine

• Easy DLL injection of the main engine and plug-ins into the target process.

• Full propagation to child processes. This allows the usage of other third-party patches which also use DLL injection, together with thcrap. (Yes, this was developed mainly for vpatch.)

• Uses JSON for all patch configuration data, making the patches themselves open-source by design. By recursively merging JSON objects, this gives us...

• Patch stacking - apply any number of patches at the same time, sorted by a priority list. Supports wildcard-based blacklisting of files in certain patches through the run configuration.

• Automatically adds transparent Unicode filename support via Win32 API wrappers to target processes using the Win32 ANSI functions, without the need for programs like AppLocale.

• Patches can support multiple builds and versions of a single program, identified by a combination of SHA-256 hashes and .EXE file sizes.

• Binary hacks for arbitrary in-memory modifications of the original program (mostly used for custom assembly).

• Breakpoints to call custom DLL functions at any instruction of the original code. These functions can read and modify the current CPU register state.

• Multiple sets of sequentially applied binary hacks and breakpoints, for working around EXE packers and DRM schemes.

• File breakpoints to replace data files in memory with replacements from patches.

• Wildcard-based file format patching hooks called on file replacements - can apply patches to data files according to a (stackable!) JSON description.

• Optional Steam integration for games that are available through Steam, but don't come with Steam integration themselves. Can be disabled by deleting steam_api.dll.

• ... and all that without any significant impact on performance. ☺

Modules included

• win32_utf8: A UTF-8 wrapper library around the Win32 API calls we require. This is a stand-alone project and can (and should) be freely used in other applications, too.
• thcrap: The main patch engine.
• thcrap_loader: A command-line loader to call the injection functions of thcrap on a newly created process.
• thcrap_configure: A GUI wizard for discovering patches, configuring patch stacks, and locating supported games.
• thcrap_tsa: A thcrap plug-in containing patch hooks for games using the STG engine by Team Shanghai Alice.
• thcrap_tasofro: A thcrap plug-in containing patch hooks for various games by Twilight Frontier.
• thcrap_update: Contains updating functionality for patches, digitally signed automatic updates of thcrap itself, as well as an updater GUI. thcrap_update.dll can be safely deleted to disable all online functionality.
• thcrap_bgmmod: A helper library to handle the non-game-specific parts of BGM modding for originally uncompressed PCM music, like codec support and loop point handling. Currently statically linked into thcrap_tsa as this module is currently the only one with support for BGM modding, but already split into a separate library to be ready for covering more engines in the future.

Building

A ready-made Visual Studio build configuration, covering all modules and their dependencies, is provided as part of this repository. To set up the build:

• Install Visual Studio Community 2019.

  • Visual Studio 2019 or newer is required to build thcrap_configure_v3. For all other components, Visual Studio Community 2017 will also work
  • You can also use Visual Studio Community 2022. During installation, make sure to install the v141 and v141_xp toolsets (v141_xp is the Visual Studio 2017 compiler with Windows XP compatibility, and v141 is required by v141_xp).

• Make sure that you've pulled all Git submodules together with this repo:

    git clone --recursive https://github.com/thpatch/thcrap.git
  

• (Optional) If your thcrap build should be able to automatically update itself, you need to create a code signing certificate. To do this, run the following commands on the Visual Studio command prompt (vcvarsall.bat) in the root directory of this repo (the one with thcrap.sln):

    makecert -n "CN=Your Name,[email protected]" -$ individual -a sha256 -len 4096 -r -cy authority -sky signature -pe -sv cert.pvk cert.cer
    pvk2pfx -pvk cert.pvk -spc cert.cer -pfx cert.pfx
  

  cert.pfx is used to sign the binaries as part of the build, so don't change the file name.

Then, open thcrap.sln, choose Debug or Release from the drop-down menu in the toolbar (or the Configuration Manager) and run Build → Build Solution from the main menu.

You can also build from the command line by running the Visual Studio tool environment batch file (vcvarsall.bat), then run

	msbuild /m /p:Configuration=Debug

or

	msbuild /m /p:Configuration=Release

in the thcrap directory. The binaries will end up in the bin/ subdirectory.

Signing a release archive for automatic updates

First, convert cert.pvk to a .pem file using OpenSSL, then use this file together with scripts/release_sign.py:

openssl rsa -inform pvk -in cert.pvk -outform pem -out cert.pem
python release_sign.py -k cert.pem thcrap.zip

Using different compilers

Visual Studio Community 2017 is recommended for building, and the build configuration references the Visual Studio 2017 platform toolset with Windows XP targeting support by default. However, the project should generally build under every version since Visual C++ 2010 Express after changing the <PlatformToolset> value in Base.props. For a list of all platform toolsets available on your system, open the Properties dialog for any included project and refer to the drop-down menu at Configuration Properties → General → Platform Toolset.

Compilation with MinGW is currently not supported. This is not likely to change in the foreseeable future as we don't see much value in it.

Dependencies

All required third-party libraries for the C/C++ code are included as Git submodules in the thcrap_external_dependencies submodule, which also contains pre-built DLLs, .lib files, export definitions and PDBs. These are:

• Jansson, required for every module apart from win32_utf8.

• libpng (>= 1.6.0), required by thcrap_tsa for image patching.

• zlib, required by thcrap_update for CRC32 verification. It's required by libpng anyway, though.

• thcrap_bgmmod currently supports the following codecs via third-party libraries:

  • FLAC via dr_flac
  • Ogg Vorbis via libogg, libvorbis and libvorbisfile
  • MP3 via libmpg123

The scripts in the scripts directory are written in Python 3. Some of them require further third-party libraries not included in this repository:

• PyCryptodome is required by release_sign.py.
• pathspec is required by repo_update.py. Can be easily installed via pip.

License

The Touhou Community Reliant Patcher and all accompanying modules are released to the Public Domain, unless stated otherwise. This means you can do whatever you want with this code without so much as crediting us.

That said, we do appreciate attribution. ☺

Backers

Love our work? Become a backer of Touhou Patch Center on Open Collective, and help us decide the priorities of thcrap's future development.

Thank you to all our backers!

The team

thcrap's Issues

Fast bootstrapping of patches

This would only download the most necessary data of each patch (game version data in particular) on initial patch selection, with the rest being only downloaded when needed (e.g. once a particular game has been located on a user's system).

In particular, this means:

1. Only pull versions.js for every patch after stack configuration
2. Use that to start searching for games
3. Download any global patch data during the search process
4. Create shortcuts
5. Download/update the rest of each game's data individually after the user started a game through a shortcut, using a blocking downloader.

Installer

We need one. I choose NSIS.

thcrap: Add a "reload" button to the JSON error message box

JSON reload option

Description

Prerequisites

None.

Implementation

thcrap_tsa: Support spell card translation

Spell cards

Description

Basically, "everything translatable in an ECL file" - also includes the spell card comments in th08, but not in th095 or th125.

Prerequisites

• Issue #1 (multi-slot breakpoints)

Implementation

ID-based lookup table, accessed via breakpoints. No reason to start parsing and patching ECLs here, and also nicely works around can include a different fix for the "in the result screen, spell cards appear in the language they were last encountered in" issue.

Translated names will be stored in game_dir/spells.js, comments in game_dir/spellcomments.js.

thcrap: Support relative paths in the run configuration

Relative paths

Description

Prerequisites

None.

Implementation

Explore the possibilities of code signing for improved security / integrity verification

Code signing

Description

Marking as both a bug and an enhancement because it may become really scary one day.

First of all, I must admit that I am a total newbie to this concept, and I'd like to thoroughly understand anything (and, most importantly, why it works or doesn't work) before implementing it.

This is disquieting not mainly because the patcher executables may be tampered with, but because every patch can contain self-updating binary hacks.

Everyone with write access to the JSON files of a patch can have code execution in the context of the game process. While this openness is certainly one of the main aspects about this project, it can just as easily be abused.

The main problem here is that we want to keep a low barrier of entry for this whole patching thing on the technical side too (even though this is hardly used), yet also want to ensure security somehow.

Once we have a working system in place, we can also have patches that come with self-updating plug-in DLLs.

Prerequisites

• Probably requires Flagged Revisions on the wiki side, combined with a way to automatically sign a patch file upon flagging, to really work.
• If automatic signing based on wiki logins even is a good idea in the first place.

Implementation

• Provide two separate versions of thcrap on thpatch.net: A regular version with all the code signing enabled, and a developer version that doesn't have these features. The latter will have a big "this version may pose a security risk, only use it if you know what you're doing warning.

thcrap_configure: Link `thcrap_update` dynamically

Dynamic linking of thcrap_update in the configuration tool

Description

We want users to opt out of any automatic updating functionality of both the patches and thcrap itself by simply deleting thcrap_update.dll. This is no problem as far as the base engine is concerned, but the configuration tool has a static link to this DLL.

Prerequisites

None.

Implementation

i18n of thcrap itself

i18n

Description

Isn't it ironic that we can translate games, yet can't translate ourselves?

Prerequisites

• A better way of templating and outputting strings than good old printf. I'm open for any kind of suggestion here - even changing the project's programming language, if you can convince me.

Implementation

thcrap: Implement hot-repatching

Hot-repatching

Description

With this feature, newly downloaded binary hacks, breakpoints and other patch data loaded on initialization could immediately be re-applied after a patch update was downloaded, without having to restart the game.

Prerequisites

• Breakpoint removal
• File change notifications also need to work with ZIP files, preferably without having to care whether patch files comes from a ZIP or a real directory structure.

Implementation

• Each plug-in (and the core) can export a thcrap_plugin_repatch() function.
• thcrap_repatch() calls each DLL's thcrap_plugin_repatch().
• Using the file change notification functions of the Win32 API, this function is also called whenever the user changes a file locally.

Credits System

Display the contributors of patches in a more accessible way.

Regex text replacement

This would allow text replacements on a finer granularity than the current translation units, and could e.g. be used to simplify the implementation of patches like the British English translation.

This issue was closed on Trello with reference to the dictionary system in #38. However, that system would only operate on complete strings pulled from the game, which match the current translation units. Sure, the regex features will be made available to translators through a dictionary system (perhaps even in the same file), but the regex part still has to be implemented separately from that.

Feel free to close/wontfix this issue now if this is still something you don't want to have, just wanted to clarify that.

Support dynamic base addresses

Dynamic base addresses

Description

Right now, thcrap_loader, the binary hack system and the breakpoint system all assume a game's main module to be loaded at a constant place in memory. Thus, a game loaded to a dynamic base address essentially can't be patched at all.

This feature is required for supporting Hopeless Masquerade and the PC-98 games at all.

Prerequisites

None.

Implementation

• For binary hack and breakpoint addresses, add rva as an alternative key for addresses relative to the module's base address.

thcrap: Support multi-slot breakpoints

Multi-slot breakpoints

Description

This will allow one breakpoint function to be triggered at multiple positions, without having to register an arbitrary number of breakpoints for the same function.

Prerequisites

None.

Implementation

• Deprecate manual breakpoint registration via breakpoint_register. Breakpoint functions now must be exported and have a name of the form BP_<name of breakpoint>. Hooray for enforced coding conventions.
• In the breakpoint name, make # work as a sentinel character, splitting the breakpoint name and the slot name

Len'en Project support

Support ZIP archives for patches

ZIP archive support

Description

This will be necessary for PC-98 support, if only to work around DOS 8.3 filenames.

Prerequisites

None.

Implementation

Fantasy Maiden Wars Support

Has been requested multiple times in multiple places. Will have to look into this once we have the time.

thcrap_tsa: Support translation of hardcoded strings

Hardcoded strings

Description

"Hardcoded strings" here means "immutable strings stored at a constant address in memory". This allows us to do a lookup solely based on the address.

Prerequisites

None. Also needs to work with #3, though.

Implementation

This consists of four parts:

1. Basic lookup function based on a string's address
2. Hooks for TextOut, MessageBox and possibly other Win32 API functions calling the lookup function
3. Breakpoint to call the lookup function (unnecessary; hardcoded strings are either fully constant (which means our hooks take care of them anyway) or format strings (see below).)
4. Custom sprintf handler

No. 4 is necessary because some of these hardcoded strings are sprintf format strings, and we shouldn't trust on the game's own buffers being large enough.

thcrap: Support right-to-left scripts

Right-to-left text

Description

It was only a matter of time until people asked for this.

Prerequisites

• #17 (text layout engine)
• #16 (Win32 dialog patching)

Implementation

• If possible, this should not require a special "rtl": true flag in the run configuration. Directionality should be automatically determined by the characters in a string. Otherwise, patch stacks that have a RTL language on top of an LTR one couldn't easily switch between both.
• Neither should it require all texts to simply be wrapped in a <r$text$> command. Once tabs come into play, we need to mirror those too.
• Add a breakpoint to read the current width of the rendering bitmap.
• According to this page, Win32 dialogs can be easily mirrored. Find the best way to do this.
• ... and then there's texture positioning. We probably could start by searching for a general "set X position" point and mirroring that value. Anything more will be part of a separate issue, though.

Feature Request: Change loader to be compatible with other patches

Currently we cannot use vsyncpatch(vpatch) or other existing patches with thcrap because they also require DLL injection.
Possible implementations may be CreateProcess hook or other similar things.

thcrap_tsa: Support ruby layout

Ruby layout

Description

By re-implementing ruby text display using the text layout engine from #17, we could get rid of the guessing code in the wiki extension and the need for a binary hack for each game to disable ZUN's 2-byte fixed-width ruby loop.

Prerequisites

#17, obviously.

Implementation

thcrap_update: Actually verify the CRC32 of downloaded files

CRC32 verification

Description

This seems to be the best way to gracefully handle broken or incomplete mirrors, and could also be the foundation of a security system (#10).

Prerequisites

None.

Implementation

Playism version of Double Dealing Character crashes immediately

For reference, I'm running 64-bit Windows 8.1. I have DDC installed at D:\Touhou\Touhou 14 - Double Dealing Character, and thcrap is at D:\Touhou\thcrap\ .

I bought and downloaded the full version of DDC from Playism, and I downloaded the latest version of thcrap from thpatch.net and ran it. I selected the English language patch and pointed it to the right directory, and it generated shortcuts that seem correct. I can run the shortcut to custom.exe without any problems, but when I try to run th14, it crashes immediately. I can run th14 by itself without any problems.

Translate strings via Japanese→$language dictionaries?

With the reorganization of the project and the progress we've made in the last year, I think it's worth raising this question again. The new project administration will need to take a stance on this before covering the next game, and it'll be important to know the background behind my decision against such a system.

It has always been my design goal for thcrap's translation functionality not to rely on any original Japanese strings. This decision stems from two observations:

• There are static translation patches for earlier games into a variety of languages.
• With the older games essentially being abandonware [citation needed] and, as of this writing, only one game being distributed as an official download release, piracy remains the only distribution channel that really matters. And it has been shown time and time again that most site owners rather provide complete, easy-to-use packages aimed to speakers of their own language than clean, original copies - which is further justified by some inadvertent "region locking" that e.g. causes 東方紅魔郷.exe and every game's custom.exe to simply not work on non-Japanese locales.

Combine the two and you have rampant piracy of unofficially translated games. We can't rely on pirated copies coming with the original text anymore. That's just how it is.

This means, however, that we have needed all sorts of alternative indexing systems to correctly assign translations:

• Dialogue and endings use a rather involved time code system that additionally gives translators the freedom to edit dialogue on a text box level and allows them to use 2 lines where the original only uses one (and vice versa). This greatly goes against the line-centric design of ZUN's original formats, and is one source of all the complexity in the dialogue patcher (with the other one being the hard line system used in th06, th07 and parts of th08).

• Spell cards use their (zero-based) index number in the game's Result screen, which are pulled out from the game's memory using a bunch of breakpoints. This works fine for games that have a result screen. All games that don't (TH095, Uwabami Breakers, TH125, TH143) coincidentally also happen to have no reliable index numbers, and therefore require even more build-specific binary hacks and breakpoints to somehow derive them from the game progression - or, in the case of Uwabami Breakers, a full copy of all .ECL danmaku scripts with corrected IDs. :(

• Hardcoded string translation assigns IDs to virtual memory addresses (stringlocs.js), then looks up translations for these IDs in a separate table (stringdefs.js). This is reliable, needs zero game- and build-specific hacks, and I have a script to locate the addresses, but they still need to be committed for every build of every game.

• Music Room translation uses a combination of hardcoded string translation (for the "No. X ??????" strings displayed for locked tracks), a separate file containing song title translations (themes.js) and a separate file for comments (musiccmt.js).

  The themes.js system was designed before thcrap to serve as a song title source for (hypothetical) third-party applications to cope with frequent translation changes (which in turn was the main motivation for thcrap in the first place).

  musiccmt.js basically uses the same format as the generic plaintext translation support that would later be developed for th143, but with a special syntax that replaces a single @ character in a line with a customizable format string, printing the title of the currently selected track.

  Not to mention that pulling the theme number out of the game also requires its fair share of build-specific breakpoints.

• Dialog resource translation makes use of the fact that the widgets (and thus, their strings) internally appear in a set order, builds a JSON array of hardcoded string IDs (dialog_*.js) in this order, then pulls the translations out of stringdefs.js. Other than that, no build-specific hacks necessary.

Using one single dictionary-based solution instead of these systems would greatly reduce the amount of effort required to support new games at the expense of both compatibility to static patches and more bloat in the translation files.

thcrap: Improve font replacement

Improved font replacements

Description

Currently, we merely have a single replacement font which overrides any font face names in CreateFont. This usually is enough for Touhou.

However, selecting a replacement font based on various parameters of the original font (face name, size, width, etc.) would allow for more detailed configuration. Text originally rendered in a smaller font could be made larger without affecting regular text, and fonts which previously looked too small would become usable.

An example of how this may look like:

{
    "fonts": {
        "'ＭＳ ゴシック'": "'UVN Chim Bien'",
        "'ＭＳ ゴシック' 10" : "'UVN Chim Bien' 16"
    }
}

Prerequisites

None.

Implementation

64-bit support

Requires OllyDbg64 and #39 with MinGW support.

Replace WinInet with Libcurl

This would fix and improve a couple of things.

thcrap_tsa: Add a text layout engine

Text layout engine

Description

This is at least necessary for horizontal alignment of text in the endings and the Music Room. th11 would also benefit for the assist character dialogue.

Prerequisites

None.

Implementation

Reverse-engineer the layout format used by the th13 English patch and add MediaWiki's syntax for bold and italic text - maybe even for Ruby.

thcrap_tsa: Support translation of the th06 ending format.

th06 ending format

Description

Prerequisites

None.

Implementation

Correct decimal and digit group separators

US and Japanese do this: 9,999.99
Some European countries do this: 9.999,99
We need to support both ways.

thcrap_tsa: Support music room translation

Music Room

Description

Prerequisites

• #17 (text layout engine)

Implementation

Write a GUI for configuration

GUI

Description

This requires careful planning: an interface that allows people to do everything the engine is capable of would result in something even harder to use without careful design.

The GUI needs to support:

• Multiple originating servers
• Blacklisting of files per patch
• Font settings
• A simple-to-use, verbose and clear wizard
• A way to suggest base_tsa, yet also allow for configurations that don't use it Dependencies
• Enable and disable self-updating
• more... ?

Prerequisites

• Settle on a GUI framework (decided on wxWidgets)

Implementation

Patches are visualized as horizontal blocks and colored according to their downloading status. These blocks contain little icons symbolizing the various types of data included in a patch:

• Version information
• Binary hacks and breakpoints (may share one icon, maybe?)
• Fonts
• Text translations
• Images

Maybe we could also indicate progress in some way?

PC-98 game support

Listing this as wontfix. Time spent trying to do this could be better spent elsewhere. If ReC98 is not released and has had no development for 2 straight years we will reconsider supporting the PC-98.

thcrap_tsa: Translate the game window title

Window title translation

Description

Sounds trivial at first - just use string table lookup and you're done, right? However, if you want to...

• minimize redundancy between game versions (we don't want to change the full string for every version)
• minimize redundancy between languages (most will probably choose to romanize the Kanji part while keeping the English part in English)
• integrate that into the wiki while still only using the title parameter in the run configuration
• and also print some additional information (e.g. the thcrap build and the currently active patch stack)

..., this becomes quite involved.

Prerequisites

• #18 (Patch dependencies)

Implementation

thcrap_configure: Support patch dependencies

Patch dependencies

Description

More connection between the individual patches.

For example, translation patches into Arabic or Hebrew could depend on a shared patch adding support for right-to-left text. In this case, the user would not have to bother downloading both base_tsa and, let's say, script_rtl, but these would be immediately added and stacked below the translation patch, if they aren't already there.

Also, we could split off the font settings and font files currently bundled in base_tsa into an own patch (script_latin).

Prerequisites

We'll probably wait for #8 (GUI configuration tool) before attempting something like this.

Implementation

thcrap_tsa: Support ending translation

Endings

Description

For time reasons, we will probably just focus on the custom MSG opcodes for endings used from th10 on.

Prerequisites

• #17 (text layout engine)

Implementation

thcrap_update: Self-updater should be able to delete obsolete files

This is for thcrap itself, not the patch updater.
The updater should be able to delete obsolete files.

Replay filename scheme

thcrap_tsa: Support translation of the th08 spell card owner

th08 spell owner

Description

Just so I don't forget, since #15 is done now. :)

Prerequisites

None.

Implementation

Support advanced image patching

Advanced image patching

Description

The basic image patching implemented in #12 doesn't have dedicated support for patch stacking. This feature goes a bit further and adds support for overlays and sprite-level replacements to save both redundancy and needless bloat.

Prerequisites

• #12 (basic image translation)

Implementation

See the development notes on Touhou Patch Center.

Support basic image translation

Images

Description

Prerequisites

Implementation

thcrap_configure: Add shortcuts to Steam library

Description

Quite a number of people have commented that integrating the thcrap shortcuts into Steam is a bit unintuitive. This point has become particularly relevant now that ZUN is investigating official download releases. In the future, Touhou games might very well be distributed through Steam, making a proper integration all the more desirable.

Prerequisites

None.

Implementation

You would think that Steam offers an API for managing the shortcuts to non-Steam applications. But no, you have to parse the Windows registry and manually write binary files to edit the shortcut list. The Ice project seems to be a "reference implementation" of how to achieve something like this - we'd only have to adapt its code.

Translation coverage/progress bars in GUI

When we get a GUI and we have thpatch 2.0 non-mediawiki edition, display the progress bars based on the information served from thpatch.net

thcrap_tsa: Support translation of the resolution dialog

Resolution dialog

Description

A fairly low priority, not even included in the C85 milestone.

Prerequisites

Implementation

Patch-level plug-ins

Allow patches to contain DLLs which are loaded into the game process. This will drastically enhance what patches can do: For example, the British English patch could be generated automatically by adding a DLL that applies a set of text replacement rules to every line of text delivered from the engine.
#10 is a prerequisite for this. We should not add a feature that executes arbitrary binaries downloaded from arbitrary servers without a way to prove their authenticity.

Restructure the repository

• Don't bundle all the games in one patch. Split them up per game, per translation/function.
• Don't download each file separately. Bundle them up in a zip.
• Namespacing.
• While we are at it, change .js and .jsdiff extensions to .json.
  • For jsdiff, either document the existing markup better or change it to something like markdown.
• There is probably something I forgot about, I'll add it later if so.

URI Scheme

Instead of directing your users to the thcrap download site, you would simply have a custom URL like

thcrap:bootstrap:https://raw.githubusercontent.com/Gamer251/thcrap-gamer251/master/mima/

and clicking that would automatically download a patch and create shortcuts.

This has the advantage of true decentralization: We would no longer need the approval of another node in the network that lists a certain repository. Instead, people can download the patch directly from your site.

The basic code to register such an URI scheme is very simple. You just have to add a bunch of registry keys and make thcrap respond to those parameters. This can be done automatically each time you run thcrap.

This might seem to forgo patch stack configuration entirely in favor of usability, but I can think of two ways to fix that:

One way would simply run thcrap_configure normally, but ensure the given repository to be present in the patch list:

thcrap_configure:https://raw.githubusercontent.com/Gamer251/thcrap-gamer251/

In fact, since all required functionality for this is already present in thcrap_configure, I could add this scheme immediately.

Another way would be to provide multiple pre-configured stacks on your site, which all would be able to be set up with one click:

thcrap:bootstrap:thpatch/lang_en https://raw.githubusercontent.com/Gamer251/thcrap-gamer251/master/mima/

This would automatically add the English patch below the Mima patch.

As you see, you can either give the complete URL to a patch, or its repo/patch ID.

With the repo/patch ID combination, thcrap would of course need a way to actually locate the patch download. For "well-known" repositories like thpatch or nmlgc, this is no problem - these are always included in the distribution. Otherwise, thcrap will do a recursive neighbor search across all the servers it knows - and, what makes this really powerful, also all the servers directly specified as URLs in the link.

This means that repositories simply need to list URLs of all the servers they need as its neighbors (which is best practice anyway), and you're guaranteed to get the right patches.

Build with Cmake

Step 1 for supporting alternative compilers.

Resolve (and bootstrap) patch dependencies on run time

Currently, thcrap_configure resolves patch dependencies and saves a complete stack in the generated run configuration file. By instead doing this every time a user starts a game, we'd only have to store what the user actually selected. This also allows patch maintainers to freely split or fork one patch into multiple ones, without requiring every user to reconfigure.

Prerequisites

Needs a blocking updater that runs in a separate process (#37). We can't start running the game before we know the exact patches the user needs to use.

Separate downloader process

I've been wanting to do this for a long time, and half of fast bootstrapping would require it as well. This is how it could work:

• When thcrap_loader.exe is started, it checks whether thcrap_downloader.exe exists and launches it if it does. Otherwise, it goes straight to launching the patched game, as it does currently. This will continue to allow people to opt out of updates.
• thcrap_downloader then determines whether there are any updates and how much to download in a blocking manner before starting the game itself.
• Once the game is started, the downloader process remains open in the background to fetch new updates while the game is runnning.
• It also remains open after the game was closed, in order to inform users about the updates that happened in the background.

The updater needs to block in the following cases:

• Technical patch changes that can't easily be repatched while the game is running, such as binary hacks and breakpoints.
• Any changes to the dependencies of selected patches, as this might potentially require user interaction (see below).
• The initial bootstrapping process of a patch. (It may be better, though, to instead define some sort of threshold value above which a patch update becomes generally blocking. This would also help to accommodate scenarios where users quit the initial blocking download after a few files, then later restart and wonder/complain why next to nothing is patched because the downloader is still running in the background...)

Reasons for putting the downloader into a separate process, rather than just keeping it in a thread within the game process:

• Stability. In rare cases, the download thread seems to cause a crash before the title screen, and it also tends to crash if the user immediately quits on the resolution dialog while an update is in progress.
• Antivirus software would be less likely to target thcrap. I looked into the situation with Norton, and it only killed the game process on the HTTP GET requests made by the uploader.
• Users would not have to add firewall exceptions for every single game executable, but only once for the downloader.

Reasons why this absolutely must have a GUI:

• Progress bars are obviously prettier.

• The Windows console subsystem still can't render Unicode properly – or at least not without requiring extensive configuration together with correct fonts, which I personally have never got working.

• We could add a nice display of the differences between the newly downloaded content and the existing files, so that users can easily see what changed in the current update. (I can already imagine that people would then be calling for some sort of spoiler protection, though?)

• We could add a "Start game now" button that is enabled once the downloader has finished all blocking parts of an update, as an alternative to just immediately running the game in that case.

• User interaction will be necessary to manage digital signatures on patches, once we have them.

• We could also give users a choice between available alternative patches that offer the same features. This would implement our new focus on namespacing, and is especially important for changes to the underlying dependencies after configuration time. (Really, it's about time that our design acknowledged the fact that maintainers of anything Touhou are most likely not going to stay around forever.)

  As you know, dependencies are stated using the syntax

  [repository/]id
  

  It is already possible to omit the repository part. Currently, this results in the patch being randomly picked from a repository that offers a patch with the same ID. With user interaction, we could instead ask the user which patch to choose.

  This question should come with a detailed overview of both patches, (rendering each patch's readme.md next to each other would do). This allows users to make a somewhat informed decision between two patches implementing the same functionality, even if they haven't followed the development of the patches in question (which they most likely haven't).

Things to consider

• Looking at Steam, it seems that gamers have largely accepted updates delaying game startups from time to time. If that's the case, we could just block on every update to simplify the design and the implementation.
• How much overlap will there be with the future configuration tool? If the answer is "a lot", why don't merge them outright?

thcrap: Multiple breakpoints on the same address?

Right now, this is undefined behavior, but who knows, maybe this might come in handy some day.