nwn2dev / nwnx4

NWN2 server extender - Continuation of the work on https://github.com/NWNX/nwnx4

NWNX4

NWNX4 is a launcher for NWN2Server that injects code into the server process, in order to provide additional features and bug fixes.

About

NWNX4 was originally written by Virusman, Grinning Fool, McKillroy, Skywing, Papillon and many contributors. This repository is based on the original codebase, but with modern build tools and new maintainers.

The original source code can be found here: https://github.com/NWNX/nwnx4

Usage

Requirements

You must install:

• Visual C++ 2005 x86 Runtime
• Visual C++ 2015 x86 Runtime
• .NET Framework 4.7.2 or above

Installation

1. Download and extract the NWNX4 zip file in C:\Program Files (x86)\nwnx4 (recommended). This folder is referenced as the "Install dir"
2. Create a new empty folder somewhere in your user profile, like %USERPROFILE%\Documents\nwnx4. This folder is referenced as the "User dir".
3. Copy all files from C:\Program Files (x86)\nwnx4\config.example\ to your User dir (%USERPROFILE%\Documents\nwnx4)
4. Customize the configuration files you just copied to suit your needs:
   • nwnx.ini:
     • plugin_list: the list of all nwnx4 plugins that will be loaded. You may want to add or remove some of them. Additional plugins should be installed in the plugins folder in User dir (you may need to create it).
     • parameters: nwn2server command line arguments. Examples:
       • -module YourModuleName to automatically load a module as a .mod file
       • -moduledir YourModuleName if your module is a directory
   • xp_*.ini: these are the plugins configuration files. Most plugins are shipped with convenient defaults, but you may need to tweak some of them. Note that the presence of a plugin configuration file does not mean the plugin is loaded (see plugin_list in nwnx.ini).

There are 3 methods for launching the server:

• Using NWNX4_GUI.exe (graphical interface)
  • Create a shortcut to NWNX4_GUI.exe, open its properties and edit the "Run in" field to set to your User dir.
  • Then Double click the shortcut to launch the server
• Using NWNX4_Controller.exe (command-line)
  • Run NWNX4_Controller.exe -userdir "%USERPROFILE%\Documents\nwnx4" -interactive in a cmd window. If -userdir is not provided, the current working directory will be used instead.
• Using a Windows service:
  • Run NWNX4_Controller.exe -userdir "%USERPROFILE%\Documents\nwnx4" -installservice in a cmd window to register the service. If -userdir is not provided, the User dir will be the current working directory.
  • Run NWNX4_Controller.exe -startservice to start the service and launch the server

Alternative install: single-folder

Before NWNX4 v1.2.0, the Install dir and User dir were the same folder. This behavior is still supported but discouraged for server setups, as it makes updating NWNX4 harder.

Double-clicking NWNX4_GUI.exe will launch NWNX4 with merged folder.

Updating NWNX4

1. Delete or rename your current NWNX4 Install dir
2. Replace it with the latest NWNX4 zip file
3. Compare the new config files in config.example/ with your existing config files in the NWNX4 User dir. Read the release notes to see exactly what changed.
4. Copy the .nss files from the nwscript/ folder into your module folder, or import nwscript/nwnx.erf into your module using the NWN2 toolset. Overwrite existing files if prompted.

Building from sources

Requirements

• Meson
• Microsoft Visual Studio 2022 Community or MSBuild tools

Building

Initialize your environment

# Initialize git submodules (if you did not clone this repo with `--recurse`)
git submodule init
git submodule update

# Bootstrap vcpkg (prepare dependencies)
vcpkg\bootstrap-vcpkg.bat

# Compile and install dependencies
vcpkg\vcpkg.exe install --triplet=x86-windows-static-md --clean-after-build

Build NWNX4

From a x86 MSBuild prompt (i.e. x86 Native Tools Command Prompt for VS 2022 if you installed Visual Studio 2022):

# Setup build files
meson builddir

# Compile project
cd builddir
meson compile

# Install nwnx4 at a given location
meson install --destdir=%cd%\..\nwnx4-dev

Developing with Visual Studio 2022

Meson can generate solutions for Visual Studio. The following command will create a NWNX4.sln that you can open with Visual Studio:

meson setup --backend=vs2022 vsbuild

How to: Distribute nwnx4 with your module

This section will guide you through the process of creating a folder / zip file containing everything players will need to quickly run a NWN2 server with your custom content, for single-player or multi-player usage.

Skeleton

Download this repository and copy the package-skel directory to any location. This folder contains some handy scripts, and the base structure for your package, that you will need to configure and fill with your content.

package-skel\
├── ClientExtension\  Contains Skywing's Client Extension
├── home\             Replaces Documents\Neverwinter Nights 2.
├── nwnx4\            Contains NWNX4 files and config
├── start-game.bat    Launcher for the game (with the client extension)
└── start-server.bat  Launcher for the server (with nwnx4)

NWNX4

Download the nwnx4 zip file, and extract it into package-skel\nwnx4\.

You must configure NWNX4 as explained in nwnx4's README.md.

In order to run nwnx4 in a portable manner, you must add -home "%NWNX4_DIR%\..\home" to the parameters option in nwnx.ini. Other arguments are recommended but not required:

# Configure the parameters to auto-start your module
# -home <DIR>           The provided path will replace the Documents\Neverwinter Nights 2\ folder
# -moduledir <MODNAME>  Your module name (if you're saving in directory mode)
# -module <MODNAME>     Your module name without the .mod extension (if you're not in directory mode)
# -publicserver         Disable server online advertising
# -maxclients           Maximum number of connected players
# -servervault 1        Makes the server automatically save the character in the servervault folder
parameters = -home "%NWNX4_DIR%\..\home" -moduledir YourModule -publicserver 0 -maxclients 1 -servervault 1

Game launcher

Skywing's Client Extension can automatically detect NWN2 installation directory and provides a better NWN2 player experience. Download it from the NWVault and extract the zip into the package-skel\ClientExtension\ folder.

As is, start-game.bat will:

1. Copy nwn2.ini and nwn2player.ini from Documents\Neverwinter Nights 2\ to package-skel\home\ (so player game settings are kept). If these files are already present in the home folder, they will not be overwritten. You can use this behavior to provide your own custom nwn2player.ini.
2. Launch the game using the Client Extension

Additionnally, start-game-autoconnect.bat will also automatically connect to the server at 127.0.0.1:5121.

Custom content

You must install your custom content (modules, campaigns, haks, tlks, override, ...) inside the package-skel\home\ folder. This custom content will be used by both the server and the game client.

nwnx4's Issues

Improve plugin management

Currently, all plugin files are in the same nwnx4 folder and all available plugins are automatically loaded. This has several drawbacks:

• Updating nwnx4 is messy: users can't simply copy & overwrite plugin files.
  • Overwriting ini files cause configuration issues
    +Copying all DLL files will re-activate some plugins that have been previously disabled (like xp_sqlite)
• NWNX4 folder is becoming crowded with DLLs, PDBs, logs, configs, executables, ...

Proposed solution:

• Move all plugins DLLs and PDBs files to a plugins/ directory
• Configure a list of all plugins that should be loaded in nwnx.ini (this can also be used to detect missing required plugins)
• Log files could be stored in a log/ directory
• UX note: older nwnx4 plugins will still expect config files to be in the nwnx4 folder, and will write logs in this folder

Implement SCORCO for xp_sqlite

Currently only xp_mysql can store and retrieve object from SQL database, but it shouldn't be very hard to implement this for xp_sqlite.

One big advantage for sqlite is that nwnx4 users don't need to configure a MySQL server. Even if sqlite performance isn't great compared with SQL servers, it would reduce the mental charge for new users and also permit the distribution of portable packages with nwnx4 + plugins + a PW or single-player module.

Provide documentation for packaging NWNX4+Module

Write some documentation in the wiki or repo explaining what to do to create a package containing nwnx4 and a module, that players use to quickly start a NWNX4+nwn2server instance that they can connect to.

The proposed config should by default start a local server that is not accessible nor listed publicly. The module author should explicitly make the package suitable for multiplayer if supported (for example by raising the maximum player count).

Distribute demos

Previous releases of nwnx4 included demo modules for testing nwnx4 features, and provided good examples to start writing nwscript code using nwnx4 features.

Auto-detect NWN2 installation directory

It would be nice if nwnx4 could automatically detect the location of the nwn2 installation directory.
I believe @SkywingvL 's client extension uses a registry key for this purpose (I don't know which one), along other eventual fallbacks.

Automatically detecting the install dir would ease the nwnx4 configuration a tiny bit, and permit the distribution of portable packages with nwnx4 + plugins + a PW or single-player module.

Support GNU/Linux with Wine

Wine fails to inject the DLL into the nwn2server process. Some investigations are required :)

Some ideas:

• There may be something to configure on Linux or Wine to allow injecting code into the process
• Maybe detours is totally incompatible with wine and there better alternatives for Linux
• Maybe we could build nwnx4 for Linux and run only nwn2server with wine
• ...

Update dependencies (zlib CVE)

Zlib compression functions are vulnerable to this CVE: https://www.cve.org/CVERecord?id=CVE-2018-25032

zlib isn't a direct dependency of nwnx4, but it's still required somewhere in the dependency graph.

We should update to a later version that include this security patch

YAML configuration

YAML could be a good candidate for replacing current ini configuration.

Benefits are:

• Tree-like configuration
  • More comprehensive structure
  • we could configure nwnx4 and all plugins with one single config file (though it's not very practical for providing plugins documentation)
• Logger could be a configured in a yaml object, that could be anchored/inherited for plugins log configuration
• Can check values types
• Supports multiline text, arrays, maps, ...

Example:

nwn2server:
  location: C:\Program Files (x86)\Atari\Neverwinter Nights 2
  parameters: -module nwnx4demo
  temp: C:\nwn2-temp
  port: 5121
  # Should NWNX disable the general protection fault error dialog ?
  # Note: The watchdogs may fail to restart the server properly if this dialog stays open
  # 0 = do not disable the GPFault dialog
  # 1 = disable the GPFault dialog
  # default: 1
  disable_error_box: true


restart:
  # How long should NWNX wait for the server to shutdown gracefully 
  # before it ends the server process forcefully ?
  # Value = seconds to wait
  # default: 10 seconds
  gracefulShutdownTimeout: 10

  # You can enter a server message that will be sent to all players
  # as NWNX tries to shutdown the server.
  # Value = message
  # default: none
  gracefulShutdownMessage: Server is shutting down NOW!

  # How long should the shutdown server message be displayed ?
  # Value = seconds to wait before actual shutdown
  # default: 5 seconds
  gracefulShutdownMessageWait: 5

  # If you want NWNX to execute a CMD or BAT file before restarting the server after
  # a crash, specify it's name here. If you leave this setting empty, no program
  # will be run. NWNX will wait for the program to finish before it continues.
  # no default
  restart_cmd: nwnx_restart.cmd


watchdogs:
  # Should NWNX restart the server if it locks up ?
  # This can happen if the server is still running, but does not react to gamespy queries any more.
  gamespy:
    enabled: true
    # How often should the gamespy watchdog query the server ?
    # Note: The server will be restarted if it does not react to a certain number of
    # queries in a row. See setting gamespyTolerance below.
    # default: every 30 seconds
    interval: 30
    # How often may the server not react to gamespy queries ?
    # Note: The server will be restarted if it does not react this many queries in a row
    # Example: gamespyInterval = 30 and gamespyTolerance = 4 means that the server will
    # be restarted if it does not react within 2 minutes 
    # default: 4 queries
    tolerance: 4
    # How long should NWNX wait before the gamespy watchdog kick in ?
    # Note: Give the server enough time to load the module, otherwise the watchdog will start to complain
    # Value = seconds to wait
    # default: 30 seconds
    delay: 30
  # Should NWNX restart the server if the server process has gone away ?
  process: 
    enabled: true

logs:
  mode: append
  max_file_size: 50M
  verbosity: error

[WIP] Changes for next CPlugin ABI breaking changes

This issue will list useful changes that will be bundled into the next NWNX4 version that needs to break the CPlugin ABI (and increments nwnxcplugin_abi_version)

NWNXCPlugin_InitInfo argument

Consider if it is necessary / useful.

NWNXCPlugin_New(NWNXCPlugin_InitInfo info);
// =>
NWNXCPlugin_New(const NWNXCPlugin_InitInfo* info);

Reorder NWNXCPlugin_InitInfo content

The current order has been built by appending new attributes at the end of the struct to improve compatibility.

Avoid strcpy for NWNXCPlugin_GetString return value

Currently NWNX4 provides a dest buffer to NWNXCPlugin_GetString so that the plugin can write its returned value into.

However in some cases the plugin already stores a null terminated string, and should be allowed to directly return this string, as it's always strcpy-ed into a NWScript string (need to double check this, just to be sure).

NWNXCPlugin_GetString should return a const char* in order to return a null terminated string. The plugin will be able to chose between returning the buffer provided as parameter, or return a custom string that's already allocated.

Note: Another solution is to give a char** result parameter, and have the plugin *result = "hello world"; or sprintf(*result, "...", ...)

Find a solution for setting the `-home` nwn2server arg to a path relative to nwnx4

Here are some ideas:

• Set an environment variable like NWNX4_DIR before starting the nwn2server. This is the most flexible approach because you would be able to use any other variable in the nwn2server arguments, however the variable substitutions may need to be done manually before starting the nwn2server process, because it is started as a program + arg list instead of a shell command.
• Replace a token in the parameters config option before executing the nwn2server process.

This would permit the distribution of portable packages with nwnx4 + plugins + a PW or single-player module.

Improve logger implementation

The current logger implementation is very basic: it just appends lines to a txt file.

Features to implement:

• Log file rotation (to prevents huge log files)
• Improve logger configuration:
  • Set file open mode: append (a) or overwrite on startup (w)
  • Set log verbosity (already implemented)
  • Configure log rotation (max size, retention duration)

Ideally there should be a "global log config" in nwnx.ini, that can be overridden by plugin-specific config files

Cannot start the server with a long parameters list

Currently, nwnx4 cannot launch correctly the nwn2server process if the parameters list is too long (over 260 characters).

nwnx4/src/controller/controller.cpp

Lines 322 to 325 in bc04e7e

	auto envRes = DoEnvironmentSubstA(params, MAX_PATH);
	if(!HIWORD(envRes)){
	logger->Err("Could not substitute environment variables in NWN2Server command line: %s", params);
	}

This is because the MAX_PATH is only 260.

Related documentation: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd

Client disconnects from server

I have a weird issue. I'm running an old PW with NWNx4. I've run this PW at various times with no issues. However, this time. Occasionally, despite being able to connect to the server. It doesn't always fully connect. I noticed this when creating a characater and instead of creating it on the server, it created a local copy. When using an existing character in this state. I can't start any dialogs.

Restarting the server fixes the issues for a time.

v1.2.0: nwnx4 crashes when restarting nwn2server

Killing the nwn2server process cause nwnx4 to exit/crash when nwnx4 tries to restart the nwn2server.

This is probably related to the installdir / userdir split, which changed how the current working directory is set

Provide plugin implementation examples in different languages

Ideally the examples should not depend on the nwnx4 codebase (i.e. can be built without cloning and building and nwnx4).
RPC client examples in different languages would be helpful too (once xp_rpc is merged).

• C
• C++
• Rust
• Go
• D (@CromFr )
• Other languages?

Those examples should be stored in separate Github repositories (so we don't dramatically increase the number of dependencies for nwnx4)

nwn2server-dll folder is not searched by the .NET runtime

This causes issues with xp_AuroraServerNWscript, that relies on the .NET runtime assembly resolution, and crashes the server during the first script execution if the dll are placed in nwn2server-dll.

Ideally this folder should be useable for any kind of DLL.

References

• How the Runtime Locates Assemblies

  You can also make a dynamic reference to an assembly by providing the calling method with only partial information about the assembly, such as specifying only the assembly name. In this case, only the application directory is searched for the assembly, and no other checking occurs. You make a partial reference using any of the various methods for loading assemblies such as Assembly.Load or AppDomain.Load.

• (Skywing) I'd probably look at whether it's possible to use _AppDomain.AppendPrivatePath https://docs.microsoft.com/en-us/dotnet/api/system._appdomain.appendprivatepath?view=netframework-4.8 from the outside world before loading anything, though that's cumbersome and annoying.