openttd / bananas-api

OpenTTD's content service API (BaNaNaS API)

OpenTTD

Table of contents

• 1.0) About
  • 1.1) Downloading OpenTTD
  • 1.2) OpenTTD gameplay manual
  • 1.3) Supported platforms
  • 1.4) Installing and running OpenTTD
  • 1.5) Add-on content / mods
  • 1.6) OpenTTD directories
  • 1.7) Compiling OpenTTD
• 2.0) Contact and community
  • 2.1) Multiplayer games
  • 2.2) Contributing to OpenTTD
  • 2.3) Reporting bugs
  • 2.4) Translating
• 3.0) Licensing
• 4.0) Credits

1.0) About

OpenTTD is a transport simulation game based upon the popular game Transport Tycoon Deluxe, written by Chris Sawyer. It attempts to mimic the original game as closely as possible while extending it with new features.

OpenTTD is licensed under the GNU General Public License version 2.0, but includes some 3rd party software under different licenses. See the section "Licensing" below for details.

1.1) Downloading OpenTTD

OpenTTD can be downloaded from the official OpenTTD website.

Both 'stable' and 'nightly' versions are available for download:

• most people should choose the 'stable' version, as this has been more extensively tested
• the 'nightly' version includes the latest changes and features, but may sometimes be less reliable

OpenTTD is also available for free on Steam, GOG.com, and the Microsoft Store. On some platforms OpenTTD will be available via your OS package manager or a similar service.

1.2) OpenTTD gameplay manual

OpenTTD has a community-maintained wiki, including a gameplay manual and tips.

1.3) Supported platforms

OpenTTD has been ported to several platforms and operating systems.

The currently supported platforms are:

• Linux (SDL (OpenGL and non-OpenGL))
• macOS (universal) (Cocoa)
• Windows (Win32 GDI / OpenGL)

Other platforms may also work (in particular various BSD systems), but we don't actively test or maintain these.

1.3.1) Legacy support

Platforms, languages and compilers change. We'll keep support going on old platforms as long as someone is interested in supporting them, except where it means the project can't move forward to keep up with language and compiler features.

We guarantee that every revision of OpenTTD will be able to load savegames from every older revision (excepting where the savegame is corrupt). Please report a bug if you find a save that doesn't load.

1.4) Installing and running OpenTTD

OpenTTD is usually straightforward to install, but for more help the wiki includes an installation guide.

OpenTTD needs some additional graphics and sound files to run.

For some platforms these will be downloaded during the installation process if required.

For some platforms, you will need to refer to the installation guide.

1.4.1) Free graphics and sound files

The free data files, split into OpenGFX for graphics, OpenSFX for sounds and OpenMSX for music can be found at:

• OpenGFX
• OpenSFX
• OpenMSX

Please follow the readme of these packages about the installation procedure. The Windows installer can optionally download and install these packages.

1.4.2) Original Transport Tycoon Deluxe graphics and sound files

If you want to play with the original Transport Tycoon Deluxe data files you have to copy the data files from the CD-ROM into the baseset/ directory. It does not matter whether you copy them from the DOS or Windows version of Transport Tycoon Deluxe. The Windows install can optionally copy these files.

You need to copy the following files:

• sample.cat
• trg1r.grf or TRG1.GRF
• trgcr.grf or TRGC.GRF
• trghr.grf or TRGH.GRF
• trgir.grf or TRGI.GRF
• trgtr.grf or TRGT.GRF

1.4.3) Original Transport Tycoon Deluxe music

If you want the Transport Tycoon Deluxe music, copy the appropriate files from the original game into the baseset folder.

• TTD for Windows: All files in the gm/ folder (gm_tt00.gm up to gm_tt21.gm)
• TTD for DOS: The GM.CAT file
• Transport Tycoon Original: The GM.CAT file, but rename it to GM-TTO.CAT

1.5) Add-on content / mods

OpenTTD features multiple types of add-on content, which modify gameplay in different ways.

Most types of add-on content can be downloaded within OpenTTD via the 'Check Online Content' button in the main menu.

Add-on content can also be installed manually, but that's more complicated; the OpenTTD wiki may offer help with that, or the OpenTTD directory structure guide.

1.5.1) Social Integration

OpenTTD has the ability to load plugins to integrate with Social Platforms like Steam, Discord, etc.

To enable such integration, the plugin for the specific platform has to be downloaded and stored in the social_integration folder.

See OpenTTD's website, under Downloads, for what plugins are available.

1.6) OpenTTD directories

OpenTTD uses its own directory structure to store game data, add-on content etc.

For more information, see the directory structure guide.

1.7) Compiling OpenTTD

If you want to compile OpenTTD from source, instructions can be found in COMPILING.md.

2.0) Contact and Community

'Official' channels

• OpenTTD website
• OpenTTD official Discord
• IRC chat using #openttd on irc.oftc.net more info about our irc channel
• OpenTTD on Github for code repositories and for reporting issues
• forum.openttd.org - the primary community forum site for discussing OpenTTD and related games
• OpenTTD wiki community-maintained wiki, including topics like gameplay guide, detailed explanation of some game mechanics, how to use add-on content (mods) and much more

'Unofficial' channels

• the OpenTTD wiki has a page listing OpenTTD communities including some in languages other than English

2.1) Multiplayer games

You can play OpenTTD with others, either cooperatively or competitively.

See the multiplayer documentation for more details.

2.2) Contributing to OpenTTD

We welcome contributors to OpenTTD. More information for contributors can be found in CONTRIBUTING.md

2.3) Reporting bugs

Good bug reports are very helpful. We have a guide to reporting bugs to help with this.

Desyncs in multiplayer are complex to debug and report (some software development skils are required). Instructions can be found in debugging and reporting desyncs.

2.4) Translating

OpenTTD is translated into many languages. Translations are added and updated via the online translation tool.

3.0) Licensing

OpenTTD is licensed under the GNU General Public License version 2.0. For the complete license text, see the file 'COPYING.md'. This license applies to all files in this distribution, except as noted below.

The squirrel implementation in src/3rdparty/squirrel is licensed under the Zlib license. See src/3rdparty/squirrel/COPYRIGHT for the complete license text.

The md5 implementation in src/3rdparty/md5 is licensed under the Zlib license. See the comments in the source files in src/3rdparty/md5 for the complete license text.

The fmt implementation in src/3rdparty/fmt is licensed under the MIT license. See src/3rdparty/fmt/LICENSE.rst for the complete license text.

The nlohmann json implementation in src/3rdparty/nlohmann is licensed under the MIT license. See src/3rdparty/nlohmann/LICENSE.MIT for the complete license text.

The OpenGL API in src/3rdparty/opengl is licensed under the MIT license. See src/3rdparty/opengl/khrplatform.h for the complete license text.

The catch2 implementation in src/3rdparty/catch2 is licensed under the Boost Software License, Version 1.0. See src/3rdparty/catch2/LICENSE.txt for the complete license text.

The icu scriptrun implementation in src/3rdparty/icu is licensed under the Unicode license. See src/3rdparty/icu/LICENSE for the complete license text.

The monocypher implementation in src/3rdparty/monocypher is licensed under the 2-clause BSD and CC-0 license. See src/3rdparty/monocypher/LICENSE.md for the complete license text.

The OpenTTD Social Integration API in src/3rdparty/openttd_social_integration_api is licensed under the MIT license. See src/3rdparty/openttd_social_integration_api/LICENSE for the complete license text.

4.0 Credits

See CREDITS.md

bananas-api's Issues

Allow uploading .zip/.tar instead of individual files

For Game Scripts and AIs this is kinda the only way to upload anything with a folder via the webinterface. The CLI does support uploading files via folders already, but this would just be a nice addition.

Add support for adding public keys to packages

Idea to enable bots/GH actions to upload packages on releases.

• Owner generates a pair of public/private keys.
• Public key is stored in authors.yaml.
• Private key is kept by uploading bots as secret.
• bananas-cli and api perform a handshake for authentication.

Support teams

For example, via GitHub Teams. Just any form that multiple people can create a single content, without them sharing a password.

Different packages with same name causes OpenTTD client to mess up

It is now possible to rename your package, and as such, people are renaming multiple packages of theirs to the same name.

I have no real problems with this, as the system considers the name a piece of metadata, but it turns out the OpenTTD client has a different vision on this.

When downloading a piece of content with the same name, it appears to be overwriting the existing file. This of course can be considered a bug on its own, but there is also not really a resolution for the client.

I want to suggest to rename the packages to <uniqueid>-<name>-<version>. This is still human-readable, and always unique.

Increase upload-token expiration time

Some content is approaching a size of 1 GB.
With some connections a upload timeout of 15 minutes is rather harsh.

It's also likely that people would start their GB upload, and go for a walk.

So I suggest to make upload-tokens expire together with the session, and not on their own.

Allow selecting newer CC licences, and deprecate older

We currently list CC v3, but v4 is out already.

Publishing updates fails

When uploading an update to an existing package, the front-end makes the same assumption as when editing meta data. "No name" means "name from package".
It looks like the API agrees in the validation step, since it does not complain about missing stuff in the GET /new-package/.

However, .../publish gives 500:

2020-04-25 21:48:46 ERROR Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/code/bananas_api/web_routes/new.py", line 194, in new_publish
publish_session(session)
File "/code/bananas_api/new_upload/session.py", line 232, in publish_session
create_tarball(session)
File "/code/bananas_api/new_upload/session_publish.py", line 112, in create_tarball
tar_path = _safe_name(session["name"]) + "-" + _safe_name(session["version"])
KeyError: 'name'
2020-04-25 21:48:46 INFO 172.17.0.1 [25/Apr/2020:21:48:46 +0000] "POST /new-package/3a6ad88d8398ed157f8e8af1f9c5157c/publish HTTP/1.1" 500 244 "-" "python-requests/2.23.0"

Uploading an update discards all meta data except "version"

I uploaded an update to an existing package, with a custom name/tags/description/...

None of the entered information was stored.

Restrict dependencies between content-type

Currently you can make a heightmap depend on an AI, which makes very little sense.

The list should be like this:

• ai, ai-library: only allow ai-library
• base-graphics, base-music, base-sounds: no dependencies allowed
• game-script, game-script-library: only allow game-script-library
• heightmap, newgrf: no dependencies allowed
• scenario: only allow newgrf, ai and game-script

Currently we have AIs that depend on a scenario. There is a case for this, but not sure it is a solid one.

Read dependencies from NewGRF / Scenarios automatically

This information is already inside these files, and we already extract it. Adding this would make uploads a lot easier for many users, so a huge QoL.

With this, we should also restrict manual editing of dependencies to only AI/AI Library/GS/GS Library.

Allow setting other authors during upload / edit

Currently you cannot modify the authors of a package, except by making a Pull Request on GitHub. Adding this functionality would be helpful, although it is not heavily used.

ReadError: file could not be opened successfully

Sentry Issue: BANANAS-API-D

ReadError: file could not be opened successfully
(1 additional frame(s) were not displayed)
...
  File "aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "bananas_api/web_routes/new.py", line 92, in tusd_handler
    add_file(
  File "bananas_api/new_upload/session.py", line 236, in add_file
    session["files"].extend(extract_tarball(new_file))
  File "bananas_api/new_upload/extract.py", line 79, in extract_tarball
    with tarfile.open(file_info["internal_filename"]) as tar:
  File "tarfile.py", line 1604, in open
    raise ReadError("file could not be opened successfully")

Error handling request

Allow uploading scenarios/heightmaps as replacement of an older

Currently you can only upload new scenarios/heightmaps (as in, get a new unique-id), but you can never update an existing.

There is no technical reason for this, other than we didn't have the code for it yet.

Package with "name" and "version" of non-ASCII characters will have a filename "-.tar.gz"

Package with "name" and "version" of non-ASCII characters will have a filename "-.tar.gz".

This is because _safe_name() removes any non-ASCII (well, give or take).

We should refuse these uploads, as this is most likely not what the uploaded intended.

Test issue

I just want to close it

Allow reading license/readme/changelog of packages via API (or give a link)

It would be nice to show in the frontend what the license is (if custom), or show the readme or changelog.

Technically, this is a bit tricky, as these files (after upload) are only available inside the tarball. Most likely it is a good idea to store those files separatly on S3, so it is easy to fetch them. Not a trivial issue to fix for sure.

GitHub commit message can have duplicated entries

See

OpenTTD/BaNaNaS-staging@e66dabd

There should only be one notice that a package changed.

Send download URL via API

For new-games, anyone should be able to get the download URLs.
For savegames-only, only the author should be able to get the download URL.

Allow setting versions to "new-games" / "savegames-only" if you like to

Currently the latest upload is always "new-games", and anything else is "savegames-only". This is not technically required by anything, it was just easier to implement.

Please create an additional to the API that allows to set any version to "new-games" and "savegames-only". Possibly it would be good if two versions are "new-games" it is enforced that their min/max compatability-versions don't overlap (as that would make no sense).

/package/<content-type>/<unique-id>/<upload-date> gives 500 for unknown unique-id/upload-date

Request:
GET /package///
with an unknown unique-id or upload-date.

Expected: Response 404

Currently: Response 500

Use information from NewGRF / Baseset to prefill name/description/version

We know this information, we already extract it from the files, but we don't do anything with it.

Would be a nice additional, and a huge QoL for many users.

Cache validation result of individual uploaded files

Validation of big files (I tried a NewGRF with 850 MB) takes several seconds (probably computing MD5). The validation is repeated for every PUT/GET, even when only updating meta-data like description.

Please cache the validation result of uploaded files, if they do not change.

Allow undoing metadata overwrite on version level

Set version members to "" should delete them (so that package data becomes active).
Not setting the fieldis used for "unchanged".

GitHub commits can be empty

See:

OpenTTD/BaNaNaS-staging@0c24b2b

There was a change and it was reverted within the graceperiod. In result, an empty commit.

Please fix this.

Base-music tarballs created are incompatible with OpenTTD client

Based on https://www.tt-forums.net/viewtopic.php?p=1234714#p1234714

A ZIP file with a music set uploaded to BaNaNaS had the directory in the root of the ZIP file stripped, so all the contents are at the root of the generated TAR file. This causes OpenTTD to discard the downloaded file (see OpenTTD/OpenTTD#8283) as invalid, so while the download at first appears to have worked, it actually failed.

Validate URL to be an URL

Currently "url" field can be anything. Validate that it starts with http/https.

/package/<content-type>/<unique-id> gives 200 for unknown unique-id

Request:
GET /package/<content-type>/<unique-id>
with an unknown unique-id.

Expected: Response 404

Currently: Response 200 with empty json {}.

API endpoint to get URL for OAuth access review

Possibly /user could return

• "review-url": "https://github.com/settings/connections/applications/:client_id" (with client-id filled)

Then the front-end can display this link on the manager site or similar.

API endpoints to get list of licenses and branches

Currently this has to be hardocded in all the clients. Please take control of this, and make it an endpoint.

Add an API endpoint to show the pending changes

Changes are not being committed immediately, so add an API endpoint that shows what changes are pending. Possibly add a button to commit the changes immediately.

Putting yourself as dependency is valid

Putting yourself as dependency is valid.

This of course is wrong, and should not be possible. This should be filtered on unique_id already, you also aren't allowed to set dependencies on an older version of your own content.

Keep audit file per entry to see who changed what when

Currently you could check the commit history; but having it per entry would be much cleaner.

Error when uploading a newgrf to bananas-api running in docker

If you run the bananas-api as suggested in the readme using:

docker run --rm -p 127.0.0.1:8080:80 -p 127.0.0.1:1080:1080 -v "${BANANAS_COMMON}/local_storage:/code/local_storage" -v "${BANANAS_COMMON}/BaNaNaS:/code/BaNaNaS" openttd/bananas-api:local

Upload of a newgrf fails with the following error:

OSError: [Errno 18] Invalid cross-device link

The issue is that the temporary upload path and the local_storage path are not on the same physical device (one is inside the virtual file system of the docker container, while the other is on a volume mounted on the host).

According to the documentation I have found, shutil.move is the correct function to use if you are moving between file systems.

Check that textfiles have \r\n for user files and \n for ini files

Most likely this is not the case, but we really should do this.

Validation warning when no readme.txt and/or changelog.txt was uploaded

It's easy to forget these files, and in-game they are as vital as the description.

So the upload validation should warn, if no readme and/or changelog is part of the upload, and recommend adding them.