decompals / slinky

Linker script generator for decompilation and modding projects

slinky

splat link yutility

Linker script generator

Table of contents

• slinky
  • Table of contents
  • What is slinky?
    • Why use slinky instead of other linker scripts generators?
  • Features
    • Planned features
  • Installing
  • Versioning and changelog

What is slinky?

slinky is a linker script generator for modern GNU (ld) and LLVM (lld) linkers. It aims to be resuable and general enough so it can be used in a variety of projects

Its main purpose is to generate linker scripts for matching decompilation projects that use either of those linkers, allowing to still generate matching binary output but adding extra quality of life features.

Why use slinky instead of other linker scripts generators?

Other linker script generators are functional and do their job, but can be rather limited or lack some features due to a number of reasons.

Two notable linker script generators on this context are splat and mkldscript (which lives inside the OoT decompilation repo and the MM decompilation repo). These two are great at their job but can be a bit annoying to work with when you use them for their intended matching mindset. For example, mkldscript doesn't have many customization options besides what has been needed to produce matching builds of the Zelda64 games, it uses a non-standard and non-documented file format as input to describe the layout of the built ROM, and since it lives inside the game's repo then other projects using it can get out of date quite easily. splat on the other side is a lot much more customizable and is a Python library so it can be used by many different projects, but its main purpose is as a binary splitting tool, meaning its input file describes the binary blob that's meant to be splitted, making it hard to produce builds with modified contents (add or remove files, move the memory layout, etc).

slinky aims to provide extra quality of life features on top of the features that other linker scripts offer, like a better modding support experience or extra shiftability features not present on other tools.

Features

• Linker script generation for modern GNU ld and LLVM lld.
• Reusable library.
  • A CLI is also available.
• Support for conditional including/excluding of file entries.
  • Allows to decide which entries should (or shouldn't) be emitted by slinky.
  • Useful for multiversion support.
• Support for replacement strings in paths.
• Allow specifying multiple "segments" required for N64 games.
  • Support for partial linking, allowing to partially link each individual segment, improving build times.
• "Single segment" support.
  • Generated linker script will resemble more a traditional linker script, each elf section mapping a normal section.
• Allow specifying custom sections and customizing the order of the sections.
  • Sections can be changed globally and/or per segment.
  • Sections of specific file entries can be changed too.
• Documented custom file format.
• Multiple alignment options: SUBALIGN, align segments start and end, align individual sections, etc.
• Generation of Makefile dependency (.d) files.
• Emit linker script symbols to allow referencing the individual segments and sections from the final built elf.
  • Two customization options for linker symbols, the splat format and the Makerom format.
  • Generation of C header file containing all the generated linker symbols. The types of these symbols on this header can be customized too.
• Support for custom list of allowed and denied elf sections.
  • Useful to cherry pick which debugging sections should be allowed.
• Archive (.a) libraries support.
  • Support for both listing a library and letting the linker grab any used file or to specify which speicific files should be linked by the linker.
• Support splat's "vram classes" for better memory layout management.

Planned features

All the planned features are added to the issue tracker. A full list can be seen here

Installing

Prebuilt binaries for the latest version of the CLI are available at the releases page on Github.

The CLI can also be installed system-wide using Rust's cargo.

cargo install slinky-cli

To use the library itself as part of a Rust project then simply add it to your project with cargo.

Versioning and changelog

This library follows Semantic Versioning. We try to always keep backwards compatibility, so no breaking changes should happen until a major release (i.e. jumping from 1.X.X to 2.0.0).

To see what changed on each release check either the CHANGELOG.md file or check the releases page on Github. You can also use this link to check the latest release.

slinky's Issues

Think on a better name

linker_script_gen is lame

Explicit deny list

Stuff that should be explicitly discarded

cli

Non hardcoded `_gp` value support

Multiversion support

Allow to specify files or segments that should be or not be including depending on the version that's being built

Maybe have a syntax like this

  - { path: src/boot/boot_main.o }
   # Only us_debug and_jp_debug have this file listed on the linker script
  - { path: src/boot/debug_menu.o, version_include: [us_debug, jp_debug] }
   # All the versions except us and eu have this file on the linker script
  - { path: src/boot/kanji.o, version_exclude: [us, eu] }

The built version could be passed as an argument to slinky.

Figure out if we somehow want to generalize this for user's arbitrary options, like modding: true, compiler: gcc, abi: n32, etc

Toggle treating bss (and relative) sections as NOLOAD

Support groups of files inside segments

Instead of having this

paths:
  base_path: build/us

segments:
- name: main_segment
  files:
  - { path: src/main_segment/replay.o }
  - { path: src/main_segment/066580.o }
  - { path: src/main_segment/066840.o }
  - { path: src/main_segment/unused/055C40.o }

  - { path: lib/libmus/src/player.o }
  - { path: lib/libmus/src/player_fx.o }
  - { path: lib/libmus/src/aud_dma.o }
  - { path: lib/libmus/src/aud_sched.o }
  - { path: lib/libmus/src/aud_thread.o }
  - { path: lib/libmus/src/lib_memory.o }
  - { path: lib/libmus/src/aud_samples.o }

  - { path: asm/rsp/rspboot.o }
  - { path: asm/rsp/aspMain.o }
  - { path: asm/rsp/f3dex2.o }
  - { path: asm/rsp/s2dex.o }

Allow to have this:

paths:
  base_path: build/us

segments:
- name: main_segment
  files:
  # No group
  - { path: src/main_segment/replay.o }
  - { path: src/main_segment/066580.o }
  - { path: src/main_segment/066840.o }
  - { path: src/main_segment/unused/055C40.o }

  # One group
  - kind: group
    dir: lib/libmus/src/
    subfiles:
    - { path: player.o }
    - { path: player_fx.o }
    - { path: aud_dma.o }
    - { path: aud_sched.o }
    - { path: aud_thread.o }
    - { path: lib_memory.o }
    - { path: aud_samples.o }

  # Another group
  - kind: group
    dir: asm/rsp/
    subfiles:
    - { path: rspboot.o }
    - { path: aspMain.o }
    - { path: f3dex2.o }
    - { path: s2dex.o }

So the paths get appended, allowing to make the paths shorter. All those files would still be flattened when emitting the linker script

Output a symbol header file

Document file format

Write down the format, all the options and how they work.

Probably use markdown files inside the docs/ folder

Check segment names are not duplicated

Add partial/incremental linking support

PR templates

`segment_end_before_align`

Consider supporting file formats other than yaml

The serde crate facilitates reading multiple file formats besides yaml, like json, toml, csvs, etc.

Would be worth to try to support anything besides yaml?

Feature parity with splat

Aim to have the same features as splat's linker writer.

Features not being exactly the same should be okay'ish

Aiming to mimic splat 0.22.1

Requires the following stuff to be done:

• #2
• #3
• #4
• #11
• #13
• #14
• #15
• #16
• #17
• #19
• #20
• #21
• #23
• #24
• #25
• #30
• #34
  • Not sure if this is possible in splat
• #35
• #36
• #41
• #42
• #43
• #44
• #53
• #67

Allow to specify that a segment should be compressed

Not very useful in the context of generating linker scripts, but can be useful if the tool ends up being a library

Hardcoded `_gp` value support

LICENSE

Allow changing what sections should be emitted for each file

• Have sensible defaults that would match most decomp projects.
• Allow to change the order, add or remove sections

Option to specify the vram of a segment should be after the end of another arbitrary segment

Something like after: boot or follows_segment: other_segment.

This is only necessary when there are a few segments in between two segments that have contiguous vram

Document the code itself

Discuss the current input format

• Is the format okay?
• Could it be improved?
• Does it have any major flaws?

Option to align the end linker symbol for each section

Option to emit an . = ALIGN(., X) before each section

Allow to specify that a specific section of a file should be put together within another section

For example, instead of putting the .data section of the file main_menu.o within the other .data sections allow to put it in between the .rodata section of the other files

Write a proper README

In no particular order:

• Explain what slinky is
• List all the features
• How to use
• Link the changelog
  • Mention semantic versioning
• How to install/build
• Link docs
• Link examples (maybe inside of docs)

Linker offset support?

Not able to set `subalign: null` on a segment to be able to disable the subalign for the said segment

This is caused by not being able to distinguish between a missing subalign and subalign: null.

The current workaround is to have a boolean member for those, in this case an use_subalign: False, but it is kinda crappy.

Maybe implement some of the options from this stackoverflow question: https://stackoverflow.com/questions/44331037/how-can-i-distinguish-between-a-deserialized-field-that-is-missing-and-one-that

Allow changing sections at the segment level

• Requires #41
• Allow changing order/which sections should be emitted in each segment.
• If this is not explicitly specified on a segment then fallback to the global setting

Figure out what is paper mario doing with its `max_vram_syms` and try to support it

Check files are not duplicated

Check there are no duplicates in the listed files, even between segments.

Generate .d (dependency) files

Figure out if this should have Python bindings

glob filepath support

Support listing globs, so everything inside a folder is included automatically.

Should it work recursively inside folders?

Does linker script support listing globs? Or would the list need to be hardcoded by slinky each time it is run (ie inspecting the existing files of the folder)?

Export list of segments using user-defined format

Not sure about this one. It may make more sense as its own independent tool.

• Allow customizing the output so it can fit the user needs.
• Add a way to specify if a segment should be emitted with a different format or if it should be omitted completely.
• Similar to OoT/MM's mkdmadata tool.

Allow to change the style of auto-generated linker symbols

Support both splat's style and makerom's style

Fill up cargo.toml

Support for padding

Just advances the . by a given amount

Probably make it a new kind of FileInfo

Setup CI for format, PRs, clippy, etc

Support to list files inside archives

Tell the linker to grab specific files from an archive.

Not sure how the syntax would look like.

Full `.a` (archive) support

Allow to just list .a files so the linker can grab whatever file it may need from the archive without needing to link them all or list them all.

Consider making slinky a library/crate for other programs to use

Publish on crates.io

Would need CI for the releases and such

Make Windows builds to use `/` instead of `\\` as path separator

Could having toggeable \\ support be useful in the future?

Issue templates

Both for bug reports and feature requests

Allow to specify a subalign

Explicit list of allowed sections

For debugging sections and such

Add support for FILL statements

Add automated tests

Setup automated tests to check there are no regressions each time we change something.

Allow to specify a fixed vram on each segment

Toggeable discard wildcard section