rust3ds / actions

actions

A set of Github Actions for working with 3DS applications and the rust3ds toolchain.

It's recommended to use the test-runner crate from ctru-rs when working with these actions, in order to get useful test output on failures.

Components

• setup: action for setting up the Rust 3DS toolchain in workflows
• run-tests: action for running test executables with Citra in workflows

Usage

This example shows the default value for each of the inputs.

jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: devkitpro/devkitarm
      volumes:
        # This is required so the test action can `docker run` the runner:
        - '/var/run/docker.sock:/var/run/docker.sock'

    steps:
      - name: Checkout branch
        uses: actions/checkout@v4

      - name: Setup Rust3DS toolchain
        uses: rust3ds/actions/setup@v1
        with:
          # Optionally use a more specific nightly toolchain here if desired
          toolchain: nightly

      - name: Build and run tests
        uses: rust3ds/actions/run-tests@v1
        with:
          # Optionally add arguments to pass to `cargo 3ds test`
          args: ''
          # Optionally set the name of the built test-runner docker image
          runner-image: test-runner-3ds
          # Optionally change to a given directory before running tests. Note
          # that this should use the environment variable ${GITHUB_WORKSPACE}
          # rather than ${{ github.workspace }} to avoid the issue described in
          # https://github.com/actions/runner/issues/2058
          working-directory: ${GITHUB_WORKSPACE}

See ci.yml to see a full lint and test workflow using these actions (including uploading output artifacts from the tests).

Caveats

• Since the custom test runner runs as part of cargo test, it won't be able to find a 3dsx that hasn't built yet. cargo-3ds doesn't build 3dsx executables until after the cargo command it runs internally, so this means that tests can't depend on any features of the 3dsx (like embedded romFS). A workaround for this is to simply build the tests as a separate step before running them, after which the runner will be able to find the 3dsx.

actions's Issues

Add support for `--movie-play` TAS files

For the sake of testing things like user input, it would be nice if the runner could be used for playing back inputs recorded via the --movie-record= flag in Citra. This would be great for e.g. running examples in CI.

Ideally the file would probably be another input to the Github action, I think?

Renaming / reorganization

There are a couple of things I want to consider here. Each of these could probably be a separate issue, but I think to some extent they'll all happen together at once so just tracking here for now:

• Moving .github/actions/* to either the top level or their own repository. I think this repo makes sense, particularly since the Citra test runner requires some level of coordination with the GDB test runner, but it's not necessarily the recommended location. This would also make it easier to version the two actions (setup and run) separately, otherwise we might need to use tags like setup@setup-v1 or something.

  Also, if actions are moved to the top level, maybe the test-runner crate should also be moved down into its own directory?

• Naming; this crate doesn't ever need to be published to https://crates.io, but if if is published it will need a sufficiently unambiguous name. test-runner-3ds is kinda long, maybe just like test3ds or something more creative?

• Moving to the rust3ds org. This seems obvious and is just a matter of making sure the CI and everything still works once moved.

• Finish documenting the actions in the README at least, to make it more obvious how they're meant to work.

Maybe consolidate all of the runners into one?

Instead of separate run_console, run_gdb etc., maybe we can just use all of them at once!

• The socket runner will simply fail to connect to 3dslink if it's running in Citra or the --server flag wasn't used
• Either gdbHioDevInit() or gdbHioDevRedirectStdStreams() should fail if running on hardware (or they are no-ops)
• Console should basically always work, unless the test needs to own the screens / Gfx for some reason (in which case it should probably be an integration test, and we can control the features more carefully there).

The only oddity would be that none of them work simultaneously, so they would need to be in some kind of priority order. Maybe still have cargo features like this?

[features]
# These three output channels will be tried, in this order, to display test output.
# Failure to set up a given output stream will be ignored, so the last successful method
# will be the one ultimately used by the test. You can skip a certain method by disabling its
# corresponding feature.

default = ["console", "gdb", "socket"] # or maybe a different order/subset
console = []
gdb = []
socket = []

Run 3dsx if present

The entrypoint script always fails to find 3dsx file even if built

Intermittent Citra crashes

Seems that we have some cases of Citra crashing for reasons unrelated to our tests or test runner, e.g.

https://github.com/rust3ds/ctru-rs/actions/runs/7936268178/job/21671177303?pr=157
https://github.com/rust3ds/ctru-rs/actions/runs/7933804707/job/21663401999

Citra output seems to indicate this is probably just a renderer bug of some sort:

�[0m�[1;33m[   5.769866] Service.APT <Warning> core/hle/service/apt/apt.cpp:NotifyToWait:345: (STUBBED) app_id=768
�[0m�[1;31m[   5.770837] Service <Error> core/hle/service/service.cpp:ReportUnimplementedFunction:166: unknown / unimplemented function 'ConfigureNew3DSCPU': port='ptm:sysm' cmd_buf={[0]=0x8180040 (0x0818, 1, 0), [1]=0x0}
�[0m�[1;33m[   5.771119] Service.APT <Warning> core/hle/service/apt/apt.cpp:AppletUtility:694: (STUBBED) called command=0X00000007, input_size=0X00000004, output_size=0X00000001
�[0m�[1;33m[   5.771256] Service.APT <Warning> core/hle/service/apt/apt.cpp:AppletUtility:694: (STUBBED) called command=0X00000004, input_size=0X00000001, output_size=0X00000001
�[0m�[1;33m[   6.183926] Service.GSP <Warning> core/hle/service/gsp/gsp_gpu.cpp:AcquireRight:803: called flag=00000000 process=11 thread_id=0
�[0m�[1;35m[   6.236768] Render.OpenGL <Critical> video_core/renderer_opengl/gl_texture_mailbox.cpp:ReloadPresentFrame:36: Failed to recreate present FBO!
�[0mUnhandled SIGSEGV at rip 0x00007f7baed657cd

Since Vulkan rendering has been implemented for a while now, maybe we can try using that instead and sidestep this issue entirely. It seems likely that Vulkan will be well-supported by Citra devs going forward so maybe we're better off using it overall anyway, but will have to see.

They also mention

Recording your gameplay using Video Dumping is not yet implemented.

But we didn't have that working for the test runner yet anyway (#2) so not really much of a loss there at this point.

Uncertain future

After rust3ds/ctru-rs#171, the future of test-runner seems dim. Download links for Citra are unreachable and the emulator is officially dead.

@ian-h-chamberlain With the current state of things, and the fact we already planned on merging this repo back into ctru-rs, what are the remaining components that we can salvage from test-runner? Glancing over the project, it seems that anything more than just building the tests needs to be taken down.

Publish container images

It might be useful for people who want to use this runner locally, to publish images to https://ghcr.io

I think Github actions should make it fairly easy to do? They should probably have tags matching the git tags / releases.

Video dumping doesn't seem to work

Example: https://github.com/ian-h-chamberlain/test-runner-3ds/suites/14871964728/artifacts/846943789

The output .webm files appear "valid", but don't play in VLC. Maybe we can just try different video formats and see if it works better?

Also, it might make sense to see if there's a way to print both to GDB and to the console for these, although really the video dump is more likely to be helpful with actual graphics and/or running examples.

Run doctests as well as other tests

Due to their kinda unique nature, doctests don't seem to have the same setup/teardown in their binaries as normal #[test] executables... It would be nice not to have to add a special setup line to every triple-backtick example just to get them to run...