mworzala / canary Goto Github PK

This is all of the things to do with managing the tests at a high level. Mainly setting up the world(s) and instance(s) for the tests.

• Arranging the tests in the world
• Assuring sufficient isolation between tests
• Allow for tests to have different environment conditions (night vs day time)

When a player is connected to a Minestom server, they can type commands into the chat using /command arg1 arg2 ... syntax. A developer can add commands to the server that can do a wide variety of different things.

We want to allow people to test these commands, or at the very least some subset of possible command behavior. There are potentially things that a server command could do, that wouldn't work in our current testing architecture. But providing at least a way to test simple commands is vital to allowing developers to write end-to-end tests for their Minestom server.

When a player is connected to a Minestom server, they can type commands into the chat using /command arg1 arg2 ... syntax. A developer can add commands to the server that can do a wide variety of different things.

We want to allow people to test these commands, or at the very least some subset of possible command behavior. There are potentially things that a server command could do, that wouldn't work in our current testing architecture. But providing at least a way to test simple commands is vital to allowing developers to write end-to-end tests for their Minestom server.

We want to provide as much debug information when tests fail. We want to hide or discard debug information from tests that pass.
There are two main parts to this, which each have their own individual components

Managing test logs / output:

• Suppress output from tests that pass
• Give full output from tests that fail
• Potentially create a report with all of the test output and info
• Associating (non assertion) logs with tests that are running

Creating test logs / debug info:

• Log every assertion made for every test
• Make assertion logs human readable
• Potentially do even more fine grained recording, such as all entity positions and block states

JUnit has the @BeforeEach and @AfterEach annotations to designate a function to be run before or after each test in that class. This is something that we want to bring to Canary. It is not hard to imagine having a group of tests that all have similar setup for every test case. After each is probably less useful, but it's addition doesn't detract from anything, and could potentially be used to clean up a test before unwanted effects happen.

We have the source project resources so we can load the file named extension.json and locate the extension main. The trickier part here is making the ExtensionManager load it (perhaps will require a mixin).

The alternative is to use the system arguments:
minestom.extension.indevfolder.classes
minestom.extension.indevfolder.resources
These require us to specify the location of those, which are build config dependent (however we may be able to get them from gradle, in which case this is probably acceptable)

Investigate potential solutions to current class path issues. Determine what problems these solutions would create and potential ways to mitigate these issues.

If someone runs the tests locally, it starts a server that they can connect to with a normal Minecraft client. One of the things we want to have, is the output of the tests to be somehow accessible from within the Minecraft world. This could be accomplished by having a book next to each test that contains the output of the test after it has completed.

We use assertions in a few places throughout the code, and they should be enabled in the sandbox server.

Each test can be associated with a set of blocks that define the start state of the test. There is also a json format used to define this block layout.

We need to be able to

• Load the associated structure for each test
• Have some way to build and save these structures from within minecraft

It is important that test cannot effect the behavior of other tests.

If, for example, there was a test that was testing if an entity went towards grass blocks, but there was a nearby test that had grass blocks, it is possible that the entity would go towards the grass blocks in the other test. This would obviously ruin the test result and so needs to be avoided.