normanmaurer / autobahntestsuite-maven-plugin

Maven plugin that runs tests of the autobahntestsuite as port of your build

Maven Autobahn Testsuite Plugin

Maven plugin which allows to run the Autobahn Testsuite as part of your maven build.

Requirements

If you want to have the Testsuite executed as part of your build you need to provide a Main class which will startup a WebSockets server which needs to behave like:

• Echo back Text/Binary messages
• Handle everything else like stated in the rfc6455

Adding it to your build

Adding the Testsuite and so make it part of your build is as easy as adding it to the pom.xml file of your maven project:

<build>
  <plugin>
    <groupId>me.normanmaurer.maven.autobahntestsuite</groupId>
    <artifactId>autobahntestsuite-maven-plugin</artifactId>
    <version>0.1.0-SNAPSHOT</version>
    <configuration>
      <!-- The class which contains a main method that accept the port as paramater and startup the -->
      <!-- the server. -->
      <mainClass>io.netty.testsuite.websockets.autobahn.AutobahnServer</mainClass>

      <!-- Optional configuration -->
      <!-- ---------------------- -->
       <!-- The IP address of the host/server-->
      <host>host</host>
      <!-- The port to bind the server on. Default is to choose a random free port. -->
      <port>-1</port>

      <!-- The number of milliseconds to wait for the server to startup. Default is 10000 ms. -->
      <waitTime>10000</waitTime>

      <!-- Specify if a JUnit compatible Xml file will be generated. This can be used by most CI's. -->
      <!-- Default is true -->
      <generateJUnitXml>true</generateJUnitXml>

      <!-- A list of cases to execute. Default is to execute all via *.-->
      <cases>
        <case>*</case>
      </cases>

      <!-- A list of cases to exclude. Default is none. --> 
      <excludeCases></excludeCases>

      <!-- Specify if the plugin should fail on non strict behaviour. Default is false. -->
      <failOnNonStrict>false</failOnNonStrict>
    </configuration>
    <executions>
      <execution>
        <phase>test</phase>
        <goals>
          <goal>fuzzingclient</goal>
        </goals>
      </execution>
    </executions>
  </plugin>
</build>

This will execute the fuzzingclient tests as part of the test phase and fail the phase if one of the test cases fails.

After the run was complete you will find test-reports in the target/autobahntestsuite-report, which contains all the details about every test case.

autobahntestsuite-maven-plugin's Issues

Add support for compression tests

plugin fails on windows because of import error in twisted

     Caused by: Traceback (most recent call last):
       File "<string>", line 1, in <module>
       File "__pyclasspath__/autobahntestsuite/__init__.py", line 23, in <module>
       File "__pyclasspath__/autobahntestsuite/wstest.py", line 24, in <module>
       File "__pyclasspath__/twisted/internet/reactor.py", line 38, in <module>
       File "__pyclasspath__/twisted/internet/default.py", line 56, in <module>
       File "__pyclasspath__/twisted/internet/default.py", line 52, in _getInstallFunction
       File "__pyclasspath__/twisted/internet/selectreactor.py", line 18, in <module>
       File "__pyclasspath__/twisted/internet/posixbase.py", line 24, in <module>
       File "__pyclasspath__/twisted/internet/udp.py", line 51, in <module>
       File "__pyclasspath__/twisted/internet/base.py", line 23, in <module>
       File "__pyclasspath__/twisted/internet/defer.py", line 1490, in <module>
       File "__pyclasspath__/twisted/python/lockfile.py", line 23, in <module>
     ImportError: cannot import name kill

Allow to automatically choose a free port

At the moment the user needs to specify the port to use for the WebSocket during tests. Would be nice if the user could just tell the plugin to use a random free port.

Set the constructed ClassLoader on the correct Thread

Ad the moment it is set on the Thread that executed the execute() method. It should be set on the Thread that class the main(...) method.

Plugin not add all test dependencies to the ClassLoader that start the mainClass

Currently we use the compile dependencies but it needs to be the test dependencies.

Add the skip configuration property

Like many other plugins so it is possible to skip the testsuite run when a certain property exists. This is particularly useful when the testsuite run has to be skipped when either skipTests or skipAutobahnTests is specified.

Generate junit xml reports

We should allow to also generate junit xml reports which can be processed by most CI systems

0.1.2

Initial release

Plugin fails on OSX

Release Version 0.14

Releasing this version and deploying on Maven central will allow it to be included in the Netty build process.

Correctly detect free port by respect configured host

At the moment the next free port is detected on "localhost" all the time. We should detect respecting the configured host.