ppazos / hl7-benchmark

Benchmark for HL7v2.x over MLLP and SOAP

hl7-benchmark

Sends messages in parallel to an HL7 server (like Mirth Connect or HAPI Test Panel) using MLLP or SOAP protocols, and throws a report of times consumed for each parallel thread and totals.

Sample call

\hl7-benchmark > run -c 5 -n 100 -s localhost:2617 -t mllp -path sample_messages\OML_O21_1.hl7

Sample report

total plan 2273 ms
plan media: 1246.5 ms
plan min: 370 ms
plan max: 2123 ms
errors: 0
----------------------
total plan 2734 ms
plan media: 1892.5 ms
plan min: 1051 ms
plan max: 2734 ms
errors: 0
----------------------
total plan 3235 ms
plan media: 1663 ms
plan min: 271 ms
plan max: 3055 ms
errors: 0
----------------------
total plan 2854 ms
plan media: 1827.5 ms
plan min: 811 ms
plan max: 2844 ms
errors: 0
----------------------
total plan 3134 ms
plan media: 1977.5 ms
plan min: 1071 ms
plan max: 2884 ms
errors: 0
----------------------
Total: 14230 ms
Percentage of transactions completed in certain time (ms)
        16.6%   < 549.4
        6.6%    < 827.8
        9.0%    < 1106.2
        12.4%   < 1384.6
        9.8%    < 1663.0
        11.4%   < 1941.4
        12.4%   < 2219.8
        8.2%    < 2498.2
        9.4%    < 2776.6
        4.2%    < 3055.0

hl7-benchmark's Issues

Measure time consumed by sending process and by processing ACK responses

For MLLP de ACK response process might not be optimized because it reads chars.

Add arg for delay between messages

The idea is to specify the delay between sending 2 messages from the same thread, trying to simulate real system delay in creating and sending messages to test real scenarios more accurately.

Add burst and burst delay

The idea is to specify the number of messages sent together on the same "burst", and the delay between bursts from the same thread, trying to simulate how real systems may work, for example a system that has an "outbox" queue of messages to send, and then sends all the messages in the outbox together, then wait until the next "burst".

Idea:
if n = 1000 (number of messages to be sent by the same thread)
and burst = 100
that thread will send a "burst" of 100 messages, 10 times (to complete n), and wait for "burst_delay" between bursts.

Improve the ACK read to do it by line instead of by char

https://github.com/ppazos/hl7-benchmark/blob/master/src/com/cabolabs/mllpclient/MLLPClient.groovy

Add total number of messages sent to final report

Exception when type is not specified

hl7-benchmark>run -c 20 -n 100 -s localhost:6661 -path sample_messages\OML_O21_1.hl7

Exception in thread "Thread-1" groovy.lang.GroovyRuntimeException: Could not find matching constructor for: com.cabolabs.hl7benchmark.SendingPlan
(java.lang.Integer, java.lang.String, java.lang.Integer, java.lang.Class, java.lang.Boolean)
at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:1723)
at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:1526)
at org.codehaus.groovy.runtime.callsite.MetaClassConstructorSite.callConstructor(MetaClassConstructorSite.java:46)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:232)
at com.cabolabs.hl7benchmark.Main$_main_closure2_closure8.doCall(Main.groovy:105)
at com.cabolabs.hl7benchmark.Main$_main_closure2_closure8.doCall(Main.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:292)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
at groovy.lang.Closure.call(Closure.java:423)
at groovy.lang.Closure.call(Closure.java:417)
at groovy.lang.Closure.run(Closure.java:504)
at java.lang.Thread.run(Unknown Source)
Exception in thread "Thread-20" groovy.lang.GroovyRuntimeException: Could not find matching constructor for: com.cabolabs.hl7benchmark.SendingPla
n(java.lang.Integer, java.lang.String, java.lang.Integer, java.lang.Class, java.lang.Boolean)

Error processing for final report

When an error sending/receiving using MLLP or SOAP is detected (excepton should be catched of HTTP error status received), we should mark the correspondent transaction/message as errored, and count the total errored messages in the final report.

Mirth is sending 2.5.1 in ACK and should be 2.5, veiry what's happening

MSH|^~&|OF|Chemistry|OP|Entero-gastric|20150521011331||ACK^O21^ACK|20150521011331|T|2.5.1
MSA|AA|2

Add support to external message specification

We need to try the benchmark with different message types, right now the message is hardcoded.
We need to read a message from a given file, and change the Contro ID for each message the benchmark sends.

Validate external messages

when path is specified and the hl7 message file is read, we should validate the contents of that file, maybe using HAPI pipe parser.