This library provides high performance event loop implementations and utility functions to help with threading and concurrency.
Concurrency is hard, and event loops provide an abstraction to make dealing with concurrency easier. Context switching between threads is expensive and best practice in a low latency system is to keep your latency-sensitive operations executing on a small number of "fast" threads and the event loop abstraction fits well with this approach.
For best latency these fast threads will be busy-spinning i.e. consuming a whole core, and the core running the fast thread is isolated from the OS and other applications.
This library contains a number of event loop implementations and additional features, all implemented with a view to minimising latency and avoiding garbage in the fast path.
An event loop can have multiple event handlers installed on it and the event loop will repeatedly execute these event handlers.
Each event handler is guaranteed to be called on only one thread. Event handlers are expected to perform a chunk of work
quickly and return without blocking, although the EventGroup does provide a mechanism to support blocking
event handlers - see
HandlerPriority.
See
EventLoop
and
EventHandler.
Figure 1 illustrates a diagram of an event loop that serves several event handlers sequentially.
In this section we explore common event handler use cases
An event handler can be created by implementing the EventHandler interface and overriding its action() method so that it executes the required work.
The action() method returns a boolean value signifying whether some work was done by the action() - an
example is an event handler that tries to poll from a Chronicle Queue - the return value of the action() should
indicate if the read succeeded and a message was processed.
|
Note
|
The event loop considers this return value by using a heuristic: if the action() did some work, it is likely to do some more
work next time, and so we should call it again as soon as we can. If it did not do some work, it is less likely
to do work next time, so it may appropriate to yield or pause before calling again - see Pausers.
|
As a rule of thumb, an action handler should do a limited amount of work then return. If it knows for sure that there is remaining work to be done at the point of return then it should return true.
public final class ExampleEventHandler implements EventHandler {
@Override
public boolean action() throws InvalidEventHandlerException {
// do work
return didWork;
}
}call the addHandler method of the event loop. See also Start event loop
el.addHandler(eh0);When an event handler wants to remove itself
from the event loop, its action() method should throw InvalidEventHandlerException. The
InvalidEventHandlerException.reusable() method returns a reusable, pre-created, InvalidEventHandlerException that is
unmodifiable and contains no stack trace. The below event handler uninstalls itself after being called 30 times.
public final class UninstallingEventHandler implements EventHandler {
private int actionCount = 0;
@Override
public boolean action() throws InvalidEventHandlerException {
if (++actionCount > 30)
throw InvalidEventHandlerException.reusable();
// do work
return didWork;
}
}Chronicle Threads contains a number of event loop implementations. These are aggregated together in the
EventGroup, and the general recommendation
is to make use of this, although the other implementations of EventLoop can of course by used, or the user can implement their own.
The EventGroup also automatically enables Performance Monitoring.
event group is created by calling the using the
EventGroupBuilder. Basic example shown below:
EventLoop eg = EventGroupBuilder.builder()
.withPauser(Pauser.busy())
.withName("my-eg")
.build()the EventLoop.start() method starts the event loop. Event handlers can be added before
and after starting the event loop but will not be executed until start() has been called.
el.start();Calling the stop() method will stop the event loop executing handlers and blocks until all handlers have finished executing.
Calling close() on an event loop first calls stop and will then call close on all event handlers.
Once an event loop has been stopped it is not expected that it can be restarted.
For more details on lifecycle please see javadoc of
EventLoop
and
EventHandler.
Event handlers have a
HandlerPriority
(the default is MEDIUM) and when an EventHandler is installed on an EventGroup, the HandlerPriority
determines which of its child event loops the EventHandler is installed on.
The second use of HandlerPriority is to enable each (child) event loop to determine how often each
EventHandler is called e.g. HandlerPriority.HIGH handlers are executed more than HandlerPriority.MEDIUM handlers.
Chronicle Threads provides a number of implementations of the
Pauser
and it is straightforward for the user to implement their own if need be.
The Pauser allows the developer to choose an appropriate trade-off between latency vs CPU consumption for when
an EventLoop is running events which exhibit "bursty" behaviour.
The recommended way to use Pauser - and this is how Chronicle Thread’s event loop implementations use it:
while (running) {
// pollForWork returns true if work was done
if (pollForWork())
pauser.reset();
else
pauser.pause();
}The Pauser implementation can choose to yield, pause (with back off if required).
In the context of the heuristic in Implement event handler above - if an
EventHandler does no work, then it may well not need to do any work for a while, as events often occur in bursts in the real world.
In this case it makes sense for the Pauser to keep track of how many times
it has been called, and progressively implement longer pauses every time its pause() is called.
This behaviour allows a back off pauser to strike a reasonable balance between handling bursts of events quickly,
but backing off and reducing CPU consumption in case of gap in incoming events.
A good example of a back off Pauser is the LongPauser which will busy-loop for minBusy events (allowing the event loop to respond
quickly if a new event arrives immediately), then will yield for minCount times before it sleeping for minTime increasing up to maxTime.
TimingPauser interface extends the Pauser interface and behaves the same, but if the
void pause(long timeout, TimeUnit timeUnit) throws TimeoutException;method is called, the TimingPauser will keep track of accumulated pause times and throw a TimeoutException if the specified timeout is exceeded.
LongPauser, TimedBusy and TimingPauser are of type TimingPauser.
PauserMode contains factory methods for Pauser implementations. Because Pauser is not an enum, and implementations
are not Marshallable, PauserMode can be used in yaml config files.
There are PauserMode for all Pauser factory methods.
Mode |
Description |
Benefits |
Downside |
Can be monitored |
Requires CPU isolation |
|
Busy-loops |
Minimises jitter |
Uses max CPU, no monitoring support |
✅ |
|
|
Same as |
Minimises jitter |
Uses max CPU, no monitoring support |
✅ |
|
|
Very briefly busy-loops then yields |
Low jitter, stateless and thus can be shared |
Uses high CPU |
✅ |
|
|
Back off pauser - implemented with |
Good balance of busy waiting and back off |
Uses less CPU, but more jitter |
✅ |
|
|
Sleeps for one millisecond, no back off |
Low CPU use |
Up to 1 ms jitter |
✅ |
|
|
Less aggressive version of |
Minimal CPU |
High jitter |
✅ |
The busy pauser minimises jitter for best performance. However, it means that an entire core is consumed and care should be taken
to ensure that there are enough cores for each busy thread. If not, the machine will perform worse.
The graph below illustrates how each different type of PauserMode backs off over time:
Event Loop Monitoring (AKA Loop Block Monitoring) is a very useful feature. This is implemented by a handler which is installed on the MonitorEventLoop
by the EventGroup. The ThreadMonitorHarness handler monitors fast event loop threads to make sure event handler latency remains within acceptable bounds. The ThreadMonitorHarness monitors latency by measuring the time the action method of the application event handlers takes to run. Whenever the method runs beyond an acceptable latency limit, ThreadMonitorHarness prints a stack trace for the event loop thread.
This output can be processed with tools to help identify hotspots in your program.
Set the monitor event interval with system property MONITOR_INTERVAL_MS.
Disable the monitor by setting the system property:
disableLoopBlockMonitor=trueYou can use any stack trace information to improve the design for efficiency.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent