yichun-aot / nifi-mongo-plugin

NiFi mongo oplog reader

Description

This is NiFI plugin for reading mongo oplog. It uses tailable await cursor type. This cursor type keeps connection open, so processor started and keeps working without exiting. If you change cursor type to non-tailable, you must save checkpoints after each received event (modify saveCheckPoint function) as outputEvents function sets cursor to lastOplogTimestamp when invoked (invoked at onTrigger function, which in turn is invoked once in interval, set at processor settings).

Read processor's fields description when configuring it.

Format of message

{"op":"i","changes":{"x":2,"y":0,"_id":"5d9c96a4735f1e9ff5493300","ts":1570543268},"collection":"events2","db":"test"}
op is event type (i == insert, d == delete, u == update)

How-to

1. mvn clean package
2. cp nifi-compose-nar/target/nifi-compose-nar-1.9.2.nar /opt/nifi/lib
3. sh /opt/nifi/bin/nifi.sh restart