nautls / ergomatic

Ergomatic is a generic off-chain execution framework for Ergo bots

Ergomatic

ergomatic is a generic off-chain execution framework for bots. Bot functionality is provided via plugins which are audited by the nautilus team.

Install

Shell (Mac, Linux):

curl -fsSL https://raw.githubusercontent.com/nautls/ergomatic/main/scripts/install/install.sh | sh

PowerShell (Windows):

irm https://raw.githubusercontent.com/nautls/ergomatic/main/scripts/install/install.ps1 | iex

Running

Ergomatic can be ran using the run CLI command. The -c flag can optionally be passed to specify the config file path, if not provided it will default to $CWD/ergomatic.yaml.

ergomatic run

Logs can be viewed in your terminal and are also persisted to disk depending on OS:

• Linux: $XDG_DATA_HOME/ergomatic
• macOS: $HOME/Library/Application Support/ergomatic
• Windows: $LOCALAPPDATA/ergomatic

Log files will automatically be rotated with a maximum of 300mb of logs kept.

Contributing

We appreciate your help!

To contribute, please read our contributing instructions.

ergomatic's Issues

Use `semantic-release` for releases

Probably should have just done this from the beginning, use https://github.com/semantic-release/semantic-release for generating releases and maintaining version.ts of ergomatic.

Should also add tools like committizen to enforce style

Validate plugin configs on their behalf

We could add a method to the plugin interface like:

get configSchema() that returns a Zod schema object, this would allow all plugins to define their schema config in a consistent way and allow ergomatic core to validate plugin configs instead of each plugin needing to do it themselves.

It should also be able to return undefined if the plugin has no config or wants to do its own validation

Add a "install" script for easy setup

Make a shell script that installs the ergomatic binary for users

Probably something like:

• download binary
• move it to a /bin on the users path
• etc

Set User-Agent in api calls

Create release pipeline

On a merge to the main branch run a action to do the following:

• Create a tag with the current ergomatic version (version.ts)
• Build ergomatic for all platforms
• Create a new release for the tag that contains the binaries for all platforms

We should also have a way to update the version of ergomatic (version.ts), either automatically or manually. This action should only run if the version has changed

CTRL-C sometimes doesnt stop `ergomatic`

Not really sure whats causing this, will need to investigate

Detail how to run `ergomatic` as a service on unix/windows

And provide any sensible default configs

Shutdown `ergomatic` if all plugins are in a not running state

If all plugins are either STOPPED or ERROR after running ergomatic then stop with a helpful message.

This could occur if the plugins throw errors when validating configuration schema or when running in general.

Add a binary self-updating "update" command

And also checks for updates/provides a message to the user

Monitoring component

This issue is to discuss implementation of the monitoring component required to be able to retrieve and pass on onBlock / onConfirmedUtxo / etc events to plugins.

Original spec that lists events on plugins: https://gist.github.com/ross-weir/258272ed82332d0b7818c23195ab80e8#plugins

Below is rough pseudo code that I have in mind for how it might work:

currentHeight = 0
mempoolDelivery = {} # map of txid -> bool indicating if mempool tx has been passed to plugins
requiredConfirmations = 3
lastBlocks = new Array(requiredConfirmations)

Every `n` seconds
do
    height = getCurrentHeight()
    
    if height > currentHeight
    do
        newBlock = getBlock(height)
        plugins.all.onBlock(newBlock)

        currentHeight = height
        lastBlocks.push(newBlock)
    end

    if lastBlocks.length >= requiredConfirmations
    do
        confirmedBlock = lastBlocks.shift()
        for tx in confirmedBlock.txs
        do
            plugins.all.onConfirmedTx(tx)
        end
    end

    mempool = getMempool()
    for tx in mempool
    do
        if mempoolDelivery[tx.txid] == false
        do
            plugins.all.onUnconfirmedTx(tx)
            mempoolDelivery[tx.txid] = true
        end
    end

    # invalidate mempoolDelivery in some way to remove confirmed or dropped txs?

end

• Still deciding on a good way to ensure mempool txns are delivered only once to plugins and cleaned up if they're dropped or included in chain
• In the original spec we had onConfirmedUtxo & onUnconfirmedUtxo methods for plugins - are these necessary if we have onConfirmedTx & onUnconfirmedTx since the txns will include the respective utxos?
  • Could we reduce the events down to OnBlock, OnConfirmedBlock, onMempoolTx? Or should we?

Add file log handler

Include file handler (probably the rotating version) in logging:

ergomatic/src/log.ts

Line 13 in f5739b5

new handlers.ConsoleHandler(level, {

Will make it easier for users to give us logs in case of errors, etc

Using RotatingFileHandler requires calling setup on the handler, which means we have to move all logger creation in async methods, currently loggers are created in the constructor

Monitor component follow-up

#15 is getting large so going to split it out, this is a issue to track todos

• Implement getBlock method for blockchain client
• Handle BlockchainMonitor.monitor throwing exceptions
• Raise mempool-tx-dropped event in BlockchainMonitor.monitor
• Add logging
• Add tests
• Use Block types when merged & released: fleet-sdk/fleet#35

Use `node` for `getBoxesByTokenId`

There is a open issue to add such a method to the node indexer: ergoplatform/ergo#2038

When this is implemented use the node for this method instead of explorer

Allow setting log level on a per plugin basis

support a logLevel config per plugin, this will allow us to debug specific plugins easier.

If the logLevel isn't supplied in the plugin config then default to the top level logLevel config value

Consider uploading each individual release artifact as a `zip`

Smaller file size, quicker download

Remove sync log rotate handler when fixed upstream

Remove:

ergomatic/src/log.ts

Lines 39 to 133 in ef39490

	interface RotatingFileHandlerOptions extends HandlerOptions {
	filename: string;
	mode?: LogMode;
	maxBytes: number;
	maxBackupCount: number;
	}
	/**
	* `RotatingFileHandler` provided by deno has async `setup()` method
	* which appears to be a bug: https://github.com/denoland/deno_std/issues/3538#issuecomment-1676942783
	* If this is addressed in deno maybe one day we can remove this class.
	*
	* Create a sync version of `RotatingFileHandler` and use that instead.
	* This is basically just `RotatingFileHandler` but using `Deno.*Sync` methods instead.
	*/
	class SyncRotatingFileHandler extends handlers.FileHandler {
	#maxBytes: number;
	#maxBackupCount: number;
	#currentFileSize = 0;
	constructor(levelName: LevelName, options: RotatingFileHandlerOptions) {
	super(levelName, options);
	this.#maxBytes = options.maxBytes;
	this.#maxBackupCount = options.maxBackupCount;
	}
	override setup() {
	if (this.#maxBytes < 1) {
	this.destroy();
	throw new Error("maxBytes cannot be less than 1");
	}
	if (this.#maxBackupCount < 1) {
	this.destroy();
	throw new Error("maxBackupCount cannot be less than 1");
	}
	super.setup();
	if (this._mode === "w") {
	// Remove old backups too as it doesn't make sense to start with a clean
	// log file, but old backups
	for (let i = 1; i <= this.#maxBackupCount; i++) {
	try {
	Deno.removeSync(this._filename + "." + i);
	} catch (error) {
	if (!(error instanceof Deno.errors.NotFound)) {
	throw error;
	}
	}
	}
	} else if (this._mode === "x") {
	// Throw if any backups also exist
	for (let i = 1; i <= this.#maxBackupCount; i++) {
	if (existsSync(this._filename + "." + i)) {
	this.destroy();
	throw new Deno.errors.AlreadyExists(
	"Backup log file " + this._filename + "." + i + " already exists",
	);
	}
	}
	} else {
	this.#currentFileSize = Deno.statSync(this._filename).size;
	}
	}
	override log(msg: string) {
	const msgByteLength = this._encoder.encode(msg).byteLength + 1;
	if (this.#currentFileSize + msgByteLength > this.#maxBytes) {
	this.rotateLogFiles();
	this.#currentFileSize = 0;
	}
	super.log(msg);
	this.#currentFileSize += msgByteLength;
	}
	rotateLogFiles() {
	this._buf.flush();
	this._file!.close();
	for (let i = this.#maxBackupCount - 1; i >= 0; i--) {
	const source = this._filename + (i === 0 ? "" : "." + i);
	const dest = this._filename + "." + (i + 1);
	if (existsSync(source)) {
	Deno.renameSync(source, dest);
	}
	}
	this._file = Deno.openSync(this._filename, this._openOptions);
	this._writer = this._file;
	this._buf = new BufWriterSync(this._file);
	}
	}

When this is merged and released: denoland/deno_std#3543

Add docs

Update README.md to include at least:

• Overview of what ergomatic is
• How to run dev tasks
• How to contribute

Add a doc detailing how to implement a ergomatic plugin