xanderio / apalis Goto Github PK

View Code? Open in Web Editor NEW

This project forked from geofmureithi/apalis

0.0 1.0 0.0 623 KB

Simple, extensible multithreaded background job processing library for Rust.

Lua 2.66% Rust 95.58% PLpgSQL 1.77%

apalis's Introduction

Apalis

Apalis is a simple, extensible multithreaded background job processing library for Rust.

Features

Simple and predictable job handling model.
Jobs handlers with a macro free API.
Take full advantage of the tower ecosystem of middleware, services, and utilities.
Fully Tokio compatible.
Optional Web interface to help you manage your jobs.

Apalis job processing is powered by tower::Service which means you have access to the tower middleware.

Apalis has support for

Redis
SQlite
PostgresSQL
MySQL
Cron Jobs
Bring Your Own Job Source eg Twitter streams

Getting Started

To get started, just add to Cargo.toml

[dependencies]
apalis = { version = "0.3", features = ["redis"] }

Usage

use apalis::prelude::*;
use apalis::redis::RedisStorage;
use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize)]
struct Email {
    to: String,
}

async fn email_service(job: Email, _ctx: JobContext) -> Result<JobResult, JobError> {
    Ok(JobResult::Success)
}

#[tokio::main]
async fn main() -> std::io::Result<()> {
    std::env::set_var("RUST_LOG", "debug");
    env_logger::init();
    let redis = std::env::var("REDIS_URL").expect("Missing env variable REDIS_URL");
    let storage = RedisStorage::new(redis).await.unwrap();
    Monitor::new()
        .register_with_count(2, move || {
            WorkerBuilder::new(storage.clone())
                .build_fn(email_service)
        })
        .run()
        .await
}

Then

//This can be in another part of the program or another application
async fn produce_route_jobs(storage: &RedisStorage<Email>) {
    let mut storage = storage.clone();
    storage
        .push(Email {
            to: "[email protected]".to_string(),
        })
        .await
        .unwrap();
}

Web UI

If you are running Apalis Board, you can easily manage your jobs. See a working Rest API here

Feature flags

tracing (enabled by default) — Support Tracing 👀
redis — Include redis storage
postgres — Include Postgres storage
sqlite — Include SQlite storage
mysql — Include MySql storage
cron — Include cron job processing
sentry — Support for Sentry exception and performance monitoring
prometheus — Support Prometheus metrics
retry — Support direct retrying jobs
timeout — Support timeouts on jobs
limit — 💪 Limit the amount of jobs
filter — Support filtering jobs based on a predicate
extensions — Add a global extensions to jobs

Storage Comparison

Since we provide a few storage solutions, here is a table comparing them:

Feature	Redis	Sqlite	Postgres	Sled	Mysql	Mongo	Cron
Scheduled jobs	✓	✓	✓	x	✓	x	✓
Retryable jobs	✓	✓	✓	x	✓	x	✓
Persistence	✓	✓	✓	x	✓	x	BYO
Rerun Dead jobs	✓	✓	✓	x	*	x	x

Thanks to

tower - Tower is a library of modular and reusable components for building robust networking clients and servers.
redis-rs - Redis library for rust
sqlx - The Rust SQL Toolkit