frodase / minion

Background job system for .NET applications

Minion

Minion is a modern, testable background job scheduler for .NET applications. Minion will handle running your background job in a reliable way with SQL Server backed storage.

Example of scenarios when you can use Minion:

• Fire web hooks
• Database cleanup
• Creating recurring automated reports
• Batch sending emails
• ... and more

Installation

Minion is available as a NuGet package. You can install it using the NuGet Package Console windows in Visual Studio:

PM> Install-Package Froda.Minion

Usage

To run the server, add the following lines of code:

MinionConfiguration.Configuration.UseSqlStorage("<connection string>");

using (var engine = new BatchEngine())
{
    Console.WriteLine("Starting ...");

    engine.Start();

    Console.ReadKey();
}

All jobs need to inherit from Job or Job<TInput>:

public class SimpleJob : Job
{
    public override async Task<JobResult> ExecuteAsync()
    {
        Console.WriteLine("Hello from simple job");

        return Finished();
    }
}

public class JobWithInput : Job<JobWithInput.Input>
{
    public class Input
    {
        public string Text { get; set; }
    }

    public override async Task<JobResult> ExecuteAsync(Input input)
    {
        Console.WriteLine("Hello from job with input: " + input.Text);

        return Finished();
    }
}

To schedule jobs you first need an instance of JobScheduler:

var scheduler = new JobScheduler();

Fire-and-forget

The job will be executed as soon as possible.

await scheduler.QueueAsync<SimpleJob>();

var input = new JobWithInput.Input { Text = "This is awesome" };
await scheduler.QueueAsync<JobWithInput, JobWithInput.Input>(input);

Scheduled job

The job will execute at the given time.

var date = new Date(2019, 04, 20);

await scheduler.QueueAsync<SimpleJob>(date);

var input = new JobWithInput.Input { Text = "This is awesome" };
await scheduler.QueueAsync<JobWithInput, JobWithInput.Input>(input, date);

Recurring jobs

Recurring jobs are scheduled as normal jobs, but need to return a new DueTime:

public class RecurringJob : Job
{
    private readonly IDateService _dateService;

    public RecurringJob(IDateService dateService) {
        _dateService = dateService;
    }

    public override async Task<JobResult> ExecuteAsync()
    {
        Console.WriteLine("Hello from recurring job, I will execute every 2 seconds");

        return Reschedule(_dateService.GetNow().AddSeconds(2));
    }
}

Sequence

This will force the jobs to run in sequnce:

var sequence = new Sequence();

sequence.Add<FirstJob>(); //This will run first
sequence.Add<SecondJob>(); //When the FirstJob is finished this will run

await scheduler.QueueAsync(sequence);

Set

These jobs will run in parallel if possible:

var set = new Set();

set.Add<FirstJob>();
set.Add<SecondJob>();

await scheduler.QueueAsync(set);

More advanced jobs

You can run a set in a sequence, and a sequence in a set:

var sequence = new Sequence();

sequence.Add<FirstJob>(); //This will run first

var set = new Set();

//Second and third job will run in parallel if possible
set.Add<SecondJob>(); 
set.Add<ThirdJob>(); 

sequence.Add(set);

sequence.Add<FourthJob>(); //Only when both second and third job is finished, this will run

await scheduler.QueueAsync(sequence);

Testing

With the TestingBatchEngine you can simulate your jobs with the ability to time travel.

For an example, if you schedule a job to send an email two days from now, you can fast forward and the batch engine will act as if two days have passed.

[Fact]
public async Task Can_Send_Email() {

    //Setup
    ...

    var startDate = new DateTime(2018, 04, 20);
    var dateSimulationService = new SimpleDateSimulationService(startDate);
    var store = new InMemoryStorage(dateSimulationService);

    var eninge = new TestingBatchEngine(store, dateSimulationService);
    var scheduler = new JobScheduler(store, dateSimulationService)

    await scheduler.QueueAsync<SendEmailJob>(startDate.AddDays(2)); //Schedule job to two days from now

    await engine.AdvanceToDateAsync(startDate.AddDays(2)); //Fast forward to two days from now

    //Assert
    ...
}

Dependency Injection

To hook up your IoC container to, you need to create a class that inherits from IDependencyResolver:

public class CustomDependencyResolver : IDependencyResolver 
{
    private readonly IContainer _container;

    public CustomDependencyResolver(IContainer container) 
    {
        _container = container;
    }

    public bool Resolve(Type type, out object resolvedType)
    {
        resolvedType = _container.Resolve(type);

        if(resolvedType == null)
        {
            return false;    
        }

        return true;
    }
}

Then you need to pass your resolver to the batch engine:

...
var container = new Container();
var resolver = new CustomDependencyResolver(container);

using(var engine = new BatchEngine(store, resolver)) {

    engine.Start();

    Console.ReadKey();

}

License

Minion goes under The MIT License (MIT).

minion's Issues

Comparison to Hangfire

Hey there, just wanted to know if there was a quick comparison between Minion and Hangfire.

Is it built to replace Hangfire at all?

thanks in advance :)

Redis Store

It would be great to have a redis implementation.
Start by creating a new project, "Minion.Redis", implement IBatchStore for redis.
Create a new nuspec file, "Froda.Minion.Redis"

Recurring job management

Just wondering if there are any plans to extend the stores so they can be queried for existing jobs and then manipulate them (deletes particularly).

For recurring jobs, I would like to avoid rescheduling the job every time the app starts up but I cannot query the store (I think) to determine if I should queue a new schedule or not.

Thanks!

Make it easier to configure/setup

To setup and run the server is a bit verbose. You should not have to pass common services around.

Measure execution time of every job

It would be great if the execution time of every job was stored along with the job. This opens up the possibility to have alerts on slow performing jobs.

Implement heart beat

Implement heart beat in IBatchStore

Remove the need for static MinionConfiguration

We often deploy logically separate services as a single deployment unit using a single service or web app as the host and then boot up the logically separate services within the same application / app domain.

Due to the static Configuration property of the MinionConfiguration, this can cause issues when the MinionConfiguration is configured by multiple logical services. I've had a cursory look and it appears that the static nature of the MinionConfiguration can be fairly easily removed. This would mean that a MinionConfiguration instance is constructed and passed into BatchEngine for example as a constructor parameter.

var minionConfiguration = new MinionConfiguration();
minionConfiguration.UseSqlStorage("<connection string>");

var engine = new BatchEngine(minionConfiguration);
engine.Start();

Is this something you'd be open to adding?

Add dashboard

Add a dashboard to view the status of all jobs, restart jobs and manage errors.