Future Features about stevejobs

[msavin]

Hey man - love it - thanks for the interest!

I created new issues to kick off the next couple of features:
1- #6
2- #5

To your points:
7 - Do you mean like, delay all the jobs? Yeah that would be cool, if a job id is not specified it should run on all jobs. It could get performance intensive if the queue has to update 10,000 documents though.
8- Indeed, though I like to reduce third party dependencies because something almost always goes wrong with them.
9- yes :D it could be really handy for development purposes too

---

10- This is a tricky one because I am not sure if jobs should have statuses. I'm thinking of jobs in this context as method calls that are being called but are running at a delay. They either return a result or fail and retry.

However, someone could implement a status system specific their needs. For example, if they are sending 10,000 emails, they can create a Emails collection, insert a document for that action, and then update the document as they need to. That way, SJ doesn't have expose its MongoDB collection and remains a lean option. And.. this does make me think about before/after hooks.

Jobs.register({
  sendEmails: {
    before: function (sender, content) {
      EmailHistory.insert({ ... })
    },
    on: function (sender, content) {
      Magic.sendEmails({}, function () {
        EmailHistory.magicUpdate({ ... })
      })
    },
    after: function (e, r) {
      if (!e) {
        Magic.sendConfirmationThatEmailsHaveBeenSentSuccessfully();
      }
    },
  }
})

In the example above, we assume that you have an application that is sending out a lot of emails on behalf of your end users. When the job is added, before runs and adds a document to your Emails collection. When the job runs, it it would update that document with the latest status. Once it completes, it will let you know the error/result and run the appropriate callback.

[luixal]

7 - Do you mean like, delay all the jobs? Yeah that would be cool, if a job id is not specified it should run on all jobs. It could get performance intensive if the queue has to update 10,000 documents though.

May be delaying all the jobs could be done by delaying just the function that queries for new jobs to run. That's why I was thinking of an specific function for that.

8- Indeed, though I like to reduce third party dependencies because something almost always goes wrong with them.

Well... maybe using a fixed version of those third party dependencies would be enough. The point is that using third party tools removes a lot of work from the table. i.e: i already have an UI for generating laterjs recurrencies, now, i would need to write some new code for supporting this package instead of using just what I have. Also momentjs makes converting times between timezones just a line of code.

9- yes :D it could be really handy for development purposes too

Or just a handy log for jobs running in a production environment :)

10- This is a tricky one because I am not sure if jobs should have statuses. I'm thinking of jobs in this context as method calls that are being called but are running at a delay. They either return a result or fail and retry.

Right now you already have two "states": already run and not run yet. I think the running right now would be handy. The before/after hook idea is great and can be used to implement that running now state. I was already thinking about implementing it with the collection-hooks package.

[msavin]

7- hmm yeah that could be trivial to implement

8- Moment.js may be an exception here, it's a great library and timezones are a crazy thing :D

10- Regarding adding a "running" state - I am still not sold on the idea. The reason is, we add latency to the job, plus I have trust issues with MongoDB. If something goes wrong with the database or the server, the job could get stuck in the "running" state. Handling that adds a lot of complexity compared to the two state method.

[msavin]

Although, if we do add some kind of "running" state, we could start running multiple jobs at once. Once the database update is complete, it can start the next job.

This also paves the way to run jobs on multiple servers, but the problem there would be, preventing jobs from running twice in case one server grabs the job while the second one is claiming it. Maybe there's something that could be done with Mongo commands to prevent this though.

[luixal]

8- Moment.js may be an exception here, it's a great library and timezones are a crazy thing :D

Totally insane thing. Once I found momentjs, never looked back xD

This also paves the way to run jobs on multiple servers, but the problem there would be, preventing jobs from running twice in case one server grabs the job while the second one is claiming it. Maybe there's something that could be done with Mongo commands to prevent this though.

It's a important thing. I have had issues with mongo when inserting a bunch of documents consecutively (even if you pass a callback to insert, it gets called once the _id has been generated, not when the inserting has been totally completed. Not sure if the same thing applies to update).

Maybe using a singleton approach for the job grabber? That would work when running jobs on a single instance, but would be a problem when running on multiple instances/servers (I've faced those problem when using cfs and job-collection packages. Found no solution but making them run only on one instance) --> this is a real problem when you use the cluster package in your app.

Another solutions would be to set a jobRunnerInstanceId when adding the job and then make each instance query for it's own instanceId but... won't leverage CPU loads.

[msavin]

Going to close this one out due to its age and the recent changes to the package. With that said, I'm always happy to discuss new features, so if you have something in mind, please open up a new ticket.