ali-essam / wtsqs Goto Github PK

Hi Ali

when we have maxConcurrency > 1
i noticed that the worker process jobs with the same group id concurrently
but to process them in order they need be processed sequentially instead

i think that happen because when the worker receive multiple messages as a batch
the code uses promise.all to process all without considering messages with the same group id

Utility function that facilitates routing different jobs to different handlers.

For example

const handler = new JobRouter({
  key: 'type',
  handlers: {
    'type_1': handler1Foo,
    'type_2': handler2Foo,
  },
  defaultHandler: defaultHandlerFoo
})

WTSQS should not fail if accessKeyId and secretAccessKey were not supplied as other authentication methods could be in place

When fetching a message from SQS, visibilityTimeout (Duration (in seconds) that the received messages are hidden from subsequent retrieve requests.) is determined at request time

In some cases, the timeout might not be clear during the request and the processing job might need more time to process the message. Currently, if the visibilityTimeout is less than the job processing time, that will cause the message to be back in the queue and visible to other workers to fetch again, causing double processing of the same message.

It's possible through SQS API to extend the timeout after fetching the message through ChangeMessageVisibility.

TODO: Implement this functionality in WTSQS.