mogadanez / sqsd Goto Github PK

Hi,

Thanks for the project. It is very useful in time of building applications via microservices and using Docker.

But we tried to use SQSd in an application with an enabled auto scaling and encountered an issue when one SQS message is processed by two workers simultaneously.

Could you tell me, product has an ability to work in environments that scale automatically?
If yes, is there a setting of how a correct behavior, like one message is processed by one worker, could be set?

Thanks in advance.

Amazon's sqsd supports scheduled jobs via a cron.yaml spec file. It looks like this would be pretty easy to support since there are at least a couple of npm packages supporting scheduling jobs according to a cron schedule.

Have the developers considered such a feature? Just wanted to check before I possibly started working on adding it.

Can you setup an automated build on Docker Hub?

It will automatically build and publish a docker image when master branch is updated.

When sending a message to SQS, the process ends immediately. I debug the process and it ends in the following code snippet.

new sqsd(mergedParams).start() .then(()=>{ console.log("HERE :("); process.exit(0); }) .catch( err=> { console.error( err ) process.exit(1); })

the command line: sqsd -w http://localhost:80 -r us-east-1 -q https://sqs.us-east-1.amazonaws.com/XXXXXXXXXXXX/xxxxxx_queue --queue-name xxxxxx_queue --ssl-enabled false

I have a message which triggers an upload of files. Since these files can be quite large the process may take some time. During this long interval of time the same message is posted to the server causing an endless loop of trying to upload these very large files. Any suggestions in regards to de-duping so only one of these messages is being processed at once?

Hi,

Please kindly reference the original repo in the documentation.

https://github.com/mozart-analytics/sqsd

In order to keep things easy to follow, is it possible to match version of the lib and tags on github?

Btw, on npmjs.com, the url of the package is still with the old username.

Is the repo still a minimum active?

and then get new batch.
Should get new message immediate when one finished.

I'm seeing a pretty large memory leak when processing large numbers of messages. So far it's exceeded 600MB. Is this a known issue?

AWS SQS daemon sends failed messages to deadLetter queue.

How will sqsd manage the failed jobs/messages? will that be sent to deadLetter like AWS sqsd?
if yes then how should we setup deadletter queue?