bbc / sqs-producer Goto Github PK

Describe the bug
Remove the warning moving await from mocha.opts.

DeprecationWarning: Configuration via mocha.opts is DEPRECATED and will be removed from a future version of Mocha. Use RC files or package.json instead.

Describe the bug
In the docs, it says that we can pass accessKeyId and secretAccessKey to the Producer.create method, but we can only pass an SQS object, the README need an update

screenshots

id should be optional for sending single message.

The problem
There is code to validate that queueUrl isn't undefined, it adds extra logic, I think it's better if this logic is directly in the ProducerOptions type
Suggested solution
Change ProducerOptions type, set queueUrl as a string instead of an undefined | string

Describe the bug

Env variable AWS_REGION is not used during producer instantiation

Your minimal, reproducible example

https://codesandbox.io/s/frosty-margulis-vrh93k?file=/src/index.js

Steps to reproduce

// those code doesn't generate the same result
Producer.create({
queueUrl: "myQueueUrl"
})

Producer.create({
queueUrl: "myQueueUrl",
region: process.env.AWS_REGION
})

Expected behavior

process.env.AWS_REGION should be used during creation

How often does this bug happen?

Every time

Screenshots or Videos

No response

Platform

all

Package version

3.1.3

AWS SDK version

No response

Additional context

No response

Question
version 1.6.1 types for .send were

send(messages: string | string[] | ProducerMessage | ProducerMessage[], cb: ProducerCallback<void>): void;

this has been changed to

send(messages: string | string[]): Promise<SendMessageBatchResultEntryList>;

does this mean to old format of

{
  id: string;
  body: string;
  messageAttributes?: { [key: string]: ProducerMessageAttribute };
  delaySeconds?: number;
  groupId?: string;
  deduplicationId?: string;
}

is no longer supported or is it just missing types?

As per contribution guide rising this important question before submitting a 1-line PR.

I found that ts-node-register is a production dependency of the module. ts-node-register has few unnecessary subdependencies.

I would assume including this was an incident. The ts-node-register is not used anywhere.

Should I submit a PR to remove this line from package.json?

1. Looking at the code I believe that the return type of the .send() method is wrong.

async send(messages: string | string[]): Promise<string[]> {

The function always return undefined. I believe the return type should be void instead of string[].

Should I submit a 1-line PR to fix this?

2. There were a number of requests from several people to return AWS raw response(s) form this function. The currently returned undefined is somewhat wasteful. It would be great to take the result variables, concat to an array, and return from the .send() method:

const result = await this.sqs.sendMessageBatch(params).promise();
results.push(result);

...

return results;

This is a non-breaking enhancement people really need.

Should I submit a PR?

Describe the bug

Install globally. Link it. The npm link fails.

Your minimal, reproducible example

npm i -g sqs-producer && npm link sqs-producer

Steps to reproduce

> npm i -g sqs-producer
> npm link sqs-producer
npm ERR! code 127
npm ERR! path ~/.nvm/versions/node/v16.18.1/lib/node_modules/sqs-producer
npm ERR! command failed
npm ERR! command sh -c -- npm run build
npm ERR! > [email protected] build
npm ERR! > npm run clean && tsc
npm ERR! 
npm ERR! 
npm ERR! > [email protected] clean
npm ERR! > rm -fr dist/*
npm ERR! sh: tsc: command not found

Expected behavior

No errors. sqs-producer installed globally and working fine.

How often does this bug happen?

Every time

Screenshots or Videos

No response

Platform

All

Package version

3.0, 3.1

AWS SDK version

No response

Additional context

Offtopic: This bug template is a bit annoying. It should not mandate for "minimal reproducible example" as a URL

Replace the prototype with a class. The public/private methods and state should be maintained.

Hello maintainers.

Looking at the Milestone "v2.0" it was recently reached: https://github.com/bbc/sqs-producer/milestones

Is there anything else you wanted to add to the milestone before publishing v2.0?
If not, how else can I help publishing v2.0?

please export type ProducerOptions

Would you consider a PR to add successful results to the callback for sending messages?

Hello!

I'm trying to send a message to a FIFO queues, which require de groupId and deduplicationId, but the package is requiring me to enter an ID which is never used.

My code:

const producer = Producer.create({
  queueUrl: 'https://sqs.us-east-1.amazonaws.com/xxxxxxxx/yyyyyyyyy.fifo',
  region: 'us-east-1'
})

const content = JSON.stringify({zzzzzzzz: 1})

const message = {
  body: content,
  id: 'woot',
  groupId: 'group1234',
  deduplicationId: shajs('sha256').update(content).digest('hex')
}

producer.send(message, err => {
  if (err) console.log(err)
})

Error: Error: Object messages must have 'id' and 'body' props
at entryFromObject (/lib/producer.js:47:11)

Greenkeeper keeps dependencies updated automatically.

It's free for open source

Migrate project to Typescript. Project configuration, style, etc, should be consistent with sqs-consumer

Currently upgrading to this version or above causes issues with the tests, we should see how best to resolve those issues to get upgraded.

Could you please publish version 1.4.0 to npm please ?

I need the Producer export from PR 15 ;)

The problem
To pass complex payloads in Message body it requires to JSON.stringify it first

            this.sqsProducer.send(
                {
                    body: JSON.stringify({
                        type: "MESSAGE_TYPE",
                        payload: {},
                    }),
                    id: "MESSAGE_ID",
                },
                (err) => {
                    if (err) { }
                },
            );

Suggested solution
Make it work with plain js objects in message body

            this.sqsProducer.send(
                {
                    body: {
                        type: "MESSAGE_TYPE",
                        payload: {},
                    },
                    id: "MESSAGE_ID",
                },
                (err) => {
                    if (err) { }
                },
            );

Extend Message.body to be of union type

export interface Message {
    id: string;
    body: string | Object;
    groupId?: string;
    deduplicationId?: string;
    delaySeconds?: number;
    messageAttributes?: SQS.MessageBodyAttributeMap;
}

Add JSON.stringify to entry

    const entry: SendMessageBatchRequestEntry = {
        Id: message.id,
        MessageBody: typeof message.body === 'string' ? message.body : JSON.stringify(message.body)
    };

Alternatives considered

Additional context

I would like to see some example using dead letter queues please.

I trying to use this package in microservice context.
I need to send a message and wait for the result, for example, send a filter and get the list of objects.
How can I wait to get the message results?

Thanks!

Describe the bug
As referenced elsewhere the types that the send method supports in typescript projects are string and string[], and that is how my team is attempting to use the library. However, when posting a string, the Id attribute is set as the string itself which means that the library can only post stringified JSON events of at most 80 characters. This is an unrealistic limitation. The object version is valid as a workaround, however we work exclusively in typescript and the compile time errors generated when attempting to utilize this api aren't worth the effort to overcome vs. using the AWS SDK directly.

To Reproduce
Steps to reproduce the behavior:

1. Import the library in to a typescript project
2. Attempt to send an event > 80 characters in length or with characters not confirming to /[A-Za-z0-9_\-]/
3. Receive this error: A batch entry id can only contain alphanumeric characters, hyphens and underscores. It can be at most 80 letters long.

Expected behavior
The types accepted are robust enough to reflect that objects can be sent instead of just strings or string arrays and that those objects have keys which are optional. When sending strings there is no limitation that the string be less than 80 characters in length, or consist only of whitelisted characters.

Recommendations
Enable strict mode in tsconfig.json. These issues would most likely have been caught prior to publication by the typescript compiler.

I was recently getting a "Failed to send message: 1" with no info.

Please consider adding debug to add optional debugging.

Adding a quick console log to the result from AWS showed that I was adding an extra param not expected, but sqs-producer would not forward this error and no other info was available.

Describe the bug
aws-sdk version is locked to version 2.677.0. This version has several security vulnerabilities. Can you unlock it in order to use a compatible version added to the project where we add this package as dependency?

Hi,

it would be very useful to have the MessageId generated by AWS when the message gets pushed. Is there any way to obtain this information in the callback notification?

Thanks!

I find it a bit weird that the create() require both a region and a complete queue url, and not relying on the getQueueUrl?

Is there a specific reason for that?

I have some typing issues on the send method of the producer and I see the bug has been resolved on master 2 months ago. When's the next release planned?

Cheers!

var uuid = require('uuid/v1');
var Producer = require('sqs-producer');
var AWS = require('aws-sdk');

exports.handler = (event, context, callback) => {

var producer = Producer.create({
  queueUrl: QUEUE_URL,
  region: 'eu-west-1'
});

var dataToSqsQueue = JSON.stringify(event);
console.log(dataToSqsQueue);

producer.send({
  id:uuid(),
  body: dataToSqsQueue,
  groupId: uuid(),
  deduplicationId: uuid() // typically a hash of the message body 
}, function(err) {
  if (err) console.log(err);
});

};

Have it all set up and suddenly got a cb is not a function TypeError. Please see code below

`E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:31
throw err;
^

TypeError: cb is not a function
at Producer._sendBatch (E:\Platform\OBSIDIAN\node_modules\sqs-producer\lib\producer.js:133:5)
at Producer.send (E:\Platform\OBSIDIAN\node_modules\sqs-producer\lib\producer.js:158:8)
at Response.sqs.getQueueUrl (E:\Platform\OBSIDIAN\application\routes\upload.js:75:34)
at Request. (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:364:18)
at Request.callListeners (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\sequential_executor.js:105:20)
at Request.emit (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\sequential_executor.js:77:10)
at Request.emit (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\state_machine.js:14:12)
at E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request. (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:38:9)
at Request. (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:685:12)
at Request.callListeners (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\sequential_executor.js:115:18)
at Request.emit (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\sequential_executor.js:77:10)
at Request.emit (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\state_machine.js:14:12)
at E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request. (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:38:9)
at Request. (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\request.js:685:12)
at Request.callListeners (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\sequential_executor.js:115:18)
at Request.emit (E:\Platform\OBSIDIAN\node_modules\aws-sdk\lib\sequential_executor.js:77:10)`

I wanted to Promisify the Producer class using Bluebird and realized that it wasn't even accessible to me to do so. Please expose class itself instead of just the create function.

Remove the note on our readme that points to V2 support from September 8th, 2024

Please join the discussion (#103) if you would like to feedback on this decision.

Node 16 will be at its end of life on 11th Sep 2023.

Due to this, support should be dropped for it from this package and we should move to support node 18 and node 20 only.