Unhandled Exception on offline start about serverless-plugins

[ormu5]

I realize this is an old ticket/post but it's one of the first hits to come up when searching for unhandled exceptions related to non existent queues and serverless-offline-sqs, so I thought I'd provide some more contemporary context for those who end up here, and mention a scenario that can cause the somewhat vague error (spoiler alert: if you're referencing a dead-letter-queue, make sure it is also accounted for - via lambda configuration - during local initialization):

AWS.SimpleQueueService.NonExistentQueue: AWS.SimpleQueueService.NonExistentQueue; see the SQS docs.

As mentioned in other posts, you do need to stand up a local queueing service for serverless-offline-sqs to leverage. Now-a-days, though, the plugin has an option to automatically create the queues during 'sls offline start'. Typical configuration in your serverless.yml plugins section might look like:

  serverless-offline-sqs:
    autoCreate: true  # create queue if not exists
    apiVersion: '2012-11-05'
    endpoint: http://localhost:9324
    region: ${self:provider.region}
    accessKeyId: root
    secretAccessKey: root
    skipCacheInvalidation: false

When serverless offline starts, serverless-offline-sqs scans any functions you are importing for any triggered by SQS events and attempts to create the referenced queue in the queue service you should have running in the background. If the queue already exists, it gracefully skips. If the queue does not exist it attempts creates it. As described at https://www.npmjs.com/package/serverless-offline-sqs, the plugin helpfully supports CloudFormation references/syntax.
There is a scenario where you might encounter the following error without a whole lot of context:

AWS.SimpleQueueService.NonExistentQueue: AWS.SimpleQueueService.NonExistentQueue; see the SQS docs.

In my case this occurred due to the following sequence of init steps:

• serverless-offline-sqs scans imported functions, finds CloudFormation reference to SQS queue
• serverless-offline-sqs uses referenced CloudFormation in an attempt to create a queue
• CloudFormation references a dead-letter-queue (which due to CloudFormation behavior is explicitly identified as a dependency via DependsOn, though I'm not sure / skeptical this has any bearing on local init behavior)
• no function is yet configured referencing the dead-letter queue as an event trigger, so serverless-offline-sqs never knew to stand it up
• the error above is returned

So as you are configuring pairs (nominal / dead-letter) of new queues - e.g., as CloudFormation - be sure to immediately add corresponding pairs of functions in order to initialize your new queues and seamlessly develop locally:

upsertCurrentInventory:
  name: ${self:custom.app}-upsert-current-inventory-lambda-${self:provider.stage}
  handler: functions/inventory/v1/handler_async.upsert_current_inventory
  events:
    - sqs:
        arn: !GetAtt CurrentInventoryReceiverQueueV1.Arn

upsertCurrentInventoryDlq:
  name: ${self:custom.app}-upsert-current-inventory-dlq-lambda-${self:provider.stage}
  handler: functions/inventory/v3/handler_async.upsert_current_inventory_dlq
  events:
    - sqs:
        arn: !GetAtt CurrentInventoryReceiverDlqQueueV1.Arn

[astuyve]

Hey @Redrazor, @brunocascio
I experienced this as well. For serverless-offline-sqs, you need to run your own message queue locally. It's opaquely described here: https://www.npmjs.com/package/serverless-offline-sqs#sqs

You can install ElasticMQ on its own or using Docker ElasticMQ
https://github.com/s12v/elasticmq-docker
If you already have Docker installed locally, you can run this

docker pull s12v/elasticmq
docker run -p 9324:9324 s12v/elasticmq

Then, modify your serverless.yml file to look something like this:

functions:
  dispatcher:
    handler: application/notification/actions/processEmail.dispatch
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - MyQueue
              - Arn

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: "MyQueue"

custom:
  serverless-offline-sqs:
    apiVersion: '2012-11-05'
    endpoint: http://0.0.0.0:9324

Then you need to create the queue:
aws sqs --endpoint-url http://localhost:9324 create-queue --queue-name MyQueue --profile serverless --region us-east-1"

Hope I've helped.

[brunocascio]

Thanks @astuyve !

I am running all my stack on docker, so I had some race conditions problems, all of them solved with the following entry point code:

#!/bin/bash

set -e

QUEUE_ENDPOINT="http://sqs:9324"

until curl -Is "$QUEUE_ENDPOINT"; do
  echo "$QUEUE_ENDPOINT is not reacheable... Trying again..."
  sleep 3
done

QUEUES="$QUEUE_1 $QUEUE_2 $QUEUE_3";

for QUEUE_NAME in $QUEUES
do 
    until aws sqs --region "$AWS_REGION" --endpoint-url ${QUEUE_ENDPOINT} get-queue-url --queue-name ${QUEUE_NAME}
    do
        echo "Creating queue $QUEUE_NAME"
        aws sqs --endpoint-url ${QUEUE_ENDPOINT} create-queue \
            --queue-name ${QUEUE_NAME} --region "$AWS_REGION"
    done
done

exec "$@"

docker-compose.yml

version: '3.6'

services:

  sqs:
    image: s12v/elasticmq
    ports:
      - "9324:9324"

  api:
    build: .
    image: custom/billing:development
    depends_on: 
      - sqs
    ports:
      - 8000:8000
      - "9229-9250:9229"
    volumes:
      - .:/usr/src/app:delegated
      - /usr/src/app/node_modules
    env_file: ../../.env.develop
    environment:
      API_ENDPOINT: http://apollo:4000/graphql

Dockerfile

FROM node:8

ENV DEBIAN_FRONTEND=noninteractive

WORKDIR /usr/src/app

RUN apt-get update >/dev/null \
  && apt-get install python-pip -y >/dev/null \
  && pip install awscli --user --upgrade >/dev/null \
  && rm -rf /var/lib/apt/lists/*

ENV PATH=~/.local/bin:$PATH

COPY package.json package-lock.json /usr/src/app/

RUN npm i && npm cache clean -f

COPY . /usr/src/app/

ENTRYPOINT ["bash", "start-dev.sh"]

[brunocascio]

same issue here

[thetumper]

aws sqs --endpoint-url http://localhost:9324 create-queue --queue-name MyQueue --profile serverless --region us-east-1"

@astuyve I'm running localstack along with serverless-offline, and having this issue. So, using http://localstack:4576 for my endpoint. What is the --profile arg for? I don't see it on the CLI docs for create-queue.

Edit: Figured out the --profile option:

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#cli-quick-configuration-multi-profiles

Still not finding the queue, as created on localstack.

[astuyve]

Yup, profile is essentially a no-op for offline usage - but their CLI requires it anyway.

Can you reach your endpoint using CURL? I'm not familiar with OpenStack, but there must be some way to test that the queue is running and proxying traffic effectively.

Perhaps try running an offline message queue outside of localstack to account for network issues?

[thetumper]

I can put a message on the queue with the CLI, from the shell. But serverless-offline-sqs is failing to find the queue in the event definition in my serverless.yml. The resource arn and event source arn appear to match. Wondering if serverless requires any additional permissions? I've allowed the lambda these:

              - sqs:DeleteMessage
              - sqs:GetQueueAttributes
              - sqs:GetQueueUrl
              - sqs:ReceiveMessage

[duythvn]

@thetumper have you figured out how to solve this? I am encountering the same issue.
I'm running ElasticMQ on Docker.

I can use CLI aws sqs --endpoint-url to create queues
Also, setting serverless-offline-sqs: autoCreate: true does create new queues for me
but my lambda functions keep getting "NonExistentQueue"

[thetumper]

@duythvn Yes, it's resolved for me. It was some combination of there being some issues with queue ARNs between localstack and moto, and maybe the way I was obtaining the arn for the event source configuration. Note - I'm using localstack for SQS, not ElasticMQ.

Current working solution for me was:

1. Take latest localstack and moto images
2. Use Fn::ImportValue for AWS stages, to get the queue ARN. Export it from cloudformation to make it available across stacks.
3. Use Fn::GetAtt in local.

I may not need 2 & 3 anymore, but just configure the global ARN, with variable stage portion in it. But I haven't gone back and tried yet -- there's always a more pressing issue each day. :/

[duythvn]

cool. I'll try localstack (currently running serverless framework with the offline plugin)