claudiajs / claudia Goto Github PK

Maybe Claudia could reuse Apex way of shiming to support Node v6 in AWS Lambda?
I started an issue there too and still hesitating to use Claudio or Apex for that (context: we have a NodeJS v6 project to try on AWS Lambda)

I sometimes get mass emails from repos I starred, but rarely two. Please stop with the spam.

I've been asked to do some fixes on a project I developed with Claudia. I've tried to deploy them and I get the above error:

> claudia update
> updating REST API
> [TypeError: Cannot redefine property: $map]

I've had a look in the source (both Claudia and my own app) and can't find anything that this might reference. I've tried to go back to 1.2.0, the last version I used to deploy successfully with no luck.

Locally I'm using node v4.4.1, although remove version is using node 4.3 (but I would not expect any major compatibility).

As an aside, one thing that might be useful is when there is an error to do the following:

console.log(error.stack)

As the stack is being swallowed up and only the error.message is being shown.

Following the tutorial at https://claudiajs.com/tutorials/hello-world-api-gateway.html

I've got this error message:

{"errorMessage":"Cannot find module 'claudia-api-builder'","errorType":"Error","stackTrace":["Function.Module._load (module.js:276:25)","Module.require (module.js:353:17)","require (internal/module.js:12:17)","Object.<anonymous> (/var/task/app.js:1:80)","Module._compile (module.js:409:26)","Object.Module._extensions..js (module.js:416:10)","Module.load (module.js:343:32)","Function.Module._load (module.js:300:12)","Module.require (module.js:353:17)"]}

I've checked the zip folder generated and it contains the module. Any clue on what can be happening?

Hi, previously I was able to use dotenv with Claudia by adding my .env file to the files property of the package.json. However, now it is ignored. I imagine this is because the file is in .gitignore. Can you please clarify if this is a desired behaviour?

I would think files property should dominate over .gitignore

Hi,

I created a lambda function + api gateway bound to it.
Here is the json file created:

{
  "lambda": {
    "role": "foobar_execution",
    "name": "foobar",
    "region": "us-east-1"
  },
  "api": {
    "id": "qmmx2fe229",
    "module": "src/server.js"
  }
}

When I run claudia update to push some new code, I get this:

{ [XMLParserError: Non-whitespace before first tag.
Line: 0
Column: 1
Char: {]
  cause:
   { [XMLParserError: Non-whitespace before first tag.
   Line: 0
   Column: 1
   Char: {]
     message: 'Non-whitespace before first tag.\nLine: 0\nColumn: 1\nChar: {',
     code: 'XMLParserError',
     retryable: true,
     time: Fri Feb 26 2016 14:46:02 GMT+0100 (CET),
     statusCode: 500 },
  isOperational: true,
  code: 'XMLParserError',
  retryable: true,
  time: Fri Feb 26 2016 14:46:02 GMT+0100 (CET),
  statusCode: 500 }

Am I doing something wrong?

Claudia behind the scene performs many steps to accomplish her task (e.g. package module, npm install, create role, policy, API Gateway, Lambda, etc, etc.) Since many of these steps involves network connections, which can be time consuming. A typical Claudia operation can take from a few seconds to over 1 minute to complete.

As a user, it will be great if Claudia can keep me inform as she is doing the various tasks. I'm thinking of making use of the elegant terminal spinner module, Ora, by Sindre Sorhus https://github.com/sindresorhus/ora to do this.

I have a draft implementation at:
aidanbon@0db54b7

Please let me know what you think, should I proceed with a PR. Thanks.

I am creating a package with claudia create.
My project have files separated in folders so I add this folders to the package.json files entry.
Claudia creates the package with the files but they are at root level instead of being in their specific folders.

• Expected behaviour: keep directory hierarchy
• What actually happens: files inside a directory are moved at root level instead of being in a directory
• Steps to reproduce the problem:
  create a directory toto with a file a.js
  add a files entry into package.json and add the directory
  the package created by claudia has the file a.js but it is at project root level and not in toto directory.

When I run the tests, I see a large number of failures (47/122), almost all of which are due to:

Failed: 1 validation error detected: Value 'arn:aws:execute-api:us-east-1:AIDAI2YYF3TYCI65FI3IM:rfj3hl9en0/*/*/*' at 'sourceArn' failed to satisfy constraint: Member must satisfy regular expression pattern: arn:aws:([a-zA-Z0-9\-])+:([a-z]{2}-[a-z]+-\d{1})?:(\d{12})?:(.*)

Indeed - the code appears to be using the Access Key ID (GetUser().User.UserId) instead of the Account ID in forming the sourceARN.

I see similar failures for any of the example-projects other than hello-world.

Is there some more specific requirement on how AWS credentials are provided to Claudia for these tests/examples to work?

In order to make use of custom authenticators, one needs access to custom $context variables for things like user id. It would be great if a custom template could be provided to claudia.

I have a service that contains multiple lambdas. For example:

example-service

• lambda1.sj
• lambda2.js
• common.js

They both share common code hence why they are zipped up in the same service. Is there a way to specify multiple lambda entires within the claudia.json. A feature like this would be very useful to support multiple lambdas within the same folder.

{
"lambda1": {
"role": "role",
"name": "lambda1",
"region": "us-west-2"
},
"lambda2": {
"role": "role",
"name": "lambda2",
"region": "us-west-2"
}
}

With the following, which sits inside a folder api/app.js, in the package.json file for the project I have "files": "api/*.js

/*global require, module*/

var APIBuilder = require('claudia-api-builder');
var api = new APIBuilder();
var Promise = require('bluebird');
var pg = require('pg');

module.exports = api;

api.get('/app', function() {

  return new Promise(function(resolve, reject) {

    var client = new pg.Client('<connection string>');

    client.connect(function(error) {
      if (error) {
        return reject(error);
      }

      client.query('SELECT * FROM apps', function(error, results) {

        if (error) {
          return reject(error);
        }

        var output = results.map(function(result) {
          return {
              // Map return in here
          }
        });

        return resolve(output);
      });
    })
  });
});

I get the error in the title after running:

claudia create --name appname --region eu-west-1 --api-module app

The full output is:

server listening on port: 4001
No apiConfig defined on module 'app'. Are you missing a module.exports?

No claudia.json file is created either

I've tried to follow the example as close as possible, although this code lives in an existing project that is a static app, and rather than have a separate repo I am keeping the lamda functions within the api folder to deploy.

Any ideas?

Hi guys,
Thank you for this awesome project.
It seems that claudia not support the dynamodb event.
Is this feature in the ToDO list or not.
Thanks btw.

Claudia is awesome. Any chance you will be adding support for Kinesis event triggers?

I have the use case where I need an API Gateway to put records onto a Kinesis stream, and a Lambda function is triggered by the Kinesis stream. Do you think Claudia can/will support something like that, or do you think I will have to do a little hacking to connect the dots myself?

Deploying the web-api example project currently fails with:

{ [BadRequestException: Resource's path part only allow a-zA-Z0-9._- and curly braces at the beginning and the end.]
  cause:
   { [BadRequestException: Resource's path part only allow a-zA-Z0-9._- and curly braces at the beginning and the end.]
     message: 'Resource\'s path part only allow a-zA-Z0-9._- and curly braces at the beginning and the end.',
     code: 'BadRequestException',
     time: Fri Feb 26 2016 12:40:14 GMT-0800 (PST),
     requestId: '23ad86d1-dcc9-11e5-afc9-35deb4d0a175',
     statusCode: 400,
     retryable: false,
     retryDelay: 78.57155506499112 },
  isOperational: true,
  code: 'BadRequestException',
  time: Fri Feb 26 2016 12:40:14 GMT-0800 (PST),
  requestId: '23ad86d1-dcc9-11e5-afc9-35deb4d0a175',
  statusCode: 400,
  retryable: false,
  retryDelay: 78.57155506499112 }

This appears to be due to the parameterized route in that example:

api.get('/people/{name}', function (request) {
    'use strict';
    return 'You wanted ' + request.pathParams.name;
});

Claudia appears to currently only support hosting API's defined through the claudia-api-builder router API.

I wonder if it would be possible to also support hosting APIs defined by other native Node.js routers, and in particular, popular ones like Express.

I'm imagining something like this - where only the first and last lines are Claudia-specific (and could easily be factored out further if needed).

var ApiBuilder = require('claudia-api-builder')
var express = require('express');
var app = express();

app.get('/', function (req, res) {
  res.send('Hello World!');
});

module.exports = new ApiBuilder(app);

There's two components to this:

1. Figuring out how to create the API Gateway routes needed to pass through to the Express API.
2. Building Node IncomingMessage and ServerResponse objects with sufficiently correct information to invoke the express app as a requestListener.

For issue (1), it's likely not possible to extract the routing information accurately enough (and even if you could, it would be framework-specific). Instead, you could maybe do something like what Zappa does and create general purpose API gateway routes that pass through all calls through to a single router. See https://github.com/Miserlou/Zappa/blob/master/zappa/zappa.py#L439.

For issue (2), Zappa again does something similar in creating a WSGI request object out of the request that comes through API gateway. See https://github.com/Miserlou/Zappa/blob/master/zappa/wsgi.py. I can't think of any reason this would not in theory be possible for Node's IncomingMessage/ServerResponse objects.

Thoughts?

Hello,

Is it possible to put authentication in front of the API ?
I would like to use oauth maybe with a lib like passport (http://passportjs.org/docs/overview) but maybe it's not so easy. So if you have any alternative roads :)

There is a 'boilerplate' repo for doing it directly with Serverless. Can it be an inspiration ?

Thanks for your answer !

BTW so far, Claudia seems a great tool to use AWS lambda

• Expected behaviour:

aws-sdk not to be included in the package if specified as an optional dependency, with --no-optional-dependencies in update options

• What actually happens:

it gets included

I operate several AWS accounts. In my ~/.aws/credentials file I only have one entry, which are credentials that are only valid for one of them. When I first ran the claudia create command, the endpoint was created under a different account than the one the credentials are valid for. When i ran the command for a different project with the credentials unchanged everything seems fine, but the lambda function is not created under the same account. Actually I cannot find the lambda function that Claudia claims to have created under any of my accounts.

I use command claudia create --region us-east-1 --api-module lambda --source ./build/src, it throws error cannot find module build/src/lambda. The reason is create command use require(path.join(source, apiModule)) to find module, which will discard '.'.

rebuildWebApi retrieving ownerId doesn't work for assumed roles, needs to use sts.getCallerIdentity() instead

HI gang, first up thanks for making claudiajs because I freaking love it. I love it so much I've made a web api with 13 endpoints, and counting. When I add a 14th endpoint, "claudia update" gives me this error, which I imagine comes from making lots of API Gateway calls during update. Anyone else run into this?

{ [TooManyRequestsException: Too Many Requests]
cause:
{ [TooManyRequestsException: Too Many Requests]
message: 'Too Many Requests',
code: 'TooManyRequestsException',
time: Wed Mar 02 2016 15:31:50 GMT-0500 (EST),
requestId: 'caa4ac2b-e0b5-11e5-a1ee-672085005625',
statusCode: 429,
retryable: false,
retryDelay: 34.01342041324824 },
isOperational: true,
code: 'TooManyRequestsException',
time: Wed Mar 02 2016 15:31:50 GMT-0500 (EST),
requestId: 'caa4ac2b-e0b5-11e5-a1ee-672085005625',
statusCode: 429,
retryable: false,
retryDelay: 34.01342041324824 }

When using a windows machine the command to create a lambda scheduled event does not work. If the following is run:

claudia add-scheduled-event --event test.json --name website-alerts --schedule 'rate(5 minutes)'

The following error is displayed as windows does not deal with the curly brackets:

' was unexpected at this time.

Using ^ escape characters works and windows validates the command but then the AWS service returns with an error that the schedule expression is invalid.

It would be ideal if the --schedule option could be read from the claudia.json file at runtime. Therefore windows users would not need to pass the brackets on the command line.

To integrate claudia into a project (which already has some tasks like Babel for processing files) it would be useful to access the different functions of the CLI tool directly in node. What do you think about exporting the different calls from the module instead of just the shell runtime ?

import { create as createLambda } from 'claudia'

createLambda(/** params **/).then(() => { .... })

Since these setup helpers contain the actual logic for working with AWS lambda & API gateway, the commandline interface could either be a different repository or check the process for its callee.

First, thanks for this awesome frictionless serverless framework. It really lower the barrier for Node.js into Lambda.

Is supporting ES6 (e.g. via Babel transpilation) in the roadmap?

IAM policies can't have dots in the name and S3 policy generator uses the bucket name (which can have dots). Policy names need to be sanitized:

ValidationException: 1 validation error detected: Value 'delme.scribe.images.com-access' at 'statementId' failed to satisfy constraint: Member must satisfy regular expression pattern: ([a-zA-Z0-9-_]+)
    at Object.extractError (/usr/local/lib/node_modules/claudia/node_modules/aws-sdk/lib/protocol/json.js:43:27)

• Expected behaviour:

It should be possible to use claudiajs (and caudia-api-builder) to deploy lambdas that expects the raw body of a POST request in a parameter.

• Use case:

An use case for this is the signature verification on a Facebook Messenger Bot for example, that sends a signature for the raw body + a secret in the header (see https://developers.facebook.com/docs/messenger-platform/webhook-reference#security )

• What actually happens:

The Body Mapping Template used by claudia is the one at https://github.com/claudiajs/claudia/blob/master/json-templates/apigw-params.txt that does not include the raw body, and the CLI don't have an option to override that template with a different one.

• Workaround

Either manually add the patch on the body mapping template at AWS website:

...
  "rawBody": "$util.escapeJavaScript($input.body).replace("\'", "'")",
...

Or patch the apigw-params.txt manually.

I'm trying to send back detailed error responses as JSON from an API Builder method but it won't return JSON representation of my result. Here's a sample API project that demonstrates the problem:

"use strict";

var ApiBuilder = require("claudia-api-builder");
var AWS = require("aws-sdk");

var api = new ApiBuilder();
module.exports = api;

api.post("/dummy", function (request) {
  if (request.body.foo === "bar") {
    return {status: "OK"};
  } else {
    throw {error: "Invalid request"};
  }
});

If you then call that API like this:

POST /latest/dummy
Request Headers & Body
Accept: */*
Accept-Encoding: gzip, deflate
Content-Type: application/json
Accept-Language: en-us

{
  "foo": "baz"
}

You get the following response:

Response Headers
HTTP/1.1 500 Internal Server Error

Connection: keep-alive
Content-Type: application/json
X-Cache: Error from cloudfront
x-amzn-RequestId: 04957e7b-28d5-11e6-b1c5-991fd1d7258a
Via: 1.1 dc7c4fb5024ff022cad1642ec506a6e8.cloudfront.net (CloudFront)
Date: Thu, 02 Jun 2016 15:16:45 GMT
Access-Control-Allow-Headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key
Content-Length: 34
X-Amz-Cf-Id: 1Y4Tb4lS7W6zDBoJkqOetlnPkKByi-R5WXeHC8fwxkbZKWGtTIGqXw==
Access-Control-Allow-Origin: *

Response Body Display Options 
{
    "errorMessage": "[object Object]"
}

I would expect to see my error there instead of the "errorMessage": "[object Object]". I've tried explicitly setting the {error: {contentType: "application/json"}} when I define the method (although I think that should be the default), but I get the same result.

Do I need to build a specific error object or is this a problem in Claudia? I couldn't find anything in the docs or example projects that demonstrated returning a complete error object.

• Expected behaviour:

When creating or updating an API method in API Gateway, the requestParameters object will be populated with a correct list of parameters. This is required if you use the gateway cache feature in order to allow cacheKeyParameters to be set on the Method Integration.

It may also be worth populating cacheKeyParameters with all path-based request parameters by default, but that is a feature request, not a bug.

• What actually happens:

The object is omitted.

• Steps to reproduce the problem:

Deploy the web-api sample, then query the API Gateway GetMethod endpoint for anything with a dynamic path parameter. requestParameters is expected to have "requestParameters": {"method.request.path.name": true}, but is blank.

https://aws.amazon.com/about-aws/whats-new/2016/09/aws-lambda-simplifies-amazon-api-gateway-integration/

Might this simplify the deployment and code base?

It'd be great if we could run Claudia routing module as server on local environment.

This would allow us to run integration tests before pushing changes to AWS.

Claudia should expose the ability to specify the memory size of the Lambda instance to override the default 128MB. This would allow users to customize the project for more endpoints that experience higher API traffic with an optional lambda.memory integer in the claudia.json configuration.

I appreciate the comprehensive list of options which are passed at creation time, example claudia create --timeout 5. Would be lovely if the same options could also be updated via claudia update --timeout 10 making things fairly easy.

Any ideas on testing the simple API examples locally?

Currently, if your lambda uses aws-sdk, and many do, it will end up in the pushed code, even tho the lambda workers have it pre-installed.

Currently the only option I came up with was to make it a devDependency so that it isn't included, but this generally will fail the validation step that require's your handler. So then you need to hack it as an optional dependency or somesuch.

Given this tool is lambda specific it seems reasonable to have an option that hunts down and removes node-modules/aws-sdk after validating and before zipping.

If I create a PR for this, is this something that you are interested in? Is it best to make it an option of the update command?

I ran:

claudia create --name storingInDynamo --region us-west-2 --api-module index

but forgot the files property. That trigger an excellent error message: package.json does not contain the files property

Great - but when I fixed that and ran the create-command again:

{ [EntityAlreadyExists: Role with name storingInDynamo-executor already exists.]
  cause: 
   { [EntityAlreadyExists: Role with name storingInDynamo-executor already exists.]
     message: 'Role with name storingInDynamo-executor already exists.',
     code: 'EntityAlreadyExists',
     time: Fri Feb 26 2016 14:24:46 GMT+0100 (CET),
     requestId: '4df7b6fb-dc8c-11e5-9b40-35d169b930d8',
     statusCode: 409,
     retryable: false,
     retryDelay: 72.52002414315939 },
  isOperational: true,
  code: 'EntityAlreadyExists',
  time: Fri Feb 26 2016 14:24:46 GMT+0100 (CET),
  requestId: '4df7b6fb-dc8c-11e5-9b40-35d169b930d8',
  statusCode: 409,
  retryable: false,
  retryDelay: 72.52002414315939 }

And now I don't know how to fix this, except updating the name of the function (--name)

Please use GitHub issues only to report bugs. To ask a general question or request assistance/support, please use the Claudia.js Gitter Chat instead.

To report a bug or a problem, please fill in the sections below. The more you provide, the better we'll be able to help.

• Expected behaviour: API Gateway endpoints defined by API Builder can be set to Authorization Type 'AWS_IAM' or 'NONE'
• What actually happens: They are hardcoded to 'NONE'
• Link to a minimal, executable project that demonstrates the problem: Project is proprietary, you can look directly at the Claudia code to see where there is a TODO to fix this bug: https://github.com/claudiajs/claudia/search?utf8=%E2%9C%93&q=authorizationtype
• Steps to install the project:
• Steps to reproduce the problem:

Recently Lambda now supports running in a given VPC.

https://aws.amazon.com/blogs/aws/new-access-resources-in-a-vpc-from-your-lambda-functions/

Are there plans to support this in Claudia?

I currently use a custom lambda function which handles OPTIONS requests so I can allow or disallow arbitrary domains for CORS. Is there a way I can do this with Claudia?

we have a naming convention for our lambda's so would like to have a different name for the lambda and the api. Is that currently possible?

As per https://aws.amazon.com/about-aws/whats-new/2016/04/aws-lambda-supports-node-js-4-3

Will Claudia support this out the box in the current version, or does it need to add support?

Hi,

I have a package locally with i've used by npm link, and linked it to my claudia package.
when i do claudia update

i get the following:
Error: Cannot find module 'intelligodb/dto/search.js' at Function.Module._resolveFilename (module.js:325:15) at Function.Module._load (module.js:276:25) at Module.require (module.js:353:17) at require (internal/module.js:12:17)

But when i browse through the node_modules
i see this:
07/05/2016 01:54 PM intelligodb [C:\Users\ran\AppData\Roaming\npm\node_modules\intelligodb]

so what the way to use a local package?
Thanks.

Easy question: I'm trying to get the current RequestId that corresponds to CloudWatch logging.

Cheers,
Daaron

Any reason you're using shelljs over fs?

For instance, you're using shelljs.ls(...) instead of fs.readdirSync(...)? Is there a reason for that or merely an oversight? I was working on a PR to resolve this, but I wanted to make sure I wasn't missing something first.

Hi,

When i create a claudia project,

1. I wish to change stage name to /v1 - instead of /latest
2. i wish to determine (since i have custom url which is api.XXX.com) the XXX. before - how?
   and how to api map it?

• Expected behaviour:
  It should create and deploy both the Api Gateway endpoint and the Lambda function
• What actually happens:
  I get an AccessDenied error due to the restrictions our operations operations team imposes. Here is the output

initialising IAM role   iam.createRole  RoleName=claudia-hello-world-executor
{ [AccessDenied: User: arn:aws:iam::XXXXXXXXX:user/francisco.calle is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::XXXXXXXXX:role/claudia-hello-world-executor]
  cause:
   { [AccessDenied: User: arn:aws:iam::XXXXXXXXX:user/francisco.calle is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::XXXXXXXXX:role/claudia-hello-world-executor]
     message: 'User: arn:aws:iam::XXXXXXXXX:user/francisco.calle is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::XXXXXXXXX:role/claudia-hello-world-executor',
     code: 'AccessDenied',
     time: Thu Sep 15 2016 17:41:22 GMT+0200 (CEST),
     requestId: 'da51dd6d-7b5a-11e6-acfe-07a6295c8f0a',
     statusCode: 403,
     retryable: false,
     retryDelay: 78.65617338102311 },
  isOperational: true,
  code: 'AccessDenied',
  time: Thu Sep 15 2016 17:41:22 GMT+0200 (CEST),
  requestId: 'da51dd6d-7b5a-11e6-acfe-07a6295c8f0a',
  statusCode: 403,
  retryable: false,
  retryDelay: 78.65617338102311 }

• Link to a minimal, executable project that demonstrates the problem: No need, I'm just using the hello world example
• Steps to reproduce the problem:
  Execute claudia --role-arn arn:aws:iam::XXXXXXXXX:role/dev-project-delegated-developer --role-arn-name corp-session-default --profile corp-session-default create --region us-east-1 --api-module index
• Extra information:
  I've made a repo based on another project to deploy a lambda function that illustrates how we have to make the deployments https://github.com/beeva-franciscocalle/beeva-aws-node-lambda

I'd be great if we could configure custom authorizer.

BTW, this project is awesome!

I know that claudia can handle the deployment of different versions of code, but what is the best practice for deploying a production or a development version of the app that needs to, say, connect to a different backend service depending on which environment it is running in? A typical 12factor solution might be to use environment variables, but this really isn't supported in Lambda. How would you recommend approaching this?

EDIT: I know that API Gateway has stage variables, but it doesn't seem like Lambda has the same thing. claudiajs/claudia-api-builder#9