The stylish Node.js middleware engine for AWS Lambda
You can read the documentation at: https://middy.js.org
Licensed under MIT License. Copyright (c) 2017-2024 Luciano Mammino, will Farrell and the Middy team.
๐ต The stylish Node.js middleware engine for AWS Lambda ๐ต
Home Page: https://middy.js.org
License: MIT License
The stylish Node.js middleware engine for AWS Lambda
You can read the documentation at: https://middy.js.org
Licensed under MIT License. Copyright (c) 2017-2024 Luciano Mammino, will Farrell and the Middy team.
Hello all,
after a PR and a private discussion i would like to propose @DavidWells from the Serverless framework to be part of the core team of this project so that he can contribute more and, most importantly, help on voting features and getting them merged.
Please add your ๐ or ๐ or ๐ . If we get 3 ๐ i will consider this to be commonly agreed. If you are against this or don't like this way of voting, please provide some rationale so that we can try to create a better framework for these kind of decisions in the future ๐
CC @acambas @dkatavic @peterjcaulfield @padraigobrien @techfort
What do you think about moving middy to a dedicated middy organisation on GitHub?
I am personally up for it and I can provide few good reasons for it:
As a downside, of course, we might lose the support from Planet 9 which might not want to keep investing (a bit of) our time in developing it.
Let me know your view and vote ๐ or ๐ to this main comment if you are in favour or not to this idea.
CC: @padraigobrien @dkatavic @acambas @peterjcaulfield @techfort @codecrunchers @augeva @daveanderson-ie
Expected implementation:
module.exports = (inputSchema, outputSchema, options) => ({
before: (handler, next) => {
// validate the input using the input schema
// if it fails throws a BadRequest using http-errors library (check json body parser for an example)
},
after: (handler, next) => {
// if there is an output schema, do the same validation logic with the response
}
})
Hey,
Typescript will complain when the tsconfig setting "noImplicitAny" is set to true.
It's because the latest release 0.10.2 does not include the middlewares.d.ts file
Workaround is adding the middlewares.d.ts file from the repo or change noImplicitAny to false
Line 3 in cc142c3
Currently, for adding error middleware we are calling error method middy.error(() ->)
. Should we rename it to onError
middy.onError(() ->)
to be more expressive?
It would be great to have a middleware that automatically adds the headers needed for enabling CORS in the response object. For example:
{
headers: {
'Access-Control-Allow-Origin': '*'
}
}
The middleware needs to be configurable with an option to specify different origins (it will default to *
).
Also, it will be good to have in place the following checks:
Coming out from a discussion in #49, it might be very useful to support async functions as middlewares.
The idea is that we might support middlewares with the following syntax:
export default function() {
return {
before: async (handler) => {
// await async stuff
return 'somevalue' // equivalent to calling next(null, 'somevalue')
}
};
}
Hi @lmammino,
I just update to the last version and notice that the file index.d.ts
is missing from the installed module.
So all the type resolution is lost.
As discussed in #82 and as described by W3C, the Content-Type
header can contain encoding information.
This also affects #75 somehow
Middy middlewares should work seamlessly in this case.
An async middleware with async
keyword or just a middleware that implements asynchronous work, doesn't seem to properly bubble up the error
object for middlewares that implement the onError
method.
This doesn't seem to be properly caught:
export default function() {
return {
before: async (handler, next) => {
throw new Error('wow');
}
};
}
Or even something like this:
export default function() {
return {
before: (handler, next) => {
setTimeout(() => {
throw new Error('wow');
}, 10);
}
};
}
Any ideas?
Since version 0.3.0
we have support for async/await
in middlewares.
We could potentially apply the same approach to the handler. Since the handler is invoked and managed by the middy wrapper, we could support different syntaxes including handlers returning promises and consequently async/await
handlers.
These are the two syntaxes I envision might be cool to support:
const handler = middy((event, context) => {
return Promise.resolve({
statusCode: 200,
body: 'hello'
})
})
and, with async/await
:
const handler = middy(async (event, context) => {
return {
statusCode: 200,
body: 'hello'
}
})
Any thoughts?
I think this is feasible by addressing the handler runtime logic here.
In a discussion in #72, it seemed that middy users would benefit from having an implementation of a warm-up middleware.
The idea of warm-up helps with addressing the cold start issue by just executing the lambda on a schedule every 5 or 10 minutes to make sure there is always at least one container bootstrapped.
When the lambda runs because of the warm-up schedule, it is not supposed to run business logic code (as that would possibly lead to undesired side effects), so it has to exit immediately.
The warmupMiddleware
should run a before
function that does basically 2 things:
Regarding point 1, we can have an option to define the check as a function that accepts the event as an argument and returns true
if the execution is caused by the warm-up schedule and false
otherwise. The default check can be compatible with serverless-plugin-warmup.
Regarding point 2, we might want to wait for #72 to be resolved so that we have an official "clean" way to stop the middleware execution and return early.
In any case, this is a potential implementation adapted from the one proposed by @DavidWells:
module.exports = (config) => {
const isWarmingUp = config.isWarmingUp || (event) => {
return event.source === 'serverless-warmup-plugin'
}
return ({
before: (handler, next) => {
if (isWarmingUp(event)) {
console.log('๐ Exiting early via warmUpMiddleware')
return handler.context.succeed() // this will probably change when #72 is closed
}
next()
}
})
}
Thoughts?
in package.json and verify that every tool associated to the old repo/url still works
I altered the httpErrorHandler
to follow the JSON spec http://jsonapi.org/examples/#error-objects-basics.
Should this change get applied across the board? Or should I just use this as my own custom middleware?
Whats the default lambda integration you guys are using? (we typically use lambda-proxy
as the serverless framework default)
Here is my altered error handler for nice errors =)
const { HttpError } = require('http-errors')
module.exports = () => ({
onError: (handler, next) => {
if (handler.error instanceof HttpError) {
// as per JSON spec http://jsonapi.org/examples/#error-objects-basics
handler.response = {
body: JSON.stringify({
errors: [{
status: handler.error.statusCode,
message: handler.error.message,
detail: handler.error.details
}]
})
}
return next()
}
return next(handler.error)
}
})
It looks like:
Before with body: handler.error.message
(no json response)
Love this freakin project! Great work ๐
Just getting started and can't seem to get validation error messages working based on the README. We're using the es7 webpack plugin + serverless-offline, but I don't think that is why.
No matter what we do, we always get a strange Unexpected 'E'
response body.
Tried debugging, going as far to the actual source and making sure all our middleware is hit in the correct order and everything, but no dice.
Anyone able to take a look at / spot a stupid mistake we're making? Thanks in advance
handler.js
const middy = require('middy')
const { urlEncodeBodyParser, validator, httpErrorHandler, cors, jsonBodyParser } = require('middy/middlewares')
const dummy = (event, context, callback) => callback(
{ statusCode: 200, body: JSON.stringify({message: 'hi'})}
)
const demo = middy( dummy )
.use(urlEncodeBodyParser()) // parses the request body when it's a JSON and converts it to an object
.use(validator({inputSchema})) // validates the input
.use(httpErrorHandler()) // handles common http errors and returns proper responses
module.exports = { demo }
serverless.yml
# NOTE: update this with your service name
service: demo
# Use the serverless-webpack plugin to transpile ES6
plugins:
- serverless-webpack
- serverless-offline
# Enable auto-packing of external modules
custom:
webpackIncludeModules: true
stage: "${opt:stage, self:provider.stage}"
provider:
name: aws
runtime: nodejs6.10
stage: dev
region: us-east-1
functions:
demo:
handler: handler.demo
events:
- http:
cors: true
path: demo/
method: POST
Hello guys. Is there a way to interrupt middleware in the middle of execution and execute callback to return something as result as in example
module.exports = (config) => {
return ({
before: (handler, next) => {
if(true) {
console.log('Lambda is warm!')
return handler.callback(null, 'Lambda is warm!')
}
next()
}
})
}
Or is there any other way to do it?
Best regards
Depends on #13
It would be great to have support to other common CORS headers like the follows:
Access-Control-Allow-Methods
Access-Control-Allow-Headers
Access-Control-Max-Age
Access-Control-Allow-Credentials
More info here
handler._middleware
is a bit unconventional name. Can we rename it to the handler.__middleware
?
Typescript grow a lot this year and is a must-have for the projects where I work to use it and is always more productive and useful if the modules that we use has the typing definition.
Following several discussions with @vladgolubev and @dkatavic (see #85 and #75) I think we should have a middleware that makes sure that the headers in the lambda-proxy integration from API Gateway are always exposed in canonical format (dash-separated-camelcasing).
The idea of this middleware would be to go over all the available headers and reconstruct the object with all the keys properly normalized.
Thoughts?
Currently generated JSDoc documentation is a bit misleading on how the API should be used.
Got this error when trying to do a build. Tried simple googling but couldnt find a suitable solution.
By first guess is that you try to copy a file to fonts
which is a folder and cant be overriden.
I think we need to copy like this instead:
node_modules/jsdoc/templates/default/static/fonts/OpenSans-Bold-webfont.eot' -> 'docs/middy/0.2.11/fonts/OpenSans-Bold-webfont.eot'
Node Version: 8.5.0
NPM Version: 5.3.0
OS: macOS Sierra
Terminal Output:
$ npm run build
> [email protected] build /Users/philipandersson/Desktop/middy
> npm run build:docs && npm run build:readme
> [email protected] build:docs /Users/philipandersson/Desktop/middy
> jsdoc --readme README.md --package package.json --destination docs src
fs.js:1919
binding.copyFile(src, dest, flags);
^
Error: EISDIR: illegal operation on a directory, copyfile '/Users/philipandersson/Desktop/middy/node_modules/jsdoc/templates/default/static/fonts/OpenSans-Bold-webfont.eot' -> 'docs/middy/0.2.11/fonts'
at Object.fs.copyFileSync (fs.js:1919:11)
at /Users/philipandersson/Desktop/middy/node_modules/jsdoc/templates/default/publish.js:516:12
at Array.forEach (<anonymous>)
at Object.exports.publish (/Users/philipandersson/Desktop/middy/node_modules/jsdoc/templates/default/publish.js:512:17)
at Object.module.exports.cli.generateDocs (/Users/philipandersson/Desktop/middy/node_modules/jsdoc/cli.js:448:35)
at Object.module.exports.cli.processParseResults (/Users/philipandersson/Desktop/middy/node_modules/jsdoc/cli.js:399:20)
at module.exports.cli.main (/Users/philipandersson/Desktop/middy/node_modules/jsdoc/cli.js:240:14)
at Object.module.exports.cli.runCommand (/Users/philipandersson/Desktop/middy/node_modules/jsdoc/cli.js:189:5)
at /Users/philipandersson/Desktop/middy/node_modules/jsdoc/jsdoc.js:105:9
at Object.<anonymous> (/Users/philipandersson/Desktop/middy/node_modules/jsdoc/jsdoc.js:106:3)
at Module._compile (module.js:624:30)
at Object.Module._extensions..js (module.js:635:10)
at Module.load (module.js:545:32)
at tryModuleLoad (module.js:508:12)
at Function.Module._load (module.js:500:3)
at Function.Module.runMain (module.js:665:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build:docs: `jsdoc --readme README.md --package package.json --destination docs src`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build:docs script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/philipandersson/.npm/_logs/2017-10-02T22_22_28_445Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `npm run build:docs && npm run build:readme`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/philipandersson/.npm/_logs/2017-10-02T22_22_28_462Z-debug.log
Hey @middyjs. Love the project. I'll keep an eye on tickets and chip in where I can help.
It appears to me that httpErrorHandler
isn't working correctly. Serverless is returning a 200 when I throw an Error. Consider the code in the middleware:
https://github.com/middyjs/middy/blob/master/src/middlewares/httpErrorHandler.js#L5
And this test code:
import HttpError from 'http-errors';
import createError from 'http-errors';
console.log( new createError.UnprocessableEntity() instanceof HttpError );
// false
console.log( new createError.UnprocessableEntity() instanceof createError.HttpError );
// true
Which to me suggests a bug.. But I may just be misusing the library. (Warning: I am not a node guy).
Cheers!
Today I added this middleware โฌ๏ธ and I discovered my Lambdas were timing out when they didn't pass validation
middy/src/middlewares/doNotWaitForEmptyEventLoop.js
Lines 1 to 3 in c9c8e70
It sets false
in after
step, meaning after handler execution. But if validation fails, handler never runs, and I get a timeout.
I changed after
to before
and it works as expected now.
Any risk making the same change in the framework itself? Was there a reason to do so specifically?
I mean it's a tiny change, it's faster to submit a PR then writing down this ๐ but who knows, was it by design?
I found, that such code:
const { doNotWaitForEmptyEventLoop } = require('middy/middlewares')
or actually importing any middleware, and any amount of it adds aditional 110 +-5 ms to code execution
My lambda without adding any middlewares running approximately 35-40 seconds. And if I want to add some middleware from you library, it becomes 110 ms seconds more. In case of high availability microservices it's not really good.
BTW middy itself without attaching any existed in library middlewares works fine.
Do you have any idea, how to solve this?
Best regards
P.S: the way you can test it:
console.time('test')
const middy = require('middy')
//const { doNotWaitForEmptyEventLoop } = require('middy/middlewares')
const test = require('tape')
const utils = require('aws-lambda-test-utils')
const mockContextCreator = utils.mockContextCreator;
let handler = function(event, context) {
context.callbackWaitsForEmptyEventLoop = false
context.succeed("succeed")
}
handler = middy(handler)
// .use(doNotWaitForEmptyEventLoop())
test('LambdaTest', function(t){
t.test("test", function(st) {
function test(result){
console.log(result)
st.end()
console.timeEnd('test')
process.exit()
};
const context = mockContextCreator({}, test); // no options and test as the callback
handler({}, context);
});
t.end();
});
This is a really great tool for getting HTTP based functions up and running on AWS.
I am curious if there is any interest/demand to support other FaaS solutions such as kubeless, fission, Azure, etc?
Not an issue as much as something we could think of to re-use some of the fantastic work that already exists in the OSS world.
We could create a middify() function that takes any express/koa middleware and makes it lambda compatible, not sure about implementation details, but it should just be a matter of translating the (req, res, next) => {}
format into (handler, next) => {}
or something.. cc @lmammino
I have a lambda running which receives XML in the body, parses it and does a few things with it.
I think it would be nice to have an xmlBodyParser
middleware (like the json one).
Im using jindw/xmldom
for parsing the xml and it work great on lambda as well.
But instead of this repo holding the dependency of xmldom
maybe require the user to install it by themselves if they ever use the xmlBodyParser? (so, kind of peerDependency)
I would love to create a PR on this if you feel its appropriate to have as a middleware.
I am currently working on a middleware that will simplify HTTP content negotiation parsing and easily allow an API gateway Lambda function to support multiple charsets, encoding (e.g. compression algorithms), languages or content-types.
My current approach is to use the module negotiator (used also by express and koa) to parse the content negotiation headers and expand the current event.
For example, the middleware will parse the incoming Accept
header and expose the resulting information in something like event.preferredMediaTypes
.
A usage example might look like the following:
const middy = require('middy')
const { httpContentNegotiation } = require('middy/middlewares')
const handler = middy((event, context, callback) => {
if (event.preferredMediaTypes[0] === 'application/json') {
return callback(null, {stastusCode: 200, body: JSON.stringify({foo: 'bar'})})
}
if (event.preferredMediaTypes[0] === 'application/xml') {
return callback(null, {stastusCode: 200, body: '<foo>bar</foo>'})
}
return callback(null, {statusCode: 415, body: 'Unsupported Media Type'})
})
handler.use(httpContentNegotiation())
module.exports = { handler }
This middleware could be the basis to create another middleware that allows to automatically serialize the API Gateway response body into the user preferred format (e.g. JSON vs XML). The combination of both middlewares should be enough to close issue number #64 (Improve HTTP error messages), as it will be easy to automatically serialize the error in the expected format.
I hope I will have a PR ready soon, meanwhile feel free to throw here suggestions or ideas ๐
When using an async handler and during the validation, an error occurs the function times out. The code below results in a timeout for me
const createHandler: Handler = async (event: any, context: Context, callback: Callback) => {
await exampleAsyncMethod
const { name } = event.body
return okResponse
}
export const handler = middy(createHandler)
.use(eventLoopMiddleware())
.use(validator({ inputSchema }))
.use(httpErrorHandler())
Changing it to a none async handler it works fine.
const createHandler: Handler = (event: any, context: Context, callback: Callback) => {
exampleAsyncMethod.then(
// code
})
}
export const handler = middy(createHandler)
.use(eventLoopMiddleware())
.use(validator({ inputSchema }))
.use(httpErrorHandler())
I am using my own middleware for the event loop because of this issue. Is there any reason why async handlers are not working with the middleware?
I noticed this package doesn't have TypeScript definitions for the middy/middlewares
submodule which makes it a little bit painful to use with TypeScript. Can I interest you in a pull request?
Here is an example of middy middleware connecting to postgres by @dschep
const pg = require('pg-promise')();
const {getParameter} = require('./ssmParameterStore');
const getDb = () => getParameter('db_password')
.then((password) => pg({
host: process.env.DB_HOST,
port: Number(process.env.DB_PORT),
database: process.env.DB_DATABASE,
user: process.env.DB_USER,
password,
}));
const dbMiddleware = () => ({
before: (handler) => getDb().then((db) => {
handler.context.db = db;
}),
after: (handler) => {
handler.context.db.$pool.end();
},
onError: (handler) => {
handler.context.db.$pool.end();
throw handler.error;
},
})
module.exports = {getDb, dbMiddleware};
We should start collecting examples in the wild and housing them somewhere =)
props to @alexdebrie for the find
Hey folks! I came across Middy a few days ago, and I've been slowly reorganizing one of my applications to take advantage of it. I've also generated a small library of middleware that someone else may be interested in. Take a look and feel free to give me some feedback.
A bunch of tools encode the content-type header using different casing, and this causes the endpoint to fail as the body is not JSON decoded before handing it over to my function code.
A common issue with Api gateway proxy event is that the keys:
pathParameters
and queryStringParameters
are created in the event if (and only if) at least one parameter exists.
this will lead to have to do the following tedious checks in the code:
console.log(event.queryStringParameters ? event.queryStringParameters.foo : undefined)
I'd love to have a middleware that takes care to normalize the event in this case and make sure we always have those key (mapped to an empty object) so that we don't need to have ternary conditions or a bunch of if statements every time we want to access one of those values in the collection.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.