tobiipro / aws-util-firecloud Goto Github PK

• v2.1.1 -> v0.2.11
• internal repos depending on aws-util-firecloud should switch asap
• remove v[^0].+ tags by 2019-12-31

we had an instance where the lambda logged a response to AWS, but then it was waiting for the event loop to become empty, which never happened, thus the lambda timed out (and the initial response was ignored)

it would be good to at least log that "hey we are now replying to AWS. but this lambda is configured to wait until the event loop is empty, and there's smth on the event loop right now" - even better if we can report some useful info about what is actually on the event loop

currently there are only some undocumented nodejs APIs - process._getActiveRequests() and process._getActiveHandles() - and even those don't work as expected. I tried in a node REPL to run a simple async function that sleeps for 10 seconds then console.log-s smth and process._getActiveRequests() remained empty, while I was expecting it to report the async function somehow.

related to emergency fix 104b634

i.e. revert the 96889a2 revert (emergency fix)

https://docs.aws.amazon.com/athena/latest/ug/data-types.html
https://prestodb.io/docs/current/language/types.html

When reading results, we should support the possibility that the data has pages.

see lodash-firecloud setup

by removing the heavy bunyan (and bunyan-slack), and replacing with https://github.com/tobiipro/minlog

Why? I see webpack.config.jsfiles having exceptions for bunyan's dtrace-provider dependency, plus dtrace-provider has native bindings, thus heavy on its own.

no config should be needed, no assumptions about storage resources etc

basically we need to get it down to "give me the Code and the Environment.Variables"

PS: breaking change, bump minor

rather than passing e.g. ctx around, just to able to call ctx.log.error, given the lambda's serial execution flow (one event at a time), maybe we could do

import {e, ctx, log} from 'aws-util-firecloud/lib/lambda';

log.error(...)

wdyt @IanSavchenko ?

https://github.com/tobiipro/aws-util-firecloud/blob/master/src/lambda.js#L59-L66

I believe node 8 has some gc changes, so maybe, just maybe we don't need to spend time running garbage collection on demand.

I suggest we try removing it, and then see the behaviour. We could also add a flag, but I don't particularly see the point in it.

// cc @tobiiasl

apparently proper stack traces only come in node 12, and under a flag (at least that's the current state)

maybe the babel-config-firecloud should go back to transforming (generators or bluebird) at least for node :(

We discussed that it should be up to the API lambda itself to setup CORS, so need to remove it here, mark the change as breaking and update lambdas.

Regarding reexporting CORS: I wouldn't do it, because clients get to decide which version to use and type-inference will probably work better (if we decide to use TS in lambda at some point).