swift-server / swift-aws-lambda-runtime Goto Github PK

requestId -> requestID
traceId -> traceID
invokedFunctionArn -> invokedFunctionARN

Expected behavior

In node you can do the following:

const secretsManager = new AWS.SecretsManager();
    const secret = await secretsManager.getSecretValue({
        SecretId: event.db_secret_arn
    }).promise();

See also: https://chrisschuld.com/2020/07/aws-retrieving-secrets-in-parameter-store-with-node/

Actual behavior

Have a way to use the SecretsManager inside a Swift Lambda as well.

Expected behavior

The ClientContext is often of well known shape:

It should handle the typing more gracefully I think, see docs:
https://docs.aws.amazon.com/lambda/latest/dg/nodejs-context.html

See also: #167 (comment) for discussion on this

Actual behavior

It's a String today:

            let clientContext: String?

Currently AWSLambdaEvents defines public symbols HTTPHeaders, HTTPMethod and HTTPResponseStatus. These clash with the symbols of the same name in NIOHTTP1. While you can differentiate between the two sets by prepending NIOHTTP1., I'm not sure modules should be defining symbols with the same name as NIO equivalents.

This is just a personal preference, but I can't get my head around these typealias. I totally understand how they work, but for me it's just not fun to work with them. Maybe we can reduce them? I need to look them up probably every 15min.

Maybe it becomes easier as soon as we use promises/futures, because LambdaResult, LambdaCallback, LambdaInitResult, LambdaInitCallback are the most painful for me.

/// A result type for a Lambda that returns a `[UInt8]`.
public typealias LambdaResult = Result<[UInt8], Error>

public typealias LambdaCallback = (LambdaResult) -> Void

/// A processing closure for a Lambda that takes a `[UInt8]` and returns a `LambdaResult` result type asynchronously.
public typealias LambdaClosure = (LambdaContext, [UInt8], LambdaCallback) -> Void

/// A result type for a Lambda initialization.
public typealias LambdaInitResult = Result<Void, Error>

/// A callback to provide the result of Lambda initialization.
public typealias LambdaInitCallBack = (LambdaInitResult) -> Void

@tomerd What do you think?

right now we have:
https://github.com/swift-server/swift-aws-lambda-examples
https://github.com/swift-server/swift-aws-lambda-deubgging-example

maybe moving to an "Examples" subdirectory would make it easier to find than separate repos? we do that in some of our other projects

Expected behavior

I tried to include the runtime into my existing Xcode iOS app through the SPM UI of Xcode.
I expected this to work, so I can dispatch lambda functions from my iOS app.

Actual behavior

In the Package.swift of your project:
let package = Package(
name: "swift-aws-lambda-runtime",
platforms: [
.macOS(.v10_13),
]

so iOS is not supported.

Steps to reproduce

Integrate into iOS app through Xcode SPM.

If possible, minimal yet complete reproducer code (or URL to code)

SwiftAWSLambdaRuntime version/commit hash

0.2.0

Swift & OS version (output of swift --version && uname -a)

Xcode 11.5

Expected behavior

Have consistent invocation duration even when lambda execution requires cold start

Actual behavior

When cold starting lambda function invocation takes more than 2s and the actual cold start only 200ms
After cold start, invocation runs lower than 30ms for a dynamo operation.

Xray cloud watch link

Steps to reproduce

Use Lambda integrated with API Gateway and perform any operation on dynamo from lambda function

If possible, minimal yet complete reproducer code (or URL to code)

https://github.com/yuriferretti/lambda-test

SwiftAWSLambdaRuntime version/commit hash

2bac896

Swift & OS version (output of swift --version && uname -a)

swift 5.2

tests are failing sometimes with what looks to be a race condition in the cancellation

10:01:00 2020-05-20T17:01:00+0000 info: AWSLambdaRuntimeCoreTests.HTTPHandler processing /2018-06-01/runtime/invocation/next
10:01:00 2020-05-20T17:01:00+0000 info: intercepted signal: ALRM
10:01:00 2020-05-20T17:01:00+0000 error: lifecycleIteration=39 could not fetch work from lambda runtime engine: cancelled
10:01:00 2020-05-20T17:01:00+0000 warning: lifecycleIteration=39 lambda invocation sequence completed with failure
10:01:00 Fatal error: invalid state, no pending request: file /code/Sources/AWSLambdaRuntimeCore/HTTPClient.swift, line 274
10:01:00 Current stack trace:
10:01:00 0    libswiftCore.so                    0x00007f620ae57db0 swift_reportError + 50
10:01:00 1    libswiftCore.so                    0x00007f620aec9f60 _swift_stdlib_reportFatalErrorInFile + 115
10:01:00 2    libswiftCore.so                    0x00007f620abdedfe <unavailable> + 1383934
10:01:00 3    libswiftCore.so                    0x00007f620abdea07 <unavailable> + 1382919
10:01:00 4    libswiftCore.so                    0x00007f620abdefe8 <unavailable> + 1384424
10:01:00 5    libswiftCore.so                    0x00007f620abdd2d0 _assertionFailure(_:_:file:line:flags:) + 520
10:01:00 6    

right now LambdaHandler creates a new instance of DispatchQueue, we should find way to reuse instead

Wondering if this is an Xcode 12 beta 3 issue:

Having had this issue with my own first attempt at a simple swift lambda I tried building and running the MyLambda example project here and had the same result. If I run the project without the LOCAL_LAMBDA_SERVER_ENABLED flag I get the expected:

2020-07-29T12:22:40+0100 info Lambda : lambda lifecycle starting with Configuration General(logLevel: info)) Lifecycle(id: 148308112161193, maxTimes: 0, stopSignal: TERM) RuntimeEngine(ip: 127.0.0.1, port: 7000, keepAlive: true, requestTimeout: nil 2020-07-29T12:22:40+0100 error Lambda : lifecycleIteration=0 could not fetch work from lambda runtime engine: badStatusCode(NIOHTTP1.HTTPResponseStatus.unprocessableEntity) 2020-07-29T12:22:40+0100 error Lambda : lifecycleIteration=0 lambda invocation sequence completed with error: badStatusCode(NIOHTTP1.HTTPResponseStatus.unprocessableEntity) 2020-07-29T12:22:40+0100 info Lambda : shutdown completed Program ended with exit code: 0

However, if I add the LOCAL_LAMBDA_SERVER_ENABLED flag and run, the process starts and finishes immediately and the only output is the following:

Program ended with exit code: 0

Some pointers on where to look to resolve this would be great.

I've been wondering what would be the best approach for deploying multiple lambdas at once, so I'm hoping to use this issue thread as a place to discuss the best strategy for this. If there is a better place, like forums.swift.org, let me know.

I don't have much experience with lambda, however, I can imagine the following scenarios:

• Swift package with single function
• Swift package with multiple, independent functions
• Swift package with multiple functions sharing resources (eg. same DynamoDB table, same AWS Gateway API, ...)

I'd like to assume for each of those scenarios, being able to deploy lambdas individually (this is, update a single lambda and deploy it) should remain an option.

Focusing on SAM for now, there are two approaches:

• One SAM template per function
• One SAM template per application, with multiple functions

What would be the recommended layout for projects with many functions? What are the pros and cons of one template for multiple lambdas vs one template per function?

with #73 and #87 we introduced the ability to start a local simulator which enables local debugging.

our initial approach was code based, ie call the withLocalServer function and gate it with #if DEBUG but that was deemed low usability
the current environment variable based approach is easy to use, but means users cannot be configured in a meaningful way, which we may want to do for example to inject middleware that simulate api gateway

we should explore more alternatives before 1.0

The AWS guide - https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html - says a custom runtime should do the following

Propagate the tracing header – Get the X-Ray tracing header from the Lambda-Runtime-Trace-Id header in the API response. Set the _X_AMZN_TRACE_ID environment variable locally with the same value. The X-Ray SDK uses this value to connect trace data between services.

The _X_AMZN_TRACE_ID environment variable is not currently set.

swift 5.1.4 on Amazon Linux 2

Foundation.JSONCoding
-----------------------------
string, cold: 25724953 (ns)
string, warm: 370171 (ns)
json, cold: 25843952 (ns)
json, warm: 423885 (ns)
-----------------------------

PureSwiftJSONCoding
-----------------------------
string, cold: 23219984 (ns)
string, warm: 372525 (ns)
json, cold: 23525811 (ns)
json, warm: 390062 (ns)
-----------------------------

boot speedup: 8-10% (not linking Foundation, statically linking PureSwiftJSONCoding)
json warm speedup: 8% should be more with larger json payloads.

Hi,
I made a small demo using Serverless framework with API Gateway, Lambda and DynamoDB.

Originally I tried to use APIGateway.Request as payload but I wasn't able to decode it as some of the properties are nil.

public let headers: HTTPHeaders
public let multiValueHeaders: HTTPMultiValueHeaders

https://github.com/swift-server/swift-aws-lambda-runtime/blob/master/Sources/AWSLambdaEvents/APIGateway.swift

I'll prepare a PR to fix it, if you agree to amend it.

My example:
https://github.com/swift-sprinter/aws-serverless-swift-api-template

Steps to reproduce

• setup a basic project that uses AWSLambdaRuntime
• in your main file, define an Input and Output type that should be used for the Lambda
• use the Lambda.run function with the non-Void callback
• the body of the run closure MUST have at least one line of code other than invoking the callback

Observed behavior

I get a compiler error: Cannot convert value of type 'Output' to expected argument type 'Void'.
There are 2 overloaded run functions: One takes a closure of type @escaping (Result<Void, Error>) -> Void and the other of type @escaping (Result<Output, Error>) -> Void. I expect the compiler to know that I am using the second function.

Example: Not compiling

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  context.logger.info("foo bar")
  callback(.success(Output(bar: input.foo.capitalized)))
}

Compiler Error: Cannot convert value of type 'Output' to expected argument type 'Void'

Notice that the order of those lines in the run body is not important, the following yields the same compiler error:

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  callback(.success(Output(bar: input.foo.capitalized)))
  context.logger.info("foo bar")
}

Example: Compiling

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  callback(.success(Output(bar: input.foo.capitalized)))
}

As long as the body of run does not contain more than one line I don't get any compiler error. Might be trivial but: Using lines of comments does not result in a compiler error, e.g. the following compiles as well:

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  // call the callback
  callback(.success(Output(bar: input.foo.capitalized)))
}

Another way to work around this problem is of course just specifying the type of callback. The following compiles without errors:

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback: @escaping (Result<Output, Error>) -> Void) in
  context.logger.info("foo bar")
  callback(.success(Output(bar: input.foo.capitalized)))
}

[EDIT 1: edited the code examples to make them compile if copy&pasted]
[EDIT 2: reworked the examples and explanation to better reflect the problem (now more directed to the compiler error rather than the styling of the code)]

The jq command line tool is required for the build/deploy scripts yet there is no mention that I could find in the documentation. The docs should be updated to reflect this.

Expected behavior

I just want to use some code from other SPM packages, for example, I want to include SwiftJWT

import AWSLambdaRuntime
import SwiftJWT

Lambda.run { (context, name: String, callback: @escaping (Result<String, Error>) -> Void) in
/// some code that are using SwiftJWT
}

I added a dependency:

import PackageDescription

let package = Package(
    name: "ThePushLambda",
    products: [
        .executable(name: "ThePushLambda", targets: ["ThePushLambda"]),
    ],
    dependencies: [
        .package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", from: "0.1.0"),
        .package(url: "https://github.com/IBM-Swift/Swift-JWT.git", from: "3.6.1"),
    ],
    targets: [
        .target(
            name: "ThePushLambda",
            dependencies: [
                .product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"),
                .product(name: "AWSLambdaEvents", package: "swift-aws-lambda-runtime"),
                "SwiftJWT" // <--- After adding this line target becomes missing in the Xcode.
            ])
    ]
)

Actual behavior

After adding SwiftJWT dependency into the target dependencies list, the target becomes invalid in the Xcode.

Is it possible to use SPM packages other than swift-aws-lambda-runtime ?

The AWSLambdaEvents define enum namespaces for the Events (S3, SNS, SES, SQS...). These mean you cannot use v4.x of AWSSDKSwift with swift-aws-lambda-runtime. This is partly the fault of AWSSDKSwift as in v4.x the framework name is the same as the main struct name, so you cannot differentiate between S3 in AWSSKDSwift and S3 in AWSLambdaEvents.

This is resolved with the AWSSDKSwift v5.0 alpha as the framework names have been prepended with AWS. Thus won't be an issue in the future.

I'm not sure if anything will be done about this now, given the issue will disappear in the future, but I must say using such general terms for a very specific part of a service wasn't necessarily the best idea.

In light of https://github.com/swift-server/async-http-client/pull/227/files#r429299831 and a number of long discussions and pondering about this...

I think we'll get the best APIs and outcome for all server libs if we stick to one style, and the "context last" indeed has some gains in Swift... I can go in depth about what convinced me today after hours of chatting with Johannes and looking at my own APIs.

This proposes to reshape the API to become:

Lambda.run { (payload: String, context, callback) in 
// this perhaps makes most sense as consistent with "last, 
// but before trailing closures (regardless who calls them)"?

// OR?
Lambda.run { (payload: String, callback, context) in

and we'd encourage the same shape in APIs I work on myself, AsyncHTTPClient, and the Baggage / Context work that's ongoing with GSoC/Moritz. (Swift gRPC already fits).

(Pretty sure people have strong feelings about this so we can talk it over?)

Specific rule wise I think we'd end up with:

• "last"
• but before trailing closures

Believe me, it took a lot of convincing for me to change my mind... As long as we end up on the same style there will be great benefits for consistency of looks of server swift code though.

Removing Backtrace I can see an improvements in the cold start times with Swift 5.1.4:

Ubuntu

with
-----------------------------
string, cold: 22710084 (ns)
string, warm: 378130 (ns)
json, cold: 24156758 (ns)
json, warm: 393128 (ns)

without
-----------------------------
string, cold: 20261595 (ns)
string, warm: 373748 (ns)
json, cold: 20569889 (ns)
json, warm: 390503 (ns)

Amazon Linux 2

with
-----------------------------
string, cold: 23968682 (ns)
string, warm: 383200 (ns)
json, cold: 29860691 (ns)
json, warm: 402576 (ns)

without
-----------------------------
string, cold: 23710979 (ns)
string, warm: 383141 (ns)
json, cold: 24595559 (ns)
json, warm: 405713 (ns)

Following https://fabianfett.de/swift-on-aws-lambda-creating-your-first-http-endpoint, there is the section about curl + 404:

curl -i http://localhost:7000/invoke

gives

HTTP/1.1 404 Not Found

Which seems to be triggered by the default branch in processRequest(context:request:):

        // unknown call
        default:
            self.writeResponse(context: context, status: .notFound)

I think this should be:

• 405 if request is not POST and url.hasSuffix(self.invocationEndpoint)
• 405 is request is not GET and url.hasSuffix(Consts.getNextInvocationURLSuffix)
• maybe a few more like that for the other branches
• 404
  Essentially a path which exists for a POST should not throw a 404 for a GET.

I'd suggest to switch on the url at the top level, and then within do the necessary guard for a resource, like:

case url.hasSuffix(Consts.postResponseURLSuffix):
  guard method == .POST else { 
     return self.writeResponse(context: context, status: .methodNotAllowed)
  }

Note: Lambda Control Plane does not send an "Connection" header. This happens when we send "Connection" = "keep-alive" and when we are not.

That's why I would suggest, we stop checking if there is a "Connection" header present in the response.

headers: [
  ("Content-Type", "application/json"), 
  ("Lambda-Runtime-Aws-Request-Id", "5cb46d87-96e0-42f3-b7ed-dbcd8df08b70"), 
  ("Lambda-Runtime-Deadline-Ms", "1583751288658"), 
  ("Lambda-Runtime-Invoked-Function-Arn", "arn:aws:lambda:eu-central-1:079477498937:function:SwiftLambdaRuntimePerforman-HelloWorldLambdaSecret-1TK4OK8NY2235"), 
  ("Lambda-Runtime-Trace-Id", "Root=1-5e662075-12412d60a99502b03e375900;Parent=09e5680f20c03edb;Sampled=0"), 
  ("Date", "Mon, 09 Mar 2020 10:54:45 GMT"), ("Content-Length", "2")
]

The runtime requires macOS 10.13, which is quite inconvenient because all consuming packages now also need to explicitly require 10.13 in the package manifest. It is especially annoying if one tries to conditionally implement Lambda support in a 10.10 package via #canImport, e.g.: https://github.com/Macro-swift/Macro/blob/feature/lambda-1/Sources/http/Lambda/lambda.swift#L9

Slack says this requirement only exists for the ISO date formatter, not sure whether you use it for rendering only or also for parsing. In any case, it should be quite easy to replace w/ a custom parser/renderer. Should be reasonable because the format is fixed and never changes.

It could be built on top of timegm and strptime, though it needs an extra processing step to capture the milliseconds:

import func   Foundation.strptime
import func   Foundation.timegm
import struct Foundation.tm

let s          = "20180905T140903.591680Z"
var parsedTime = tm()

s.withCString { cstr in
  strptime(cstr, "%Y%m%dT%H%M%S.%fZ", &parsedTime) // this needs to extract the ms
}
let time = timegm(&parsedTime)
let ti   = TimeInterval(time) // + milliseconds

You could also provide this, and still use the ISO formatter if available (via if #available(macOS 10.13, ...)), which is what I do in Macro.

In October 2020 AWS has released Lambda Extensions.

The official Lambda Runtimes already support the new Extension API. We should investigate the support use cases and consider supporting the new Extensions API.

Hi-- I am really appreciative of this effort for Swift AWS lambda support! Thanks so much.

Please take the following question in that appreciative light and also with the fact that I'm new to AWS lambda and not fully sure of the way it works.

I am wondering if database support will (eventually) be part of this effort or part of some other package? E.g., could one just use database access within Swift AWS lambda code from say Vapor (https://github.com/vapor/vapor) with the swift-aws-lambda-runtime package or would that support have to be tailored to AWS lambda?

Thanks!
Chris.

2020-04-03T15:55:18+0000 info: lambda lifecycle starting with Configuration
2020-04-03T17:55:18 General(logLevel: info))
2020-04-03T17:55:18 Lifecycle(id: 358103301969, maxTimes: 0, stopSignal: TERM)
2020-04-03T17:55:18 RuntimeEngine(ip: 127.0.0.1, port: 9001, keepAlive: true, requestTimeout: nil

We currently log internal information on startup. The best way in my opinion would be to create two Loggers (one for internal usage with default log level error – which can be changed for debug) and one exposed to the user with default level info.

@tomerd wdyt?

Awesome work everyone! Thank you so much 🙏
I might be doing something wrong but the local server seems to only support POST

Expected behavior

Th local server should support the other HttpMethods.

Actual behavior

If I comment out request.httpMethod = "POST" so that I can try GET in the example ContentView.swift I get "resource exceeds maximum size"

Steps to reproduce

Comment out request.httpMethod = "POST" so that I can try GET in the example ContentView.swift
Run iOS project again, fill in the fields and hit Regiser

SwiftAWSLambdaRuntime version/commit hash

Latest f3c68d6

Swift & OS version (output of swift --version && uname -a)

Apple Swift version 5.3 (swiftlang-1200.0.25.2 clang-1200.0.27.1)
Target: x86_64-apple-darwin20.0.0
Darwin Joels-MBP.lan 20.0.0 Darwin Kernel Version 20.0.0: Thu Jul 30 22:49:28 PDT 2020; root:xnu-7195.0.0.141.5~1/RELEASE_X86_64 x86_64

Currently the JWT struct on the Context's Authorizer is public, but the jwt instance is not. Is this a design decision?
Is there a better approach? Currently I am getting the token from the request authorization header and decoding manually to get claims.

I have several lambdas on an HTTP APIGateway API.

SwiftAWSLambdaRuntime version/commit hash

0.2.0

Swift & OS version (output of swift --version && uname -a)

Swift 5.2.4@MacOS 10.5/Swift 5.2.5@Amazonlinux2

The local server currently does not expose an error endpoint.

That means if a local lambda throws an error the lambda runtime tries to report an error, but get's a 404. This makes the Runtime shutdown.

TBD: What error message do we want to send back to the invoker for an error within the lambda? Probably 500?

this issue is to track finding on cold start performance

• reusable buffer
• reusable encoder/decoder

Currently we store a lifecycleIteration in the Logger metadata. For me this is always 0 on Lambda. I wonder if we should remove it...

https://github.com/swift-server/swift-aws-lambda-runtime/pull/46/files#diff-46c15e7ad33f3c55f1eff630cb7c7d6aR30

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-attributes.html under "Message Attribute Data Types"

Number attributes can store positive or negative numerical values. A number can have up to 38 digits of precision, and it can be between 10^-128 and 10^+126.

right now using string as a stop gap, could use Foundation's Decimal or come up with our own BigNumber abstraction

Expected behavior

Obtain information passed from a Lambda Authorizer into Lambda.

Note: This is the REST API (V1)

Actual behavior

Inability to access information due to parameter not existing on APIGateway.Request

If possible, minimal yet complete reproducer code (or URL to code)

{
...
"requestContext": {
      "authorizer": {
        "principalId": "user-id",
        "integrationLatency": 557
      }
}
...
}

Maybe it makes sense to add this to APIGateway.Request.

public let authorizer: Authorizer?

public struct Authorizer: Codable {
    public let principalId: String
    public let context: [String: String]?
}

// OR

public let authorizer: [String: String]?

We should log with debug priority if the connection to the control plane is kept alive to verify that we reuse the same connection over and over.

Can you make HTTP requests with this? Eg. to respond to a webhook, integrate APIs

I see there's the HTTPClient, which looks like it does HTTP requests, but it's currently internal.

As per PR #150, which needs to be fixed in more places.

Note that this is API semver major, so for 2.x.

e.g. https://github.com/awslabs/aws-lambda-cpp/blob/fe55d84c808243baf48468369e98e09018e959a3/src/runtime.cpp#L127-L131

this issue is to track details re. warm start performance

According to the documentation we must set a special header ("Lambda-Runtime-Function-Error-Type: Unhandled") when we report an error.

This must be done for both:

• initialization error
• invocation error

#98 (comment)

We currently don't respect the chosen handler and rely on the executable being named bootstrap. We should at least document that we don't care about the given handler name, or that developers might want to use it in their lambda startup method to decide which handler they wanna attach to the runtime.

In a non-trivial Lambda we might use an AHC, which we initialize with a factory method like this:

class MyLambdaHandler: EventLoopLambdaHandler {
    typealias In = SQS.Event
    typealias Out = Void

    let httpClient: HTTPClient

    static func create(_ eventLoop: EventLoop) -> EventLoopFuture<ByteBufferLambdaHandler> {
        let httpClient = HTTPClient(eventLoopGroupProvider: .shared(eventLoop))
        return eventLoop.makeSucceededFuture(MyLambdaHandler(httpClient: httpClient))
    }

    private init(httpClient: HTTPClient) {
        self.httpClient = httpClient
    }

    func handle(context: Lambda.Context, payload: SQS.Event) -> EventLoopFuture<Void> {
        // not interesting
    }
}

Lambda.run(MyLambdaHandler.create)

We currently don't have any chance to stop the httpClient in a coordinated way, if we stop the LocalTestServer, which closes the Lambda.

This is why I propose the following:

We expand the ByteBufferLambdaHandler protocol with a function syncShutdown() throws which will be invoked, if the Lambda is stopped. We supply a default empty implementation so that developers only need to deal with it if they run into a problem.

@tomerd @weissi @ktoso @adam-fowler wdyt?

We need to discuss how we want to replace Date if we don't want to link against Foundation at all. The problem is that we get a Deadline from AWS:

Lambda-Runtime-Deadline-Ms – The date that the function times out in Unix time milliseconds.
Source: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html

Users of this library might want to know how much time they have left to execute their function. Since I can't see a way to get access to the Unix time in milliseconds, it seems like we have to import <time.h>.

https://pubs.opengroup.org/onlinepubs/7908799/xsh/clock_gettime.html

I've never wrapped a C call before, so this is something I'd be interested in trying if this is a way forward for us.

@fabianfett:

How much do we want to expose SwiftNIO. Currently one can not initialise the Lambda with an EventLoopGroup, and the potential callbacks are sync or async, but there is no EventLoopFuture option. Considering that in lot’s of use cases developers want to make calls to external services (http/databases) and so on, I’m not sure if this should be our default option.

@tomerd:

this is a great question, and one that i have been thinking about too. originally i refrained from it because i wanted it to be dead simple for iOS developers - so they dont need to understand futures and nio to write a lambda backend for their iOS apps. that said, as you mention these days most database client etc are nio based so its a impedance mismatch for someone that wants to make use of such. one option we should consider is to also offer a hander variant that returns a future (or fulfills a promise which would be my choice).

@fabianfett:

That’s why I opted for the SwiftNIO design in my runtime first, because that’s want I wanted to use. In other languages AWS encourages developers to set up as much as possible during cold start and to reuse those setups during invocations. For example an AsyncHTTPClient would be setup before Lambda.run and injected into/captured by the handler. If we don’t expose the EventLoopGroup the LambdaRuntime and AsyncHTTPClient will run on different EventLoopGroups – I don’t think we want that.

@tomerd:

yes, one thing i def want to change is to include the evenloop in the context object so that you can share it with the handlers

@fabianfett asked:

How many batteries do we want to include and which ones? Logging for sure I guess (SSWG graduated). Backtrace? (SSWG not even in sandbox but within the guides)

@tomerd answered:

logging “feels” right but we need to make sure perf is not hurt

linux-backtrace will be added to sswg soon (eventually just be part of the language), but we need it otherwise you are blind on crashes

Given that:

• Foundation JSON encoder/decoder has terrible performance on Linux (~10 times slower than PureJSONSwift and IkigaJSON
• swift-aws-lambda-runtime runs on Linux
• AWS Lambda handler does a lot of JSON decoding/encoding, including:
  • AWS Lambda Runtime API
  • JSON payloads delivered as String (APIGateway, DynamoDB, SNS, SQS, ...)
• most people dont change the default implementation assuming libraries are well tuned
• swift-aws-lambda-runtime performance is compared against other runtimes: https://www.twitch.tv/videos/661855412?t=04h53m04s
• as far as AWS Lambda is concerned, time literally costs money

Consider:

• replacing Foundation JSON encoder/decoder with PureJSONSwift or IkigaJSON
• recommend not to use it describing how to change it
• improve performance of Foundation JSON on Linux

Performance comparison:

From PureJSONSwift

Encoding

Decoding

From aws-xray-sdk-swift

(Average) results in seconds of EncodingTests from run 191224345

I have also posted this on the aws-swift-sdk repo
https://github.com/swift-aws/aws-sdk-swift/issues/337

Describe the bug
When using swift-aws-lambda-runtime and aws-swift-sdk I get an error when building the package for release.

error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again

I have tried deleting the Package.resolved file and the same error occurs.

To Reproduce
Steps to reproduce the behavior:

Here is my Package.swift file

// swift-tools-version:5.2
// The swift-tools-version declares the minimum version of Swift required to build this package.`

import PackageDescription
   
let package = Package(
  name: "sf-api",
  platforms: [
      .macOS(.v10_13),
  ],
  products: [
    .executable(name: "sf-api", targets: ["sf-api"]),
  ],
  dependencies: [
    .package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", .upToNextMajor(from: "0.2.0")),
    .package(url: "https://github.com/swift-aws/aws-sdk-swift.git", from: "5.0.0-alpha")
  ],
  targets: [
    .target(
      name: "sf-api",
      dependencies: [
        .product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"),
        .product(name: "AWSLambdaEvents", package: "swift-aws-lambda-runtime"),
        .product(name: "AWSDynamoDB", package: "aws-sdk-swift"),
      ]
    ),
  ]
)

Here is my build script

#!/bin/bash

set -eu

executable=sf-api
workspace="$(pwd)"

echo "-------------------------------------------------------------------------"
echo "preparing docker build image"
echo "-------------------------------------------------------------------------"
docker build . -t builder
echo "done"

echo "-------------------------------------------------------------------------"
echo "building \"$executable\" lambda"
echo "-------------------------------------------------------------------------"
docker run --rm -v "$workspace"/:/workspace -w /workspace builder \
       bash -cl "swift build --product $executable -c release"
echo "done"

echo "-------------------------------------------------------------------------"
echo "packaging \"$executable\" lambda"
echo "-------------------------------------------------------------------------"
docker run --rm -v "$workspace"/:/workspace -w /workspace builder \
       bash -cl ./scripts/package.sh
echo "done"

and package.sh script

#!/bin/bash

set -eu

executable=sf-api

target=".build/lambda/$executable"
rm -rf "$target"
mkdir -p "$target"
cp ".build/release/$executable" "$target/"
# add the target deps based on ldd
ldd ".build/release/$executable" | grep swift | awk '{print $3}' | xargs cp -Lv -t "$target"
cd "$target"
ln -s "$executable" "bootstrap"
zip --symlinks lambda.zip *

Full Output

./scripts/build-and-package.sh
-------------------------------------------------------------------------
preparing docker build image
-------------------------------------------------------------------------
Sending build context to Docker daemon  160.8kB
Step 1/2 : FROM swiftlang/swift:nightly-amazonlinux2
 ---> 0f45823328d8
Step 2/2 : RUN yum -y install      git      libuuid-devel      libicu-devel      libedit-devel      libxml2-devel      sqlite-devel      python-devel      ncurses-devel      curl-devel      openssl-devel      tzdata      libtool      jq      tar      zip
 ---> Using cache
 ---> 939f6ea94a42
Successfully built 939f6ea94a42
Successfully tagged builder:latest
done
-------------------------------------------------------------------------
building "sf-api" lambda
-------------------------------------------------------------------------
Fetching https://github.com/apple/swift-metrics.git
Fetching https://github.com/apple/swift-nio.git
Fetching https://github.com/swift-aws/aws-sdk-swift-core.git
Fetching https://github.com/swift-server/swift-backtrace.git
Fetching https://github.com/apple/swift-nio-extras.git
Fetching https://github.com/apple/swift-nio-transport-services.git
Fetching https://github.com/apple/swift-nio-ssl.git
Fetching https://github.com/apple/swift-log.git
Fetching https://github.com/swift-server/async-http-client.git
Fetching https://github.com/swift-server/swift-aws-lambda-runtime.git
Fetching https://github.com/swift-aws/aws-sdk-swift.git
Cloning https://github.com/swift-aws/aws-sdk-swift.git
Resolving https://github.com/swift-aws/aws-sdk-swift.git at 5.0.0-alpha.5
Cloning https://github.com/swift-server/swift-aws-lambda-runtime.git
Resolving https://github.com/swift-server/swift-aws-lambda-runtime.git at 0.2.0
Updating https://github.com/swift-server/swift-aws-lambda-runtime.git
Updating https://github.com/swift-aws/aws-sdk-swift.git
Updating https://github.com/apple/swift-metrics.git
Updating https://github.com/swift-server/async-http-client.git
Updating https://github.com/apple/swift-nio-transport-services.git
Updating https://github.com/apple/swift-log.git
Updating https://github.com/swift-server/swift-backtrace.git
Updating https://github.com/apple/swift-nio-ssl.git
Updating https://github.com/apple/swift-nio-extras.git
Updating https://github.com/swift-aws/aws-sdk-swift-core.git
Updating https://github.com/apple/swift-nio.git
Fetching https://github.com/apple/swift-crypto.git
Cloning https://github.com/swift-server/async-http-client.git
Resolving https://github.com/swift-server/async-http-client.git at 1.1.1
Cloning https://github.com/apple/swift-nio.git
Resolving https://github.com/apple/swift-nio.git at 2.19.0
Cloning https://github.com/apple/swift-nio-transport-services.git
Resolving https://github.com/apple/swift-nio-transport-services.git at 1.7.0
Cloning https://github.com/swift-aws/aws-sdk-swift-core.git
Resolving https://github.com/swift-aws/aws-sdk-swift-core.git at 5.0.0-alpha.5
Cloning https://github.com/apple/swift-metrics.git
Resolving https://github.com/apple/swift-metrics.git at 2.0.0
Cloning https://github.com/swift-server/swift-backtrace.git
Resolving https://github.com/swift-server/swift-backtrace.git at 1.2.0
Cloning https://github.com/apple/swift-crypto.git
Resolving https://github.com/apple/swift-crypto.git at 1.0.2
Cloning https://github.com/apple/swift-log.git
Resolving https://github.com/apple/swift-log.git at 1.3.0
Cloning https://github.com/apple/swift-nio-ssl.git
Resolving https://github.com/apple/swift-nio-ssl.git at 2.8.0
error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again

Additional context
I have also tried the following package variations

.package(url: "https://github.com/swift-aws/aws-sdk-swift.git", .branch("5.x.x"))
.package(url: "https://github.com/swift-aws/aws-sdk-swift.git", .branch("swift-5.3-package-resolution"))