Coder Social home page Coder Social logo

swift-aws-lambda-runtime's People

Contributors

adam-fowler avatar andrea-scuderi avatar fabianfett avatar finestructure avatar fitomad avatar florentmorin avatar glbrntt avatar jareyesda avatar jbosecker avatar joannis avatar jsonfry avatar marwanekoutar avatar mattmassicotte avatar maxdesiatov avatar mr-j-tree avatar mufumade avatar normanmaurer avatar pmarrufo avatar pokryfka avatar proggeramlug avatar ro-m avatar sebsto avatar sja26 avatar stefannienhuis avatar stevapple avatar tachyonics avatar tomerd avatar tonychol avatar weissi avatar yim-lee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

swift-aws-lambda-runtime's Issues

Name clashes with NIOHTTP1

Currently AWSLambdaEvents defines public symbols HTTPHeaders, HTTPMethod and HTTPResponseStatus. These clash with the symbols of the same name in NIOHTTP1. While you can differentiate between the two sets by prepending NIOHTTP1., I'm not sure modules should be defining symbols with the same name as NIO equivalents.

Discussion: Typealias

This is just a personal preference, but I can't get my head around these typealias. I totally understand how they work, but for me it's just not fun to work with them. Maybe we can reduce them? I need to look them up probably every 15min.

Maybe it becomes easier as soon as we use promises/futures, because LambdaResult, LambdaCallback, LambdaInitResult, LambdaInitCallback are the most painful for me.

/// A result type for a Lambda that returns a `[UInt8]`.
public typealias LambdaResult = Result<[UInt8], Error>

public typealias LambdaCallback = (LambdaResult) -> Void

/// A processing closure for a Lambda that takes a `[UInt8]` and returns a `LambdaResult` result type asynchronously.
public typealias LambdaClosure = (LambdaContext, [UInt8], LambdaCallback) -> Void

/// A result type for a Lambda initialization.
public typealias LambdaInitResult = Result<Void, Error>

/// A callback to provide the result of Lambda initialization.
public typealias LambdaInitCallBack = (LambdaInitResult) -> Void

@tomerd What do you think?

Support as a dependency of an (existing) iOS app

Expected behavior

I tried to include the runtime into my existing Xcode iOS app through the SPM UI of Xcode.
I expected this to work, so I can dispatch lambda functions from my iOS app.

Actual behavior

In the Package.swift of your project:
let package = Package(
name: "swift-aws-lambda-runtime",
platforms: [
.macOS(.v10_13),
]

so iOS is not supported.

Steps to reproduce

Integrate into iOS app through Xcode SPM.

If possible, minimal yet complete reproducer code (or URL to code)

SwiftAWSLambdaRuntime version/commit hash

0.2.0

Swift & OS version (output of swift --version && uname -a)

Xcode 11.5

Invocation time longer than expected on cold starts

Expected behavior

Have consistent invocation duration even when lambda execution requires cold start

Actual behavior

When cold starting lambda function invocation takes more than 2s and the actual cold start only 200ms
After cold start, invocation runs lower than 30ms for a dynamo operation.

Xray cloud watch link

Steps to reproduce

Use Lambda integrated with API Gateway and perform any operation on dynamo from lambda function

If possible, minimal yet complete reproducer code (or URL to code)

https://github.com/yuriferretti/lambda-test

SwiftAWSLambdaRuntime version/commit hash

2bac896

Swift & OS version (output of swift --version && uname -a)

swift 5.2

race condition in cancellation

tests are failing sometimes with what looks to be a race condition in the cancellation

10:01:00 2020-05-20T17:01:00+0000 info: AWSLambdaRuntimeCoreTests.HTTPHandler processing /2018-06-01/runtime/invocation/next
10:01:00 2020-05-20T17:01:00+0000 info: intercepted signal: ALRM
10:01:00 2020-05-20T17:01:00+0000 error: lifecycleIteration=39 could not fetch work from lambda runtime engine: cancelled
10:01:00 2020-05-20T17:01:00+0000 warning: lifecycleIteration=39 lambda invocation sequence completed with failure
10:01:00 Fatal error: invalid state, no pending request: file /code/Sources/AWSLambdaRuntimeCore/HTTPClient.swift, line 274
10:01:00 Current stack trace:
10:01:00 0    libswiftCore.so                    0x00007f620ae57db0 swift_reportError + 50
10:01:00 1    libswiftCore.so                    0x00007f620aec9f60 _swift_stdlib_reportFatalErrorInFile + 115
10:01:00 2    libswiftCore.so                    0x00007f620abdedfe <unavailable> + 1383934
10:01:00 3    libswiftCore.so                    0x00007f620abdea07 <unavailable> + 1382919
10:01:00 4    libswiftCore.so                    0x00007f620abdefe8 <unavailable> + 1384424
10:01:00 5    libswiftCore.so                    0x00007f620abdd2d0 _assertionFailure(_:_:file:line:flags:) + 520
10:01:00 6    

reusable DispatchQueue

right now LambdaHandler creates a new instance of DispatchQueue, we should find way to reuse instead

Local Server Immediate Exit code: 0 (Xcode 12 b3)

Wondering if this is an Xcode 12 beta 3 issue:

Having had this issue with my own first attempt at a simple swift lambda I tried building and running the MyLambda example project here and had the same result. If I run the project without the LOCAL_LAMBDA_SERVER_ENABLED flag I get the expected:

2020-07-29T12:22:40+0100 info Lambda : lambda lifecycle starting with Configuration General(logLevel: info)) Lifecycle(id: 148308112161193, maxTimes: 0, stopSignal: TERM) RuntimeEngine(ip: 127.0.0.1, port: 7000, keepAlive: true, requestTimeout: nil 2020-07-29T12:22:40+0100 error Lambda : lifecycleIteration=0 could not fetch work from lambda runtime engine: badStatusCode(NIOHTTP1.HTTPResponseStatus.unprocessableEntity) 2020-07-29T12:22:40+0100 error Lambda : lifecycleIteration=0 lambda invocation sequence completed with error: badStatusCode(NIOHTTP1.HTTPResponseStatus.unprocessableEntity) 2020-07-29T12:22:40+0100 info Lambda : shutdown completed Program ended with exit code: 0

However, if I add the LOCAL_LAMBDA_SERVER_ENABLED flag and run, the process starts and finishes immediately and the only output is the following:

Program ended with exit code: 0

Some pointers on where to look to resolve this would be great.

Deploying multiple Lambdas at once

I've been wondering what would be the best approach for deploying multiple lambdas at once, so I'm hoping to use this issue thread as a place to discuss the best strategy for this. If there is a better place, like forums.swift.org, let me know.

I don't have much experience with lambda, however, I can imagine the following scenarios:

  • Swift package with single function
  • Swift package with multiple, independent functions
  • Swift package with multiple functions sharing resources (eg. same DynamoDB table, same AWS Gateway API, ...)

I'd like to assume for each of those scenarios, being able to deploy lambdas individually (this is, update a single lambda and deploy it) should remain an option.

Focusing on SAM for now, there are two approaches:

  • One SAM template per function
  • One SAM template per application, with multiple functions

What would be the recommended layout for projects with many functions? What are the pros and cons of one template for multiple lambdas vs one template per function?

improve local debugging

with #73 and #87 we introduced the ability to start a local simulator which enables local debugging.

our initial approach was code based, ie call the withLocalServer function and gate it with #if DEBUG but that was deemed low usability
the current environment variable based approach is easy to use, but means users cannot be configured in a meaningful way, which we may want to do for example to inject middleware that simulate api gateway

we should explore more alternatives before 1.0

_X_AMZN_TRACE_ID is not set

The AWS guide - https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html - says a custom runtime should do the following

Propagate the tracing header – Get the X-Ray tracing header from the Lambda-Runtime-Trace-Id header in the API response. Set the _X_AMZN_TRACE_ID environment variable locally with the same value. The X-Ray SDK uses this value to connect trace data between services.

The _X_AMZN_TRACE_ID environment variable is not currently set.

Performance difference Foundation JSON vs. PureSwiftJSON

swift 5.1.4 on Amazon Linux 2

Foundation.JSONCoding
-----------------------------
string, cold: 25724953 (ns)
string, warm: 370171 (ns)
json, cold: 25843952 (ns)
json, warm: 423885 (ns)
-----------------------------

PureSwiftJSONCoding
-----------------------------
string, cold: 23219984 (ns)
string, warm: 372525 (ns)
json, cold: 23525811 (ns)
json, warm: 390062 (ns)
-----------------------------

boot speedup: 8-10% (not linking Foundation, statically linking PureSwiftJSONCoding)
json warm speedup: 8% should be more with larger json payloads.

Nullability of AWSGateway.Request properties

Hi,
I made a small demo using Serverless framework with API Gateway, Lambda and DynamoDB.

Originally I tried to use APIGateway.Request as payload but I wasn't able to decode it as some of the properties are nil.

public let headers: HTTPHeaders
public let multiValueHeaders: HTTPMultiValueHeaders

https://github.com/swift-server/swift-aws-lambda-runtime/blob/master/Sources/AWSLambdaEvents/APIGateway.swift

I'll prepare a PR to fix it, if you agree to amend it.

My example:
https://github.com/swift-sprinter/aws-serverless-swift-api-template

Lambda.run cannot be used without explicit types

Steps to reproduce

  • setup a basic project that uses AWSLambdaRuntime
  • in your main file, define an Input and Output type that should be used for the Lambda
  • use the Lambda.run function with the non-Void callback
  • the body of the run closure MUST have at least one line of code other than invoking the callback

Observed behavior

I get a compiler error: Cannot convert value of type 'Output' to expected argument type 'Void'.
There are 2 overloaded run functions: One takes a closure of type @escaping (Result<Void, Error>) -> Void and the other of type @escaping (Result<Output, Error>) -> Void. I expect the compiler to know that I am using the second function.

Example: Not compiling

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  context.logger.info("foo bar")
  callback(.success(Output(bar: input.foo.capitalized)))
}

Compiler Error: Cannot convert value of type 'Output' to expected argument type 'Void'

Notice that the order of those lines in the run body is not important, the following yields the same compiler error:

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  callback(.success(Output(bar: input.foo.capitalized)))
  context.logger.info("foo bar")
}

Example: Compiling

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  callback(.success(Output(bar: input.foo.capitalized)))
}

As long as the body of run does not contain more than one line I don't get any compiler error. Might be trivial but: Using lines of comments does not result in a compiler error, e.g. the following compiles as well:

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback) in
  // call the callback
  callback(.success(Output(bar: input.foo.capitalized)))
}

Another way to work around this problem is of course just specifying the type of callback. The following compiles without errors:

import AWSLambdaRuntime

struct Input: Codable {
  let foo: String
}

struct Output: Codable {
  let bar: String
}

Lambda.run { (context, input: Input, callback: @escaping (Result<Output, Error>) -> Void) in
  context.logger.info("foo bar")
  callback(.success(Output(bar: input.foo.capitalized)))
}

[EDIT 1: edited the code examples to make them compile if copy&pasted]
[EDIT 2: reworked the examples and explanation to better reflect the problem (now more directed to the compiler error rather than the styling of the code)]

How to use other SPM packages in Swift lambda function?

Expected behavior

I just want to use some code from other SPM packages, for example, I want to include SwiftJWT

import AWSLambdaRuntime
import SwiftJWT

Lambda.run { (context, name: String, callback: @escaping (Result<String, Error>) -> Void) in
/// some code that are using SwiftJWT
}

I added a dependency:

import PackageDescription

let package = Package(
    name: "ThePushLambda",
    products: [
        .executable(name: "ThePushLambda", targets: ["ThePushLambda"]),
    ],
    dependencies: [
        .package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", from: "0.1.0"),
        .package(url: "https://github.com/IBM-Swift/Swift-JWT.git", from: "3.6.1"),
    ],
    targets: [
        .target(
            name: "ThePushLambda",
            dependencies: [
                .product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"),
                .product(name: "AWSLambdaEvents", package: "swift-aws-lambda-runtime"),
                "SwiftJWT" // <--- After adding this line target becomes missing in the Xcode.
            ])
    ]
)

Actual behavior

After adding SwiftJWT dependency into the target dependencies list, the target becomes invalid in the Xcode.

Is it possible to use SPM packages other than swift-aws-lambda-runtime ?

Name clash with AWSSDKSwift v4.x

The AWSLambdaEvents define enum namespaces for the Events (S3, SNS, SES, SQS...). These mean you cannot use v4.x of AWSSDKSwift with swift-aws-lambda-runtime. This is partly the fault of AWSSDKSwift as in v4.x the framework name is the same as the main struct name, so you cannot differentiate between S3 in AWSSKDSwift and S3 in AWSLambdaEvents.

This is resolved with the AWSSDKSwift v5.0 alpha as the framework names have been prepended with AWS. Thus won't be an issue in the future.

I'm not sure if anything will be done about this now, given the issue will disappear in the future, but I must say using such general terms for a very specific part of a service wasn't necessarily the best idea.

Reorder context, future-proof for a tracing context world

In light of https://github.com/swift-server/async-http-client/pull/227/files#r429299831 and a number of long discussions and pondering about this...

I think we'll get the best APIs and outcome for all server libs if we stick to one style, and the "context last" indeed has some gains in Swift... I can go in depth about what convinced me today after hours of chatting with Johannes and looking at my own APIs.

This proposes to reshape the API to become:

Lambda.run { (payload: String, context, callback) in 
// this perhaps makes most sense as consistent with "last, 
// but before trailing closures (regardless who calls them)"?

// OR?
Lambda.run { (payload: String, callback, context) in

and we'd encourage the same shape in APIs I work on myself, AsyncHTTPClient, and the Baggage / Context work that's ongoing with GSoC/Moritz. (Swift gRPC already fits).

(Pretty sure people have strong feelings about this so we can talk it over?)

Specific rule wise I think we'd end up with:

  • "last"
  • but before trailing closures

Believe me, it took a lot of convincing for me to change my mind... As long as we end up on the same style there will be great benefits for consistency of looks of server swift code though.

Include Backtrace by default?

Removing Backtrace I can see an improvements in the cold start times with Swift 5.1.4:

Ubuntu

with
-----------------------------
string, cold: 22710084 (ns)
string, warm: 378130 (ns)
json, cold: 24156758 (ns)
json, warm: 393128 (ns)

without
-----------------------------
string, cold: 20261595 (ns)
string, warm: 373748 (ns)
json, cold: 20569889 (ns)
json, warm: 390503 (ns)

Amazon Linux 2

with
-----------------------------
string, cold: 23968682 (ns)
string, warm: 383200 (ns)
json, cold: 29860691 (ns)
json, warm: 402576 (ns)

without
-----------------------------
string, cold: 23710979 (ns)
string, warm: 383141 (ns)
json, cold: 24595559 (ns)
json, warm: 405713 (ns)

404 on a GET even though resource URL is valid for a POST

Following https://fabianfett.de/swift-on-aws-lambda-creating-your-first-http-endpoint, there is the section about curl + 404:

curl -i http://localhost:7000/invoke

gives

HTTP/1.1 404 Not Found

Which seems to be triggered by the default branch in processRequest(context:request:):

        // unknown call
        default:
            self.writeResponse(context: context, status: .notFound)

I think this should be:

  • 405 if request is not POST and url.hasSuffix(self.invocationEndpoint)
  • 405 is request is not GET and url.hasSuffix(Consts.getNextInvocationURLSuffix)
  • maybe a few more like that for the other branches
  • 404
    Essentially a path which exists for a POST should not throw a 404 for a GET.

I'd suggest to switch on the url at the top level, and then within do the necessary guard for a resource, like:

case url.hasSuffix(Consts.postResponseURLSuffix):
  guard method == .POST else { 
     return self.writeResponse(context: context, status: .methodNotAllowed)
  }

Lambda Control Plane does not send a "Connection" header

Note: Lambda Control Plane does not send an "Connection" header. This happens when we send "Connection" = "keep-alive" and when we are not.

That's why I would suggest, we stop checking if there is a "Connection" header present in the response.

headers: [
  ("Content-Type", "application/json"), 
  ("Lambda-Runtime-Aws-Request-Id", "5cb46d87-96e0-42f3-b7ed-dbcd8df08b70"), 
  ("Lambda-Runtime-Deadline-Ms", "1583751288658"), 
  ("Lambda-Runtime-Invoked-Function-Arn", "arn:aws:lambda:eu-central-1:079477498937:function:SwiftLambdaRuntimePerforman-HelloWorldLambdaSecret-1TK4OK8NY2235"), 
  ("Lambda-Runtime-Trace-Id", "Root=1-5e662075-12412d60a99502b03e375900;Parent=09e5680f20c03edb;Sampled=0"), 
  ("Date", "Mon, 09 Mar 2020 10:54:45 GMT"), ("Content-Length", "2")
]

Remove requirement for macOS 10.13

The runtime requires macOS 10.13, which is quite inconvenient because all consuming packages now also need to explicitly require 10.13 in the package manifest. It is especially annoying if one tries to conditionally implement Lambda support in a 10.10 package via #canImport, e.g.: https://github.com/Macro-swift/Macro/blob/feature/lambda-1/Sources/http/Lambda/lambda.swift#L9

Slack says this requirement only exists for the ISO date formatter, not sure whether you use it for rendering only or also for parsing. In any case, it should be quite easy to replace w/ a custom parser/renderer. Should be reasonable because the format is fixed and never changes.

It could be built on top of timegm and strptime, though it needs an extra processing step to capture the milliseconds:

import func   Foundation.strptime
import func   Foundation.timegm
import struct Foundation.tm

let s          = "20180905T140903.591680Z"
var parsedTime = tm()

s.withCString { cstr in
  strptime(cstr, "%Y%m%dT%H%M%S.%fZ", &parsedTime) // this needs to extract the ms
}
let time = timegm(&parsedTime)
let ti   = TimeInterval(time) // + milliseconds

You could also provide this, and still use the ISO formatter if available (via if #available(macOS 10.13, ...)), which is what I do in Macro.

Database support?

Hi-- I am really appreciative of this effort for Swift AWS lambda support! Thanks so much.

Please take the following question in that appreciative light and also with the fact that I'm new to AWS lambda and not fully sure of the way it works.

I am wondering if database support will (eventually) be part of this effort or part of some other package? E.g., could one just use database access within Swift AWS lambda code from say Vapor (https://github.com/vapor/vapor) with the swift-aws-lambda-runtime package or would that support have to be tailored to AWS lambda?

Thanks!
Chris.

Leaking logs on startup

2020-04-03T15:55:18+0000 info: lambda lifecycle starting with Configuration
2020-04-03T17:55:18 General(logLevel: info))
2020-04-03T17:55:18 Lifecycle(id: 358103301969, maxTimes: 0, stopSignal: TERM)
2020-04-03T17:55:18 RuntimeEngine(ip: 127.0.0.1, port: 9001, keepAlive: true, requestTimeout: nil

We currently log internal information on startup. The best way in my opinion would be to create two Loggers (one for internal usage with default log level error – which can be changed for debug) and one exposed to the user with default level info.

@tomerd wdyt?

Other HttpMethod support for local server

Awesome work everyone! Thank you so much 🙏
I might be doing something wrong but the local server seems to only support POST

Expected behavior

Th local server should support the other HttpMethods.

Actual behavior

If I comment out request.httpMethod = "POST" so that I can try GET in the example ContentView.swift I get "resource exceeds maximum size"

Steps to reproduce

Comment out request.httpMethod = "POST" so that I can try GET in the example ContentView.swift
Run iOS project again, fill in the fields and hit Regiser

SwiftAWSLambdaRuntime version/commit hash

Latest f3c68d6

Swift & OS version (output of swift --version && uname -a)

Apple Swift version 5.3 (swiftlang-1200.0.25.2 clang-1200.0.27.1)
Target: x86_64-apple-darwin20.0.0
Darwin Joels-MBP.lan 20.0.0 Darwin Kernel Version 20.0.0: Thu Jul 30 22:49:28 PDT 2020; root:xnu-7195.0.0.141.5~1/RELEASE_X86_64 x86_64

APIGateway.V2.Request Authorizer jwt visibility

Currently the JWT struct on the Context's Authorizer is public, but the jwt instance is not. Is this a design decision?
Is there a better approach? Currently I am getting the token from the request authorization header and decoding manually to get claims.

I have several lambdas on an HTTP APIGateway API.

SwiftAWSLambdaRuntime version/commit hash

0.2.0

Swift & OS version (output of swift --version && uname -a)

Swift 5.2.4@MacOS 10.5/Swift 5.2.5@Amazonlinux2

LocalServer does not expose `/error` endpoint

The local server currently does not expose an error endpoint.

That means if a local lambda throws an error the lambda runtime tries to report an error, but get's a 404. This makes the Runtime shutdown.

TBD: What error message do we want to send back to the invoker for an error within the lambda? Probably 500?

better representation for numeric sqs-message-attributes

https://github.com/swift-server/swift-aws-lambda-runtime/pull/46/files#diff-46c15e7ad33f3c55f1eff630cb7c7d6aR30

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-attributes.html under "Message Attribute Data Types"

Number attributes can store positive or negative numerical values. A number can have up to 38 digits of precision, and it can be between 10^-128 and 10^+126.

right now using string as a stop gap, could use Foundation's Decimal or come up with our own BigNumber abstraction

Lambda Authorizer - APIGateway.Request

Expected behavior

Obtain information passed from a Lambda Authorizer into Lambda.

Note: This is the REST API (V1)

Actual behavior

Inability to access information due to parameter not existing on APIGateway.Request

If possible, minimal yet complete reproducer code (or URL to code)

{
...
"requestContext": {
      "authorizer": {
        "principalId": "user-id",
        "integrationLatency": 557
      }
}
...
}

Maybe it makes sense to add this to APIGateway.Request.

public let authorizer: Authorizer?

public struct Authorizer: Codable {
    public let principalId: String
    public let context: [String: String]?
}

// OR

public let authorizer: [String: String]?

Question/Request: Make HTTP requests

Can you make HTTP requests with this? Eg. to respond to a webhook, integrate APIs

I see there's the HTTPClient, which looks like it does HTTP requests, but it's currently internal.

Document what to do with AWS "Handler" name.

We currently don't respect the chosen handler and rely on the executable being named bootstrap. We should at least document that we don't care about the given handler name, or that developers might want to use it in their lambda startup method to decide which handler they wanna attach to the runtime.

Shutdown procedure for local testing

In a non-trivial Lambda we might use an AHC, which we initialize with a factory method like this:

class MyLambdaHandler: EventLoopLambdaHandler {
    typealias In = SQS.Event
    typealias Out = Void

    let httpClient: HTTPClient

    static func create(_ eventLoop: EventLoop) -> EventLoopFuture<ByteBufferLambdaHandler> {
        let httpClient = HTTPClient(eventLoopGroupProvider: .shared(eventLoop))
        return eventLoop.makeSucceededFuture(MyLambdaHandler(httpClient: httpClient))
    }

    private init(httpClient: HTTPClient) {
        self.httpClient = httpClient
    }

    func handle(context: Lambda.Context, payload: SQS.Event) -> EventLoopFuture<Void> {
        // not interesting
    }
}

Lambda.run(MyLambdaHandler.create)

We currently don't have any chance to stop the httpClient in a coordinated way, if we stop the LocalTestServer, which closes the Lambda.

This is why I propose the following:

We expand the ByteBufferLambdaHandler protocol with a function syncShutdown() throws which will be invoked, if the Lambda is stopped. We supply a default empty implementation so that developers only need to deal with it if they run into a problem.

@tomerd @weissi @ktoso @adam-fowler wdyt?

Replacing `Date`

We need to discuss how we want to replace Date if we don't want to link against Foundation at all. The problem is that we get a Deadline from AWS:

Lambda-Runtime-Deadline-Ms – The date that the function times out in Unix time milliseconds.
Source: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html

Users of this library might want to know how much time they have left to execute their function. Since I can't see a way to get access to the Unix time in milliseconds, it seems like we have to import <time.h>.

https://pubs.opengroup.org/onlinepubs/7908799/xsh/clock_gettime.html

I've never wrapped a C call before, so this is something I'd be interested in trying if this is a way forward for us.

Discussion: Make API SwiftNIOer

@fabianfett:

How much do we want to expose SwiftNIO. Currently one can not initialise the Lambda with an EventLoopGroup, and the potential callbacks are sync or async, but there is no EventLoopFuture option. Considering that in lot’s of use cases developers want to make calls to external services (http/databases) and so on, I’m not sure if this should be our default option.

@tomerd:

this is a great question, and one that i have been thinking about too. originally i refrained from it because i wanted it to be dead simple for iOS developers - so they dont need to understand futures and nio to write a lambda backend for their iOS apps. that said, as you mention these days most database client etc are nio based so its a impedance mismatch for someone that wants to make use of such. one option we should consider is to also offer a hander variant that returns a future (or fulfills a promise which would be my choice).

@fabianfett:

That’s why I opted for the SwiftNIO design in my runtime first, because that’s want I wanted to use. In other languages AWS encourages developers to set up as much as possible during cold start and to reuse those setups during invocations. For example an AsyncHTTPClient would be setup before Lambda.run and injected into/captured by the handler. If we don’t expose the EventLoopGroup the LambdaRuntime and AsyncHTTPClient will run on different EventLoopGroups – I don’t think we want that.

@tomerd:

yes, one thing i def want to change is to include the evenloop in the context object so that you can share it with the handlers

Include Logging and Backtrace?

@fabianfett asked:

How many batteries do we want to include and which ones? Logging for sure I guess (SSWG graduated). Backtrace? (SSWG not even in sandbox but within the guides)

@tomerd answered:

logging “feels” right but we need to make sure perf is not hurt

linux-backtrace will be added to sswg soon (eventually just be part of the language), but we need it otherwise you are blind on crashes

Consider replacing Foundation JSON encoder

Given that:

  • Foundation JSON encoder/decoder has terrible performance on Linux (~10 times slower than PureJSONSwift and IkigaJSON
  • swift-aws-lambda-runtime runs on Linux
  • AWS Lambda handler does a lot of JSON decoding/encoding, including:
    • AWS Lambda Runtime API
    • JSON payloads delivered as String (APIGateway, DynamoDB, SNS, SQS, ...)
  • most people dont change the default implementation assuming libraries are well tuned
  • swift-aws-lambda-runtime performance is compared against other runtimes: https://www.twitch.tv/videos/661855412?t=04h53m04s
  • as far as AWS Lambda is concerned, time literally costs money

Consider:

  • replacing Foundation JSON encoder/decoder with PureJSONSwift or IkigaJSON
  • recommend not to use it describing how to change it
  • improve performance of Foundation JSON on Linux

Performance comparison:

From PureJSONSwift

Encoding

macOS Swift 5.1 macOS Swift 5.2 Linux Swift 5.1 Linux Swift 5.2
Foundation 2.61s 2.62s 13.03s 12.52s
PureSwiftJSON 1.23s 1.25s 1.13s 1.05s
Speedup ~2x ~2x ~10x ~10x

Decoding

macOS Swift 5.1 macOS Swift 5.2 Linux Swift 5.1 Linux Swift 5.2
Foundation 2.72s 3.04s 10.27s 10.65s
PureSwiftJSON 1.70s 1.72s 1.39s 1.16s
Speedup ~1.5x ~1.5x ~7x ~8x

From aws-xray-sdk-swift

(Average) results in seconds of EncodingTests from run 191224345

Test test-macos (5.2.4) test-linux (swift:5.2) test-linux (swiftlang/swift:nightly-5.3-bionic)
testEncodingUsingFoundationJSON 0.884 2.258 2.293
testEncodingUsingIkigaJSON 0.518 0.262 0.248
testEncodingUsingPureSwiftJSON 0.497 0.347 0.328

swift-aws-lambda-runtime and aws-swift-sdk: release build issue

I have also posted this on the aws-swift-sdk repo
https://github.com/swift-aws/aws-sdk-swift/issues/337

Describe the bug
When using swift-aws-lambda-runtime and aws-swift-sdk I get an error when building the package for release.

error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again

I have tried deleting the Package.resolved file and the same error occurs.

To Reproduce
Steps to reproduce the behavior:

Here is my Package.swift file

// swift-tools-version:5.2
// The swift-tools-version declares the minimum version of Swift required to build this package.`

import PackageDescription
   
let package = Package(
  name: "sf-api",
  platforms: [
      .macOS(.v10_13),
  ],
  products: [
    .executable(name: "sf-api", targets: ["sf-api"]),
  ],
  dependencies: [
    .package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", .upToNextMajor(from: "0.2.0")),
    .package(url: "https://github.com/swift-aws/aws-sdk-swift.git", from: "5.0.0-alpha")
  ],
  targets: [
    .target(
      name: "sf-api",
      dependencies: [
        .product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"),
        .product(name: "AWSLambdaEvents", package: "swift-aws-lambda-runtime"),
        .product(name: "AWSDynamoDB", package: "aws-sdk-swift"),
      ]
    ),
  ]
)

Here is my build script

#!/bin/bash

set -eu

executable=sf-api
workspace="$(pwd)"

echo "-------------------------------------------------------------------------"
echo "preparing docker build image"
echo "-------------------------------------------------------------------------"
docker build . -t builder
echo "done"

echo "-------------------------------------------------------------------------"
echo "building \"$executable\" lambda"
echo "-------------------------------------------------------------------------"
docker run --rm -v "$workspace"/:/workspace -w /workspace builder \
       bash -cl "swift build --product $executable -c release"
echo "done"

echo "-------------------------------------------------------------------------"
echo "packaging \"$executable\" lambda"
echo "-------------------------------------------------------------------------"
docker run --rm -v "$workspace"/:/workspace -w /workspace builder \
       bash -cl ./scripts/package.sh
echo "done"

and package.sh script

#!/bin/bash

set -eu

executable=sf-api

target=".build/lambda/$executable"
rm -rf "$target"
mkdir -p "$target"
cp ".build/release/$executable" "$target/"
# add the target deps based on ldd
ldd ".build/release/$executable" | grep swift | awk '{print $3}' | xargs cp -Lv -t "$target"
cd "$target"
ln -s "$executable" "bootstrap"
zip --symlinks lambda.zip *

Full Output

./scripts/build-and-package.sh
-------------------------------------------------------------------------
preparing docker build image
-------------------------------------------------------------------------
Sending build context to Docker daemon  160.8kB
Step 1/2 : FROM swiftlang/swift:nightly-amazonlinux2
 ---> 0f45823328d8
Step 2/2 : RUN yum -y install      git      libuuid-devel      libicu-devel      libedit-devel      libxml2-devel      sqlite-devel      python-devel      ncurses-devel      curl-devel      openssl-devel      tzdata      libtool      jq      tar      zip
 ---> Using cache
 ---> 939f6ea94a42
Successfully built 939f6ea94a42
Successfully tagged builder:latest
done
-------------------------------------------------------------------------
building "sf-api" lambda
-------------------------------------------------------------------------
Fetching https://github.com/apple/swift-metrics.git
Fetching https://github.com/apple/swift-nio.git
Fetching https://github.com/swift-aws/aws-sdk-swift-core.git
Fetching https://github.com/swift-server/swift-backtrace.git
Fetching https://github.com/apple/swift-nio-extras.git
Fetching https://github.com/apple/swift-nio-transport-services.git
Fetching https://github.com/apple/swift-nio-ssl.git
Fetching https://github.com/apple/swift-log.git
Fetching https://github.com/swift-server/async-http-client.git
Fetching https://github.com/swift-server/swift-aws-lambda-runtime.git
Fetching https://github.com/swift-aws/aws-sdk-swift.git
Cloning https://github.com/swift-aws/aws-sdk-swift.git
Resolving https://github.com/swift-aws/aws-sdk-swift.git at 5.0.0-alpha.5
Cloning https://github.com/swift-server/swift-aws-lambda-runtime.git
Resolving https://github.com/swift-server/swift-aws-lambda-runtime.git at 0.2.0
Updating https://github.com/swift-server/swift-aws-lambda-runtime.git
Updating https://github.com/swift-aws/aws-sdk-swift.git
Updating https://github.com/apple/swift-metrics.git
Updating https://github.com/swift-server/async-http-client.git
Updating https://github.com/apple/swift-nio-transport-services.git
Updating https://github.com/apple/swift-log.git
Updating https://github.com/swift-server/swift-backtrace.git
Updating https://github.com/apple/swift-nio-ssl.git
Updating https://github.com/apple/swift-nio-extras.git
Updating https://github.com/swift-aws/aws-sdk-swift-core.git
Updating https://github.com/apple/swift-nio.git
Fetching https://github.com/apple/swift-crypto.git
Cloning https://github.com/swift-server/async-http-client.git
Resolving https://github.com/swift-server/async-http-client.git at 1.1.1
Cloning https://github.com/apple/swift-nio.git
Resolving https://github.com/apple/swift-nio.git at 2.19.0
Cloning https://github.com/apple/swift-nio-transport-services.git
Resolving https://github.com/apple/swift-nio-transport-services.git at 1.7.0
Cloning https://github.com/swift-aws/aws-sdk-swift-core.git
Resolving https://github.com/swift-aws/aws-sdk-swift-core.git at 5.0.0-alpha.5
Cloning https://github.com/apple/swift-metrics.git
Resolving https://github.com/apple/swift-metrics.git at 2.0.0
Cloning https://github.com/swift-server/swift-backtrace.git
Resolving https://github.com/swift-server/swift-backtrace.git at 1.2.0
Cloning https://github.com/apple/swift-crypto.git
Resolving https://github.com/apple/swift-crypto.git at 1.0.2
Cloning https://github.com/apple/swift-log.git
Resolving https://github.com/apple/swift-log.git at 1.3.0
Cloning https://github.com/apple/swift-nio-ssl.git
Resolving https://github.com/apple/swift-nio-ssl.git at 2.8.0
error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again

Additional context
I have also tried the following package variations

.package(url: "https://github.com/swift-aws/aws-sdk-swift.git", .branch("5.x.x"))
.package(url: "https://github.com/swift-aws/aws-sdk-swift.git", .branch("swift-5.3-package-resolution"))

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.