Coder Social home page Coder Social logo

serverless / serverless-openwhisk Goto Github PK

View Code? Open in Web Editor NEW
143.0 17.0 47.0 417 KB

Adds Apache OpenWhisk support to the Serverless Framework!

Home Page: http://openwhisk.org/

License: MIT License

JavaScript 99.60% Python 0.22% Shell 0.18%
serverless-functions openwhisk serverless-framework serverless serverless-plugin

serverless-openwhisk's Introduction

Serverless Apache OpenWhisk Plugin

Build Status codecov

This plugin enables support for the Apache OpenWhisk platform within the Serverless Framework.

Getting Started

Register account with Apache OpenWhisk

Before you can deploy your service to Apache OpenWhisk, you need to have an account registered with the platform.

Set up account credentials

Account credentials for OpenWhisk can be provided through a configuration file or environment variables. This plugin requires the API endpoint, namespace and authentication credentials.

Do you want to use a configuration file for storing these values? Please follow the instructions for setting up the OpenWhisk command-line utility. This tool stores account credentials in the .wskprops file in the user's home directory. The plugin automatically extracts credentials from this file at runtime. No further configuration is needed.

Do you want to use environment variables for credentials? Use the following environment variables to be pass in account credentials. These values override anything extracted from the configuration file.

  • OW_APIHOST - Platform endpoint, e.g. openwhisk.ng.bluemix.net
  • OW_AUTH - Authentication key, e.g. xxxxxx:yyyyy
  • OW_NAMESPACE - Namespace, defaults to user-provided credentials
  • OW_APIGW_ACCESS_TOKEN - API gateway access token (optional)
  • OW_IAM_NAMESPACE_API_KEY - IBM Cloud IAM API key (optional & overrides auth).

Install Serverless Framework

$ npm install --global serverless

This framework plugin requires Node.js runtime version 6.0 or above.

Create Service From Template

Using the create command, you can create an example service from the following template.

serverless create --template openwhisk-nodejs --path my_service
cd my_service
npm install

More service examples are available in the serverless-examples repository.

Using a self-hosted version of the platform?

Ensure you set the ignore_certs option in the serverless.yaml prior to deployment.

provider:
  name: openwhisk
  ignore_certs: true

Deploy Service

The sample service from the template can be deployed without modification.

serverless deploy

If the deployment succeeds, the following messages will be printed to the console.

$ serverless deploy
Serverless: Packaging service...
Serverless: Compiling Functions...
Serverless: Compiling API Gateway definitions...
Serverless: Compiling Rules...
Serverless: Compiling Triggers & Feeds...
Serverless: Deploying Functions...
Serverless: Deployment successful!

Service Information
platform:	openwhisk.ng.bluemix.net
namespace:	_
service:	my_service

actions:
my_service-dev-hello

triggers:
**no triggers deployed***

rules:
**no rules deployed**

endpoints:
**no routes deployed**

web-actions:
**no web actions deployed**

Test Service

Use the invoke command to test your newly deployed service.

$ serverless invoke --function hello
{
    "payload": "Hello, World!"
}
$ serverless invoke --function hello --data '{"name": "OpenWhisk"}'
{
    "payload": "Hello, OpenWhisk!"
}

Add the -v or --verbose flag to show more invocation details, e.g. activation id and duration details.

$ serverless invoke --function hello -v
=> action (<ACTION_NAME>) activation (<ID>) duration: 96ms (init: 83ms, wait: 35ms)
{
    "payload": "Hello, OpenWhisk!"
}

Writing Functions - Node.js

Here's an index.js file containing an example handler function.

function main(params) {
  const name = params.name || 'World';
  return {payload:  'Hello, ' + name + '!'};
};

exports.main = main;

Modules should return the function handler as a custom property on the global exports object.

In the serverless.yaml file, the handler property is used to denote the source file and module property containing the serverless function.

functions:
  my_function:
    handler: index.main

Request Properties

OpenWhisk executes the handler function for each request. This function is called with a single argument, an object containing the request properties.

function main(params) {
  const parameter = params.parameter_name;
  ...
};

Function Return Values

The handler must return an object from the function call. Returning undefined or null will result in an error. If the handler is carrying out an asynchronous task, it can return a Promise.

// synchronous return
function main () {
  return { payload: "..." }
}

// asychronous return
function main(args) {
  return new Promise(function(resolve, reject) {
    setTimeout(function() {
      resolve({ done: true });
     }, 2000);
  })
}

If you want to return an error message, return an object with an error property with the message. Promise values that are rejected will be interpreted as runtime errors.

// synchronous return
function main () {
  return { error: "..." }
}

// asychronous return
function main(args) {
  return new Promise(function(resolve, reject) {
    setTimeout(function() {
      reject("error message");
     }, 2000);
  })
}

Using NPM Modules

NPM modules must be installed locally in the node_modules directory before deployment. This directory will be packaged up in the deployment artefact. Any dependencies included in node_modules will be available through require() in the runtime environment.

OpenWhisk provides a number of popular NPM modules in the runtime environment. Using these modules doesn't require them to be included in the deployment package. See this list for full details of which modules are available.

const leftPad = require("left-pad")

function pad_lines(args) {
    const lines = args.lines || [];
    return { padded: lines.map(l => leftPad(l, 30, ".")) }
};

exports.handler = pad_lines;

Writing Functions - PHP

Here's an index.php file containing an example handler function.

<?php
function main(array $args) : array
{
    $name = $args["name"] ?? "stranger";
    $greeting = "Hello $name!";
    echo $greeting;
    return ["greeting" => $greeting];
}

In the serverless.yaml file, the handler property is used to denote the source file and function name of the serverless function.

functions:
  my_function:
    handler: index.main
    runtime: php

Request Properties

OpenWhisk executes the handler function for each request. This function is called with a single argument, an associative array containing the request properties.

function main(array $args) : array
{
    $name = $args["name"] ?? "stranger";
    ...
}

Function Return Values

The handler must return an associative array from the function call.

func main(args: [String:Any]) -> [String:Any] {
	...
    return ["foo" => $bar];
}

If you want to return an error message, return an object with an error property with the message.

Writing Functions - Python

Here's an index.py file containing an example handler function.

def endpoint(params):
    name = params.get("name", "stranger")
    greeting = "Hello " + name + "!"
    print(greeting)
    return {"greeting": greeting}

In the serverless.yaml file, the handler property is used to denote the source file and module property containing the serverless function.

functions:
  my_function:
    handler: index.endpoint
    runtime: python:3

Request Properties

OpenWhisk executes the handler function for each request. This function is called with a single argument, a dictionary containing the request properties.

def endpoint(params):
    name = params.get("name", "stranger")
    ...

Function Return Values

The handler must return a dictionary from the function call.

def endpoint(params):
    ...
    return {"foo": "bar"}

If you want to return an error message, return an object with an error property with the message.

Writing Functions - Ruby

Here's an hello.rb file containing an example handler function.

def main(args)
  name = args["name"] || "stranger"
  greeting = "Hello #{name}!"
  puts greeting
  { "greeting" => greeting }
end

In the serverless.yaml file, the handler property is used to denote the source file and function name of the serverless function.

functions:
  my_function:
    handler: hello.main
    runtime: ruby

Request Properties

OpenWhisk executes the handler function for each request. This function is called with a single argument, which is a hash containing the request properties.

def main(args)
  name = args["name"] || "stranger"
  ...

Function Return Values

The handler must return a hash from the function call.

def main(args)
  ...
  { "greeting" => greeting }
end

If you want to return an error message, return an error property string in the return hash.

Writing Functions - Swift

Here's an index.swift file containing an example handler function.

func main(args: [String:Any]) -> [String:Any] {
    if let name = args["name"] as? String {
      return [ "greeting" : "Hello \(name)!" ]
    } else {
      return [ "greeting" : "Hello stranger!" ]
    }
}

In the serverless.yaml file, the handler property is used to denote the source file and module property containing the serverless function.

functions:
  my_function:
    handler: index.main
    runtime: swift

Request Properties

OpenWhisk executes the handler function for each request. This function is called with a single argument, a dictionary containing the request properties.

func main(args: [String:Any]) -> [String:Any] {
    let prop = args["prop"] as? String
}

Function Return Values

The handler must return a dictionary from the function call.

func main(args: [String:Any]) -> [String:Any] {
	...
    return ["foo": "bar"]
}

If you want to return an error message, return an object with an error property with the message.

Codable Support

Swift 4 runtimes support Codable types to handle the converting between JSON input parameters and response types to native Swift types.

struct Employee: Codable {
  let id: Int?
  let name: String?
}
// codable main function
func main(input: Employee, respondWith: (Employee?, Error?) -> Void) -> Void {
    // For simplicity, just passing same Employee instance forward
    respondWith(input, nil)
}

Pre-Compiled Swift Binaries

OpenWhisk supports creating Swift actions from a pre-compiled binary. This reduces startup time for Swift actions by removing the need for a dynamic compilation step.

In the serverless.yaml file, the handler property can refer to the zip file containing a binary file produced by the build.

functions:
  hello:
    handler: action.zip

Compiling a single Swift file to a binary can be handled using this Docker command with the OpenWhisk Swift runtime image. main.swift is the file containing the swift code and action.zip is the zip archive produced.

docker run -i openwhisk/action-swift-v4.2 -compile main < main.swift > action.zip

Swift packages containing multiple source files with a package descriptor (Package.swift ) can be built using the following command.

zip - -r * | docker run -i openwhisk/action-swift-v4.2 -compile main > action.zip

Writing Functions - Java

Here's an src/main/java/HelloWorld.java file containing an example handler function.

import com.google.gson.JsonObject;

public class HelloWorld {

  public static JsonObject main(JsonObject args) throws Exception {

    final String name = args.getAsJsonPrimitive("name").getAsString();

    final JsonObject response = new JsonObject();
    response.addProperty("greeting", "Hello " + name + "!");

    return response;
  }
}

Here is a simple pom.xml file that will allow you to use Maven to build it. You will notice that gson is excluded from the uberjar. That is because OpenWhisk already provides this dependency.

<project>
 <modelVersion>4.0.0</modelVersion>
 <groupId>hello</groupId>
 <artifactId>hello-world</artifactId>
 <version>1.0</version>

 <dependencies>
  <dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.2</version>
  </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <minimizeJar>true</minimizeJar>
              <artifactSet>
                <excludes>
                  <exclude>com.google.code.gson:gson</exclude>
                </excludes>
              </artifactSet>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
 </project>

In the serverless.yaml file (see below), the handler property is the uberjar produced by calling mvn clean package, a colon, and then the fully qualified class name of the class with the main function. If you do not provide a class name after the jar, it will look for a class in the default package called Main.

service: my-java-service
provider:
  name: openwhisk
  runtime: java
functions:
  hello:
    handler: target/hello-world-1.0.jar:HelloWorld
plugins:
  - serverless-openwhisk

Request Properties

OpenWhisk executes the handler function for each request. This function is called with a single argument, a com.google.gson.JsonObject containing the request properties.

import com.google.gson.JsonObject;

public class MyActionClass {
  public static JsonObject main(JsonObject args) throws Exception
  {
    final String name = args.getAsJsonPrimitive("name").getAsString();
    ...
  }
}

Function Return Values

The handler must return an com.google.gson.JsonObject from the function call.

import com.google.gson.JsonObject;

public class MyActionClass {
  public static JsonObject main(JsonObject args) throws Exception
  {
    ...
    final JsonObject response = new JsonObject();
    response.addProperty("greeting", "Hello " + name + "!");

    return response;
  }
}

If you want to return an error message, throw an exception.

Writing Functions - Binary

OpenWhisk supports executing a compiled binary for the function handler. Using a Python wrapper, the file will be invoked within the openwhisk/dockerskeleton Docker container.

The binary must be compiled for the correct platform architecture and only link to shared libraries installed in the openwhisk/dockerskeleton runtime.

In the serverless.yaml file, the handler property is used to denote the binary file to upload.

functions:
  my_function:
    handler: bin_file
    runtime: binary

Request Properties

OpenWhisk executes the binary file for each request. Event parameters are streamed to stdio as a JSON object string.

Function Return Values

The handler must write a JSON object string with the response parameters to stdout before exiting.

If you want to return an error message, return an object with an error property with the message.

Custom Runtime Images

OpenWhisk actions can use custom Docker images as the runtime environment. This allows extra packages, libraries or tools to be pre-installed in the runtime environment. Using a custom runtime image, with extra libraries and dependencies built-in, is useful for overcoming the maximum deployment size on actions.

Images must implement the API used by the platform to interact with runtime environments. Images must also be available on Docker Hub. OpenWhisk does not support private Docker registries.

OpenWhisk publishes the existing runtime images on Docker Hub. Using these images in the FROM directive in the Dockerfile is an easy way to create new images compatible with the platform.

In the serverless.yaml file, the image property is used to denote the custom runtime image.

functions:
  my_function:
    handler: source.js
    runtime: nodejs
    image: dockerhub_user/image_name

Node.js, Swift, Python and Binary runtimes support using a custom image property.

Writing Functions - Docker

OpenWhisk supports creating actions from public images on Docker Hub without handler files. These images are expected to support the platform API used to instantiate and invoke serverless functions.

All necessary files for execution must be provided within the image. Local source files will not be uploaded to the runtime environment.

In the serverless.yaml file, the handler property is used to denote the image label.

functions:
  my_function:
    handler: repo/image_name
    runtime: docker

Working With Packages

OpenWhisk provides a concept called "packages" to manage related actions. Packages can contain multiple actions under a common identifier in a namespace. Configuration values needed by all actions in a package can be set as default properties on the package, rather than individually on each action.

Packages are identified using the following format: /namespaceName/packageName/actionName.

Rules and triggers can not be created within packages.

Implicit Packages

Actions can be assigned to packages by setting the function name with a package reference.

functions:
  foo:
    handler: handler.foo
    name: "myPackage/foo"
  bar:
    handler: handler.bar
    name: "myPackage/bar"

In this example, two new actions (foo & bar) will be created using the myPackage package.

Packages which do not exist will be automatically created during deployments. When using the remove command, any packages referenced in the serverless.yml will be deleted.

Explicit Packages

Packages can also be defined explicitly to set shared configuration parameters. Default package parameters are merged into event parameters for each invocation.

functions:
  foo:
    handler: handler.foo
    name: "myPackage/foo"

resources:
  packages:
    myPackage:
      name: optionalCustomName
      parameters:
        hello: world

Explicit packages support the following properties: name, parameters, annotations, services and shared.

Binding Packages

OpenWhisk also supports "binding" external packages into your workspace. Bound packages can have default parameters set for shared actions.

For example, binding the /whisk.system/cloudant package into a new package allows you to set default values for the username, password and dbname properties. Actions from this package can then be invoked with having to pass these parameters in.

Define packages explicitly with a binding parameter to use this behaviour.

resources:
  packages:
    mySamples:
      binding: /whisk.system/cloudant
      parameters:
        username: bernie
        password: sanders
        dbname: vermont

For more details on package binding, please see the documentation here.

Binding Services (IBM Cloud Functions)

This feature requires the IBM Cloud CLI and IBM Cloud Functions plugin to be installed.

IBM Cloud Functions supports automatic binding of service credentials to actions using the CLI.

Bound service credentials will be passed as the __bx_creds parameter in the invocation parameters.

This feature is also available through the serverless.yaml file using the bind property for each function.

functions:
  my_function:
    handler: file_name.handler    
    bind:
      - service:
          name: cloud-object-storage
          instance: my-cos-storage

The service configuration supports the following properties.

  • name: identifier for the cloud service
  • instance: instance name for service (optional)
  • key: key name for instance and service (optional)

If the instance or key properties are missing, the first available instance and key found will be used.

Binding services removes the need to manually create default parameters for service keys from platform services.

More details on binding service credentials to actions can be found in the official documentation and this blog post.

Packages defined in the resources section can bind services using the same configuration properties.

resources:
  packages:
    myPackage:
      bind:
        - service:
            name: cloud-object-storage
            instance: my-cos-storage

Runtime Configuration Properties

The following OpenWhisk configuration properties are supported for functions defined in the serverless.yaml file.

functions:
  my_function:
    handler: file_name.handler_func
    name: "custom_function_name"
    runtime: 'runtime_label' // defaults to nodejs:default
    namespace: "..." // defaults to user-provided credentials
    memory: 256 // 128 to 512 (MB).
    timeout: 60 // 0.1 to 600 (seconds)
    concurrency: 1 // 1 to 500, default is 1
    parameters:
      foo: bar // default parameters
    annotations:
      foo: bar // action annotations
    bind:
      - service:
          name: cloud-object-storage
          instance: my-cos-storage

Writing Sequences

OpenWhisk supports a special type of serverless function called sequences.

These functions are defined from a list of other serverless functions. Upon invocation, the platform executes each function in series. Request parameters are passed into the first function in the list. Each subsequent function call is passed the output from the previous step as input parameters. The last function's return value is returned as the response result.

Here's an example of the configuration to define a sequence function, composed of three other functions.

functions:
  my_function:
    sequence:
      - parse_input
      - do_some_algorithm
      - construct_output

Sequence functions do not have a handler file defined. If you want to refer to functions not defined in the serverless project, use the fully qualified identifier e.g. /namespace/package/action_name

Connecting HTTP Endpoints

Functions can be bound to public URL endpoints using the API Gateway service. HTTP requests to configured endpoints will invoke functions on-demand. Requests parameters are passed as function arguments. Function return values are serialised as the JSON response body.

HTTP endpoints for functions can be configured through the serverless.yaml file.

functions:
  my_function:
    handler: index.main
    events:
      - http: GET /api/greeting

HTTP event configuration also supports using explicit parameters.

  • method - HTTP method (mandatory).
  • path - URI path for API gateway (mandatory).
  • resp - controls web action content type, values include: json, html, http, svgor text (optional, defaults to json).
functions:
  my_function:
    handler: index.main
    events:
      - http:
          method: GET
          path: /api/greeting
          resp: http

API Gateway hosts serving the API endpoints will be shown during deployment.

$ serverless deploy
...
endpoints:
GET https://xxx-gws.api-gw.mybluemix.net/service_name/api/greeting --> service_name-dev-my_function

Calling the configured API endpoints will execute the deployed functions.

$ http get https://xxx-gws.api-gw.mybluemix.net/api/greeting?user="James Thomas"
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Date: Mon, 19 Dec 2016 15:47:53 GMT

{
    "message": "Hello James Thomas!"
}

Functions exposed through the API Gateway service are automatically converted into Web Actions during deployment. The framework secures Web Actions for HTTP endpoints using the require-whisk-auth annotation. If the require-whisk-auth annotation is manually configured, the existing annotation value is used, otherwise a random token is automatically generated.

URL Path Parameters

The API Gateway service supports path parameters in user-defined HTTP paths. This allows functions to handle URL paths which include templated values, like resource identifiers.

Path parameters are identified using the {param_name} format in the URL path. The API Gateway sends the full matched path value in the __ow_path field of the event parameters.

functions:
  retrieve_users:
    handler: users.get
    events:
      - http:
          method: GET
          path: /users/{id}
          resp: http

This feature comes with the following restrictions:

  • Path parameters are only supported when resp is configured ashttp.
  • Individual path parameter values are not included as separate event parameters. Users have to manually parse values from the full __ow_path value.

CORS Support

API Gateway endpoints automatically include CORS headers for all endpoints under the service base path. This property can be disabled by manually configuring the resources.apigw.cors property.

resources:
    apigw:
        cors: false

Application Authentication

API endpoints can be protected by API keys with a secret or API keys alone.

Setting the HTTP headers used to pass keys and secrets automatically enables API Gateway authentication.

This parameter configures the HTTP header containing the API key. Without the additional secret header, authentication uses an API key alone.

resources:
    apigw:
        auth:
            key: API-Key-Header

Adding the secret header parameter enables authentication using keys with secrets.

resources:
    apigw:
        auth:
            key: API-Key-Header
            secret: API-Key-Secret-Header

See the API Gateway configuration panel to manage API keys and secrets after authentication is enabled.

Application Authentication with OAuth

API endpoints can also be protected by an external OAuth providers.

OAuth tokens must be included as the Authorization header of each API request. Token will be validated with the specified token provider. If the token is invalid, requests are rejected with response code 401.

The following OAuth providers are supported: IBM Cloud App ID, Google, Facebook and Github.

resources:
  apigw:
    oauth:
      provider: app-id || google || facebook || github

If the app-id provider is selected, the tenant identifier must be provided as an additional configuration token. This can be retrieved from the tenantId property of provisioned service credentials for the instance

resources:
  apigw:
    oauth:
      provider: app-id
      tenant: uuid

Application Authentication with keys (and secrets) and OAuth support are mutually exclusive configuration options.

Rate Limiting

API Gateways endpoints support rate limiting to reject excess traffic. When rate limiting is enabled, API calls falling outside of the limit will be rejected and response code 429 will be returned.

Rate limiting is on a per-key basis and application authentication (without oauth) must be enabled.

The leaky bucket algorithm is used to prevent sudden bursts of invocations of APIs. If the limit is set as 10 calls per minute, users will be restricted to 1 call every 6 seconds (60/10 = 6).

resources:
  apigw:
    rate_limit:
      rate: 100
      unit: minute || second || hour || day
  • rate: number of API calls per unit of time.
  • unit: unit of time (minute, second, hour, day) used to threshold API calls with rate.

Base Path

All API Gateway endpoints defined as HTTP events in the serverless.yml are deployed under the default base path (/). This basepath can be configured explicitly using the following parameter.

resources:
  apigw:
    basepath: /api

API Name

The service name is used as the API identifier in the API Gateway swagger files. This can be configured explicitly using the following parameter.

resources:
  apigw:
    name: my-api-name

Exporting Web Actions

Functions can be turned into "web actions" which return HTTP content without use of an API Gateway. This feature is enabled by setting an annotation (web-export) in the configuration file.

functions:
  my_function:
    handler: index.main
    annotations:
      web-export: true

Functions with this annotation can be invoked through a URL template with the following parameters.

https://{APIHOST}/api/v1/web/{USER_NAMESPACE}/{PACKAGE}/{ACTION_NAME}.{TYPE}
  • APIHOST - platform endpoint e.g. openwhisk.ng.bluemix.net.
  • USER_NAMESPACE - this must be an explicit namespace and cannot use the default namespace (_).
  • PACKAGE - action package or default.
  • ACTION_NAME - default form ${servicename}-${space}-${name}.
  • TYPE - .json.html.text or .http.

Return values from the function are used to construct the HTTP response. The following parameters are supported.

  1. headers: a JSON object where the keys are header-names and the values are string values for those headers (default is no headers).
  2. code: a valid HTTP status code (default is 200 OK).
  3. body: a string which is either plain text or a base64 encoded string (for binary data).

Here is an example of returning HTML content:

function main(args) {
    var msg = "you didn&#39;t tell me who you are."
    if (args.name) {
        msg = `hello ${args.name}!`
    }
    return {body:
       `<html><body><h3><center>${msg}</center></h3></body></html>`}
}

Here is an example of returning binary data:

function main() {
   let png = <base 64 encoded string>
   return {
      headers: { "Content-Type": "image/png" },
      body: png };
}

Functions can access request parameters using the following environment variables.

  1. __ow_method - HTTP method of the request.
  2. __ow_headers - HTTP request headers.
  3. __ow_path - Unmatched URL path of the request.
  4. __ow_body - Body entity from request.
  5. __ow_query - Query parameters from the request.

Full details on this feature are available in this here.

Scheduled Invocations

Functions can be set up to fire automatically using the alarm package. This allows you to invoke functions with preset parameters at specific times (12:00 each day) or according to a schedule (every ten minutes).

Scheduled invocation for functions can be configured through the serverless.yaml file.

The schedule event configuration is controlled by a string, based on the UNIX crontab syntax, in the format cron(X X X X X). This can either be passed in as a native string or through the rate parameter.

functions:
  my_function:
    handler: index.main
    events:
      - schedule: cron(* * * * *) // fires each minute.

This above example generates a new trigger (${service}_crawl_schedule_trigger) and rule (${service}_crawl_schedule_rule) during deployment.

Other schedule event parameters can be manually configured, e.g trigger or rule names.

functions:
  aggregate:
    handler: statistics.handler
    events:
      - schedule:
          rate: cron(0 * * * *) // call once an hour
          trigger: triggerName
          rule: ruleName
          max: 10000 // max invocations, default: 1000, max: 10000
          params: // event params for invocation
            hello: world

IBM Message Hub Events

IBM Bluemix provides an "Apache Kafka"-as-a-Service called IBM Message Hub. Functions can be connected to fire when messages arrive on Kafka topics.

IBM Message Hub instances can be provisioned through the IBM Bluemix platform. OpenWhisk on Bluemix will export Message Hub service credentials bound to a package with the following name:

/${BLUEMIX_ORG}_${BLUEMIX_SPACE}/Bluemix_${SERVICE_NAME}_Credentials-1

Rather than having to manually define all the properties needed by the Message Hub trigger feed, you can reference a package to use instead. Credentials from the referenced package will be used when executing the trigger feed.

Developers only need to add the topic to listen to for each trigger.

# serverless.yaml
functions:
    index:
        handler: users.main
        events:
            - message_hub:
                package: /${BLUEMIX_ORG}_${BLUEMIX_SPACE}/Bluemix_${SERVICE_NAME}_Credentials-1
                topic: my_kafka_topic

The plugin will create a trigger called ${serviceName}_${fnName}_messagehub_${topic} and a rule called ${serviceName}_${fnName}_messagehub_${topic}_rule to bind the function to the message hub events.

The trigger and rule names created can be set explicitly using the trigger andrule parameters.

Other functions can bind to the same trigger using the inline trigger event referencing this trigger name.

# serverless.yaml
functions:
    index:
        handler: users.main
        events:
            - message_hub:
                package: /${BLUEMIX_ORG}_${BLUEMIX_SPACE}/Bluemix_${SERVICE_NAME}_Credentials-1
                topic: my_kafka_topic
                trigger: log_events
                rule: connect_index_to_kafka
     another:
        handler: users.another
        events:
            - trigger: log_events

Using Manual Parameters

Parameters for the Message Hub event source can be defined explicitly, rather than using pulling credentials from a package.

# serverless.yaml
functions:
    index:
        handler: users.main
        events:
            - message_hub:
                topic: my_kafka_topic
                brokers: afka01-prod01.messagehub.services.us-south.bluemix.net:9093
                user: USERNAME
                password: PASSWORD
                admin_url:  https://kafka-admin-prod01.messagehub.services.us-south.bluemix.net:443
                json: true
                binary_key: true
                binary_value: true

topic, brokers, user, password and admin_url are mandatory parameters.

Cloudant DB Events

IBM Cloudant provides a hosted NoSQL database, based upon CouchDB, running on IBM Bluemix. Functions can be connected to events fired when the database is updated. These events use the CouchDB changes feed to follow database modifications.

IBM Cloudant instances can be provisioned through the IBM Bluemix platform. OpenWhisk on Bluemix will export Cloudant service credentials bound to a package with the following name:

/${BLUEMIX_ORG}_${BLUEMIX_SPACE}/Bluemix_${SERVICE_NAME}_Credentials-1

Rather than having to manually define all the properties needed by the Cloudant trigger feed, you can reference a package to use instead. Credentials from the referenced package will be used when executing the trigger feed.

Developers only need to add the database name to follow for modifications.

# serverless.yaml
functions:
    index:
        handler: users.main
        events:
            - cloudant:
                package: /${BLUEMIX_ORG}_${BLUEMIX_SPACE}/Bluemix_${SERVICE_NAME}_Credentials-1
                db: my_db_name

The plugin will create a trigger called ${serviceName}_${fnName}_cloudant_${topic} and a rule called ${serviceName}_${fnName}_cloudant_${topic}_rule to bind the function to the Cloudant update events.

The trigger and rule names created can be set explicitly using the trigger andrule parameters.

Other functions can bind to the same trigger using the inline trigger event referencing this trigger name.

Using Manual Parameters

Parameters for the Cloudant event source can be defined explicitly, rather than using pulling credentials from a package.

# serverless.yaml
functions:
    index:
        handler: users.main
        events:
            - cloudant: // basic auth example
                host: xxx-yyy-zzz-bluemix.cloudant.com
                username: USERNAME
                password: PASSWORD
                db: db_name
			- cloudant: // iam auth example
                host: xxx-yyy-zzz-bluemix.cloudant.com
                iam_api_key: IAM_API_KEY
                db: db_name

username and password or iam_api_key parameters can be used for authentication.

Adding Optional Parameters

The following optional feed parameters are also supported:

  • max - Maximum number of triggers to fire. Defaults to infinite.
  • filter - Filter function defined on a design document.
  • query - Optional query parameters for the filter function.
# serverless.yaml
functions:
    index:
        handler: users.main
        events:
            - cloudant:
                ...
                max: 10000
                query:
                   status: new
                filter: mailbox/by_status

Custom Event Triggers

Functions are connected to event sources in OpenWhisk using triggers and rules. Triggers create a named event stream within the system. Triggers can be fired manually or connected to external data sources, like databases or message queues.

Rules set up a binding between triggers and serverless functions. With an active rule, each time a trigger is fired, the function will be executed with the trigger payload.

Event binding for functions can be configured through the serverless.yaml file.

functions:
  my_function:
    handler: index.main
    events:
      - trigger: my_trigger

This configuration will create a trigger called servicename-my_trigger with an active rule binding my_function to this event stream.

Customising Rules

Rule names default to the following format servicename-trigger-to-action. These names be explicitly set through configuration.

functions:
  my_function:
    handler: index.main
    events:
      - trigger:
        name: "my_trigger"
        rule: "rule_name"

Customing Triggers

Triggers can be defined as separate resources in the serverless.yaml file. This allows you to set up trigger properties like default parameters.

functions:
  my_function:
    handler: index.main
    events:
      - trigger: my_trigger

resources:
  triggers:
    my_trigger:
      parameters:
        hello: world            

Trigger Feeds

Triggers can be bound to external event sources using the feed property. OpenWhisk provides a catalogue of third-party event sources bundled as packages.

This example demonstrates setting up a trigger which uses the /whisk.system/alarms/alarm feed. The alarm feed will fire a trigger according to a user-supplied cron schedule.

resources:
  triggers:
    alarm_trigger:
      parameters:
        hello: world
      feed: /whisk.system/alarms/alarm
      feed_parameters:
        cron: '*/8 * * * * *'

Commands

The following serverless commands are currently implemented for the OpenWhisk provider.

serverless-openwhisk's People

Contributors

aml2732 avatar austencollins avatar bvennam avatar cjelger avatar csantanapr avatar dbachko avatar herzog31 avatar jthomas avatar junoyoon avatar keonhee avatar macdonst avatar pmuens avatar rabbah avatar sarveshbhatnagar avatar shazron avatar siegesmund avatar swiftpengu avatar volteanu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

serverless-openwhisk's Issues

Add verbose output to Openwhisk provider deploy command

When running serverless deploy with the openwhisk provider, the --verbose switch does not have any effect. It would be really helpful if this could add some details while performing the "Deploying Functions..." and "Deploying Sequences..." steps (and probably also all other steps too), so that it's clear what actions/functions are exactly being deployed. The summary displayed after the deployment is done is indeed confusing, because it displays a list of all actions deployed in openwhisk, and not only the actions deployed by serverless deploy.

cc: @jthomas as discussed on Slack

Add Swift plugin for automatic compilation

Swift execution can be speed up by compiling and deploying the binary, rather than the source files.

Using Docker to run the container locally, users can build the executable for the correct runtime. This approach is detailed here:
https://github.com/openwhisk/openwhisk/blob/master/docs/actions.md#packaging-an-action-as-a-swift-executable

As this relies on Docker and external images, having this as the main compilation step for Swift files is not realistic. However, it would be possible to write a new serverless fw plugin that hooked into the build step and performed this automatically.

Instructions incorrect with new authentication model of whisk service

Setting up the cli according to the document doesn't work when following the current CLI instructions that are linked:
https://console.ng.bluemix.net/openwhisk/cli

The rules fail to deploy unless I'm explicitly setting a namespace in .wskprops. (Which is not what the CLI suggests)

Error is:
Serverless: Packaging service...
Serverless: Compiling Functions...
Serverless: Compiling API Gateway definitions...
Serverless: Compiling Rules...
Serverless: Compiling Triggers & Feeds...
Serverless: Deploying Functions...
Serverless: Deploying Triggers...
Serverless: Deploying Rules...

Serverless Error ---------------------------------------

 Failed to deploy rule (my_service_event_name_to_hello_world)
 due to error: OpenWhisk authentication failed, check
 API key? The supplied authentication is not authorized
 to access this resource.

Stack Trace --------------------------------------------

ServerlessError: Failed to deploy rule (my_service_event_name_to_hello_world) due to error: OpenWhisk authentication failed, check API key? The supplied authentication is not authorized to access this resource.
at disableRule.then.catch.err (/home/dpittner/whisk/serverless-openwhisk-boilerplate/node_modules/serverless-openwhisk/deploy/lib/deployRules.js:9:15)

Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues

Your Environment Information -----------------------------
OS: linux
Node Version: 7.3.0
Serverless Version: 1.4.0

Re-work event support for inline triggers

Currently the event configuration in the YAML file relies on the user to define triggers separately. Users define the rule and reference an external trigger. It would be nicer to support inline trigger definition in addition to this approach.

We should support two event types, trigger and HTTP (when API GW implementation is ready). Users should not have to define as little as possible to get up and running.

Add default events for common triggers.

Allow users to define events like schedule that will be converted to the rules and triggers that define the alarm trigger feed.

This will make it easier for developers to move from other providers.

Exception when deploying Message Hub trigger

Deploying a Message Hub trigger:

resources:
  triggers:
    sm-message-hub-incoming:
      parameters:
      feed: /whisk.system/messaging/messageHubFeed
      feed_parameters:
        endpoint: openwhisk.ng.bluemix.net
        kafka_brokers_sasl: '["kafka01-prod02.messagehub.services.eu-gb.bluemix.net:9093", "kafka02-prod02.messagehub.services.eu-gb.bluemix.net:9093", "kafka03-prod02.messagehub.services.eu-gb.bluemix.net:9093" ,"kafka04-prod02.messagehub.services.eu-gb.bluemix.net:9093", "kafka05-prod02.messagehub.services.eu-gb.bluemix.net:9093"]'
        kafka_admin_url: https://kafka-admin-prod02.messagehub.services.eu-gb.bluemix.net:443
        isJSONData: true
        topic: kb-tweets
        user: ${env:KAFKA_USER}
        password: ${env:KAFKA_PASSWORD}
        api_key: ${env:KAFKA_API_KEY}

Causes the following exception:

ServerlessError: Failed to deploy feed (messaging/messageHubFeed) due to error: Invalid URL for API called, OpenWhisk returned HTTP 404 response. The requested resource does not exist.
    at ow.feeds.create.catch.err (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/deploy/lib/deployFeeds.js:19:15)
    at tryCatcher (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/promise.js:510:31)
    at Promise._settlePromiseAt (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/promise.js:584:18)
    at Promise._settlePromises (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/promise.js:700:14)
    at Async._drainQueue (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/async.js:123:16)
    at Async._drainQueues (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues (/Users/shawdm/Documents/Projects/kappa-bluemix/code/serverless-openwhisk/node_modules/request-promise/node_modules/bluebird/js/main/async.js:15:14)
    at runCallback (timers.js:651:20)
    at tryOnImmediate (timers.js:624:5)
    at processImmediate [as _immediateCallback] (timers.js:596:5)
From previous event:
    at PluginManager.run (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:160:22)
    at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:91:31)
    at serverless.init.then (/usr/local/lib/node_modules/serverless/bin/serverless:23:50)
    at process._tickCallback (internal/process/next_tick.js:103:7)

Trying to put some debug information in deployFeeds.js I changed the deployFeed method to

deployFeed(feed) {
    return this.provider.client().then(function(ow){
      ow => ow.feeds.create(feed).catch(err => {
        throw new this.serverless.classes.Error(
          `Failed to deploy feed (${feed.feedName}) due to error: ${err.message}`
        );
      })
    });
  }

...so I could add some logging. Adding the normal function rather than the big arrow one seems to make it work, or at least not throw any exceptions, but I can't understand why. I'm sure this is my lack of understanding the big arrow function.

Confusing API Gateway Endpoint Logging

On deployment, the provider plugin prints the base API gateway path to the console. It doesn't list the individual function paths and methods. This can be confusing for a new user.

Here is the equivalent from AWS Lambda's provider....

[12:50:04 /tmp/hello]$ serverless deploy
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
.....
Serverless: Stack create finished...
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading service .zip file to S3 (409 B)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..................................
Serverless: Stack update finished...
Service Information
service: hello
stage: dev
region: us-east-1
api keys:
  None
endpoints:
  GET - https://99urhq7on5.execute-api.us-east-1.amazonaws.com/dev/path
functions:
  hello-dev-hello
[12:51:35 /tmp/hello]$

We should match this behaviour.

Support New API Gateway Service.

OpenWhisk has now moved the API Gateway service from Experimental to Production.

This has changed the API and other options. I need to update the API event plugin to send the Swagger YAML to the new endpoint.

Add Logging support

Serverless now supports retrieving logs for deployed functions.

[12:25:53 ~/code/bluemix/serverless-test/new_plugin]$ serverless logs --help
Plugin: Logs
logs .......................... Output the logs of a deployed function
    --function / -f (required) ......... The function name
    --stage / -s ....................... Stage of the service
    --region / -r ...................... Region of the service
    --tail / -t ........................ Tail the log output
    --startTime ........................ Logs before this time will not be displayed
    --filter ........................... A filter pattern
    --interval / -i .................... Tail polling interval in milliseconds. Default: `1000`

OpenWhisk has an API for activation logs, need to hook this up as a new command.

Add runtime support for non-Node.js languages.

Each runtime will need packaging in a different way to the current package bundle.
Native runtimes (Python, Java, Swift) should be easy to add.
Binaries runtimes could also be supported through dockerskeleton image.

serverless deploy results in syntax error due to strict mode not being supported

$ serverless deploy

Syntax Error -------------------------------------------

 Block-scoped declarations (let, const, function, class)
 not yet supported outside strict mode

 For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues

 Please report this error. We think it might be a bug.

Your Environment Information -----------------------------
OS: darwin
Node Version: 5.6.0
Serverless Version: 1.6.1

Release process 0.5 version

  • Update website documentation
  • Add examples for new features to serverless-examples repo
  • Write blog post about new release
  • NPM publish 0.5

Support HTTP event object parameter

This implementation currently supports the HTTP event in the following format.

events:
  - http: GET some/path

It does not support the secondary style that AWS Lambda's implementation also supports.

events:
  - http: 
    path: some/path
    method: GET

This implementation should be updated to support this.

Update README

serverless invoke --function my_function

refers to a function that isn't defined.

Other items...

  • The boilerplate also isn't necessary anymore with the template creation support.

Schedule event fails to redeploy

Updating schedule events fails as the API doesn't support updating these triggers.
The framework will have to ensure it is removed before deploying again.

Handle returned error code differently based on the serverless command

In the case where, for example, the API gateway functionality is disabled, calling serverless deploy fails when the command tries to get the api-gw endpoints. The command also returns an error code to the Linux shell, so any automated deployment script (for example in Jenkins) considers the deployment to have failed. However, the deployment succeeded, what failed was the call to display the summary info printed to the standard output.

The command should maybe only abort and return an error code if the "main" serverlesscommand failed. For example, serverless deploy should only fail when the deployment failed, while serverless info should fail if getting the api-gw endpoints has failed.

In the case where serverless deploy works but getting the api-gw info fails, this should still be displayed in the output of the command, the only difference should be that the command does not abort and does not return an error code to the Linux shell.

problems running invoke local -f hello

Steps to reproduce

$ serverless create --template openwhisk-nodejs
...
$ ls
README.md      handler.js     package.json   serverless.yml
$ serverless invoke local -f hello

Get errors

  Type Error ---------------------------------------------

     Cannot read property 'namespace' of undefined

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues

     Please report this error. We think it might be a bug.

  Your Environment Information -----------------------------
     OS:                 darwin
     Node Version:       6.7.0
     Serverless Version: 1.6.1

Running remotely after deploy works, but it doesn't locally

$ serverless deploy
$ serverless invoke -f hello
{
    "payload": "Hello, World!"
}

As workaround using this to run locally with a test.js

const action = require('./handler');
//const params = require('./params.json');
Promise.resolve(action.hello({name:"csantanapr"}))
  .then(result => console.log("test passed",JSON.stringify(result)))
  .catch(error => console.err("test failed",error));
$ node test.js
test passed {"payload":"Hello, csantanapr!"}

env:
~/foo/serverless
$ npm ls -g serverless
/Users/csantanapr/.nvm/versions/node/v6.7.0/lib
└── [email protected]

~/foo/serverless
$ npm ls -g serverless-openwhisk
/Users/csantanapr/.nvm/versions/node/v6.7.0/lib
└── [email protected]

Use packages to implement "services"?

The Serverless Framework uses the concept of services to logically group serverless functions running on the serverless platform.

On AWS, this corresponds to a separate cloudformation template and service definition for each service.

OpenWhisk has a different model (packages) for managing related serverless resources (actions, triggers, rules).

The current provider implementation uses service and stage identifiers within the resource name to group resources for a service. It does not deploy these resources into a custom package.

This leads to a large number of resources being deployed under the root namespace package. I can see this being challenging as the number of services grows.

One option would be to use the service identifer as a package identifier for each deployment. This would also make it easier to check which resources were deployed for a service.

Add support for zip package deployments

OpenWhisk now supports deploying a zip file to create an action. This can contain third-party NPM modules.

Need to review the deployment plugins to work out how we can take advantage of this.

Update documentation.

Need to review and update documentation with all the changes.
Also need to write a README for the project.

Move OpenWhisk client into provider

Serverless now provides a 'provider' API for registering clients with the frameworks.
Need to migrate from using the OpenWhisk API directly to this approach.

API management

Could you provide more details on the API mgmt implementation? From the endpoint URL I'm assuming that Bluemix IBM API Connect is being used, however I don't see the API connect service provisioned under my Bluemix account. Is some other instance of the service being used? How can I use my provisioned instance API Connect in Bluemix?

Thanks in advance.

Handle API errors without failing.

During the "remove" operation, if one of the API calls fails we want to allow the other operations to continue. It currently bails out after the first fail.

For example, if the user has manually removed one of the resources, we shouldn't block until they re-deploy.

CORS support

There needs to be support for CORS in serverless.yaml, similar to that provided by AWS

Add Info command

Serverless framework now as an info command.

This will provide output for the deployed functions from the service provider.

See here for the Google provider implementation.

Need to implement this for OpenWhisk.

Add ignore_certs constructor option (or make it available)

Add ignore_certs constructor option (or make it available) like wsk -i). Currently I get an error during deploy of the boilerplate.

Failed to deploy function (localess-dev-hello_world)
due to error: Error encountered calling OpenWhisk: Error:
self signed certificate

Add Event Type: Feed

Simplify creating event feeds by allowing user to provide trigger details as inline event.
This improves upon current experience of having to use resources section.

release checklist

  • docs on serverless.com
  • coordinate story for posts
  • blog post on serverless.com
  • blog post on IBM property
  • how-to video

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.