Coder Social home page Coder Social logo

apache / openwhisk-package-cloudant Goto Github PK

View Code? Open in Web Editor NEW
15.0 38.0 38.0 446 KB

The Apache OpenWhisk cloudant package enables you to work with a Cloudant/CouchDB database

Home Page: https://openwhisk.apache.org/

License: Apache License 2.0

JavaScript 40.89% Shell 5.93% Java 4.93% Scala 47.97% Dockerfile 0.29%
openwhisk apache serverless faas functions-as-a-service cloud serverless-architectures serverless-functions

openwhisk-package-cloudant's Introduction

Apache OpenWhisk package for Cloudant

License Build Status

The /whisk.system/cloudant package enables you to work with a Cloudant database. It includes the following actions and feeds.

Entity Type Parameters Description
/whisk.system/cloudant package dbname, host, username, password Work with a Cloudant database
/whisk.system/cloudant/read action dbname, id Read a document from a database
/whisk.system/cloudant/write action dbname, overwrite, doc Write a document to a database
/whisk.system/cloudant/changes feed dbname, iamApiKey, iamUrl, filter, query_params, maxTriggers Fire trigger events on changes to a database

Firing a trigger on database changes

Use the changes feed to configure a service to fire a trigger on every change to your Cloudant database. The parameters are as follows:

  • dbname (required): The name of the Cloudant database.

  • iamApiKey (optional): The IAM API key for the Cloudant database. If specified will be used as the credentials instead of username and password.

  • iamUrl (optional): The IAM token service url that is used when iamApiKey is specified. Defaults to https://iam.bluemix.net/identity/token.

  • maxTriggers (optional): Stop firing triggers when this limit is reached. Defaults to infinite.

  • filter (optional): Filter function that is defined on a design document.

  • query_params (optional): Extra query parameters for the filter function.

The following topics walk through setting up a Cloudant database, configuring an associated package, and using the actions and feeds in the /whisk.system/cloudant package.

Setting up a Cloudant database in Bluemix

If you're using OpenWhisk from Bluemix, OpenWhisk automatically creates package bindings for your Bluemix Cloudant service instances. If you're not using OpenWhisk and Cloudant from Bluemix, skip to the next step.

  1. Create a Cloudant service instance in your Bluemix dashboard.

Be sure to create a Credential key, after creating a new service instance.

  1. Refresh the packages in your namespace. The refresh automatically creates a package binding for each Cloudant service instance that has a credential key defined.
wsk package refresh
created bindings:
Bluemix_testCloudant_Credentials-1
wsk package list
packages
/myBluemixOrg_myBluemixSpace/Bluemix_testCloudant_Credentials-1 private binding

Your package binding now contains the credentials associated with your Cloudant service instance.

  1. Check to see that the package binding that was created previously is configured with your Cloudant Bluemix service instance host and credentials.
wsk package get /myBluemixOrg_myBluemixSpace/Bluemix_testCloudant_Credentials-1 parameters
ok: got package /myBluemixOrg_myBluemixSpace/Bluemix_testCloudant_Credentials-1, displaying field parameters
[
    {
        "key": "username",
        "value": "cdb18832-2bbb-4df2-b7b1-Bluemix"
    },
    {
        "key": "host",
        "value": "cdb18832-2bbb-4df2-b7b1-Bluemix.cloudant.com"
    },
    {
        "key": "password",
        "value": "c9088667484a9ac827daf8884972737"
    }
]

Setting up a Cloudant database outside Bluemix

If you're not using OpenWhisk in Bluemix or if you want to set up your Cloudant database outside of Bluemix, you must manually create a package binding for your Cloudant account. You need the Cloudant account host name, user name, and password.

  1. Create a package binding that is configured for your Cloudant account.
wsk package bind /whisk.system/cloudant myCloudant -p username MYUSERNAME -p password MYPASSWORD -p host MYCLOUDANTACCOUNT.cloudant.com
  1. Verify that the package binding exists.
wsk package list
packages
/myNamespace/myCloudant private binding

Using CouchDB

CouchDB databases are compatible with this package but you need to ensure the CouchDB works via HTTPS and runs on the standard HTTP ports.

The simplest way to do this is to place a reverse proxy infront of the CouchDB server and then place a signed SSL certificate on it via a standard provider or a free service like LetsEncrypt.

Listening for changes to a Cloudant database

Filter database change events

You can define a filter function, to avoid having unnecessary change events firing your trigger.

To create a new filter function you can use an action.

Create a json document file design_doc.json with the following filter function

{
  "doc": {
    "_id": "_design/mailbox",
    "filters": {
      "by_status": "function(doc, req){if (doc.status != req.query.status){return false;} return true;}"
    }
  }
}

Create a new design document on the database with the filter function

wsk action invoke /_/myCloudant/write -p dbname testdb -p overwrite true -P design_doc.json -r

The information for the new design document is printed on the screen.

{
    "id": "_design/mailbox",
    "ok": true,
    "rev": "1-5c361ed5141bc7856d4d7c24e4daddfd"
}

Create the trigger using the filter function

You can use the changes feed to configure a service to fire a trigger on every change to your Cloudant database. The parameters are as follows:

  • dbname: Name of Cloudant database.
  • maxTriggers: Stop firing triggers when this limit is reached. Defaults to infinite.
  • filter: Filter function defined on a design document.
  • query_params: Optional query parameters for the filter function.
  1. Create a trigger with the changes feed in the package binding that you created previously including filter and query_params to only fire the trigger when a document is added or modified when the status is new. Be sure to replace /_/myCloudant with your package name.
wsk trigger create myCloudantTrigger --feed /_/myCloudant/changes \
--param dbname testdb \
--param filter "mailbox/by_status" \
--param query_params '{"status":"new"}'
ok: created trigger feed myCloudantTrigger
  1. Poll for activations.
wsk activation poll
  1. In your Cloudant dashboard, either modify an existing document or create a new one.

  2. Observe new activations for the myCloudantTrigger trigger for each document change only if the document status is new based on the filter function and query parameter.

Note: If you are unable to observe new activations, see the subsequent sections on reading from and writing to a Cloudant database. Testing the following reading and writing steps will help verify that your Cloudant credentials are correct.

You can now create rules and associate them to actions to react to the document updates.

The content of the generated events has the following parameters:

  • id: The document ID.
  • seq: The sequence identifier that is generated by Cloudant.
  • changes: An array of objects, each of which has a rev field that contains the revision ID of the document.

The JSON representation of the trigger event is as follows:

{
    "id": "6ca436c44074c4c2aa6a40c9a188b348",
    "seq": "2-g1AAAAL9aJyV-GJCaEuqx4-BktQkYp_dmIfC",
    "changes": [
        {
            "rev": "2-da3f80848a480379486fb4a2ad98fa16"
        }
    ]
}

Writing to a Cloudant database

You can use an action to store a document in a Cloudant database called testdb. Make sure that this database exists in your Cloudant account.

  1. Store a document by using the write action in the package binding you created previously. Be sure to replace /_/myCloudant with your package name.
wsk action invoke /_/myCloudant/write --blocking --result --param dbname testdb --param doc "{\"_id\":\"heisenberg\",\"name\":\"Walter White\"}"
ok: invoked /_/myCloudant/write with id 62bf696b38464fd1bcaff216a68b8287
{
  "id": "heisenberg",
  "ok": true,
  "rev": "1-9a94fb93abc88d8863781a248f63c8c3"
}
  1. Verify that the document exists by browsing for it in your Cloudant dashboard.

The dashboard URL for the testdb database looks something like the following: https://MYCLOUDANTACCOUNT.cloudant.com/dashboard.html#database/testdb/_all_docs?limit=100.

Reading from a Cloudant database

You can use an action to fetch a document from a Cloudant database called testdb. Make sure that this database exists in your Cloudant account.

  • Fetch a document by using the read action in the package binding that you created previously. Be sure to replace /_/myCloudant with your package name.

    wsk action invoke /_/myCloudant/read --blocking --result --param dbname testdb --param id heisenberg
    
    {
      "_id": "heisenberg",
      "_rev": "1-9a94fb93abc88d8863781a248f63c8c3",
      "name": "Walter White"
    }

Using an action sequence and a change trigger to process a document from a Cloudant database

You can use an action sequence in a rule to fetch and process the document associated with a Cloudant change event.

Here is a sample code of an action that handles a document:

function main(doc){
  return { "isWalter:" : doc.name === "Walter White"};
}

Create the action to process the document from Cloudant:

wsk action create myAction myAction.js

To read a document from the database, you can use the read action from the Cloudant package. The read action may be composed with myAction to create an action sequence.

wsk action create sequenceAction --sequence /_/myCloudant/read,myAction

The action sequenceAction may be used in a rule that activates the action on new Cloudant trigger events.

wsk rule create myRule myCloudantTrigger sequenceAction

Note The Cloudant changes trigger used to support the parameter includeDoc which is not longer supported. You will need to recreate triggers previously created with includeDoc. Follow these steps to recreate the trigger:

wsk trigger delete myCloudantTrigger
wsk trigger create myCloudantTrigger --feed /_/myCloudant/changes --param dbname testdb

The example illustrated above may be used to create an action sequence to read the changed document and call your existing actions. Remember to disable any rules that may no longer be valid and create new ones using the action sequence pattern.

Building from Source

To build this package from source, execute the command ./gradlew distDocker

openwhisk-package-cloudant's People

Contributors

abaruni avatar anthonyamanse avatar buggtb avatar cbickel avatar chetanmeh avatar csantanapr avatar dgrove-oss avatar dubee avatar jasonpet avatar jbampton avatar johnsolm avatar markusthoemmes avatar mhenke1 avatar mrutkows avatar rabbah avatar style95 avatar vipulkashyap111 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openwhisk-package-cloudant's Issues

Feature request: Cloudant package, changes feed: add filtering parameters

@bseyb commented on Wed Oct 19 2016

Why

I was trying to listen to changes of a cloudant feed and then send an email using sendgrid, when one flag 'process' was set to true.
For this I needed to implement one openwhisk action to filter out whether 'process' was set to true and 'mailsend' set to false.

Suggestion

Looking at the documentation of cloudant I saw that you can add add filter and doc_ids to a changefeed. It would be very awesome to add these parameters, so that filtering can already be taken care of at the change feed instead of the user created action.

Bonus

I also saw that you can pass oldDoc as a parameter. This could be helpful if you want to compare what exactly changed in an action.


@csantanapr commented on Thu Nov 24 2016

Hi @bseyb I started looking into the design on providing a way to filter changes for the cloudant trigger.

Looking at the spec of the changes API

It looks like a filter and docs_id is doable, a filter will be a filter function on a design document.

What I didn't find was oldDoc during my testing the changes feed only provide the id and deleted=true and the new revision marked as deleted.

I was not able to find a reference to oldDoc in the changes API params.
Could you share a bit more where you saw this reference?


@bseyb commented on Tue Dec 13 2016

Hi @csantanapr I must have misread it, I cannot find oldDoc in the correct context either.

Having that Filter function available would help a whole lot.


@rabbah commented on Tue Jan 24 2017

@csantanapr move this to openwhisk-catalog-cloudant? FYI @johnsolm @dubeejw.

Write with Overwrite = true fails if doc doesn't exist

If you do a write with --param overwrite 'true' on a non-existent doc id:

wsk action invoke cloudant/write --blocking --result --param overwrite 'true' --param doc '{"foo": "bar", "_id": "nonexistent_docid"}'

it gives an error:

        "stack": "Error: missing\n    at Request._callback (/nodejsAction/node_modules/cloudant-nano/lib/nano.js:247:15)\n    at Request.self.callback (/nodejsAction/node_modules/request/request.js:186:22)\n    at emitTwo (events.js:106:13)\n    at Request.emit (events.js:191:7)\n    at Request.<anonymous> (/nodejsAction/node_modules/request/request.js:1081:10)\n    at emitOne (events.js:96:13)\n    at Request.emit (events.js:188:7)\n    at IncomingMessage.<anonymous> (/nodejsAction/node_modules/request/request.js:1001:12)\n    at IncomingMessage.g (events.js:291:16)",
        "statusCode": 404

But if I remove the --param overwrite 'true', it works. And then subsequent writes with --param overwrite 'true' succeed and update the revision.

In this case, I think it's safe to ignore the 404 error trying to do the GET to fetch the existing revision, since it's expected that it might be the first revision, and no existing revisions exist.

Possible bug around empty query_params when filter is present

This issue has two parts:

  1. It was discovered that when trying to create a Cloudant Trigger if one specifies a "filter" but leaves the "query_params" blank the server will return an error complaining that the "query_params" are not valid JSON.

I believe this happened after the recent fix for validating the "query_params" to be correct JSON.

From glancing at the code it appears the fix would possibly be to add an extra condition to check for undefined query params and to allow them through when the filter is specified.

  1. A user was reporting that when they would supply {} as their "query_param" (so that they could get past the Invalid JSON error) the filter will no longer fire on their Trigger.

pick up from the last change handled by using since

We should record the last-sequence and save it in the trigger DB.
This way if the server goes away (i.e restarted) when it returns and start following changes, it will pick up from the last changes handled.

In the handler where the trigger is fired, also save the changes.last_seq , then use this value if present when starting to follow changes, if not set then use "now"

Support for Trigger filters

Maybe the CouchDB / Cloudant Design filter function for change feed can be utilized to specify desired filter function. Filter function can also take query parameter(s), which makes it easier to do more dynamic filtering "on demand"...

Filter functions

https://docs.cloudant.com/design_documents.html#filter-functions

Query parameters:
https://docs.cloudant.com/cloudant_query.html#query-parameters

Filter function with query parameter:

GET /$DATABASE/_changes?filter=$DESIGN_ID/$FILTER_FUNCTION&status=new

Filter function described in CouchDB The Definitive Guide

http://guide.couchdb.org/draft/notifications.html

CouchDB on filter functions with query parameter sample:

http://couchdb.readthedocs.io/en/latest/couchapp/ddocs.html#filterfun

So e.g. when creating OpenWhisk trigger, a "filter" parameter could specify the filter function:

wsk -v trigger create myTrigger --feed /org_space/cloudantPackage/changes --param dbname mydb --param includeDoc true --params filter mailbox/by_status&status=new

Do not error when feed action is invoke with namespace `_`

Have the feed action go get the value for namespace if the one pass as parameter is _

We are instructing users to unset the namespace here:
https://console.ng.bluemix.net/openwhisk/learn/cli

wsk property unset --namespace

The feed action will need to handle when user uses the new auth key method and no namespace is configured in the CLI.

Options:

  1. Have the feed action do a a OW API call to get the namespace for the authkey
  2. Have the runtime provide the namespace string associated with the authkey being used to invoke the feedaction. Today there is a namespace environment variable but it doesn't have the value that we need it has the value of the owner of the action not the namespace associated with the authkey.

remove step to set namespace from the docs

Now the feed action takes the namespace from the env variable in the action process.env.__OW_NAMESPACE there is no requirement for the whisk client to send the namespace.

Also now that openwhisk has a unique api key per namespace, user can't switch to a different namespace with the same key, and this is why all clients should be configured with namespace _ and the identity is retrieve using api key.

update annotations for feed action changes

currently in installCatalog.sh

$WSK_CLI -i --apihost "$APIHOST" action update --auth "$AUTH" cloudant/changes "$PACKAGE_HOME/actions/changes.js" \
    -t 90000 \
    -a feed true \
    -a description 'Database change feed' \
    -a parameters '[ {"name":"dbname", "required":true} ]'

Need to add two more parameters filter and query_params

Cloudant property misnamed in help

castrop commented:

For "wsk package get $cloudant binding name", a required parameter for "exec-query-view" is incorrectly listed as "view". It should be "viewname"

whisk object is deprecated

the actions in this package use the now deprecated whisk object for nodejs actions.
as part of removing these references and moving the code to promises, refactoring many lines of duplicated functionality is in order.

install catalog for this package doesn't conform to openwhisk-catalog

the installation script for this catalog is different and doesn't reuse any of the utils from the openwhisk-catalog --- this is bad because it means one needs a completely different process to install this catalog (locally) compared to the other package.

for better or worse, all our packages should follow a standard process for deployment. so we should have only one (whether it uses the pattern here or in the other repo), and make every package adapt accordingly.

Following the README.md, the first time to write the doc '_design/mailbox' will get an error

Following the document README.md to create a filter, when I type wsk action invoke /_/myCloudant/write -p dbname testdb -p overwrite true -P design_doc.json -r and return, I will get an error:

"Error: missing\n    at Request._callback (/nodejsAction/node_modules/cloudant-nano/lib/nano.js:247:15)\n    at Request.self.callback (/nodejsAction/node_modules/request/request.js:186:22)\n    at emitTwo (events.js:106:13)\n    at Request.emit (events.js:191:7)\n    at Request.<anonymous> (/nodejsAction/node_modules/request/request.js:1081:10)\n    at emitOne (events.js:96:13)\n    at Request.emit (events.js:188:7)\n    at IncomingMessage.<anonymous> (/nodejsAction/node_modules/request/request.js:1001:12)\n    at IncomingMessage.g (events.js:291:16)",

I think here we should remove -p overwrite true since it's the first time to create the doc '_design/mailbox'. README.md should be updated.

Package should prefer _selector filters over JavaScript filter functions

Cloudant / CouchDB has supported using Query selectors as an alternative to JavaScript filter functions for over a year. Query selectors have a significant performance advantage over using JavaScript filters - they can be roughly an order of magnitude more efficient - and do not require users to inject a design document prior to reading changes.

The OpenWhisk package should strongly push users towards using Query selectors in favour of JS functions. Essentially the changes would be:

  • use filter=_selector instead of asking users to specify a design doc/filter
  • make a POST request to _changes instead of GET
  • in the request body, include a "selector" field containing the Query selector to match documents against.

Exclude views and design documents from when querying for trigger documents

Creation of a design document caused the trigger service to crash because the design document is interpreted as a trigger definition.

/cloudantTrigger/lib/utils.js:66
            dbHost = dbHost.replace('https://','');
                           ^

TypeError: Cannot read property 'replace' of undefined
    at createTrigger (/cloudantTrigger/lib/utils.js:66:25)
    at /cloudantTrigger/lib/utils.js:230:28
    at Array.forEach (native)
    at /cloudantTrigger/lib/utils.js:218:27
    at Request._callback (/cloudantTrigger/node_modules/nano/lib/nano.js:217:16)
    at Request.self.callback (/cloudantTrigger/node_modules/request/request.js:199:22)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:191:7)
    at Request.<anonymous> (/cloudantTrigger/node_modules/request/request.js:1036:10)
    at emitOne (events.js:101:20)

in-code docs have hrefs to non-existant pages

An error in Dockerfile

The image built based on Dockerfile in the root folder is not runnable now because /logs/cloudantTrigger_logs.log is not exist.

Below changes are an option to fix this issue.

--- a/Dockerfile
+++ b/Dockerfile
@@ -16,6 +16,7 @@ RUN apt-get update --fix-missing && \
 # only package.json
 ADD package.json /cloudantTrigger/
 RUN cd /cloudantTrigger; npm install
+RUN touch ~/alarmsTrigger_logs.log

 # App
 ADD provider/. /cloudantTrigger/
@@ -23,4 +24,4 @@ ADD provider/. /cloudantTrigger/
 EXPOSE 8080

 # Run the app
-CMD ["/bin/bash", "-c", "node /cloudantTrigger/app.js >> /logs/cloudantTrigger_logs.log 2>&1"]
+CMD ["/bin/bash", "-c", "node /cloudantTrigger/app.js >> ~/cloudantTrigger_logs.log 2>&1"]

Specify doc id as parameter outside of doc body when creating doc via write action

I'm trying to build a sequence of actions where one action produces:

{
    "doc": {
        "IsTruncated": false,
        "Marker": null,
        "Users": [
            {
                "Arn": "arn:aws:iam::9798798:user/[email protected]",
                "CreateDate": "2016-01-11T23:49:40Z",
                "PasswordLastUsed": "2017-06-07T17:41:08Z",
                "Path": "/",
                "UserId": "ERXEAIDAHGJJK87878KKW",
                "UserName": "[email protected]"
            },
        ...
    ]
}

and the output is connected to a cloudant write action.

Everything works, however it creates a new unique doc id every time the action sequence is invoked, which isn't what I want in this case. The only way that I can tell to have it write to the same doc id is to modify the upstream action to produce:

{
    "doc": {
        "_id": "aws_user_list",
        "IsTruncated": false,
        "Marker": null,
        "Users": [
            {
                "Arn": "arn:aws:iam::9798798:user/[email protected]",
                "CreateDate": "2016-01-11T23:49:40Z",
                "PasswordLastUsed": "2017-06-07T17:41:08Z",
                "Path": "/",
                "UserId": "ERXEAIDAHGJJK87878KKW",
                "UserName": "[email protected]"
            },
        ...
    ]
}

It would be a "nice-to-have" option to be able to bind the doc id parameter to a cloudantdb write action so that when I call the action it will use that doc id, rather than having to include the _id parameter in the doc contents.

Actually I just realized this is assuming it's even possible to create an action based on an existing cloudant write action and bind extra parameters to it. I'm not sure how to do that, but I thought I remember reading that it was possible.

Replace the binary Wsk with REST interface WskRest

All the non-CLI related tests should sunset the wsk, and re-implement them with wskrest.

import common.wsk 
val wsk = new Wsk

need to be changed to

import common.rest.WskRest
val wsk = new WskRest

plus there may be changes to the rest of the tests as well, since the response result may have been changed.

improve cloudant changes feed error handling (especially for bogus host or dbname)

when creating a cloudant.changes trigger, if one provides a dbname that does not exist, cloudant now rejects the request. but the openwhisk changes action does not propagate this error through in a way that allows a user to know what went wrong:

firstly, the statusCode is 502, which implies author error, i.e. in some code i wrote (because i get the same 502 e.g. if i have a typo in my action source), whereas this is a case of invoker error. i don't think we have error codes that distinguish between author error and invoker error.

secondly, the response body has no logs or error text that'd help intuit what went wrong:

{"name":"changes","subject":"[email protected]","activationId":"9b6bce0c279940b6
b830eed71e5ceb12","publish":false,"annotations":[],"version":"0.0.45","response":{"result":{"error":null},"success":false,"status":"application error"},"end":
1467120637640,"logs":[],"start":1467120637039,"namespace":"[email protected]"}

@markus-thoemmes found the underlying backend log (that does not appear in what the client sees):

[2016-06-28T13:30:37.644Z] [ERROR] [??] [addDataTrigger] Error occurred for trigger /uitest3_uitest3/whisk_system-cloudant-changes_nodejs__trigger (db xxx): {"code":"ENOTFOUND","errno":"ENOTFOUND","syscall":"getaddrinfo","hostname":"xxx"}`

pragmatic suggestion: at the very least, the errno:ENOTFOUND should be passed through to the client; better, perhaps, would be to pass through the entire error response. in this case, the error was actually not in the dbname, but in the hostname; i.e. the getaddrinfo syscall failed to find hostname xxx.

ideal suggestion: changes should interpret syscall==getaddrinfo && errno==ENOTFOUND as "hostname field invalid"; similarly, for whatever error cloudant passes back for a bogus dbname, the response should be "dbname field invalid".

update backoff strategy to be exponential

The strategy backoff currently is linear with default values of 1second delay with 10 retries.

It should use an exponential backoff instead.
With default values of 10seconds defayl with 10 retries.

Because of OpenWhisk rate limit on trigger limits, it will return 429 statusCode.
OpenWhisk window for the rate limit is 1 minute.

I would make it on first 429 delay for 1 minute, if after 1 minute delay there is a another 429 then I would start exponential back off of 1 seconds (i.e. 2s, 4s, 8s, 16s, 32s, 64s etc..)

Refactor action names and update document

The are a few actions that are not documented but installed.

I think it would be better to rename some of the actions to be nounVerb
For example:
read-document -> documentRead
delete-document -> documentDelete
read-attachment -> attachmentRead
etc..

After this rename we can update the docs to for the actions, and list the old ones as deprecated, having both install for a while.

manage-bulk-documents action fails with incorrect Cloudant request body

When I use the manage-bulk-documents action, it fails because the query sent to Cloudant is not a JSON object. The action expects a JSON object with a docs key that contains an array of documents and so does Cloudant. However, the action only sends the array without the container object and the docs key. So what Cloudant receives is:

[{"_id":"0399a368b906664af2d0e39de79a8ccf","_rev":"1-d55be7137d8447de7062c9ffed88341c","_deleted":true}]

while it expects:

{"docs":[{"_id":"0399a368b906664af2d0e39de79a8ccf","_rev":"1-d55be7137d8447de7062c9ffed88341c","_deleted":true}]}

The action should retain the container object with the docs key to ensure that what is passed to Cloudant is correct.

Steps to reproduce

Send the manage-bulk-documents action a query like:

{
  "docs": [ actual docs go here ],
  "dbname": "mydb"
}

It will fail with the error indicated below.

Workaround

A workaround is to send the action an object like this:

{
  "docs": {
    "docs": [ actual docs go here ]
  },
  "dbname": "mydb"
}

Error details

uri: "https://XXXXXX:[email protected]/codes-db/_bulk_docs",
body: "[{"_id":"0399a368b906664af2d0e39de79a8ccf","_rev":"1-d55be7137d8447de7062c9ffed88341c","_deleted":true}]"
},
description: "couch returned 400",
scope: "couch",
reason: "Request body must be a JSON object",
error: "bad_request",
stack: "Error: Request body must be a JSON object at Request._callback (/nodejsAction/node_modules/cloudant-nano/lib/nano.js:247:15) at Request.self.callback (/nodejsAction/node_modules/request/request.js:186:22) at emitTwo (events.js:106:13) at Request.emit (events.js:191:7) at Request.<anonymous> (/nodejsAction/node_modules/request/request.js:1081:10) at emitOne (events.js:96:13) at Request.emit (events.js:188:7) at IncomingMessage.<anonymous> (/nodejsAction/node_modules/request/request.js:1001:12) at IncomingMessage.g (events.js:292:16)",
message: "Request body must be a JSON object",
errid: "non_200",
headers: {
statusCode: 400,
x-cloudant-request-class: "write",
cache-control: "must-revalidate",
uri: "https://XXXXXX:[email protected]/codes-db/_bulk_docs",
x-cloudant-backend: "bm-cc-uk-04",
date: "Fri, 13 Apr 2018 07:58:18 GMT",
content-type: "application/json",
via: "1.1 lb1.bm-cc-uk-04 (Glum/1.50.4)",
x-couch-request-id: "9564b6b77f",
x-frame-options: "DENY",
x-content-type-options: "nosniff",
strict-transport-security: "max-age=31536000"
}
}
},
success: false,
status: "application error"
},
cause: "a09af6f4223f42d69af6f4223f42d637",
end: 1523606299620,
logs: [
"2018-04-13T07:58:19.612240359Z stdout: Error: { Error: Request body must be a JSON object",
"2018-04-13T07:58:19.612292573Z stdout: at Request._callback (/nodejsAction/node_modules/cloudant-nano/lib/nano.js:247:15)",
"2018-04-13T07:58:19.612318225Z stdout: at Request.self.callback (/nodejsAction/node_modules/request/request.js:186:22)",
"2018-04-13T07:58:19.612326284Z stdout: at emitTwo (events.js:106:13)",
"2018-04-13T07:58:19.612332269Z stdout: at Request.emit (events.js:191:7)",
"2018-04-13T07:58:19.612338124Z stdout: at Request.<anonymous> (/nodejsAction/node_modules/request/request.js:1081:10)",
"2018-04-13T07:58:19.612344609Z stdout: at emitOne (events.js:96:13)",
"2018-04-13T07:58:19.612350314Z stdout: at Request.emit (events.js:188:7)",
"2018-04-13T07:58:19.612356059Z stdout: at IncomingMessage.<anonymous> (/nodejsAction/node_modules/request/request.js:1001:12)",
"2018-04-13T07:58:19.612362053Z stdout: at IncomingMessage.g (events.js:292:16)",
"2018-04-13T07:58:19.6123677Z stdout: name: 'Error',",
"2018-04-13T07:58:19.61237329Z stdout: error: 'bad_request',",
"2018-04-13T07:58:19.612378858Z stdout: reason: 'Request body must be a JSON object',",
"2018-04-13T07:58:19.612384286Z stdout: scope: 'couch',",
"2018-04-13T07:58:19.612389608Z stdout: statusCode: 400,",
"2018-04-13T07:58:19.612395147Z stdout: request:",
"2018-04-13T07:58:19.612400364Z stdout: { method: 'POST',",
"2018-04-13T07:58:19.612405802Z stdout: headers:",
"2018-04-13T07:58:19.612411188Z stdout: { 'content-type': 'application/json',",
"2018-04-13T07:58:19.612416727Z stdout: accept: 'application/json' },",
"2018-04-13T07:58:19.612422952Z stdout: uri: 'https://XXXXXX:[email protected]/codes-db/_bulk_docs',",
"2018-04-13T07:58:19.612428666Z stdout: body: '[{"_id":"0399a368b906664af2d0e39de79a8ccf","_rev":"1-d55be7137d8447de7062c9ffed88341c","_deleted":true}]' },",
"2018-04-13T07:58:19.612435273Z stdout: headers:",
"2018-04-13T07:58:19.612440656Z stdout: { 'cache-control': 'must-revalidate',",
"2018-04-13T07:58:19.612446081Z stdout: 'content-type': 'application/json',",
"2018-04-13T07:58:19.612451549Z stdout: date: 'Fri, 13 Apr 2018 07:58:18 GMT',",
"2018-04-13T07:58:19.612456953Z stdout: 'x-couch-request-id': '9564b6b77f',",
"2018-04-13T07:58:19.61246244Z stdout: 'x-frame-options': 'DENY',",
"2018-04-13T07:58:19.612467943Z stdout: 'strict-transport-security': 'max-age=31536000',",
"2018-04-13T07:58:19.612473238Z stdout: 'x-content-type-options': 'nosniff',",
"2018-04-13T07:58:19.61247879Z stdout: 'x-cloudant-request-class': 'write',",
"2018-04-13T07:58:19.612484218Z stdout: 'x-cloudant-backend': 'bm-cc-uk-04',",
"2018-04-13T07:58:19.612489575Z stdout: via: '1.1 lb1.bm-cc-uk-04 (Glum/1.50.4)',",
"2018-04-13T07:58:19.612495081Z stdout: statusCode: 400,",
"2018-04-13T07:58:19.61250041Z stdout: uri: 'https://XXXXXX:[email protected]/codes-db/_bulk_docs' },",
"2018-04-13T07:58:19.612511338Z stdout: errid: 'non_200',",
"2018-04-13T07:58:19.612521052Z stdout: description: 'couch returned 400' }"
],
start: 1523606299173,
namespace: "consideratehoteliers.com_dev"
}

Is it possible to override cloudant actions with default params?

I'm planning to invoke a cloudant/write action as the second action in a two action sequence, and I want to specify the -p overwrite 'true' parameter, but I don't want to modify the first action in the action sequence to add the -p overwrite 'true' parameter to it's output for reasons of loose coupling.

The only way I can see to do that is during the binding phase:

wsk package bind /whisk.system/cloudant myCloudant -p overwrite 'true' -p username MYUSERNAME -p password MYPASSWORD -p host MYCLOUDANTACCOUNT.cloudant.com

which means that all actions in the myCloudant bound package will have always have -p overwrite 'true' set, which isn't what I necessarily want. As a workaround I was thinking of creating two bindings:

  • myCloudant
  • myCloudantOverwrite

and calling myCloudantOverwrite/write when I needed to write with the overwrite parameter set to true.

Or is there a better way to customize the behavior of cloudant/write via default parameters? Is it possible to create a new action based on an existing action, and customize the default parameters?

Improve error handling on trigger fire, do not delete trigger mark it paused instead

Currently the trigger is being deleted from the DB on certain error scenarios

We need to improve how do we handle each use case, I think there are 3 use cases when firing

  1. Trigger has being removed from OpenWhisk, Controller is reachable and returns Not Found (404)
  2. Errors that are not recoverable (wrong auth 403, etc..), then delete trigger from db
  3. Errors that with a retry can be succesfully fire on a retry, but never delete the trigger from db, and instead mark the trigger doc in db "paused: true"
  4. others?

paused state for triggers is something that could be use when redudancy providers is implemented

cloudant action annotations for dbname erroneously omit bindTime: true

@starpit commented on Wed Jun 08 2016

we would like a way to distinguish dbname as a special parameter, one that we suggest the user provide at package binding time, rather than invocation time.

for example, the read-document action throws an exception if dbname is not provided, and it is unlikely that in normal scenarios dbname will be provided as input from a prior action in a sequence, or by a trigger as part of a rule.

whereas the docID field is much more likely to be in-flow data. therefore, dbname should be prompted for at package binding time

Solution: every cloudant action that specifies dbname as required should also specify it as bindTime: true


@starpit commented on Wed Jun 08 2016

All of the cloudant actions that specify dbName as required must also specify bindTime: true


@markusthoemmes commented on Fri Mar 10 2017

@csantanapr move to catalog?

includeDocs getting set to 'false' string

The code in initTrigger from util.js contains the following

        if (obj.includeDoc) {
            includeDoc = ((obj.includeDoc === true || obj.includeDoc.toString().trim().toLowerCase() === 'true')) || 'false';
        }

includeDocs is getting set to string 'false' instead of boolean false
Needs to be change to

        if (obj.includeDoc) {
            includeDoc = ((obj.includeDoc === true || obj.includeDoc.toString().trim().toLowerCase() === 'true')) || false;
        }

Confused on namespace instructions

I'm trying to configure my local openwhisk running under Vagrant to connect to Cloudant.

From the instructions:

Be sure to remember the name of the service instance and the Bluemix organization and space you're in.

Make sure your OpenWhisk CLI is in the namespace corresponding to the Bluemix organization and space that you used in the previous step.

wsk property set --namespace myBluemixOrg_myBluemixSpace

How do I find the service instance and bluemix org? Is this the Cloudant service instance, Bluemix OpenWhisk service instance, or Bluemix overall service instances?

increase default trigger limit

Suggest we bump the default limit for cloudant to infinity.

Marking critical as users are being affected and we should do this ASAP.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.