Coder Social home page Coder Social logo

nimbella-sdk-python's Introduction

Nimbella SDK for Python

A Python package to interact with nimbella.com services.

Installation

pip install nimbella

Usage

This SDK provides access to the following cloud services on Nimbella.com:

Redis

The nimbella.redis() method returns a pre-configured Redis client for use in your application. See the Redis client library documentation for the methods provided.

import nimbella

# Redis
redis = nimbella.redis()
redis.set("key", "value")
value = redis.get("key")

Object Storage (GCP & S3)

The nimbella.storage() method returns a pre-configured object storage client. This client exposes a high-level storage API (details below) - which hides the underlying storage provider implementation. The storage client is automatically configured to use the storage service for the cloud it is running on - which is GCS on GCP and S3 on AWS.

import nimbella

# Storage
bucket = nimbella.storage()
filename = "test.txt"
file = bucket.file(filename)
file.save('Expected %s contents' % filename, 'text/plain')

The nimbella.storage() constructor takes a single parameter web to determine whether the storage bucket is for a website (nimbella.storage(web=True)) or files (nimbella.storage()). Website buckets can be used for store web content (e.g. HTML & JS files) to host static websites.

Object Storage API

# Storage API
class StorageProvider(web=False):    
    # External bucket URL
    @property
    def url() -> Union[str, None]:

    # Configure website for web storage buckets
    def setWebsite(mainPageSuffix, notFoundPage):

    # Remove all files from the bucket (using optional prefix)    
    def deleteFiles(force, prefix):

    # Upload new file from path to bucket destination.
    def upload(path, destination, contentType, cacheControl):

    # Return storage file instance from bucket
    def file(destination) -> StorageFile:

    # Return all storage files (with optional prefix) instance from bucket
    def getFiles(prefix) -> list:
      
# Storage File Class
class StorageFile():
    # Name of the bucket file
    @property
    def name() -> str:

    # key/value pairs for provider-specific object metadata
    @property
    def metadata() -> dict:

    # does file exist?
    def exists() -> bool:

    # delete file from bucket
    def delete() -> None:

    # update file contents from string or bytes with content-type
    def save(data: Union[str, bytes], contentType: str) -> None:

    # return file contents as bytes
    def download() -> bytes:

    # return pre-signed url from file for external access
    def signed_url(version: str, action: str, expires: int, contentType: str) -> str:

Embedded SQL support

You can access embedded sql with: sql = nimbella.esql()

Available methods are:

sql.exec(sql, *arg)

Excecute an sql statement. The statement is either a string in SQL or an id returned by sql.prep (a prepared statement).

You can also pass multiple additional args, for parametric statementns.

It returns an array of [lastId, changedRows]

Values are significant where relevant (for insert or delete, but not for create for example).

Example:

# single statement
res = sql.exec("create table t(i int)")
# parametric statement
res = sql.exec("insert into t(i) values(?),(?),(?)",1,2,3)
# returns [3,3]

sql.map(sql, *args [,limit=<n>])

Execute an sql statement. The statement is either a string in SQL or an id returned by sql.prep (a prepared statement).

You can also pass multiple additional args, for parametric statementns.

It returns the result of an SQL query (like a SELECT) as an an array of dictionaries. Each dictionary is a record, where the keys are the fields and the values are the field values.

The optional keyword argument is the maximum number of records returned, it will limit the size of the returned array to the first <n>

Examples:

sql.map("select * from t")
# returns [{"i":1},{"i":2},{"i":3}]
sql.map("select * from t where i >?",1)
# returns [{"i":2},{"i":3}]
sql.map("select * from t where i >?",1,limit=1)
# returns [{"i":2}]

sql.arr(sql, *args [,limit=<n>])

Execute an sql statement. The statement is either a string in SQL or an id returned by sql.prep (a prepared statement).

You can also pass multiple additional args, for parametric statements.

It returns the result of an SQL query (like a SELECT) as an an array of arrays. Each array includes the field values of a record.

The optional keyword argument is the maximum number of records returned, it will limit the size of the returned array to the first <n>

Examples:

sql.map("select * from t")
# returns [[1],[2],[3]]
sql.map("select * from t where i >?",1)
# returns [[2],[3]]
sql.map("select * from t where i >?",1,limit=1)
# returns [[2]]

prep(sql)

You can prepare statements to save time from precompiling.

ins =  sql.prep("insert into t(i) values(?)")
sel = sql.prep("select * from t where i>?")

The returned value is a number and can be used to execute the statement with exec, map and arr.

# executing statement
res = sql.exec(ins,1)
# executing query
m = sql.map(sel,1,limit=1)

When you do not need any more you can close the statement running prep again with the returned value.

# closing prepared statements
sql.prep(ins)
sql.prep(sel)

Note that you can prepare up to 10000 statement at the same time without closing them, otherwise you will get an error too many prepared statement. In the unfortunate accident you fill the prepared statement cache, you can clear it with prep("clean_prep_cache")

Support

We're always happy to help you with any issues you encounter. You may want to join our Slack community to engage with us for a more rapid response.

License

Apache-2.0. See LICENSE to learn more.

nimbella-sdk-python's People

Contributors

joshuaauerbachwatson avatar jthomas avatar rabbah avatar satyarohith avatar sciabarracom avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar

nimbella-sdk-python's Issues

alias .redis to .kv

Is your feature request related to a problem? Please describe.
To lineup with nomenclature used through out the cli and docs, .redis() should be aliase to .kv().

Support endpoint_url in S3 client constructor

If you want to use the S3 storage client with a newer region - we need to explicitly set the endpoint_url parameter in the client config. This is an undocumented "feature" of the S3 service. boto/boto3#2728

We should also change the presigned url API parameters to be more generic. Rather than having an explicit "ContentType" - allow users to pass in any parameters they want. This will make it easier to decouple the implementation of the storage API from the providers API.

Bucket access AWS region related problem

Describe the bug
If nimbella is deployed in AWS in a generic region (for example eu-south-1) you get an error

An error occurred (IllegalLocationConstraintException) when calling the PutObject operation: The eu-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.

To Reproduce
Steps to reproduce the behavior:

  1. Install nimbella in AWS in another region (for example EKSItaly is in eu-south-1)
  2. Run this code:
def main(arg):
    try:
        bucket = nimbella.storage()
        filename = "test.txt"
        file = bucket.file(filename)
        file.save('Expected %s contents' % filename, 'text/plain')
        return { "ok": True}
    except Exception as e:
        return { "error": str(e) }

Expected behavior
ok: True

Screenshots
If applicable, add screenshots to help explain your problem.

Extra informations:

This is what the __NIM_STORAGE_KEY looks like (stripped down of credentials)

NIM_STORAGE_KEY="{\"provider\":\"@nimbella/storage-s3\",\"region\":\"eu-south-1\",\"endpoint\":\"\",\"weburl\":\"https://iogw-eksitaly.nimbella.io\",\"cloudfrontId\":\"XXXX\",\"iamUser\":\"app-buckets-iogw-Y0avZmVH\",\"policyArn\":\"arn:aws:iam::332320267357:policy/app-buckets-iogw-Y0avZmVH\",\"credentials\":{\"accessKeyId\":\"XXX\",\"secretAccessKey\":\"jXXX"}}"

SDK Website methods for setting website don't seem to work.

Finishing the implementation for the storage providers - I struggled to get the website methods to work despite calling the underlying SDK methods correctly. Need to go back and investigate this and write a new integration test to verify it works.

Support pluggable storage providers

Replicate work done here in the Node.js SDK to de-couple storage API from a single storage provider. Python SDK should also support pluggable storage providers. Introduce S3 storage provider alongside exsting GCP store.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.