serverless-operations / jeffy Goto Github PK

This project forked from marcy-terui/jeffy

Jeffy is Serverless "Application" Framework for Python, which is suite of Utilities for Lambda functions to make it easy to develop serverless applications.

Python 100.00%

jeffy's People

Contributors

Stargazers

Watchers

Forkers

cm-iwata apriljo sinofseven t3yamoto manbo34 keisuke69 daiki-mori watanabe3tipapa twkiiim isabella232

jeffy's Issues

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I get an error when trying to use sqs decorator.

You can reproduce this error with this simple code.

from jeffy.framework import get_app
app = get_app()

@app.handlers.sqs()
def handler(event, context):
    return event['messageId']

An error is the below.

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/jeffy/encoding/json.py", line 41, in decode
    return json.loads(payload.decode('utf-8'))
  File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

jeffy.sdk.sqs.send_message message argument causes a little confuse

jeffy.sdk.sqs.send_message() is able to accept a dict object as message arg.

But this makes a little confuse. I feel like not intuitive. If I find 'send_message', then imagine this method has same interface as boto3's one.

I propose that add docstring more detailed at jeffy's send_message (It maybe not essential approach...)

[Logging] Add output log target(Firehose).

I want Kinesis Firehose.

Debug logs are going to info logger

Debug logs should go to debug.

If the object name is Japanese, an error occurs with `@app.handlers.s3()`.

I tried this sample code.

from jeffy.framework import get_app


app = get_app()


@app.handlers.s3()
def handler(event, context):
    print(event['key'])
    print(event['bucket_name'])
    print(event['body'])

It worked fine when the file test.txt was uploaded to s3, but the An error occurred when I uploaded a file called テスト.txt.

[ERROR] ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
Traceback (most recent call last):
  File "/var/task/jeffy/handlers/s3.py", line 59, in wrapper
    raise e
  File "/var/task/jeffy/handlers/s3.py", line 40, in wrapper
    response = s3.get_resource().get_object(Bucket=bucket, Key=key)
  File "/var/task/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/task/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)

A permission error occurs, but it looks like a different error.
If the object name is in Japanese, need to decode the URL.

Decorator seems not to work well.

In my environment, all decorators does not work.

I execute this simple source code locally with serverless framework.

from jeffy.framework import get_app
app = get_app()

@app.handlers.common
def handler(event, context):
    return event

I got a following error.

Traceback (most recent call last):
  File "/usr/local/lib/node_modules/serverless/lib/plugins/aws/invokeLocal/invoke.py", line 86, in <module>

    result = handler(input['event'], context)
TypeError: _common() takes 1 positional argument but 2 were given

There is no error when executing the same code with commenting out "@app.handlers.common".

[Feature] Add a decorator that can handle "events.Records" as an array

For events that receive records in array format, such as Kinesis/SQS, you may want to PUT the results of each event into DynamoDB.

In these cases, BatchWriter is used to write to the event, and it is more convenient in some cases to handle the records as they are.

Rename the existing SQS handler jeffy.handlers.sqs to another (e.g jeffy.handlers.sqs_each )
Re-define existing handler jeffy.handlers.sqs as the handler receives records as an array

Add handler to receive records as an array (e.g jeffy.handlers.sqs_batch / jeffy.handlers.sqs_records )

Add a decorator argument that tells the lambda handler receives an event object single record or arrayed records

As mentioned above, how about providing handlers that can be received both arrays/each element?

Related discussion: serverless-operations/jeffy-ja#1

from jeffy.sdk.sqs import Sqs

def handler(event, context):
    Sqs().send_message(
        queue_url=os.environ['QUEUE_URL'],
        message='hello world'
    )

Errors I got are the below.

Traceback (most recent call last):
  File "/usr/local/lib/node_modules/serverless/lib/plugins/aws/invokeLocal/invoke.py", line 72, in <module>

    module = import_module(args.handler_path.replace('/', '.'))
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "./sqs-publish.py", line 1, in <module>
    from jeffy.sdk.sqs import Sqs
  File "/usr/local/lib/python3.8/site-packages/jeffy/sdk/__init__.py", line 1, in <module>
    from jeffy import framework
  File "/usr/local/lib/python3.8/site-packages/jeffy/framework.py", line 1, in <module>
    from jeffy.handlers import Handlers
  File "/usr/local/lib/python3.8/site-packages/jeffy/handlers/__init__.py", line 7, in <module>
    from jeffy.handlers.s3 import S3HandlerMixin
  File "/usr/local/lib/python3.8/site-packages/jeffy/handlers/s3.py", line 6, in <module>
    from jeffy.sdk.s3 import S3
  File "/usr/local/lib/python3.8/site-packages/jeffy/sdk/s3.py", line 5, in <module>
    from jeffy.sdk import SdkBase
ImportError: cannot import name 'SdkBase' from partially initialized module 'jeffy.sdk' (most likely due to a circular import) (/usr/local/lib/python3.8/site-packages/jeffy/sdk/__init__.py)

Slack integration test

test

S3 Decorator Error: AttributeError: 'NoneType' object has no attribute 'logger' Traceback

I got an error when trying to use s3 decorator.

My code is below. (This is nearly equal to sample code in
https://github.com/serverless-operations/jeffy)

# Using python-requirements plugin of Serverless Framework.
# Unzipping package because 'pythonRequirements.zip' parameter is set true in 
# Serverless Framework
# Probably, unzipping operation effects nothing. 
# (In case of rest_api decorator, no error raised.) 
try:
    import unzip_requirements
except ImportError:
    pass

from jeffy.framework import get_app
app = get_app()

@app.handlers.s3()
def handler(event, context):

    print(event['bucket_name'])
    print(event['key'])

# Instead of using print method, I tried to use app.logger.info method, but the same error raised. 
# app.logger.info({'bucket': event['bucket_name']})

And error is the below.

[ERROR] AttributeError: 'NoneType' object has no attribute 'logger' Traceback (most recent call last):   
  File "/tmp/sls-py-req/jeffy/handlers/s3.py", line 50, in wrapper self.app.logger.exception(e)

Additional information:
In case of executing the code below, no error raised.

try:
    import unzip_requirements
except ImportError:
    pass

import json
from jeffy.framework import get_app
app = get_app()

def handler(event, context):

    app.logger.info({'bucket': event['Records'][0]['s3']['bucket']['name']})

[Feature] with_metadata handler for sns, sqs and kinesis_stream

Now, SNS, SQS and Kinesis handlers are only handling the body of message.
However, sometimes we need the other metadata, such as Message ID.