kodemore / chocs Goto Github PK

A tiny change to README.

Chocs installation depends on bjoern lib.
Bjoern build fails on Ubuntu as it's required additional lib libev-dev

Pls., add install of lib libev-dev in the README.
apt install -y libev-dev

Bjoern issue discussion:
jonashaag/bjoern#166

In my automated tests I'd like to compare the HttpResponse that a controller returns with the expected HttpResponse that I set up in my test. However this doesn't seem possible because HttpResponse objects use object identity for equality checks:

>>> from chocs import HttpResponse, HttpStatus
>>> HttpResponse(status=HttpStatus.OK) == HttpResponse(status=HttpStatus.OK)
False

Here's an example of a pytest assertion failure:

assert <chocs.http_response.HttpResponse object at 0x10d0389d0> == <chocs.http_response.HttpResponse object at 0x10d240880>

What do you think about making these classes more like dataclasses in this regard? (Value objects?)

An empty body does not raise any validation error even if the definition contains required fields.

openapi: 3.0.3

info:
  description: example
  version: "1.0.0"
  title: chocs_example

paths:
  /test:
    post:
      summary: test
      requestBody:
        description: test
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - email
              properties:
                email:
                  type: string
                  format: email

An empty body to this endpoint does not fail any validation.

http://json-schema.org/understanding-json-schema/reference/object.html#dependencies

Long term goal to be achieve cloud agnostic approach.

https://cloud.google.com/functions/docs/writing

Each endpoint may define x-auth property which will contain value of scope(s) that are required in order to access the endpoint. Additionally every schema in the component can additionally implement x-auth:* property to limit access to certain fields.

Examples:

/pets/{id}:
    get:
      description:
         ...
      x-auth: pet:read

In the above example pet:read scope is required to access GET pet/{id} resource.

NewPet:
      type: object
      required:
        - name
      properties:
        name:
          type: string
        tag:
          type: string
          x-auth:read: pet:read
          x-auth:write: pet:manage

In the above example pet:read is required to see tag property and pet:manage is required to change tag property.

Programmatically assinging scopes may happen in the middleware layer as shown below:

def authorise(request: HttpRequest, next) -> HttpResponse:
    token = request.headers["Api-Token"]
    is_valid = db.tokens.search(token)

    if not is_valid:
        return next(request)

    scopes = db.scopes.search(token)

    for scope in scopes:
        request.auth.add_scope(scope)

    return next(request)

To allow multiple Set-Cookie headers we need to allow non-unique header keys.

https://github.com/fatcode/chocs/blob/609e942e6100e20daa18944ba64cfb32c39c0a7d/chocs/headers.py#L38

See HTTP RFC2616 https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

Multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It MUST be possible to combine the multiple header fields into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy MUST NOT change the order of these field values when a message is forwarded.

Middleware at the moment is only supported in wsgi based applications, we should also support it in aws lambda.

Aws allows for defining any method route, this should be also reflected in the current implementation.

Long term goal to be achieve cloud agnostic approach.

Adding support for azure functions:

https://docs.microsoft.com/en-us/azure/azure-functions/create-first-function-cli-python?tabs=azure-cli%2Cbrowser

When using oneOf, the nested options are not validated at all.

The following openapi.yaml file demonstrate this behaviour:

```yaml
openapi: 3.0.3

info:
  description: example
  version: "1.0.0"
  title: chocs_example

paths:
  /test:
    post:
      summary: test
      requestbody:
        description: test
        required: true
        content:
          application/json:
            schema:
              oneOf:
                - $ref: '#/components/schemas/test_1'
                - $ref: '#/components/schemas/test_2'

components:
  schemas:
    test_1:
      type: object
      required:
        - id
      properties:
        id:
          type: string
    test_2:
      type: object
      required:
        - email
      properties:
        email:
          type: string
          format: email

```Python
import json
import logging
from typing import Any, Dict, Callable

import gunicorn
from chocs import Application, HttpRequest, HttpResponse, create_wsgi_handler
from chocs.middleware import OpenApiMiddleware
from gunicorn.app.base import BaseApplication


class StandaloneApplication(gunicorn.app.base.BaseApplication):
    def __init__(self, app: Any, _options: Dict[str, Any] = None):
        self.options = _options or {}
        self.application = app
        super().__init__()

    def load_config(self) -> None:
        config = {key: value for key, value in self.options.items() if key in self.cfg.settings and value is not None}

        for key, value in config.items():
            self.cfg.set(key.lower(), value)

    def load(self) -> Any:
        return self.application


def handle_errors(request: HttpRequest, _next: Callable) -> HttpResponse:
    try:
        return _next(request)
    except Exception as error:
        logging.getLogger().error(error)
        return HttpResponse(
            json.dumps({"error: ": error}),
            status=500,
            headers={"content-type": "application/json"},
        )


http = Application(
    handle_errors,
    OpenApiMiddleware("./openapi.yaml"),
)


@http.post("/test")
def create_withdraw_method(request: HttpRequest) -> HttpResponse:
    return HttpResponse(
        "{}",
        headers={"content-type": "application/json"},
        status=201,
    )


if __name__ == "__main__":
    host = "127.0.0.1"
    port = "8085"
    options = {"bind": f"{host}:{port}", "workers": 1, "reload": True}

    StandaloneApplication(create_wsgi_handler(http), options).run()

When running this example, I am able to pass anything in the body (including an empty body), without triggering any validation error.

The same bug is observed if the oneOf is used for a field in the body.

Route definition should accept additional parameter which is used to provide maximum execution time allowed for the function. This time can be also mapped into serverless configuration and should also work in dockerised environment.

from chocs import HttpRequest
from chocs import HttpResponse
from chocs import http

@http.post("/users", timeout=30)
def create_user(request: HttpRequest) -> HttpResponse:
    # if function takes more than 30 sec to execute it should time out with appropriate http response

    return HttpResponse("OK");

timeout should accept any positive integer value.

Unified logger fro wsgi and lambda integrations should be introduced. Possibility to filter PII information should be on of the features of such a logger, as well as integrating it with sentry or any other external provider.

There should be a way to have generic pagination system to use either, offset based pagination or cursor based pagination.

Route definition should accept additional parameter which is used to provide json schema compatible definition to validate input. Similar feature is available in aws rest api

from chocs import HttpRequest
from chocs import HttpResponse
from chocs import http

@http.post("/users", schema="./openapi.json#definitions/ExampleSchema")
def create_user(request: HttpRequest) -> HttpResponse:
    user = request.parsed_body # at this stage parsed body should be valid input mapped to corresponding type

    return HttpResponse("OK");

schema should either accept json schema definition file path, json schema definition or dataclass. We should also consider mapping input to corresponding dataclass when and if possible.

Automation cli tool should allow to:

• scrap all the routes definition from application code and dump them into routes configuration for aws
• use openapi.yml file to build application template with routes

Travis lately become unreliable and sluggish, there is already ready ci/cd pipeline in gata which can be reused here.

Right now the only way to define route's handler is by decorators, but to fully support open api spec operationId should also respected and used to define route's handler.

There are two possible problems with adding support for operationId:

• declaring two handlers for one route (if decorator and operation id is being used)
• pointing to invalid handler in operationId attribute (eg. handle name does not match any function name defined in your code)

Declaring two handlers for one route

This problem can be fixed by ignoring decorator if route already has a handler.

Pointing to invalid handler

There is no easy way to recover from this scenario and should end up with an exception being thrown. Additionally we might simply ignore support for operationId if developers are planning to use it for something else.

When using oneOf options and a string field with format that is present in both oneOf options, there is an incorrect validation, due to that field being transformed by the format validation.

The following example works if no string format is used, but fails as soon as string format is introduced:

openapi: 3.0.3

info:
  description: example
  version: "1.0.0"
  title: chocs_example

paths:
  /test:
    post:
      summary: test
      requestBody:
        description: test
        required: true
        content:
          application/json:
            schema:
              oneOf:
                - type: object
                  required:
                    - flag
                    - name
                  properties:
                    flag:
                      type: string
                    name:
                      type: string
                - type: object
                  required:
                    - flag
                    - pin
                  properties:
                    flag:
                      type: string
                    pin:
                      type: number
      responses:
        201:
          description: OK

Introducing string format for the flag field makes it fail:

openapi: 3.0.3

info:
  description: example
  version: "1.0.0"
  title: chocs_example

paths:
  /test:
    post:
      summary: test
      requestBody:
        description: test
        required: true
        content:
          application/json:
            schema:
              oneOf:
                - type: object
                  required:
                    - flag
                    - name
                  properties:
                    flag:
                      type: string
                      format: boolean
                    name:
                      type: string
                - type: object
                  required:
                    - flag
                    - pin
                  properties:
                    flag:
                      type: string
                      format: boolean
                    pin:
                      type: number
      responses:
        201:
          description: OK

Run cleanup for the codebase, assure mypy isnt failing and all tests are passing.

I've noticed that init_dataclass fails to initialise a dataclass with Optional types for ObjectId or enum classes, but works fine with others like str

from dataclasses import dataclass
from enum import Enum
from typing import Optional

from chocs.dataclasses import init_dataclass


class MyEnum(Enum):
    ELEM = 'hello'


@dataclass
class TestOk:
    something: MyEnum


@dataclass
class TestFails:
    something: Optional[MyEnum]


print(init_dataclass({"something": "hello"}, TestOk))
print(init_dataclass({"something": "hello"}, TestFails))

The following example fails when initialising TestFails when using the enum defined above, but works if I replace the enum type in the dataclass definition by str.

I see the same problem when using ObjectId