nerium-data / nerium Goto Github PK

View Code? Open in Web Editor NEW

7.0 7.0 4.0 423 KB

The little business intelligence engine that could

License: Apache License 2.0

Python 98.07% Dockerfile 1.93%

nerium's People

Contributors

Stargazers

Watchers

Forkers

gitter-badger nick-desteffen millerjs navah

nerium's Issues

500 INTERNAL SERVER ERROR in Takei Line Item Endpoint Responses

A 500 INTERNAL SERVER ERROR precedes the 200 OK response from the following query:

http 'http://nerium-takei:8081/v1/takei/takei-line-item-history/affix/?client_id={$client_id}&ad_id={$ad_id}'

301 MOVED PERMANENTLY Responses Heard as Errors at Takei Organizations Endpoint

At the "takei organizations" endpoint, I noticed query strings and the base path to the affix endpoint both return a 301 MOVED PERMANENTLY instead of whatever the more appropriate response would be. Addressing these points will allow the front end to handle things a bit better should the JS or browser behavior lead to breaks in those ways.

HTTP/1.1 301 MOVED PERMANENTLY
Connection: close
Content-Length: 315
Content-Type: text/html; charset=utf-8
Date: Mon, 22 Jan 2018 18:06:13 GMT
Location: http://nerium-takei:8081/v1/takei/takei-organizations/
Server: gunicorn/19.7.1

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="http://nerium-takei:8081/v1/takei/takei-organizations/">http://nerium-takei:8081/v1/takei/takei-organizations/</a>.  If not click the link.

Switch all Reporting Platform components to Apache 2.0 license

I'm starting to think about what needs to happen to make the Reporting Platform fully public — by which, for clarity, we'll say means pushing semantically versioned releases to major public package repositories (e.g., PyPi, npm, Docker Hub). As the product of a commercial venture itself, and in the interest of wider adoption, the copyleft requirements of the GPL aren't appropriate. Among non-copyleft, permissive, "pushover" licenses, Apache 2.0 is the recommendation of the FSF and their reasoning looks sound to me. I also notice that it seems to be popular among the caliber and type of projects that I'd like to be playing among, — viz. Luigi, Superset, &c.

Refactor error handling to allow an error code per exception type.

Put format in the path instead of the slightly awkward query param

I was thinking about this and it occurred to me that the format could be specified as an optional extension to the path rather than using the awkward ne_format query param

/v1/query/<query-name>?a=b would use the default formatter while
/v1/query/<query-name>/toml?a=b would use the toml formatter

Separate serialization schema from serialization mechanism

When Neirum formatters were implemented as methods, it wasn't so noticeable that we were combining schema and output type (JSON, YAML, CSV, etc) into a single class called "format". Now that has changed to using schema definitions with marshmallow, the app no longer provides a general way to request a different serialization system than JSON (although, owing to the way Responder works, any available schema may be returned as either JSON or YAML by changing the Accept header of the request, and there is an endpoint for CSV).

Not clear how important this is, but wanted to note it here.

Pipfile.lock not copied into build

Set Uncaught Exceptions' JSON Responses

If nerium can capture uncaught exceptions, most often seen as "Internal Server Error" messages, that would allow the client to manage these errors like any other, by looking at the error property. As is, the JS will need to take this potential response into account repeatedly, at the expense of maintainability and, potentially, performance.

http 'http://nerium-takei:8081/v1/takei/takei-orders/affix/?client_id=361&advertiser_name=Visa'

HTTP/1.1 500 INTERNAL SERVER ERROR
Connection: close
Content-Length: 37
Content-Type: application/json
Date: Mon, 22 Jan 2018 17:38:02 GMT
Server: gunicorn/19.7.1

{
    "message": "Internal Server Error"
}

Add detailed gunicorn config, to pass along access and error logs, work async, etc.

Add flexible query script storage

It would be cool to be able to fetch query files from a remote URL, GCS (using google-api-client rather than FUSE), etc. Local filesystem would still be the default.

I'm thinking another subclassable abstract method that returns a file-like object, and ResultSet can use whatever the app is configured for.

Use namedtuple instead of munch for query object

Why add a dependency when something from the standard library does the same job?

PUT/POST endpoints for adding/modifying backend definitions and new query scripts

While the simplicity/built-in nature of flat-file query script storage is dear to my heart, I could see where it would be handy to be able to upload new queries directly to Nerium itself via its API. (Would allow for limiting access to the filesystem for some report developers, for example.)

With this, we could also a develop a simple CLI that would upload a file to a running Nerium instance from a user's local system, keeping with the staticgen/text editor UI model.

Move all OAO deployment pieces to a private repo.

Again related to public release: The public Nerium is a package, and can be installed to a Dockerfile in one pipenv command (via -e git currently, and eventually PyPi). This is what we should be building and deploying.

ModuleNotFoundError: No module named 'starlette.lifespan'

Actually an issue with underlying responder dependency, noted here.

Until Responder merges in a fix, there is a workaround:

$ pipenv install nerium --pre
$ pipenv uninstall starlette --pre
$ pipenv install 'starlette<0.9' --pre
$ nerium