soixantecircuits / bifrost Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 1.0 15.38 MB

A cache gateway for posting in the past

Home Page: http://soixantecircuits.fr

JavaScript 78.83% CSS 7.64% HTML 13.53%

bifrost's People

Contributors

Stargazers

Watchers

Forkers

philip-kazakov

bifrost's Issues

[nedb-migration-feature] request fails

Some post requests that works on the master branch seems to fail on nedb-migration-feature with no error, just proxy.js - fail - but saved

the request was made in meteor like this:

request.post({url: url, formData: formData}, function optionalCallback (err, httpResponse, body) {...})

Bifrost should implement mdns discovery

At startup bifrost should declare it's service and broadcast a bifrost service with port information.
This would allow other apps to seamlessly connect to bifrost if needed and post the content to save it

npm test fail

running npm test

proxy.js - launchRequest:  http://46.101.182.71/api/item/
/Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:331
          length = nativeMax(args.length - start, 0),
                   ^

RangeError: Maximum call stack size exceeded
    at max (native)
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:331:20
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:840:18
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/app/queue.js:42:9
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:3781:7
    at Object.q.process (/Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:2127:19)
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:2076:17
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:338:29
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/node_modules/async/dist/async.js:840:18
    at /Volumes/Macintosh HD/Users/gabrielstuff/Sources/node/bifrost/app/queue.js:42:9

npm ERR! Darwin 15.5.0
npm ERR! argv "/Users/gabrielstuff/.nvm/versions/v5.12.0/bin/node" "/Users/gabrielstuff/.nvm/versions/v5.12.0/bin/npm" "start"
npm ERR! node v5.12.0
npm ERR! npm  v3.8.6
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: `node $NODE_DEBUG_OPTION app.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script 'node $NODE_DEBUG_OPTION app.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the bifrost package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node $NODE_DEBUG_OPTION app.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs bifrost
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls bifrost
npm ERR! There is likely additional logging output above.

Form Data should accept multiple form type (content-type) and save the corresponding form type

Instead of considering only one type of posting, bifrost should care for :

Content-Type: multipart/form-data
Content-Type: application/x-www-form-urlencoded
Content-Type: raw
Content-Type: binary
Content-Type: binary
Content-Type: application/json

Simple POST script test could reflect the corresponding case

Add reason for saving file for posting later

The reason why the post failed should be indicate : network issue or remote server error

bodyparserLimit in config

Set bodyparserLimit in option config.

unexpected token d

when sending a base64 image like this: $http.post(url, base64) bifrost shows unexpected token d

hotfix was to use $http.post(url, {"formData": data})

Display details about failure on the Bifrost front

Currently the front does not show the reason for a POST failure. The front should display all the errors met while trying to post content to the remote URI.

Default value should follow a common pattern

Instead of :
if (!postData.timestamp) postData.timestamp = timestamp
We should :
postData.timestamp = timestamp || new Date()

The same principle should apply for every field.

request is sent twice

bifrost 0.0.1 is running on http://192.168.8.102:3002
queue.js - clear timer
proxy.js - post from proxy
proxy.js - launchRequest:  http://10.196.60.41:3000/api/storeMedia
proxy.js - fail - but saved
proxy.js - post from proxy - queue
proxy.js - launchRequest:  http://10.196.60.41:3000/api/storeMedia
queue.js: all requests have been processed
proxy.js - post from proxy - queue
proxy.js - launchRequest:  http://10.196.60.41:3000/api/storeMedia
queue.js: all requests have been processed
proxy.js - success + deleted
proxy.js - success + deleted
queue.js - clear timer

and I can see the image twice on the server

JSON.parse prone to errors

Line 34, we use JSON.parse without try catch.
https://github.com/soixantecircuits/bifrost/blob/master/app/proxy.js#L34

This is error prone and should be avoid.

Add a test with lot of failing query jobs

We need to test if a one day connection issue could be properly handle. This means around 1000 posts that should be queued.
Thanks !

Place the listening port in the config file

The port, line 152, should be placed in the config file

Add relevant test

The test folder should include :

a server script that accept POST file
a server script that accept base64 file
a server script with all form type

A test script for each POST type
A test script that shows that the response from the server is well forwarded
A test script that shows that if the test server fail, bifrost return that it saved it

The queue should use a NeDB nosql database instead of files

This would allow to:

Save content in database and meta data
Save files as blob or link to files needed to be uploaded