shanek / matador Goto Github PK

Front-end web interface for Bull Job Manager

License: MIT License

JavaScript 99.62% CSS 0.38%

matador's Issues

Current knockout data-bound tables are having issues where they sometimes get into infinite loops. For some reason, sometimes even if you only have one row it just keeps adding rows to the table forever. Not sure why.

I think converting the tables to react components would help with this issue. React components are also pretty fast in general.

Knockout error: pending is not defined

I'm getting this error now:

knockout.min.js:60 Uncaught ReferenceError: Unable to process binding "text: function (){return pending }"
Message: pending is not defined

Basic auth example

I would like to password protect the interface. I tried to copy the http-auth and express method but it didn't work. How can I achieve this?

Make matador available on npm

In the README it says you can use require('matador/app') which is great but the app isn't available on npm (as far as I know, matador on npm is something else). It would be a great way to include the ui as part of a worker to also make the upgrade path easy when you don't have any need for customizations anyway.

URL parsing maybe broken

I do not think that Matador can handle URL parsing when the input is as follows:

var app = require('bull-ui/app')({
  redis: {
    url: 'redis://:password@host:6379'
  }
});

It automatically falls back on 127.0.0.1:6379 after failing to parse the URL. I realized this after I tried to create a job from the /newjob page.

Here are my dependencies:

"dependencies": {
    "bull-ui": "1.2.3"
},

There is obviously a workaround:

var app = require('bull-ui/app')({
  redis: {
    host: 'pulkit-dev-3.shoppinpal.com',
    port: '6379',
    password: 'r3dis'
  }
});

But it thought it worth mentioning the breaking behaviour above.

Broken on 3.0.0 beta

2017-05-25T14:05:44.041Z - error: uncaughtException: failedJobs.keys[explodedKeys[1]].indexOf is not a function date=Thu May 25 2017 15:05:44 GMT+0100 (BST), pid=92140, uid=501, gid=20, cwd=/Users/etienne/Github/panelise/express, execPath=/usr/local/bin/node, version=v6.9.4, argv=[/usr/local/bin/node, /Users/etienne/Github/panelise/express/bin/www], rss=76759040, heapTotal=67305472, heapUsed=56975568, loadavg=[3.0810546875, 3.7080078125, 3.27099609375], uptime=1452267, trace=[column=106, file=/Users/etienne/Github/panelise/express/node_modules/bull-ui/models/redis.js, function=null, line=230, method=null, native=false, column=54, file=/Users/etienne/Github/panelise/express/node_modules/q/q.js, function=_fulfilled, line=794, method=null, native=false, column=30, file=/Users/etienne/Github/panelise/express/node_modules/q/q.js, function=self.promiseDispatch.done, line=823, method=promiseDispatch.done, native=false, column=13, file=/Users/etienne/Github/panelise/express/node_modules/q/q.js, function=Promise.promise.promiseDispatch, line=756, method=promise.promiseDispatch, native=false, column=44, file=/Users/etienne/Github/panelise/express/node_modules/q/q.js, function=null, line=564, method=null, native=false, column=17, file=/Users/etienne/Github/panelise/express/node_modules/q/q.js, function=flush, line=110, method=null, native=false, column=7, file=internal/process/next_tick.js, function=_combinedTickCallback, line=67, method=null, native=false, column=9, file=internal/process/next_tick.js, function=process._tickDomainCallback, line=122, method=_tickDomainCallback, native=false], stack=[TypeError: failedJobs.keys[explodedKeys[1]].indexOf is not a function,     at /Users/etienne/Github/express/node_modules/bull-ui/models/redis.js:230:106,     at _fulfilled (/Users/etienne/Github/express/node_modules/q/q.js:794:54),     at self.promiseDispatch.done (/Users/etienne/Github/express/node_modules/q/q.js:823:30),     at Promise.promise.promiseDispatch (/Users/etienne/Github/express/node_modules/q/q.js:756:13),     at /Users/etienne/Github/express/node_modules/q/q.js:564:44,     at flush (/Users/etienne/Github/express/node_modules/q/q.js:110:17),     at _combinedTickCallback (internal/process/next_tick.js:67:7),     at process._tickDomainCallback (internal/process/next_tick.js:122:9)]
Program node ./bin/www exited with code 1

Support for selecting redis db number

Currently there is no way to have Matador listen to another redis db other than db 0, since bull now supports using user defined user db, Matador should be able to do so as well.

TODO: Dynamic width for the memory chart

Need to add password option to redis config

Need to add password option for people who have password protected redis databases. Making this an issue because I keep forgetting about it.

Sidebar reports negative number of stuck jobs

Is this intended? Let me know if not - and I'll provide details to help debug. Thanks!

fix the screenshot image

Need to change how data is being refreshed

Currently data is just instantly refreshing itself pretty quickly. It's spamming the server and it makes it difficult to look at stuff in the networking tab.

This isn't actually too terrible, but it could be done better. I think switching to long polling or socket.io would be a better solution than what's currently in place.

license?

Hi Shane as you know I built those 2 other components off your core... but I am just realizing there is no license in this project... Could you please add a license ?

Thank you

low severity vulnerability message when installing

It looks like the version of lodash included has a low severity vulnerability. After npm install bull-ui and npm audit:

                       === npm audit security report ===                        
                                                                                
┌──────────────────────────────────────────────────────────────────────────────┐
│                                Manual Review                                 │
│            Some vulnerabilities require your attention to resolve            │
│                                                                              │
│         Visit https://go.npm.me/audit-guide for additional guidance          │
└──────────────────────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.17.5                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ bull-ui                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ bull-ui > lodash                                             │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/577                       │
└───────────────┴──────────────────────────────────────────────────────────────┘

I can certainly submit a pull request for this, but I haven't done anything with Matador other than install it, so don't know what kind of ramifications there could be from bumping lodash up two major versions...

Namespace issues.

It does not show anything if I'm using bull queue in a different namespace (e.g dev:bull and prod:bull).

Regards.

cannot "delete all failed jobs"

repro:
in Failed screen, click the gear icon dropdown, then click "delete all failed jobs"

expected: the jobs disappear

actual: the jobs are still there. I can see in the browser console there is an error:

Uncaught TypeError: Cannot read property 'name' of undefined
    at Object.deleteByStatus (RedisHandler.js:35)
    at HTMLAnchorElement.onclick (VM1131 failed:1)
deleteByStatus @ RedisHandler.js:35
onclick @ VM1131 failed:1

using versions:

    "bull": "2.2.6",
    "bull-ui": "1.2.1",

here is one of the failed jobs i have:

Job ID: 4
Type: test job
Status: failed

Data: {
  "num": 39906
}

Stack Trace: 
["TypeError: _.rand is not a function\n    at Queue.testJobBullQueue.process.job (/Users/brad/repos/nd-api/api/controllers/test-job.js:71:37)\n    at Queue.tryCatcher (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/util.js:16:23)\n    at Queue.handler (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/method.js:15:34)\n    at /Users/brad/repos/nd-api/node_modules/bull/lib/queue.js:742:30\n    at bound (domain.js:280:14)\n    at runBound (domain.js:293:12)\n    at tryCatcher (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/promise.js:512:31)\n    at Promise._settlePromise (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/promise.js:569:18)\n    at Promise._settlePromise0 (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/promise.js:614:10)\n    at Promise._settlePromises (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/promise.js:693:18)\n    at Async._drainQueue (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/async.js:133:16)\n    at Async._drainQueues (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/async.js:143:10)\n    at Immediate.Async.drainQueues (/Users/brad/repos/nd-api/node_modules/bluebird/js/release/async.js:17:14)\n    at runCallback (timers.js:637:20)\n    at tryOnImmediate (timers.js:610:5)\n    at processImmediate [as _immediateCallback] (timers.js:582:5)",null]

Graceful shutdown

Is there a way to gracefully shutdown Matador? An interval is still running while attempting to do a graceful shutdown of the server:

Intervals:
- (60000 ~ 60 s) (anonymous) @ /Development/integrations/node_modules/bull-ui/lib/updateInfo.js:54

Paused jobs are shown as stuck while they are not really stuck

Example code to reproduce this.

'use strict';

var Bull = require('bull');

var queue = new Bull('test', 6379, '127.0.0.1');

queue.pause();
queue.add({ name: 'first one' });

Can't npm install

There is some dependency problems on a fresh install. After a clone, I run npm install and get this:

npm ERR! peerinvalid The package express does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer [email protected] wants express@^3.16.1

npm ERR! System Linux 3.13.0-24-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! cwd /mnt/data/code/Matador
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.3.10
npm ERR! code EPEERINVALID
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /mnt/data/code/Matador/npm-debug.log
npm ERR! not ok code 0

I can chase it down locally, but it is something of a barrier to adoption ;)

Thinking about changing the front end

Greetings!

I'm considering redoing the front end, since the current front end is relying Knockout, which isn't particularly well known and fails in comparison to some modern solutions -- such as Anular or React.

I'm opening this issue to get an idea of what you guys think about this. I don't currently use this project anymore, but I understand that it is used, and if it would make it easier to use and contribute to in the future then I think this could be a valuable change to this project.

So, what do you think? Also, if you like the idea of the front end framework being changed, would you prefer Angular or React? I currently know most about Angular, but I'm not opposed to learning React.

Hook Up Socket.Io for real-time updates in the browser

Here's an example:
https://github.com/sayanee/Kraken_Example_SocketIO

TODO: Update Matador

Express updated and changed the way Kraken hooks into Express. The temporary solution is requiring Matador to use older versions of some of its dependencies (resolved issue #3), but this can't be relied on for long because eventually they'll remove the older versions of their packages (and also this is a crummy way to fix things). There is also a deprecation warning when starting Matador with this temporary solution, although Matador still works.

Need to update:
Make Matador use the latest versions of Kraken/Express

Security issues

We are using nsp to check for vulnerabilities and bull has 3 of them due to old version of express. Is there any plan to upgrade to a newer version of express?

> nsp check

(+) 3 vulnerabilities found

                Regular Expression Denial of Service                  

 Name           ms                                                    

 Installed      0.7.0                                                 

 Vulnerable     <=0.7.0                                               

 Patched        >0.7.0                                                

 Path            > [email protected] > [email protected] > d 

 More Info      https://nodesecurity.io/advisories/46                 


                Root Path Disclosure                                  

 Name           send                                                  

 Installed      0.10.1                                                

 Vulnerable     <0.11.1                                               

 Patched        >=0.11.1                                              

 Path            > [email protected] > [email protected] > s 

 More Info      https://nodesecurity.io/advisories/56                 


                Regular Expression Denial of Service                  

 Name           ms                                                    

 Installed      0.6.2                                                 

 Vulnerable     <=0.7.0                                               

 Patched        >0.7.0                                                

 Path            > [email protected] > [email protected] > s 

 More Info      https://nodesecurity.io/advisories/46

Delayed jobs are shown as "stuck"

Need to implement getDelayedKeys function that looks for redis keys of the form bull:*:delayed. Currently Matador lists delayed jobs as "stuck"

Locked up? Too many clients?

I love the Matador UI. But it's slow for me.

It worked great when I was prototyping. But now in production, I have 6 queues, each with a worker, running in Redis, handling about 2-3 jobs per second each.

The workers run fine, but once they get going, maybe 15-30 mniutes later, the Matador UI gets stuck. The express path just times out at 30s. Refresh the browser and rebooting the workers doesn't seem to fix it.

It seems to get stuck after several hundred thousand jobs have completed, or when a lot of jobs are running at once. During this time, the jobs keep working hunky-dory. It's just the Madator UI

Once stuck, even if I stop all the workers, still stuck. Only clearing redis completely seems to unstick it.

Could there be some sort of redis locking happening? Or are there scaling issues with Bull or something?

Maybe Matador is trying to load hundreds and hundreds of job detaisl under the hood, eagerly? I would figure it should only lazy-load details once I drill down, and just get counts at first load.

require path question

hi there. thanks for having built this. quick question though -- why are you doing your requires this way

updateInfo = require(process.cwd()+'/lib/updateInfo.js');

instead of, say, this way

updateInfo = require('../lib/updateInfo.js'); (depending of course where the file is)

I guess one has to choose whether do requires relative to an expected root (e.g., process.cwd()) versus relative to files. I think the latter is better.

Reason I think so, is what if I've required your Matador in another app, now all the paths are wrong.

I just want to know if there's something im unaware of as to why you did it this way?

thanks

shanek / matador Goto Github PK

matador's Issues

Recommend Projects

Recommend Topics

Recommend Org