Coder Social home page Coder Social logo

stephenpascoe / luigi Goto Github PK

View Code? Open in Web Editor NEW

This project forked from spotify/luigi

1.0 1.0 1.0 7.06 MB

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

License: Apache License 2.0

Python 91.37% JavaScript 5.95% HTML 2.32% Shell 0.19% CSS 0.17%

luigi's People

Contributors

anyman avatar berkerpeksag avatar davefnbuck avatar dchentech avatar deacondesperado avatar demoray avatar dhruvg avatar dhurlburtusa avatar erikbern avatar f355 avatar freider avatar gouthambs avatar gpoulin avatar graingert avatar hadesbox avatar jackdanger avatar jcrobak avatar jkukul avatar jonnelson avatar kawaa avatar mikekap avatar miku avatar przemekpastuszka avatar rantav avatar sisidra avatar tarrasch avatar themalkolm avatar tommyengstrom avatar ulzha avatar vitling avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

Forkers

migueldickson

luigi's Issues

sidebar not completely hidden when menu icon clicked

From spotify#1086:

The hamburger menu item has an extremely unintuitive purpose and I'm not sure why it exists. It might make a little sense if it completely hid the side bar, but it's confusing and unnecessary as-is.

This is a bug introduced when I fixed long task lists being inaccessible. It should completely hide the sidebar. I will fix.

Worker tab needs restyling to match tasks tab

From spotify#1086:

The workers tab did not survive the transition very well. The tasks are fairly hard to read compared to the nice layout of everything else in the new vis. It's a bit jarring.

I haven't worked on the worker tab at all. I would rather see this addressed as a future update unless it is completely broken.

Greater feedback on current filter constraints

From spotify#1086

It's hard to tell when something in the side bar is selected. I got confused a few times as to why my searches were giving such limited results until I realized I had left something selected. This would be more obvious if the highlight color stood out more than black on gray.

I agree, some more visual feedback on what the search constraints are would be useful. I can do that.

Same issue with disabled and upstream disabled, it's not obvious that they're selected because the highlight color blends in with the background.

I can change the colours but they look clear to me.

Not all Pending tasks shown

From spotify#1086

First, I have a couple thousand pending tasks but only 367 show up as pending on the main page. The worker page shows more than that. Did the internal representation change in a way that requires clearing the pickle?

Are any of these pending tasks upstream failures or disabled? I'm counting these independently. Maybe this is also related to the server limit (see below)?

Visualiser doesn't work with disabled tasks

From spotify#1086:

The task page seems to be ignoring tasks that were disabled by the scheduler. They're missing when I search for them and don't show up in the disabled task count. I need to be able to see them and re-enable them with an action button.

I haven't tested with disabled tasks because my workflows typically don't use them. Providing they are easy to simulate I should be able to get this fixed.

Allow users with large graphs to use SVG backend by default

From spotify#1086

Using D3 as the default graph is way too slow and makes the graph view unusable for me.

We know the D3 visualisation doesn't scale to thousands of tasks but one of the visualisations has to be default. It's awkward to make it configurable without changing how tornado serves the content.
Would it suffice for the visualiser to be specified in the URL fragment (http:/.../index.html#visType=...)?

Visualiser does not search tasks beyond server limit

From spotify#1086:

My done task count is beyond the limit where the server sends them all, so I get a question mark rather than a count on the tasks page.
I've lost the ability to search within done tasks when there are too many to display. This used to trigger a call to the server to fetch tasks that matched the query but no longer does.

I wasn't aware that the server has a limit to how many tasks are returned. I'll have to take a look at how this could be overcome. Although DataTables can work with server-side queries it works best when all data is retrieved and sorting/filtering happens in the client. An extra box to "filter server side" might be needed. Is there a way to tell from the API that not all results have been returned?

More informative titles in visualiser

From spotify#1086

The sidebar has two titles but neither one is informative and the link in the top one duplicates the Task List link. They can probably be safely removed.

The intention is that the "Luigi Task Status" title is part of the top bar, not the sidebar. I'm agnostic on whether we need the "TASKS" title or what it should say. Is "Task Family" well understood? The code uses "Task Name" interchangeably.

Sidebar should hide when on graph tab

spotify#1086

The sidebar should probably go away on the graph and worker tabs, as it doesn't affect them.

It could be hidden automatically when you select the graph tab -- providing I can fix the hamburger menu issue above.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.