datajoely / kedro-rich Goto Github PK

Was wondering since no commits for over a year

In order to show some of the things that are possible via the rich library, some of the core Kedro library have been patched dynamically.

1. Logging Handler Override

This is already documented in #5 , the easiest solution would be to change this to the default in Kedro going forward.

2. Overriding the Node to string method:

kedro.pipeline.node.Node.__str__

This results in the node execution information being a bit cleaner and to the point. I've dropped the namespace information here since it is still sandwiched between catalog load/save operations that log that.

vs

2. Overriding the DataCatalog load/save methods:

kedro.io.data_catalog.DataCatalog.load|save

In the same vein changing the colours and the quote style instantly makes this look much snazzier:

vs

3. Provide nicer experience outside of a project

KedroCLI.get_command

I've always felt this behaviour has always catches beginners out. My implementation does this quite a drastic way, there are neater ways to do this here.

vs

4. Use rich_click where we can:

rich_click import RichGroup, RichCommand

This one I'm not as wedded to as the library is only on v1.2.1 at the time of writing. However it provides a really neat drop in replacement for click that seriously prettifies the help interface users get via the CLI:

vs

Description

• Currently kedro list-datasets is active and works
• It really should replace / override the kedro catalog list command
• I'm currently not sure if we can replace a command within a group via a plugin

Potential solutions

• Work out way to override
• Keep as is

Description

Due to the way the ParallelRunner works in Kedro via multi-processing the existing hook implementation breaks when interacting with whatever UI thread rich needs to work.

Context

Currently if the user specifies parallel runner the progress bar logic is disabled.

Potential solutions to evaluate:

1. Work out a way of making the existing implementation using Locks (preferred)
2. Subclass the ParallelRunner implementation and do it there (not ideal)

Description

• Kedro exposes kedro.line_magic as an entrypoint for a plugin to activate.
• This is the way the Kedro ipython extension works.
• I want to find a way of running this import statement at the time the user jumps into an ipython session so that the print statement is overridden.

Potential solutions to evaluate:

• Follow the route the ipython extension works and reverse engineer it.

Description

In order to enable the rich logging handler via an external plugin we currently intercept the way that the KedroSession parses the contents of conf/base/logging.yml and ensure that the console key is updated to use the right handler.

I tried for agest to intercept the handlers after creation and change them dynamically through approaches like this but this is the only mechanism I could get to work.

Potential solutions to evaluate:

• Work out the correct way to mutate the global logging objects so that the given log-handler is ignored without the need for monkey patching.
• Given rich is becoming quite mature at this point I think there is a decent argument for this becoming the default handler in the core Kedro library.