sirex / databot Goto Github PK

Probably the best choice is to use Colander. Colander does not preserve primitive JSON types when serializing, see Pylons/colander#80 , so instead of importing things directly from Colander, something like this could be used:

import databot.schema as ds

Here ds would provide all Colander classes and would override primitive types to support serialization preserving primitive types.

bot.define will accept Colander SchemaNode as second argument.

And for the semantic information, probably something like JSON-LD could be used and this could be provided as third argument in bot.defined called context.

If Ctrl+C is press, script should stop running and exit. Now instead of ending work, it just marks rows as errors.

When defining a pipe, it should be possible to compress values. This would save a lot of space for html pages.

After compact, dedup or clean, errors are left detached from rows. A method is needed to clean up all stale errors.

If handler in call method for all of the rows returns no items, then state should be updated anyway.

This bug is already fixed, but tests needs to be written.