ryshoooo / treemodel Goto Github PK

Numpy strings of any lengths can be represented as '<U', i.e the longest_string parameter is not necessary.

NB: This will be very helpful in retyping and uniformization of trees.

Find a consistent way to apply a new schema to a built tree row and be able to safely transform old values to the new schema values.

FIrst questions #48 #49 #50 have to be answered.

The idea is having trees T1 and T2, then:

• T1 <= T2 iff T1 is a subtree of T2
• T1 < T2 iff T1 is a subtree of T2 and T1 != T2
• vice-versa for >= and >

This might heavily influence how method mul is defined, i.e. link to issue #23

Currently the child nodes do not have attribute level, which should be passed on when they are being gathered via forks.

This task is to allow level attribute at child nodes as well. It is essential for union method.

As a consequence, the intersection and union methods were actually with bugs, fix them as well.

Uniformize by:

• maximum common subtree
• minimum common suptree

The idea is that the user can write user-defined python function, which is then executed on each TreeRow and results in a new value with specified DataType.

Ex.

def multiply(a, b):
    return a * b
mul_udf = TreeUDF(multiply, FloatDataType())
tree_dataset['base/new_feature'] = mul_udf('base/old_feature_1', 'base/old_feature_2')

The current implementation of the intersection (mul in tree schema and fork node) matches from top to bottom, i.e. if the top doesn't match, nothing in the branches match.

Should this be the case or not?
What if there is a equivalent branch in other, which equals to the tree in self or vice versa? Should that be included? How do Arrays and Lists of trees fit into this concept?

1. All errors should be tested against
2. High probability of wrong levelling
3. Impossible error is achievable

Currently to initialize TreeDataType input TreeSchema is necessary, however the TreeDataType is not utilizing any of the schema possibilities.

Furthermore schemas should be only essential in datasets, not in data types, since datatypes are building blocks of schemas.