grouplens / samantha Goto Github PK

For the movielens dataset, SGD (and ParallelSGD) will fail to converge when using the following settings with a timestamp ordered training set.

learningRate = 0.01
l2coef = 0.001

Randomizing the order of the training set fixes the problem—SGD converges and the resulting model has good performance.

We should (at the very least) make a note in the documentation that users may need to randomize the order of their datasets prior to training, particularly if you get poor performance or the algorithm fails to converge.

We may also want to offer a DAO that handles this randomization for the user.

It looks to me like the problem is that the evaluation portion of the code expects that all extracted features are used. It doesn't limit itself to only those features present in biasFeas, ufactFeas, and ifactFeas like it should (since that's what samantha does when training the model).

• Allow user to specify all available cores by passing in 0, null, or just not specifying a number
• Allow user to specify a specific number of cores by passing in a number >= 1 (up to the number of available cores)
• Allow user to specify a percentage of cores by passing in a number x between (0, 1), in which case we will multiply the number of available cores by x, rounding to the nearest integer (but not rounding down to 0 or up to the number of available cores).

//TODO: warn if attr is not present
//TODO: currently mostly feature extractors are using attrName in data to be the key internal representation, consider separate them and use attrName as default
public interface FeatureExtractor extends Serializable {
Map<String, List> extract(JsonNode entity, boolean update,
IndexSpace indexSpace);
}

I think we should consider making models configurable outside of predictors, recommenders, etc. Conceptually, I think this makes samantha much easier to explain and understand. It will be important to consider the consequences of this change to make sure that it is a good option.