giacbrd / shallowlearn Goto Github PK

I installed shallowlearn. But, I am getting an import error when trying to import the models.

replace fasttext.py with the official https://github.com/facebookresearch/fastText/tree/master/python

Iterating over documents and hashing words is almost an order of magnitude slower.
What can we do:

• the hash function used for hashing trick and also for matrices initialization should be written in Cython/C (but also the fastest hash functions seem slow)

implement https://arxiv.org/abs/1612.03651
see https://code.facebook.com/posts/162370720961195

Given #1, implementing n-grams of words

http://dl.acm.org/citation.cfm?id=2983755 (but it is only for linked documents!)

Taking into account piskvorky/gensim#847, implementing the hashing trick. See also http://radimrehurek.com/gensim/corpora/hashdictionary.html

Hello!

Sorry for a little bit strange question.
But tell me, please, in which case would it be better to use your library?

use both fastText and Gensim names in GensimFastText, raise a warning when two names for the same parameter are set, make it retro-compatible with current (wrong) names

If I load a model, with the load function, it cannot be saved with save. It misses of the file it creates after initialization.

In GensimFastText predicting more documents is a sequence of prediction calls. It would be preferable to implement as deeply as possible in the core (i.e. matrix computations in Cython) the computations of multiple inputs.

When I try to predict some sentence I get the prediction for each word.
Is this a bug?

Given a training set, it is possible to train the word embeddings (unsupervised) first, and then doing the label learning. It is similar to pass a pre-trained model, but more transparent

It should follow this convention: http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression.predict_proba

this would be interesting to integrate https://github.com/facebookresearch/Starspace, even if results are not better than fastText, and it is slower

Documents can be structured, made of different sections (title, body, tags (tags positions are not significant, but they dependent on the document body)), learned independently

Implementation:

• the input data is a dict of the sections
• in the learning algorithm: concatenate labels to words according to their section

reset_from is used, so all numeric vectors are initialized but not copied from the pre-trained model

fastText supervised model does not take into account of the document and words representation, it just learns bag of words and labels.
embeddings are computed only on the relation word->label. it would be interesting to learn jointly the semantic relation label<->document<->word<->context.
for now it is only possible to pre-train word embeddings and then use them as initial vectors for the classification algorithm

Your implemetion GensimFastText can use sentence iterator to feed and train fasttext model, that's Great. Because we may have very large text corpus, it's very not convenient to first convert them to a file and then train the model. Do you know why their official fasttext python wrapper cannot have a api which directly accept sentence iterator as corpus feeder?

a fastText tutorial has been published: https://github.com/facebookresearch/fastText/blob/master/tutorials/supervised-learning.md

Do a Jupyter Notebook!

from shallowlearn.models import GensimFastText

Traceback (most recent call last):
File "", line 1, in
File "/home/usr/anaconda2/lib/python2.7/site-packages/shallowlearn/models.py", line 15, in
import fasttext
File "/home/usr/anaconda2/lib/python2.7/site-packages/fasttext/init.py", line 1, in
from .fasttext import skipgram
ImportError: /home/usr/anaconda2/lib/python2.7/site-packages/fasttext/fasttext.so: undefined symbol: _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE

Tried with both 0.8.2 & 0.8.3 version of fasttext
and 0.13.4 & 1.0.1 version of gensim

Installed ShallowLearn using both 'pip' and by running the 'setup.py'

This could open to scenarios like:

• one-class classification (the training set is only made of positive examples)
• multi-label classification: learn with multiple labels but, given a test sample, the predicted label may not be the one with the highest probability (it could be not associable to any label)

@giacbrd @tmylk While running this document classification script, the performance graph generated shows training time to be lesser than test time.

But if we look closely, the training time is actually longer than test time for all the classifier. Here is the log

I think this is not the right way to convert the result into a range of (0,1) on the same axis

training_time = np.array(training_time) / np.max(training_time)
test_time = np.array(test_time) / np.max(test_time)

it contains interesting stuff for online training