ynqa / wego

Word Embeddings in Go!

wego's Issues

Scalability / Datasets larger than RAM

Hello,

I have not tried this out yet (I am excited to) but was wondering how this dealt with very large datasets, particularly those larger than RAM. Can wego handle this use case?

Thanks,
Glen

Neighbor attributes are private

Could you make Neighbor word and similarity accessible?

How to use `word-embedding` for French?

Do you think it is possible to use word-embedding for French? The difficulty is to find a corpus in French.

What do you think ?

Error when running example/example.go

I raised this issue on the gorgonia repo but the fix does not solve the problem. Any help appreciated.

gorgonia.org/gorgonia

..\gorgonia.org\gorgonia\walker.go:43:43: cannot use g (type *ExprGraph) as type graph.Directed in argument to topo.SortStabilized:
*ExprGraph does not implement graph.Directed (wrong type for Edge method)
have Edge(graph.Node, graph.Node) graph.Edge
want Edge(int64, int64) graph.Edge
..\gorgonia.org\gorgonia\walker.go:53:33: cannot use g (type *ExprGraph) as type graph.Directed in argument to topo.Sort:
*ExprGraph does not implement graph.Directed (wrong type for Edge method)
have Edge(graph.Node, graph.Node) graph.Edge
want Edge(int64, int64) graph.Edge

Example doesn't work

I ran the example with the text8 file and i get the following error

"read 17005207 words 2.4695522s
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc00029a008)
c:/go/src/runtime/sema.go:56 +0x49
sync.(*WaitGroup).Wait(0xc00029a000)
c:/go/src/sync/waitgroup.go:130 +0x6b
github.com/ynqa/wego/pkg/model/word2vec.(*word2vec).batchTrain(0xc00007c100, 0x2710, 0xa)
D:/FMI/golang_workspace/src/github.com/ynqa/wego/pkg/model/word2vec/word2vec.go:176 +0x14c
github.com/ynqa/wego/pkg/model/word2vec.(*word2vec).Train(0xc00007c100, 0x2c32e0, 0xc000006028, 0x0, 0xc000006028)
D:/FMI/golang_workspace/src/github.com/ynqa/wego/pkg/model/word2vec/word2vec.go:128 +0x5af
main.main()
D:/FMI/golang_workspace/src/project/main.go:71 +0x1cb

goroutine 50 [chan receive]:............................................."
and so on...
I have tried my own files with the same format as the one in text8 but I get the same mistake at some time during the execution.
Am I doing something wrong or there is some kind of issue with the example?

Move `similarity` to use `tensor.Tensor`

Along the way this also means fixing up the serialization strategy

Dependency broken

go get -u github.com/ynqa/word-embedding
package github.com/chewxy/gorgonia/tensor: cannot find package "github.com/chewxy/gorgonia/tensor" in any of:
.../chewxy/gorgonia/tensor (from $GOROOT)
.../src/github.com/chewxy/gorgonia/tensor (from $GOPATH)

Allow `io.Writer` as an output

Overview

Currently, it is not possible to flush the output to other sinks than file by providing its name as an argument of Save(outputFile string). Would it make a sense to provide a brother Save signature like the following one?

func (w *<Model>) Save(output io.Writer) error

This would handle files as well.

I would love to help with the implementation.

Improvement: doc2vec

Could you add also doc2vec (aka paragraph vectors)?

Use precalculated sigmoid tables

I was noticing some numerical issues yesterday while doing some profiling on this code. I seriously think we can get a speed boost by pre-calculating sigmoids and using a sigmoid table.

How can i use this package load a word2vec trained bin-package by python?

I used python to train a word2vec package. I want to use golang for servitization, but I don't know how to load the package I have trained，Anybody have any experience with that？ pls help me，thanks very much

Sentence simalirity

Dear,

is it possibile to do sentence similarity using embedding ?

Unsafe Cbow channel

Overview

Current implementation of Cbow use a channel of type []float64 for data exchange, but every trainOne function call needs to send and receive two []float64 vectors. Such communications are non-atomic. When multi-threads race at this channel, a pair of such vectors could be separated by another pair of vectors from other threads.

Abnormal Behaviors of Software Caused by This Issue

The Cbow training Example of this package could freeze due to this issue. It suspends during training and cannot proceed.

Actually, all Cbow trainings using this package could freeze.

Locations of Problematic Code

pkg/model/word2vec/model.go, line 80 ~ 83 (the data structure):

type cbow struct {
	ch     chan []float64
	window int
}

pkg/model/word2vec/model.go, line 103 ~ 107 (the go routine):

	agg, tmp := <-mod.ch, <-mod.ch
	defer func() {
		mod.ch <- agg
		mod.ch <- tmp
	}()

See, these sends and receives are non-atomic.

Possible Solutions to Overcome this Issue

We could define an auxiliary struct to avoid such race conditions, such as this:

type cbowToken struct {
	agg []float64
	tmp []float64
}

Revise the data structure of Cbow as:

type cbow struct {
        ch    chan cbowToken
        window int
}

Then in the go routine:

	token := <-mod.ch
	agg, tmp := token.agg, token.tmp
	defer func() {
		token := cbowToken{agg, tmp}
		mod.ch <- token
	}()

In this way, all sends and receives at this channel is atomic. Race conditions are eliminated.

How can I save model every epoch.

Overview

I am tryign to save my model every epoch, how can I achieve this?