chrisjmccormick / word2vec_commented Goto Github PK
View Code? Open in Web Editor NEWCommented (but unaltered) version of original word2vec C implementation.
License: Apache License 2.0
Commented (but unaltered) version of original word2vec C implementation.
License: Apache License 2.0
Hi ,
First of all, Thanks for the great post and commentary. The example in the word2phrase.c - getWordHash function seems wrong to me.
Line 108 in word2phrase.c
hash = ((((h * 257) + a) * 257) + t) % 30E6
i guess the correct version should be :
hash = (((((hash * 257) + h) * 257) + a)*257+t) % 30E6
Please correct me if i am wrong.
word2vec_commented/word2phrase.c
Line 108 in 07e9576
The comment says that for word 'hat'
(((h * 257) + a) * 257 + t)
but the code has hash
initialized to 1 which makes the comment wrong
By the way thank you very much for comments sir. 👍
Dear Chris
I came across papers saying word2vec (SGNS) weighs context word different based on the distance to the centered word (higher weight for closer ones). I am quite curious on that and want to check their code, but had a hard time to figure this out...
I find your word2vec repo with many great comments. If possible, could you please point me/make a comment on where that weighing scheme is implemented in word2vec.c? (Or maybe they did not implement this at all...)
Thanks,
Matt
Hi , I read over your comment on the subsampling part.
The c implementation is
(sqrt(vocab[word].cn / (sample * train_words)) + 1) * (sample * train_words) / vocab[word].cn
However, in your comment, you says
Using the default 'sample' value of 0.001, the equation for ran is:
ran = (sqrt(x / 0.001) + 1) * (0.001 / x)
Should it be ran = (sqrt(x / 0.001 *train_words ) + 1) * (0.001 * train_words / x)
instead ?
Thank you
I think there's a small mistake in your writeup.
So, we do not require all sentences to be separated by two newline characters. Since we unget(ch) whenever ch == '\n', a single newline character will suffice. That is, when we encounter a single newline character, we put it back. In the next invocation of ReadWord(), we start at a=0 and immediately find our old newline character, so we return the EOS token.
Therefore, it suffices for every sentence to be on a line by itself.
I usually add a space to the end of the line before adding the newline character, but that's just my personal preference.
Let me know if I'm wrong, but this was my understanding of the code.
In the comment in line-1547, don't you think it should be sigmoid instead of softmax?
In word2vec.c file,it seems that variabe d is not declared.
Hey, I was just going through your code, and I think it's great! Just a minor correction here.
The Formatting of the markdown for the header seems to be off.
Regards
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.