n0obcoder / skip-gram-model-pytorch

PyTorch implementation of the Word2Vec (Skip-Gram Model) and visualizing the trained embeddings using TSNE

Skip-Gram-Model-PyTorch

PyTorch implementation of the word2vec (skip-gram model) and visualization of the trained embeddings using TSNE !

My TensorFlow implemntation of Skip-Gram Model can be found here.

Requirements

• torch >= 1.4
• numpy >= 1.18
• matplotlib
• tqdm
• nltk
• gensim

Training

python main.py

Visualizing real-time training loss in Tensorboard

tensorboard --logdir <PATH_TO_TENSORBOARD_EVENTS_FILE>

NOTE: By default, PATH_TO_TENSORBOARD_EVENTS_FILE is set to SUMMARY_DIR in config.py

Testing

python test.py

Inference

war	india	crime	guitar	movies	desert	physics	religion	football	computer
fight	europe	despite	band	movie	region	theory	religious	team	program
battle	central	help	play	series	along	mathematics	christian	win	systems
army	western	seek	record	show	western	mathematical	regard	sport	available
force	indian	challenge	piece	film	southern	study	tradition	club	design
ally	part	fail	star	feature	plain	science	christianity	league	information

Blog-Post

Check out my blog post on word2vec here.

skip-gram-model-pytorch's Issues

noise_dist

what`s the 'noise_dist' function？