hankcs / cs224n

CS224n: Natural Language Processing with Deep Learning Assignments Winter, 2017

CS224n

CS224n: Natural Language Processing with Deep Learning Assignments Winter, 2017

Requirements

• Python 2.7
• TensorFlow r1.2

Assignment #1

1. Softmax
2. Neural Network Basics
3. word2vec
4. Sentiment Analysis

Assignment #2

1. Tensorflow Softmax
2. Neural Transition-Based Dependency Parsing

924/924 [==============================] - 49s - train loss: 0.0631    
Evaluating on dev set - dev UAS: 88.54
New best dev UAS! Saving model in ./data/weights/parser.weights
================================================================================
TESTING
================================================================================
Restoring the best model weights found on the dev set
Final evaluation on test set - test UAS: 88.92
Writing predictions
Done!

3. Recurrent Neural Networks: Language Modeling

Assignment #3

1. A window into NER

DEBUG:Token-level confusion matrix:
go\gu   PER     ORG     LOC     MISC    O    
PER     2968    26      84      16      55   
ORG     147     1621    131     65      128  
LOC     48      88      1896    26      36   
MISC    37      40      54      1030    107  
O       42      46      18      39      42614
DEBUG:Token-level scores:
label   acc     prec    rec     f1   
PER     0.99    0.92    0.94    0.93 
ORG     0.99    0.89    0.77    0.83 
LOC     0.99    0.87    0.91    0.89 
MISC    0.99    0.88    0.81    0.84 
O       0.99    0.99    1.00    0.99 
micro   0.99    0.98    0.98    0.98 
macro   0.99    0.91    0.89    0.90 
not-O   0.99    0.89    0.87    0.88 
INFO:Entity level P/R/F1: 0.82/0.85/0.84

2. Recurrent neural nets for NER

DEBUG:Token-level confusion matrix:
go\gu   PER     ORG     LOC     MISC    O    
PER     2987    32      47      12      71   
ORG     136     1684    90      70      112  
LOC     39      83      1907    21      44   
MISC    43      45      47      1031    102  
O       36      56      15      34      42618
DEBUG:Token-level scores:
label   acc     prec    rec     f1   
PER     0.99    0.92    0.95    0.93 
ORG     0.99    0.89    0.80    0.84 
LOC     0.99    0.91    0.91    0.91 
MISC    0.99    0.88    0.81    0.85 
O       0.99    0.99    1.00    0.99 
micro   0.99    0.98    0.98    0.98 
macro   0.99    0.92    0.89    0.91 
not-O   0.99    0.90    0.88    0.89 
INFO:Entity level P/R/F1: 0.85/0.86/0.85

3. Grooving with GRUs

DEBUG:Token-level confusion matrix:
go\gu	PER  	ORG  	LOC  	MISC 	O    
PER  	2920 	41   	57   	12   	119  
ORG  	101  	1716 	73   	64   	138  
LOC  	22   	95   	1908 	16   	53   
MISC 	37   	45   	53   	1017 	116  
O    	21   	67   	14   	39   	42618

DEBUG:Token-level scores:
label	acc  	prec 	rec  	f1   
PER  	0.99 	0.94 	0.93 	0.93 
ORG  	0.99 	0.87 	0.82 	0.85 
LOC  	0.99 	0.91 	0.91 	0.91 
MISC 	0.99 	0.89 	0.80 	0.84 
O    	0.99 	0.99 	1.00 	0.99 
micro	0.99 	0.98 	0.98 	0.98 
macro	0.99 	0.92 	0.89 	0.90 
not-O	0.99 	0.91 	0.88 	0.89 

INFO:Entity level P/R/F1: 0.86/0.85/0.85

4. Easter Egg Hunt!
   • Run python q3_gru.py dynamics to unfold your candy eggs

References

CS224n official website

• http://web.stanford.edu/class/cs224n/index.html

Many code snippets come from

• https://github.com/rymc9384/DeepNLP_CS224N
• https://github.com/gxlzj/cs224n-hw3​

cs224n's Issues

The link for the sh file of assignment1 is down. Where can I find the zip file? Can you help. Thx.

The link for the sh file of assignment1 is down. Where can I find the zip file? Can you help. Thx.

what are the inputVectors and outputVectors?

can someone explain what are the inputVectors and outputVectors? why the vectors be split to two parts? thank you

np.exp(x)遇到了RuntimeWarning: overflow encountered in exp

楼主有遇到吗？test2的1001和1002一用exp函数就溢出了...

Assignment 1 q2_neural.py softmax gradient not explicitly calculated

In calculating gradients, the gradient of the softmax function is not calculated using the formula that is derived in the lecture notes. It seems like in the code, this step is skipped over, and the gradient of the cost function with respect to yhat is used only ('d3' variable). Am I missing something here?

AttributeError: 'Namespace' object has no attribute 'func'

this error shows in q2_rnn

哪里可以找到空的作业

哪里可以找到没有答案的作业

关于assignment1中下载的文本的编码问题

博主,您好.我现在在做assignment1.我想用python3做完.然后就出现了一些编码问题...
主要就是下载的文本中,datasetSentences.txt 和dictionary.txt 这两个文本中的句子没办法match.
然后我发现datasetSentences.txt中的文本显示有少量乱码,因此我估计该文本编码方式有问题.然后在作业给的程序中我发现,读dictionary.txt文本时直接用read()方法即可(treebank.py中120行往下).但是在读datasetSentences.txt时的操作却让人很费解: read().decode("utf8").encode("latin1").程序中解释是"Deal with some peculiar encoding issues with this file"(treebank.py中54行往下),但没具体说为什么..博主能不能帮忙理一下,这个操作的逻辑是什么?
然后,在python3中我应该怎样正确读取datasetSentences.txt呢?因为python3中read()是不能接decode操作的,我试了下read().encode("latin1")和直接read()都不成功...
非常感谢啦!

关于作业1中求matrix的softmax

Hi，hankcs.我在做作业1的时候参考学习你的代码，其中关于softmax的矩阵运算(q1_softmax.py中35行)。
请问此处denom=lambda x:1.0/np.sum(x)是不是归一化分母，如果按照归一化分母的写法，是不是应该是denom=lambda x:1.0/np.sum(exp_minmax)或者denom=lambda x:1.0/np.sum(np.exp(x))呢？请指教

请问作业一的数学推到 怎么我总是推的和结果对不上呢?

请问作业一的数学推到 怎么我总是推的和结果对不上呢?
含有一个隐藏层的网络 和word2vec 怎么推?
答案给的结果 怎么感觉有点诡异呢
谢谢啊

顺便问一句,你有微信没 可以直接加好友没?

q3_gru.py add_prediction_op() 中dynamic_rnn应该取output作为值而不是state吧?

outputs, state = tf.nn.dynamic_rnn(cell, x, dtype=tf.float32)
output = outputs[:, -1]
preds = tf.sigmoid(output)

虽然gru中两者是等同的

求问Assignment3的问题

在q2_rnn.py 中的Config里面
n_word_features = 2 # Number of features for every word in the input.
这个feature是什么feature？每个word用两个feature来表示是什么意思。。。想了好久都没有想通。。。

在后面做embedding的时候发现这个feature也不是word——embedding的length。。

Returns:
embeddings: tf.Tensor of shape (None, max_length, n_features*embed_size)

谢谢大神。。

怎么打分

代码里好像没有自动打分的？请问你怎么测试的呢

博主您好，很感谢您的分享！

貌似资料链接 http://pan.hankcs.com/jump.php?4 失效了，您能再分享下吗，谢谢！

原来是国人哈，正在学习，谢谢！

想问下，代码全部都是正确无误的么？

请问呢，作业一中word2vec梯度检测错误的原因

大佬您好，我参考了您得assignment1中得word2vec.py得实现。但是在运行过程中梯度检测报错了。
==== Gradient check for skip-gram ====
Gradient check failed.
First gradient error found at index (0, 0)
Your gradient: -0.087147 Numerical gradient: 1254.567123
我是用py3实现的，之前的所有代码几乎一致，也都正确通过了，唯独这里通过不了。之后我将您的代码直接全部拷贝下来运行，同样报以上错误，请问您知道怎么回事嘛，您当时运行通过了吗？