abooundev / cs224n

cs224n

• Stanford cs224n: Natural Language Processing with Deep Learning (Winter 2019)

  • youtube
  • syllabus, Winter 2019 // youtube version
  • syllabus, Winter 2020

• 강필성 교수님 세미나

  • http://dsba.korea.ac.kr/seminar/?mod=document&uid=42

study

• 20/10/13~12/16(매주 수 저녁 8시)
• 온라인 스터디

[Winter 2020 기준]

날짜	주차	Description	파트	발표자
20/10/13	1주차	Introduction and Word Vectors Gensim word vectors example:	week1-1	유인혁
20/10/13	1주차	Word Vectors 2 and Word Senses	week1-2	유인혁
		Python review session		최슬기
20/10/21	2주차	Word Window Classification, Neural Networks, and PyTorch	week2-1	장건희
20/10/21	2주차	Matrix Calculus and Backpropagation	week2-2	유인혁
20/10/28	3주차	Linguistic Structure: Dependency Parsing	week3-1	최슬기
20/10/28	3주차	The probability of a sentence? Recurrent Neural Networks and Language Models	week3-2	장건희
20/11/04	4주차	Vanishing Gradients and Fancy RNNs	week4-1	유인혁
20/11/04	4주차	Machine Translation, Seq2Seq and Attention	week4-2	최슬기
20/11/11	5주차	Practical Tips for Final Projects	week5-1	장건희
20/11/11	5주차	Question Answering, the Default Final Project, and an introduction to Transformer architectures	week5-2	유인혁
20/11/18	6주차	ConvNets for NLP	week6-1	최슬기
20/11/18	6주차	Information from parts of words (Subword Models)	week6-2	장건희
20/11/25	7주차	Contextual Word Representations: BERT (guest lecture by Jacob Devlin)	week7-1	유인혁
20/11/25	7주차	Modeling contexts of use: Contextual Representations and Pretraining. ELMo and BERT.	week7-2	최슬기
20/12/2	8주차	Natural Language Generation	week8-1	장건희
20/12/2	8주차	Reference in Language and Coreference Resolution	week8-2	유인혁
20/12/09	9주차	Fairness and Inclusion in AI (guest lecture by Vinodkumar Prabhakaran)	week9-1	최슬기
20/12/09	9주차	Constituency Parsing and Tree Recursive Neural Networks	week9-2	장건희
20/12/16	10주차	Recent Advances in Low Resource Machine Translation (guest lecture by Marc'Aurelio Ranzato)	week10-1	유인혁
20/12/16	10주차	Analysis and Interpretability of Neural NLP	week10-2	최슬기

cs224n's Issues

ww_classifier hidden layer

negative log likelihood, cross entropy

• http://www.awebb.info/probability/2017/05/18/cross-entropy-and-log-likelihood.html
  • H(p,q) == H(q,p)
• https://glassboxmedicine.com/2019/12/07/connections-log-likelihood-cross-entropy-kl-divergence-logistic-regression-and-neural-networks/

Multi-class, multi-label