Comments (4)
When i implement the equation in paper and the way in code , it seems that there are some commen feature in the result.
-
The result is matrix with 2 dimensions and its shape is [sentence_size, embedding_size] . when compare the matrix from 2 ways, the value decreases as the index of word increased until the medium index of matrix, and increase as the index of word increased until final index.
-
It means that the value in matraix are symmetrical. when i first find the rule in the matrix, it arouses my curiosity.
-
i read the data and it seems that the index of all the answer is first of the sentence or last of the sentence.Therefore, the formular is result from the data ,in other words, the postion encoding is another attention which makes the first or last word more important
-
So ,if the answer is mostly in the midturm index of the word ,when we want to get the representation of the sentence ,we can give the midturm word more attention
from memn2n.
It came from https://github.com/facebook/MemNN/blob/master/MemN2N-babi-matlab/build_model.m#L9. It's been a while but I think it worked better than what was written in the paper. I'm not sure why they did it that way, didn't explain it afaik.
from memn2n.
If anything it was a slight improvement, I don't think it made much of a difference.
from memn2n.
Thank you.
from memn2n.
Related Issues (20)
- Test
- Change name to more general HOT 1
- Add tutorial to download data before running HOT 1
- Pandas not imported in joint.py
- To see memory slot probabilities HOT 6
- Difference between code and paper
- running joint.py throws an error HOT 1
- Dialog tasks
- Found joint.py errors HOT 1
- how to run inference/test independently
- is this possible to modeling multi turn dialog.......here is new dataset HOT 1
- something wrong at nonlin
- Compare Results HOT 1
- Puzzled about the attention part HOT 1
- Support for Ragged/Jagged arrays
- tokenize function code in data_utils.py is incorrect
- Add gradient noise HOT 1
- fix 0 logits in input module HOT 1
- Separate memory from model? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from memn2n.