Comments (3)
Hi Yejun,
Thank you for the issue, and your kind interest in GAT!
The phrase "without depending on the graph structure upfront" refers to the training/testing routine. Namely, GAT is an inductive method---the mechanism it learns is in principle not conditioned on the graph it has been trained on. This means that, at test time, you can apply GAT to any structure you'd like (including ones unseen at training time). This is in stark contrast to many methods that were published before (which were transductive, and wouldn't in theory work outside of the graph they were trained on).
Regarding the second phrase, I believe you've misread the paper a little bit. From what I recall, the phrase appears here:
"In its most general formulation, the model allows every node to attend on every other node, dropping all structural information"
This is the formulation before masked attention is introduced (i.e. we just do all-pairs self-attention as in the Transformer paper). Indeed, in this version the graph is not used at all. Afterwards we introduce the neighbourhoods, and the graph structure is injected.
So, to confirm, the GAT model does not drop all structural information. It uses the local adjacency information of every node to determine which other nodes to attend over. That being said, it only needs the local information (i.e. a node does not need to know anything about a node that is outside of its neighbourhood).
Hope that helps! Let me know more clarification is needed.
Thanks,
Petar
from gat.
thanks Petar!
btw, how does GAT consider about edge with an arrow. My understanding is that it depends on the definition of 'neighborhood', it means that alpha of one direction is learned/calculated, while alpha of the other direction is just zero.
from gat.
Hi Yejun,
The general answer is "it's up to you". :)
In the simplest case, as you suggested, the attention is simply not computed over one direction. Other authors like to include a notion of two "edge types" (inbound/outbound) and learn a separate set of attention heads for each edge type. I'd say -- it really depends on the problem you're trying to solve (and how expressive the edges actually are semantically), but ultimately the framework is quite flexible with respect to how you choose to approach this.
Thanks,
Petar
from gat.
Related Issues (20)
- How to output labels updated after this algorithm?
- output the uodated label
- ValueError: Dimensions must be equal, but are 122904 and 12 for 'mul' (op: 'Mul') with input shapes: [122904], [12].
- multi-graph node classsification HOT 3
- adj_to_bias "return -1e9 * (1.0 - mt)" term
- Weighted adjacency graphs HOT 2
- The attention value of all edges of a node is almost equal HOT 2
- t-SNE HOT 2
- Unseen node embedding
- Usage with image data HOT 2
- how to sample neighbours(nodes)?
- issue in Generating test.index file HOT 3
- Please follow this to get the "Solution to creating the files under the data folder"
- Need help in creating evaulation metrics
- How to batch large datasets to apply GAT?
- transform to other scope dataset
- Why using undirected graph for transductive learning?
- Why using undirected graph for transductive learning?
- How to get node embeddings? HOT 1
- Code fails with tensorflow-2.9.1: module 'tensorflow' has no attribute 'layers' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gat.