shen-lab / graphcl Goto Github PK
View Code? Open in Web Editor NEW[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
License: MIT License
[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
License: MIT License
I follow the instructions in readme, but I can not achieve 0.7417 accuracy in paper on PROTEINS dataset, I only get about 0.73
The running environment is not mentioned here. Can you talk about the configuration of the running environment?
For GraphCL paper, the positive pairs are both augmentations views using some augmentations in GraphCL paper.
But when I read code in unsupervised_TU/gsimclr.py, I found only one view is augmented, the other view is original without augmentation.
I wonder if I'd like to reproduct the results in paper, should I change x to another augmentaion view using some augmentations?
In other words, such implementations in code means one original view and one augmentaion view is enough for experiement?
Besides, In GraphCL_Automated/unsupervised/gsimclr.py, I have similar questions. In code , there is only compute one augmentation view's distribution, and the other view is original view. I think there are some differences between paper and code. Could you help me explain such cases?
By the way, I'm so thankful to all of you for such great and inspirable work! Best wishes for you !
Hi:
In GraphCL/unsupervised_TU/gsimclr:
def loss_cal(self, x, x_aug):
T = 0.2
batch_size, _ = x.size()
x_abs = x.norm(dim=1)
x_aug_abs = x_aug.norm(dim=1)
sim_matrix = torch.einsum('ik,jk->ij', x, x_aug) / torch.einsum('i,j->ij', x_abs, x_aug_abs)
sim_matrix = torch.exp(sim_matrix / T)
pos_sim = sim_matrix[range(batch_size), range(batch_size)]
loss = pos_sim / (sim_matrix.sum(dim=1) - pos_sim)
loss = - torch.log(loss).mean()
return loss
Why the denominator of loss need subtract the pos_sim?
Dear authors, It seems that you still follow a local-global contrastive manner when applying the model to node classification tasks(it's similar to 'mvgrl' but adopts different data augmentations). Is there any reason for not directly computing the similarities of nodes' representations within a batch and applying the NT-XENT loss?
Hi @Yuning You,
As the title shows, I have a question about the data augmentation in unsupervised_graph_TU experiments. If we drop an edge or a node and that edge happens to be in a structural motif, it will drastically change the attributes/labels of the molecule. Could you pls give some explainations? Thanks!
Dear Authors,
Thanks for the code, in your paper, node dropping is "Given the graph G, node dropping will randomly discard certain portion of vertices along with their connections".
But in your current implementation, in unsupervised_TU/aug.py, the drop_nodes method only cut-off the edges, but still keeps the nodes, this would create isolated nodes, wouldn't that be a problem? I think if you want to remove a node, maybe you should re-index the nodes and the edges, otherwise this augmentation is some kind of "edge dropping", right?
def drop_nodes(data):
node_num, _ = data.x.size()
_, edge_num = data.edge_index.size()
drop_num = int(node_num / 10)
idx_drop = np.random.choice(node_num, drop_num, replace=False)
idx_nondrop = [n for n in range(node_num) if not n in idx_drop]
idx_dict = {idx_nondrop[n]:n for n in list(range(node_num - drop_num))}
edge_index = data.edge_index.numpy()
adj = torch.zeros((node_num, node_num))
adj[edge_index[0], edge_index[1]] = 1
adj[idx_drop, :] = 0
adj[:, idx_drop] = 0
edge_index = adj.nonzero().t()
data.edge_index = edge_index
return data
Hi,
I'm interested in self-supervised learning on the graph and GraphCL
is an excellent work that combining self-supervised learning with augmentation. However, I'm also curious about its usage in large graphs such as knowledge graph and so on. May I ask you about the suggestion on the augmentation operation on the knowledge graph? Because in the way of GraphCL
, it needs 4 GNN encoders at the same time per epoch. Thanks a lot in advance~
hello,@yyou1996, thanks for your great work!
I have a question about the GraphCL in transferLearning_MoleculeNet_PPI/chem.When I run the code,the loss of each epoch is negative number. I wonder if I did something wrong,or that's what it is.
thanks!
Hi! Thanks for sharing the code. The data loader raises an AttributeError when I run the code in unsupervised_TU. It seems that the data in the Batch is a tuple (data and data_aug):
(Data(edge_attr=[38, 4], edge_index=[2, 54], x=[17, 7], y=[1]), Data(edge_attr=[38, 4], edge_index=[2, 54], x=[17, 1], y=[1]))
But in the source code of torch geometric, the data is used as "keys = [set(data.keys) for data in data_list]". This raises the error:
AttributeError: 'tuple' object has no attribute 'keys'.
Could you please help me with this bug? The version of my torch geometric is 1.3.2. Thanks!
Hi GraphCL team, thanks for your excellent work.
I have some questions about the loss function in unsupervised_TU/gsimclr.py:
in your article eq.(3), you claim that the positive pairs and negative pairs are composed of augmentations, but in the code I find that you use the original sample and one augmentation to form a positive pair. I don't understand the differences between these two methods and wish you could give me some suggestions or explanation. Thank you in advance.
I have run the command
python gsimclr.py --DS github_stargazers --lr 0.01 --local --num-gc-layers 3 --aug dnodes --seed 0
but got an error. The detailed report are listed as follows.
C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [272,0,0], thread: [96,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [272,0,0], thread: [97,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [272,0,0], thread: [98,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [272,0,0], thread: [99,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [272,0,0], thread: [100,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [96,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [97,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [98,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [99,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [100,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [101,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [102,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [103,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [104,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [105,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [106,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [107,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [108,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [109,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [110,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [111,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [112,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [113,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [114,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [115,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [116,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [117,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [118,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [119,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [120,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [121,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [122,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [123,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [124,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [125,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [126,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [307,0,0], thread: [127,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [96,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [97,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [98,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [99,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [100,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [101,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [102,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [103,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [104,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [105,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [106,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [107,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [108,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [109,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [110,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [111,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [112,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [113,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [114,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [115,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [116,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [117,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [118,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [119,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [120,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [121,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [122,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [123,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [124,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [125,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [126,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [127,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [0,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [1,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [2,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [3,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [4,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [5,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [6,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [7,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [8,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [9,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [10,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [11,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [12,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [13,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [14,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [15,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [16,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [17,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [18,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [19,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [20,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [21,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [22,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [23,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [24,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [25,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [26,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [27,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [28,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [29,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [30,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [31,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [64,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [65,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [66,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [67,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [68,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [69,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [70,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [71,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [72,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [73,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [74,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [75,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [76,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [77,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [78,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [79,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [80,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [81,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [82,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [83,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [84,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [85,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [86,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [87,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [88,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [89,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [90,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [91,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [92,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [93,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [94,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [311,0,0], thread: [95,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [96,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [97,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [98,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [99,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [100,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [101,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [102,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [103,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [104,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [105,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [106,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [107,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [108,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [109,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [110,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [111,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [112,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [113,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [114,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [115,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [116,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [117,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [118,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [119,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [120,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [121,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [122,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [123,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [124,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [125,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [126,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [127,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [64,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [65,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [66,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [67,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [68,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [69,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [70,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [71,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [72,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [73,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [74,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [75,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [76,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [77,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [78,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [79,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [80,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [81,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [82,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [83,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [84,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [85,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [86,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [87,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [88,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [89,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [90,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [91,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [92,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [93,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [94,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [95,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [32,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [33,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [34,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [35,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [36,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [37,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [38,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [39,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [40,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [41,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [42,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [43,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [44,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [45,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [46,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [47,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [48,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [49,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [50,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [51,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [52,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [53,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [54,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [55,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [56,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [57,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [58,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [59,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [60,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [61,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [62,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [63,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [64,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [65,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [66,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [67,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [68,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [69,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [70,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [71,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [72,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [73,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [74,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [75,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [76,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [77,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [78,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [79,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [80,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [81,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [82,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [83,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [84,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [85,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [86,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [87,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [88,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [89,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [90,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [91,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [92,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [93,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [94,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [319,0,0], thread: [95,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [0,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [1,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [2,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [3,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [4,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [5,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [6,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [7,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [8,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [9,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [10,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [11,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [12,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [13,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [14,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [15,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [16,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [17,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [18,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [19,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [20,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [21,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [22,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [23,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [24,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [25,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [26,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [27,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [28,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [29,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [30,0,0] Assertion
srcIndex < srcSelectDimSizefailed. C:/w/b/windows/pytorch/aten/src/ATen/native/cuda/Indexing.cu:662: block: [287,0,0], thread: [31,0,0] Assertion
srcIndex < srcSelectDimSize` failed.
Traceback (most recent call last):
File "gsimclr.py", line 189, in
emb, y = model.encoder.get_embeddings(dataloader_eval)
File "D:\Document\Works\Codes\Python\GCN-related\GraphCL-master\unsupervised_TU_exhsp\gin.py", line 82, in get_embeddings
x, _ = self.forward(x, edge_index, batch)
File "D:\Document\Works\Codes\Python\GCN-related\GraphCL-master\unsupervised_TU_exhsp\gin.py", line 54, in forward
x = F.relu(self.convs[i](x, edge_index))
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch_geometric\nn\conv\gin_conv.py", line 69, in forward
return self.nn(out)
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch\nn\modules\container.py", line 119, in forward
input = module(input)
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch\nn\modules\linear.py", line 94, in forward
return F.linear(input, self.weight, self.bias)
File "D:\App\Anaconda3\envs\GraphCL_unsupervised\lib\site-packages\torch\nn\functional.py", line 1753, in linear
return torch._C._nn.linear(input, weight, bias)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)``
Hi @yyou1996,
I wonder how to speed up the pretraining on chem data. How long did you use to pretrain one epoch on it? Which gpu version did you use? I think the speed bottleneck is the CPU or io. I tried increasing the num_workers
while it seems no effects.
Hi @yyou1996,
I tired pretraining on transferlearning experiments yet found: 1) on Bio benchmark, the loss and acc are always 0.0 and nan; 2) On chem dataset, the training process stopped at the 2nd epoch. Could you pls check the code? Btw, the finetuning command worked well. Thanks a lot!
Ps: My environment is cuda=10.0
and pyG has the same version just except it is for cu100
.
I have a question about these two datasets. there are three subset trainset, valset, and testset. How to combine these dataset to one whole set? Thank you. @yongduosui
Hi,
I have the problem of #4 (comment) and #1 trying lunching semisupervised TU pre training. I launch python main.py --dataset MUTAG --aug1 random2 --aug2 random2 --lr 0.001 --suffix 0 --exp test
and I get this error:
[INFO] running single test..
-----
Total 1 experiments in this run:
1/1 - MUTAG - deg+odeg100+ak3+reall - ResGFN
Here we go..
-----
1/1 - MUTAG - deg+odeg100+ak3+reall - ResGFN
None None
Traceback (most recent call last):
File "main.py", line 338, in <module>
run_exp_single_test()
File "main.py", line 316, in run_exp_single_test
run_exp_lib([('MUTAG', 'deg+odeg100+ak3+reall', 'ResGFN')])
File "main.py", line 165, in run_exp_lib
dataset = get_dataset(
File "C:\Users\alessandro\Developments\GraphCL\semisupervised_TU\pre-training\datasets.py", line 57, in get_dataset
dataset = TUDatasetExt(
File "C:\Users\alessandro\Developments\GraphCL\semisupervised_TU\pre-training\tu_dataset.py", line 49, in __init__
super(TUDatasetExt, self).__init__(root, name, transform, pre_transform,
File "C:\_______\envs\torch\lib\site-packages\torch_geometric\datasets\tu_dataset.py", line 66, in __init__
self.data, self.slices = torch.load(self.processed_paths[0])
File "C:\_______\envs\torch\lib\site-packages\torch\serialization.py", line 579, in load
with _open_file_like(f, 'rb') as opened_file:
File "C:\_______\envs\torch\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "C:\_______\envs\torch\lib\site-packages\torch\serialization.py", line 211, in __init__
super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'data\\MUTAG\\MUTAG\\processed\\data_deg+odeg100+ak3+reall.pt'
I have installed all, read the other two issues but I can't understand what I have to do in order to make it work (if there is anything I can do). I have all installed in a python env
Hi Yuning,
There are some errors in the code.
When I run mask
and edge
in unsupervised_Cora_Citeseer
using python -u execute.py --dataset citeseer --aug_type mask --drop_percent 0.20 --seed 39 --save_name cite_best_dgi.pkl --gpu 0
, there will have unassignment error as follows:
Traceback (most recent call last): File "execute.py", line 189, in <module> sparse, None, None, None, aug_type=aug_type) File "/anaconda3/envs/graph/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/GraphCL/unsupervised_Cora_Citeseer/models/dgi.py", line 51, in forward ret1 = self.disc(c_1, h_0, h_2, samp_bias1, samp_bias2) UnboundLocalError: local variable 'c_1' referenced before assignment
Here are my env version: python==3.7, torch==1.5.0, torch-geometric==1.5.0. I hope you will notice and fix them.
哈喽,请问graphcl中报告的实验结果,是最优的learning rate和最好的augmentation下,取不同seed得到的结果的平均值么?
还是,最优的learning rate,不同的augmentation相同的seed得到的结果的平均值呢?
或者不同的learning rate,不同的augmentation,不同的seed得到的结果的平均值呢?
同样的问题对于半监督场景下的结果是在什么样的设置下得到的报告结果呢?
Hi~ here are several questions about semi-supervised learning
For MUTAG dataset as instance:
(1)As for the semi-supervised result reported in the paper, which dataset did you use?Did you use MUTAG deg+odeg100+ak3+recall or deg+odeg100+ak3+reall?And what does the dataset name means?What are the differences between these two datasets in deed?
(2)Why not use the same datasets with unsupervised experiements? What are the differeces between these datasets used in semi-supervised learning and supervised learning ? Can I replace the dataset used in semi-supervised directly by the datasets used in unsupervised learning? Can I get same results?
(3)In semi-supervised learning,I get several pretraining weights after pretraining ,Why it still takes very long time when I finetuing ?It ‘s so weried. It should be very quick for finetuing. Where should I modify to get a quick finetuning process? (4)
(4)In semi-supervised learning ,which model did you use to pretrain ,GFN or ResGCN to get the result report in the paper?Why not use GIN as same as in unsupervised learning?
Thanks a lot!! I'm very puzzled!
Hi, @yyou1996. Thanks for your excellent work!
I have a question about the similarity changes when I run the unsupervised_TU codes:
First, as I run gsimclr.py with COLLAB dataset, I find that the loss is gradually decreasing as we expected;
Then, I try to print the pos_sim and the neg_sim values (the numerator and denominator of the loss, respectively) during the training process to observe their changes
GraphCL/unsupervised_TU/gsimclr.py
Line 132 in 1d43f79
So my question is, whether the training process has been reducing the similarity between positive and negative pairs without increasing the similarity between positive pairs? If so, does the conclusion that GraphCL tries to improve the consistency between different graph views still hold?
Look forward to your reply! Thanks.
Hi, @yyou1996
When I run the code of unsupervised_TU experiments with a fixed random seed (e.g., 0), the outputs, including loss, accuracy, etc., may be different every time.
How about your opinion on the issue? Thanks a lot!
Hi @yyou1996,
Thanks for the amazing work and released code. They are really interesting.
However, I find that it's hard to reproduce the results of Table. 5 on PPI dataset, where GraphCL gets 67.88 ± 0.85 ROC-AUC. I follow the instruction in README to run the script cd ./bio; ./finetune.sh
in two versions of PyG.
The results and details of result.log
of my reproducing experiments are as following:
In this experiment, I had torch_geometric == 1.0.3, torch == 1.0.1
and all the codes are as original.
0 0.6488228587091137 0.6437029455712953
1 0.6453625892746733 0.637906301310777
2 0.6653588896536515 0.6491496412192561
3 0.6712478235166237 0.6609652991518822
4 0.6551347790238357 0.648166151211857
5 0.6496970788807328 0.6285455615711776
6 0.6377575006466477 0.6259403911657462
7 0.6356370096455746 0.629560479239631
8 0.6442826009871101 0.6378061247726416
9 0.6421840662455275 0.6328729467542645
In this experiment, I had torch_geometric == 1.6.3, torch == 1.7.1
, and I changed the codes a little bit following #14.
0 0.6442808014337911 0.6321503395158243
1 0.644493673207877 0.6267153949242326
2 0.6453095524067848 0.6349091275301133
3 0.6438850211437335 0.6420234098635493
4 0.6411963663200242 0.6328391671224951
5 0.66046061449877 0.6512654813973853
6 0.6473159630814896 0.6400854279154624
7 0.6308149754938717 0.6191641371576463
8 0.6486396922222195 0.6367294139925784
9 0.6357155854879776 0.6267330172938124
Results of my experiments are calculated according to the rightest column of result.log
, which are results of test_acc_hard_list
following Hu et.al. ICLR 2020 [1]. Just to make sure if I have missed some important details to reproduse the results presented in the literature. Looking forward to your reply, thanks!
[1] Strategies for Pre-training Graph Neural Networks. Weihua Hu, Bowen Liu et.al. ICLR 2020. arxiv.
Dear Authors,
I've been reproducing your work and the original GFN work (ResGCN) for months. The idea is inspiring. But I think some of the results are not accurate. Your variance is too small comparing with standard baselines.
This is the result of GFN (ResGCN) trained using 100% training data:
This is my reproduced result of the ResGCN in GFN paper, with default settings, which is the backbone you're using, quite close to the results reported by GFN.
And these are the results of yours, the Table 3 in your paper.
The variance is inevitable since 10-fold cross validation is used, even if the random seed is fixed. I also have some reproduced results using your original code, I'll just put some of the results on NCI1 as an example.
So I think the std should be at least 10x larger if you're using 10-fold cross val correctly. Do you have any explanation on this issue? Otherwise I can't use these results. Thanks!
Hi @yyou1996,
Thanks for your efforts in this project. I have some questions as follows:
I notice that the training & evaluation process of GraphCL is slightly different from InfoGraph, which evaluates per epoch
. GraphCL doesn't save the model during training and evaluates per 10 epochs
. I wonder if I could evaluate per epoch
and choose the corresponding test acc.
to the highest val acc. as the final test result. Or I can choose the highest test acc. directly, though testing many times during training could be weird.
I tried evaluating per epoch on IMDB-B
and the training process was a bit weird to me. The validate acc. didn't improve through more training epochs, which probably means the training process didn't benefit the representation learning well in my opinion. Could you pls give some analysis or explanations?
Epoch 1, Loss 420.73120760917664
acc_val, acc = 0.72 0.709
Epoch 2, Loss 407.65890550613403
acc_val, acc = 0.701 0.696
Epoch 3, Loss 395.5375609397888
acc_val, acc = 0.673 0.679
Epoch 4, Loss 383.75150847435
acc_val, acc = 0.707 0.7020000000000001
Epoch 5, Loss 371.76993465423584
acc_val, acc = 0.6759999999999999 0.688
Epoch 6, Loss 361.50709533691406
acc_val, acc = 0.7020000000000001 0.701
Epoch 7, Loss 352.6328740119934
acc_val, acc = 0.707 0.705
Epoch 8, Loss 340.92082047462463
acc_val, acc = 0.72 0.711
Epoch 9, Loss 334.9960980415344
acc_val, acc = 0.694 0.688
Epoch 10, Loss 325.21264362335205
acc_val, acc = 0.7 0.7070000000000001
Epoch 11, Loss 318.04585337638855
acc_val, acc = 0.7089999999999999 0.7230000000000001
Epoch 12, Loss 310.82839179039
acc_val, acc = 0.7070000000000001 0.722
Epoch 13, Loss 304.04966139793396
acc_val, acc = 0.72 0.7250000000000001
Epoch 14, Loss 299.41626381874084
acc_val, acc = 0.744 0.715
Epoch 15, Loss 293.7600963115692
acc_val, acc = 0.6910000000000001 0.7220000000000001
Epoch 16, Loss 288.6614990234375
acc_val, acc = 0.703 0.7270000000000001
Epoch 17, Loss 285.12760519981384
acc_val, acc = 0.717 0.726
Epoch 18, Loss 280.9172787666321
acc_val, acc = 0.736 0.7289999999999999
Epoch 19, Loss 277.5624203681946
acc_val, acc = 0.71 0.715
Epoch 20, Loss 274.18093502521515
acc_val, acc = 0.7150000000000001 0.7289999999999999
Btw, the final result of the seed should be 0.7289(epoch 18)
due to the highest val_acc.(0.736)
, right? Thanks in advance!
Traceback (most recent call last):
File "gsimclr.py", line 188, in
emb, y = model.encoder.get_embeddings(dataloader_eval)
File "/home/m10915046/GraphCL/unsupervised_TU/gin.py", line 76, in get_embeddings
x, _ = self.forward(x, edge_index, batch)
File "/home/m10915046/GraphCL/unsupervised_TU/gin.py", line 58, in forward
xpool = [global_add_pool(x, batch) for x in xs]
File "/home/m10915046/GraphCL/unsupervised_TU/gin.py", line 58, in
xpool = [global_add_pool(x, batch) for x in xs]
File "/home/m10915046/anaconda3/envs/graphcl2/lib/python3.6/site-packages/torch_geometric/nn/glob/glob.py", line 26, in global_add_pool
return scatter(x, batch, dim=0, dim_size=size, reduce='add')
File "/home/m10915046/anaconda3/envs/graphcl2/lib/python3.6/site-packages/torch_scatter/scatter.py", line 154, in scatter
return scatter_sum(src, index, dim, out, dim_size)
RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #3 'index' in call to th_scatter_add
The above operation failed in interpreter.
Traceback (most recent call last):
File "/home/m10915046/anaconda3/envs/graphcl2/lib/python3.6/site-packages/torch_scatter/scatter.py", line 27
size[dim] = int(index.max()) + 1
out = torch.zeros(size, dtype=src.dtype, device=src.device)
return out.scatter_add_(dim, index, src)
~~~~~~~~~~~~~~~~ <--- HERE
else:
return out.scatter_add_(dim, index, src)
When I run main_superpixels_contrastive.py
I got
f = pickle.load(f)
ModuleNotFoundError: No module named 'data.superpixels'
I don't know why "f = pickle.load(f)" can cause this problem?
Hi @yyou1996,
When I ran the unsupervised experiments for Biochemical Molecules datasets, I met the bug of size mismatch
as follows:
python gsimclr.py --DS MUTAG --lr 0.01 --local --num-gc-layers 3 --aug dnodes --seed 0
Traceback (most recent call last):
File "gsimclr.py", line 190, in
emb, y = model.encoder.get_embeddings(dataloader_eval)
File "/data3/linshuai/GraphCL/unsupervised_TU/gin.py", line 87, in get_embeddings
x, _ = self.forward(x, edge_index, batch)
File "/data3/linshuai/GraphCL/unsupervised_TU/gin.py", line 61, in forward
x = F.relu(self.convs[i](x, edge_index))
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch_geometric/nn/conv/gin_conv.py", line 69, in forward
return self.nn(out)
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch/nn/modules/container.py", line 100, in forward
input = module(input)
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "/data/linshuai/anaconda3/envs/mvgrl/lib/python3.6/site-packages/torch/nn/functional.py", line 1370, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: size mismatch, m1: [2258 x 1], m2: [7 x 32] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:290
It seems that the aug
is 'none' here in the testing and thus the second dim of data_aug.x
was set to 1
. So I removed this line and the bug was gone.
What's your opinion about the issue and solution? Thanks in advance!
Is it training the downstream svm model with 1) random embeddings or 2) no training GNN embeddings 3) using fully supervised trained GNN embeddings.
I want to know if it is any of the above or something different.
There are no the script "from train.train_SBMs_node_classification import train_epoch, evaluate_network"
I have respectively run the codes using commands python -u execute.py --dataset citeseer --aug_type node --drop_percent 0.20 --seed 39 --save_name cite_best_dgi.pkl --gpu 0
and python -u execute.py --dataset cora --aug_type node --drop_percent 0.20 --seed 39 --save_name cite_best_dgi.pkl --gpu 0
on the following two environments:
CUDA 11.1, Python 3.6.7, PyTorch 1.8.0
, hard env: RTX 3080 laptop
CUDA 10.0, Python 3.7.6, Pytorch 1.3.1
, hard env: RTX 2080ti
and respectively got the results as the following:
Citeseer
Average accuracy:[0.7279]
Mean:[72.7880]
Std :[0.1394]
Cora
Average accuracy:[0.8124]
Mean:[81.2360]
Std :[0.1274]
Citeseer
Average accuracy:[0.7256]
Mean:[72.5620]
Std :[0.1615]
Cora
Average accuracy:[0.8137]
Mean:[81.3660]
Std :[0.1649]
But according to the paper's Appendix G, the results should be 73.14±0.15 on Citeseer and 81.76±0.17 on Cora. How can I reproduce the results?
Hi, I have read the paper and I want to know how to change the label rate of this?
See the opening issue, sorry for my opening the issue carelessly.
Hello, I have read the paper "Graph Contrastive Learning with Augmentations" with the appendix, but implementation of contrastive learning in unsupervised_Cora_Citeseer looks a bit unclear for me. Could you please clarify a few details?
Hi, Shen. Thanks for your efforts in this program. I noticed there might have a logical bug in the aug_random_edge function.
“for i in index_list:
single_index_list.append(i)
index_list.remove((i[1], i[0]))”
Removing items in a list while iterating that list may cause problems, as the list changed but the current index is not changed.
This blog illustrated well.
https://thispointer.com/python-remove-elements-from-a-list-while-iterating/
Thanks for your help.
pip install torch-scatter==1.1.0 -f https://pytorch-geometric.com/whl/torch-1.4.0.html
pip install torch-sparse==0.4.4 -f https://pytorch-geometric.com/whl/torch-1.4.0.html
pip install torch-cluster==1.4.5 -f https://pytorch-geometric.com/whl/torch-1.4.0.html
pip install torch-spline-conv==1.1.0 -f https://pytorch-geometric.com/whl/torch-1.4.0.html
pip install torch-vision==0.5.0
I am trying to install these but all failed, I am asking for help if there is someone who successfully installed these pakeges? Thank you.
I'd like to reproduce the result of transfer learning, could you release the code of transfer learning and settings?
File "/home/zhenghua/pythoncode/unsupervised_graph_TU/gsimclr.py", line 166, in
dataset = TUDataset(path, name=DS, aug=args.aug).shuffle()
File "/home/zhenghua/pythoncode/unsupervised_graph_TU/aug.py", line 73, in init
pre_filter)
File "/home/zhenghua/.conda/envs/graphcontra/lib/python3.6/site-packages/torch_geometric/data/in_memory_dataset.py", line 56, in init
pre_filter)
File "/home/zhenghua/.conda/envs/graphcontra/lib/python3.6/site-packages/torch_geometric/data/dataset.py", line 76, in init
self.raw_dir = osp.join(self.root, 'raw')
AttributeError: can't set attribute
when I run gsimclr I got this error, I don't know why, can someone help me?
Dear Authors,
The permute_edges in unsupervised_TU/aug.py is not adding the edges, the "idx_add" variable is not used.
GraphCL/unsupervised_TU/aug.py
Line 330 in aedd92a
I think your implementation in semisupervised_TU is correct, so may I ask why there are different implementations, why not just use a same set of augmentation implementation and just import them in different tasks?
Hi @yyou1996, thanks for your great work. Just have a question about the GraphCL for node classification in unsupervised_Cora_Citeseer. The default batch_size is set to 1. How do you generate negative pairs?
Hi,
when browsing the code, I found there might be bugs in the subgraph augmentation.
In Python 3.7+, the union
operation is not an inplace operation, so that idx_neigh
will not be updated properly. In this situation, only 1-hop subgraph of a random centor node will be generated.
Some examples:
GraphCL/unsupervised_TU/aug.py
Line 371 in e9e598d
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [105,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [55,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [56,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [57,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [58,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [59,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [60,0,0] Assertion srcIndex < srcSelectDimSize
failed.
/opt/conda/conda-bld/pytorch_1623448224956/work/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [158,0,0], thread: [61,0,0] Assertion srcIndex < srcSelectDimSize
failed.
Traceback (most recent call last):
File "/home/zhenghua/pythoncode/unsupervised_graph_TU/gsimclr.py", line 190, in
emb, y = model.encoder.get_embeddings(dataloader_eval)
File "/home/zhenghua/pythoncode/unsupervised_graph_TU/gin.py", line 76, in get_embeddings
x, _ = self.forward(x, edge_index, batch)
File "/home/zhenghua/pythoncode/unsupervised_graph_TU/gin.py", line 52, in forward
x = F.relu(self.convs[i](x, edge_index))
File "/home/zhenghua/.conda/envs/pytorchgeo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in call_impl
return forward_call(*input, **kwargs)
File "/home/zhenghua/.conda/envs/pytorchgeo/lib/python3.7/site-packages/torch_geometric/nn/conv/gin_conv.py", line 64, in forward
out = self.propagate(edge_index, x=x, size=size)
File "/home/zhenghua/.conda/envs/pytorchgeo/lib/python3.7/site-packages/torch_geometric/nn/conv/message_passing.py", line 253, in propagate
out = self.aggregate(out, **aggr_kwargs)
File "/home/zhenghua/.conda/envs/pytorchgeo/lib/python3.7/site-packages/torch_geometric/nn/conv/message_passing.py", line 288, in aggregate
reduce=self.aggr)
File "/home/zhenghua/.conda/envs/pytorchgeo/lib/python3.7/site-packages/torch_scatter/scatter.py", line 153, in scatter
return scatter_sum(src, index, dim, out, dim_size)
File "/home/zhenghua/.conda/envs/pytorchgeo/lib/python3.7/site-packages/torch_scatter/scatter.py", line 21, in scatter_sum
return out.scatter_add(dim, index, src)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Can anyone help me with what wrong with the algorithm or the enviroment?
the environment as follows:
Jinja2 | 3.0.1 | 3.0.1 |
---|---|---|
MarkupSafe | 2.0.1 | 2.0.1 |
Pillow | 8.2.0 | 8.2.0 |
PySocks | 1.7.1 | 1.7.1 |
brotlipy | 0.7.0 | 0.7.0 |
certifi | 2020.6.20 | 2021.5.30 |
cffi | 1.14.5 | 1.14.5 |
chardet | 4.0.0 | 4.0.0 |
cryptography | 3.4.7 | 3.4.7 |
cycler | 0.10.0 | 0.10.0 |
decorator | 4.4.2 | 5.0.9 |
googledrivedownloader | 0.4 | 0.4 |
idna | 2.10 | 3.2 |
joblib | 1.0.1 | 1.0.1 |
kiwisolver | 1.3.1 | 1.3.1 |
matplotlib | 3.4.2 | 3.4.2 |
mkl-fft | 1.3.0 | 1.3.0 |
mkl-random | 1.2.1 | 1.2.2 |
mkl-service | 2.3.0 | 2.4.0 |
networkx | 2.5.1 | 2.6rc2 |
numpy | 1.20.2 | 1.21.0 |
olefile | 0.46 | 0.47.dev4 |
pandas | 1.2.5 | 1.3.0rc1 |
pip | 21.1.2 | 21.1.3 |
pyOpenSSL | 20.0.1 | 20.0.1 |
pycparser | 2.20 | 2.20 |
pyparsing | 2.4.7 | 3.0.0b2 |
python-dateutil | 2.8.1 | 2.8.1 |
python-louvain | 0.15 | 0.15 |
pytz | 2021.1 | 2021.1 |
requests | 2.25.1 | 2.25.1 |
scikit-learn | 0.24.2 | 0.24.2 |
scipy | 1.6.2 | 1.7.0 |
seaborn | 0.11.0 | 0.11.1 |
setuptools | 52.0.0.post20210125 | 57.0.0 |
six | 1.16.0 | 1.16.0 |
threadpoolctl | 2.1.0 | 2.1.0 |
torch | 1.9.0 | 1.9.0 |
torch-cluster | 1.5.9 | 1.5.9 |
torch-geometric | 1.7.2 | 1.7.2 |
torch-scatter | 2.0.7 | 2.0.7 |
torch-sparse | 0.6.10 | 0.6.10 |
torch-spline-conv | 1.2.1 | 1.2.1 |
torchaudio | 0.9.0a0+33b2469 | 0.9.0 |
torchvision | 0.10.0 | 0.10.0 |
tornado | 6.1 | 6.1 |
tqdm | 4.61.1 | 4.61.1 |
typing-extensions | 3.7.4.3 | 3.10.0.0 |
urllib3 | 1.26.6 | 1.26.6 |
wheel | 0.36.2 | 0.36.2 |
Hello! I'm studying your paper and code. I have a few doubts to ask:
Finally, thank you very much for your answer, thank you!
Hi:
In GraphCL/semisupervised_TU/pre-training/
I take NCI1 as an example as you stated, run CUDA_VISIBLE_DEVICES=$GPU_ID python main.py --dataset NCI1 --aug1 random2 --aug2 random2 --lr 0.001 --suffix 0. There is an error that "No such file or directory: data/NCI1/NCI1/processed/data_deg+odeg100.pt". How could I get the file data_deg+odeg100.pt? Should I run another code before getting .pt file?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.