yifeiacc / costa

Code for KDD'22 paper, COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

Covariance-Preserving Feature Augmentation for Graph Contrastive Learning.

Overview

This repo contains an example implementation for KDD'22 paper: COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning. This code provides the multiview MV-COSTA. The SV-COSTA can be easily obtained by modifying the MV-COSTA.

Overview

COSTA is a feature augmentation method that generates augmented samples in the feature space (latent space). It produced a bias-free and covariance-bounded augmentation to alleviate the bias problem in the typical graph augmentation (e.g., edge permutations).

Dependencies

Our implementation works with PyTorch>=1.0.0 Install other dependencies: $ pip install -r requirement.txt

Reproduce our results

We provide several datasets to reproduce our results. We provide wandb logs to show the performance. See following to see the detail

https://wandb.ai/yifeiacc/COSTA_public?workspace=user-yifeiacc

The detailed settings (including hyper-parameters and GPUs) and the results can be found in these logs. You can directly checkout to the corresponding branch(commit).

Usage

To run our code, just run the following

$ cd src 
$ python main.py --root path/to/COSTA/dir --dataset Cora --model COSTA --config COSTA_default.yaml

costa's Issues

Hyperparameters for other datasets

• Q1: About the training parameters of other datasets

• I noticed that only the hyperparameters of some datasets are released in the yaml file. Could you please release the training parameters of other datasets?

• Q2: I'm trying to run this work in colab, but find F1-score can't reach 79.12.

config['dataset']='WikiCS'
config

{'seed': 12345,
 'learning_rate': 0.0001,
 'num_hidden': 256,
 'num_input_dim': 1433,
 'num_proj_hidden': 256,
 'num_layers': 2,
 'num_proj_layers': 2,
 'drop_edge_rate_1': 0.2,
 'drop_edge_rate_2': 0.4,
 'drop_feature_rate_1': 0.3,
 'drop_feature_rate_2': 0.4,
 'tau': 0.4,
 'num_epochs': 700,
 'eta': 0.1,
 'K': 1433,
 'm': 1,
 'gpu_id': 0,
 'root': '/content/drive/MyDrive/COSTA-main',
 'model': 'COSTA',
 'dataset': 'WikiCS',
 'data_dir': '/content/drive/MyDrive/COSTA-main/data',
 'test_every_epoch': False}

Runner(conf=config).execute()

/usr/local/lib/python3.7/dist-packages/torch_geometric/datasets/wikics.py:39: UserWarning: The WikiCS dataset now returns an undirected graph by default. Please explicitly specify 'is_undirected=False' to restore the old behaviour.
  f"The {self.__class__.__name__} dataset now returns an "
Initializing and building model...
Load Dataset...
Training Model...
(T): 100%|██████████| 700/700 [07:49<00:00,  1.49it/s, loss=6.93]
Testing...
(LR): 100%|██████████| 5000/5000 [00:07<00:00, best test F1Mi=0.278, F1Ma=0.0923](E): Best test F1Mi=0.2784, F1Ma=0.0923

SV-COSTA

Hi, i am a little confused about the inplementation of SV-COSTA. In Figure 3 caption, you claim that SV-GCL performs self-contrast. Does that mean the latent Z after projection P, then input into the loss function with a (P(Z),P(Z)) , the total same Z works in contrastive learning? If not, how to self-contrast? If yes, it is really counterintuitive lol...

长尾分布图

你好，想问一下论文当中的长尾分布图是如何制作的，谢谢~

Questions about the proof of Lemma 4.3

Symbols and characters are very confusing and misleading. Also, it appears to be summing and then squaring, rather than squaring and then summing. Please check carefully to complete the proof or clarify a misunderstanding.