chen742 / pipa Goto Github PK

Official Implementation of PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation

Home Page: https://arxiv.org/abs/2211.07609

Python 99.71% Shell 0.29%

contrastive-learning deep-learning domain-adaptation semantic-segmentation transformer

pipa's People

Contributors

Stargazers

Watchers

Forkers

kagawa588 jlqzzz sibo888 tianjiu233 kuangsoh jinyoon002 fanglimin2 tongtongxxi ludolp layumi jayantteotia16 deepeshkalura aftott itsanastasiaminina bepuro

pipa's Issues

out of memory

May I ask the author of PIPA how much video memory is required for this work? Why does the 24G 3090 report an error of out of memory?

patch loss

Why your patch loss use the feature of teacher network？And how to train your self.classifier in dacs.py？

About reproducing the performance.

Great project! I am very insterested in your work and thanks for the release.

However, after reproducing the experiment with the code(HRDA+PiPa), I was not able to achieve the result reported in the paper. Specifically, I reproduced the experiment by directly run

python run_experiments.py --config configs/pipa/gtaHR2csHR_hrda.py

with the random seed 0, 1, 2, and achieved mean intersection over union score 74.52, 74.34, 74.73 respectively. Here is my logs:

with seed=0, mIoU=74.52 20230109_160110.log
with seed=1, mIoU=74.34 20230208_233817.log
with seed=2, mIoU=74.73 20230210_153346.log

I don't know the reason of the performance drop. Could you please tell me the possible reason or any hint to reproduce the results?

Thanks in advance.

Best,
Yuanbing

About GPU device

Dear authors,

Thanks for your impressive work and code for the community, I am a student studying your paper and your work inspires me a lot.

I want to learn more about your work by running the code. Could I train it with my 3080 GPU (laptop)？Do I need to use more GPUS for distributed?

Questions about embedding features for contrastive learning

Hi, thanks for your sharing codes, it's really an awesome work for UDA.

I have noticed that the embedding features for “Source Pixel Contrast” and "Target Patch Contrast" are not the same.

Concretely, embedding features for “Source Pixel Contrast” are obtained by concatenating 4 features from the backbone by yourself.

PiPa/PiPa_DAFormer/mmseg/models/segmentors/encoder_decoder.py

Lines 174 to 181 in 4eb532e

    
           feat1 = x[0] 
        
           feat2 = F.interpolate(x[1], size=(h, w), mode="bilinear", align_corners=True) 
        
           feat3 = F.interpolate(x[2], size=(h, w), mode="bilinear", align_corners=True) 
        
           feat4 = F.interpolate(x[3], size=(h, w), mode="bilinear", align_corners=True) 
        
           feats = torch.cat([feat1, feat2, feat3, feat4], 1) 
        
           out = self.cls_head(feats) 
        
           emb = self.proj_head(feats)

However, embedding features for “Target Patch Contrast” are directly obtained from the fuse_layer

PiPa/PiPa_DAFormer/mmseg/models/decode_heads/daformer_head.py

Lines 176 to 180 in 4eb532e

    
           x = self.fuse_layer(torch.cat(list(_c.values()), dim=1)) 
        
           fuse_x = x 
        
           x = self.cls_seg(x) 
        
           return x, fuse_x

Why not use the same embedding features?

About reproducing the performance about pixel-wise contrast

Hi, I do the experiment about the pixel-wise contrast on GTA to Cityscapes, but I can only get 69.8 mIoU finally. I only delete the code about the patch-wise contrast and don't modify the other code. What is wrong in my experiment? Looking forward to your reply, thanks.

Can I train PiPa with more than one GPU?

Hi,

Thanks for your great work!
I am doing a research project and I would like to adopt your model as my baseline, but I would like to know if this model could only be trained on one GPU.

how to run this code on CPU only

Dear Author,

This is an excellent piece work and thanks a lot for sharing it. I was trying to run your code for experimentation. Can you please let me know how to run this code on CPU only device?

--
Best Regards,
Dinesh

Code of DAFormer + PiPa

Hi, thanks for your awesome code.

I noticed that the released code is disigned for HRDA, could you please provide the code for DAFormer?

Can I generate realistic images instead of segmentation masks?

Hello,

I wanted to know if it's possible to generate realistic images (like cityscape images) instead of segmentation masks?

Thanks

patch loss + pixel loss

How to train your self.classifier in dacs.py？
And How to train your self.cls_head in encoder_decoder.py？
I did not find related loss such as cross entropy.

Loss problem

I only found PixelContrastLoss but did not find regional loss. The code of the article here is the same as that of Exploring Cross-Image PixelContrast for Semantic Segmentation. What the article describes conflicts with the code implementation.There is also no bank that stores positive and negative pixel comparisons

Running Error

Hi authors,

I installed the codebase following your instruction, but got the below error,
could you helpme ? any suggestion is good.

"RuntimeError: nms is not compiled with GPU support"

Thanks

Have you used imagenet feature distance?

From the code you released, the imagenet feature distance is used in your work. But in your paper, the fd loss is not included in the total loss. Can you explain it. Thank you very much!

Questions about ablation study

Thank you for your outstanding work! I would like to ask you whether the ablation result in DAFormer use the original settings of DAFormer with 40k iterations and a 512x512 crop size or keep same with PiPa's 60k iterations and 640x640 crop size. I'd appreciate it if you replied to me.

about performance

I want to know the results you report are the best checkpoint or the last checkpoint?

Google drive link to download the [MiT weights] is not working

The google drive link to download the [MiT weights] is not working,please share working link @chen742

	feat1 = x[0]
	feat2 = F.interpolate(x[1], size=(h, w), mode="bilinear", align_corners=True)
	feat3 = F.interpolate(x[2], size=(h, w), mode="bilinear", align_corners=True)
	feat4 = F.interpolate(x[3], size=(h, w), mode="bilinear", align_corners=True)
	feats = torch.cat([feat1, feat2, feat3, feat4], 1)

	out = self.cls_head(feats)
	emb = self.proj_head(feats)

	x = self.fuse_layer(torch.cat(list(_c.values()), dim=1))
	fuse_x = x
	x = self.cls_seg(x)

	return x, fuse_x