abaldrati / clip4cirdemo Goto Github PK

What image DB did you use when you implemented it?

Hi!

Thanks for this interesting work!

I am trying to replicate your results from CIRR and was wondering if you could release the training code? I have evaluated the pre-trained weights from the demo on the evaluation server and get roughly the same results. However, when training the combiner from scratch, I am only getting about 31.5% (global R@1, test set, CLIPR50x4, only train 'combiner'). I am training with full precision on top of pre-extracted features so I can fit the batch size of 4096.

hello. As mentioned in the paper you used the original fashionIQ dataset containing around 77000 images. but there are many missing images in the current dataset (from the main GitHub page for the dataset and also other sources). do you have access to the full dataset? I would appreciate it if you share the whole dataset to be used in future works.

when I run extract_features.py，I found the elements of batch_features are all nan:

batch_features = clip_model.encode_image(images)

result: tensor([[nan, nan, nan ....nan, nan]]
then I use the code below to test. When I load RN50x4 model, the result is the same as above(both wild image and dataset image). But when I load another model like ViT-B/32, the result is not nan:

import clip
import torch
import PIL.Image
import PIL.ImageOps
from data_utils import targetpad_transform, server_base_path

if torch.cuda.is_available():
    device = torch.device("cuda")
    data_type = torch.float16
else:
    device = torch.device("cpu")
    data_type = torch.float32

#RN50x4
#"ViT-B/32"
clip_model, clip_preprocess = clip.load("RN50x4")
clip_model = clip_model.eval().to(device)

image_path = 'woman.jpg'
pil_image = PIL.Image.open(image_path).convert('RGB')
image = targetpad_transform(1.25, clip_model.visual.input_resolution)(pil_image).to(device)
reference_features = clip_model.encode_image(image.unsqueeze(0))

I would like to know how to solve this problem? And I would appreciate it if you could test it and tell me the solution.