lukejaffe / gfn Goto Github PK
View Code? Open in Web Editor NEWGallery Filter Network for Person Search
License: MIT License
Gallery Filter Network for Person Search
License: MIT License
(bcsj) [G19830015@admin1 GFN-main]$ osr_run --trial_config=./configs/cuhk_train_final.yaml
Traceback (most recent call last):
File "/public/home/G19830015/.local/bin/osr_run", line 33, in
sys.exit(load_entry_point('osr-lib==1.1.0', 'console_scripts', 'osr_run')())
File "/public/home/G19830015/.local/bin/osr_run", line 25, in importlib_load_entry_point
return next(matches).load()
File "/public/home/G19830015/miniconda3/envs/bcsj/lib/python3.7/site-packages/importlib_metadata/init.py", line 209, in load
module = import_module(match.group('module'))
File "/public/home/G19830015/miniconda3/envs/bcsj/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 668, in _load_unlocked
File "", line 638, in _load_backward_compatible
File "/public/home/G19830015/.local/lib/python3.7/site-packages/osr_lib-1.1.0-py3.7.egg/osr/engine/main.py", line 21, in
ModuleNotFoundError: No module named 'ray.tune.integration.torch'
Unable to download dataset PRW via link
Hello, I would like to ask, if I am doing it based on SeqNet, how to set up the two data sets of Occlusion and Resolution in SeqNet?
In Readme.md, in checkpoints link,
for PRW dataset, checkpoint is cuhk...
for CUHK dataset, checkpoint is prw...
Pls rename checkpoint or update link to avoid confusion.
Tks.
The error when getting the person sim
person sim: tensor([], device='cuda:0')
# For each query image
for query_detect in query_output_list:
print(query_detect)
# Get query person embeddings
query_person_emb = query_detect['det_emb']
print("query person emb:", query_person_emb )
print(gallery_output_list)
# For each gallery image
for gallery_output_dict in gallery_output_list:
## Get gallery person embeddings
gallery_person_emb = gallery_output_dict['det_emb']
## Compute person similarity: cosine similarity of person embeddings
person_sim = torch.mm(
F.normalize(query_person_emb, dim=1),
F.normalize(gallery_person_emb, dim=1).T
).flatten()
print("person sim:", person_sim)
## Store person similarity
if 'person_sim' not in gallery_output_dict:
gallery_output_dict['person_sim'] = []
gallery_output_dict['person_sim'].append(person_sim)
Here is the error
[{'det_boxes': tensor([], device='cuda:0', size=(0, 4)), 'det_scores': tensor([], device='cuda:0'), 'det_labels': tensor([], device='cuda:0'), 'det_emb': tensor([], device='cuda:0', size=(0, 2048)), 'gt_emb': tensor([[-3.4526e-02, 2.3393e-03, 7.4060e-05, ..., -1.4855e-02,
3.2582e-02, -1.1846e-02]], device='cuda:0'), 'scene_emb': tensor([[-0.0055, 0.0030, 0.0254, ..., 0.0161, -0.0118, -0.0061]],
device='cuda:0')}, {'det_boxes': tensor([], device='cuda:0', size=(0, 4)), 'det_scores': tensor([], device='cuda:0'), 'det_labels': tensor([], device='cuda:0'), 'det_emb': tensor([], device='cuda:0', size=(0, 2048)), 'gt_emb': tensor([[-3.4526e-02, 2.3393e-03, 7.4060e-05, ..., -1.4855e-02,
3.2582e-02, -1.1846e-02]], device='cuda:0'), 'scene_emb': tensor([[-0.0055, 0.0030, 0.0254, ..., 0.0161, -0.0118, -0.0061]],
device='cuda:0')}]
query berpson embedding: tensor([], device='cuda:0', size=(0, 2048))
tensor([], size=(0, 1))
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
[<ipython-input-82-ec31bd52fe00>](https://localhost:8080/#) in <cell line: 3>()
16 with torch.no_grad():
17 print(model.gfn.get_scores(query_person_emb, query_scene_emb, gallery_scene_emb))
---> 18 qg_scene_sim = model.gfn.get_scores(query_person_emb, query_scene_emb, gallery_scene_emb).flatten().item()
19
20 ## Store query-scene similarity
RuntimeError: a Tensor with 0 elements cannot be converted to Scalar
Hi there, I have tried train GFN in my own device(3090ti-24G) by using
osr_run --trial_config=./configs/prw_train_final.yaml
set batch size to 4 and change debug to True,
while I got failed as below, have you ever meet the same problem? Hoping for your reply.
==> Computing detections and embeddings
100% 6112/6112 [11:01<00:00, 9.24it/s]
==> Computing detection performance
100%|██████████| 6112/6112 [00:09<00:00, 654.58it/s]
num_det_tot: 27972
gt/tot: 23809 25062
100%|██████████| 6112/6112 [00:07<00:00, 798.01it/s]
num_det_tot: 25062
gt/tot: 25062 25062
det:
{'[email protected]': 0.9250273006517983, '[email protected]': 0.9500039901045407}
gt:
{'[email protected]': 1.0, '[email protected]': 1.0}
==> Computing retrieval performance (protocol)
==> Protocol: test
0%| | 0/2057 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/.local/bin/osr_run", line 33, in <module>
sys.exit(load_entry_point('osr-lib==1.0.0', 'console_scripts', 'osr_run')())
File "/home/.local/lib/python3.7/site-packages/osr_lib-1.0.0-py3.7.egg/osr/engine/main.py", line 439, in main
File "/home/.local/lib/python3.7/site-packages/osr_lib-1.0.0-py3.7.egg/osr/engine/main.py", line 333, in run
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/.local/lib/python3.7/site-packages/osr_lib-1.0.0-py3.7.egg/osr/engine/evaluate.py", line 201, in evaluate_performance
File "/home/.local/lib/python3.7/site-packages/osr_lib-1.0.0-py3.7.egg/osr/engine/evaluate.py", line 505, in evaluate_retrieval_orig
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 233, in average_precision_score
average_precision, y_true, y_score, average, sample_weight=sample_weight
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/metrics/_base.py", line 75, in _average_binary_score
return binary_metric(y_true, y_score, sample_weight=sample_weight)
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 206, in _binary_uninterpolated_average_precision
y_true, y_score, pos_label=pos_label, sample_weight=sample_weight
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 859, in precision_recall_curve
y_true, probas_pred, pos_label=pos_label, sample_weight=sample_weight
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 737, in _binary_clf_curve
assert_all_finite(y_score)
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/utils/validation.py", line 134, in assert_all_finite
_assert_all_finite(X.data if sp.issparse(X) else X, allow_nan)
File "/home/anaconda3/envs/osr/lib/python3.7/site-packages/sklearn/utils/validation.py", line 116, in _assert_all_finite
type_err, msg_dtype if msg_dtype is not None else X.dtype
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
I don't quite understand how hard threshold of gfn are used in your code. Your paper mentions that GFN will use a hard threshold, low-scoring scenes are discarded and no need to perform detection.
But in inference, I think all scenes are detected and no variable like gfn_score_thresh had been have been mentioned or used.
def inference(self, images: List[Tensor], targets:Optional[List[Dict[str, Tensor]]]=None, inference_mode:str='both') -> List[Dict[str, Tensor]]:
#
original_image_sizes = [(img.shape[-2], img.shape[-1]) for img in images]
num_images = len(original_image_sizes)
images, targets = self.transform(images, targets)
bb_features = self.backbone(images.tensors)
# Get image features from the GFN
if self.use_gfn:
scene_emb = self.gfn.get_scene_emb(bb_features).split(1, 0)
else:
scene_emb = [torch.empty(0) for _ in range(num_images)]
#
reid_features = bb_features
detections = [{} for _ in range(num_images)]
embeddings = [torch.empty(0) for _ in range(num_images)]
if (inference_mode in ('gt', 'both')) and (targets is not None):
# query
boxes = [t["boxes"] for t in targets]
section_lens = [len(b) for b in boxes]
box_features = self.roi_heads.reid_roi_pool(reid_features, boxes, images.image_sizes)
box_features = self.roi_heads.reid_head(box_features)
_embeddings, _ = self.roi_heads.embedding_head(box_features)
embeddings = _embeddings.split(section_lens, dim=0)
if (inference_mode in ('det', 'both')) or (inference_mode in ('gt', 'both')):
# gallery
rpn_features = bb_features
proposals, _ = self.rpn(images, rpn_features, targets)
detections, _ = self.roi_heads(
bb_features, proposals, images.image_sizes, targets
)
detections = self.transform.postprocess(
detections, images.image_sizes, original_image_sizes
)
# Reorganize outputs into single list of dict
output_list = [{
'det_boxes': d['boxes'],
'det_scores': d['scores'],
'det_labels': d['labels'],
'det_emb': d['embeddings'],
'gt_emb': e,
'scene_emb': s,
} for d, e, s in zip(detections, embeddings, scene_emb)]
# Return output
return output_list
In evaluate, it is the same situation. Although the gfn_scores are calculated, they are only used as weights and do not seem to be compared with any hard thresholds.
The only variable related to gfn_score's thresh I can find is gfn_filter_thresh
, which keeps 99% of positive gallery scenes, as you mentions in your paper. But it seems that this variable has not been further used.
Your explanation can help me better understand. Looking forward to your reply and assistance.
Hello, I would like to know how the occlusion and low-resolution datasets used in Table 1 were set up. Do I need to manually gather statistics from the original database, or is there a parameter setting during training that takes care of it? I look forward to your response. Thank you very much!
Hi for web_demo.ipynb, downloading torchscript files from readme says file is in owner's trash
(the file for torchscript_path = '../torchscript/cuhk_final.pt')
the checkpoints file.pt is available, tried just in case it was swapped with torchscript, but it produces error "PytorchStreamReader failed locating file constants.pkl: file not found"
Thank you!
Hello, I have tested your code and tested it on the PRW dataset. I want to know which indicators I should focus on, namely, the Map and Top-1 that you refer to in your paper.
input: osr_run --trial_config=./configs/prw_test_final.yaml
``
(func pid=19147) ==> SUCCESS!!!
Result for WrappedDistributedTorchTrainable_cda35_00000:
date: 2023-03-22_21-59-16
done: true
experiment_id: 56fe3ccccab14c33a1421bb441dc2be7
experiment_tag: '0'
hostname: user-virtual-machine
iterations_since_restore: 1
node_ip: 10.60.150.135
pid: 18464
[email protected]: 0.934801125125824
test_cross_cam_id_det_gfn_mAP: 0.5643779315391021
test_cross_cam_id_det_gfn_match: 1.0
test_cross_cam_id_det_gfn_recall: 0.9788652820880083
test_cross_cam_id_det_gfn_top1: 0.8215848322800194
test_cross_cam_id_det_mAP: 0.5525456165277787
test_cross_cam_id_det_match: 1.0
test_cross_cam_id_det_recall: 0.9788652820880083
test_cross_cam_id_det_top1: 0.8045697617890131
test_cross_cam_id_gt_gfn_mAP: 0.5782933392838289
test_cross_cam_id_gt_gfn_match: 1.0
test_cross_cam_id_gt_gfn_recall: 1.0
test_cross_cam_id_gt_gfn_top1: 0.8230432668935342
test_cross_cam_id_gt_mAP: 0.5664176155812705
test_cross_cam_id_gt_match: 1.0
test_cross_cam_id_gt_recall: 1.0
test_cross_cam_id_gt_top1: 0.8045697617890131
test_cross_cam_id_image_gfn_frac_filter: 0.11711236787108889
test_cross_cam_id_image_gfn_mAP: 0.16209892104473658
test_cross_cam_id_image_gfn_thresh_filter: 0.4776982069015503
test_cross_cam_id_image_gfn_top1: 0.42343218279047157
[email protected]: 0.9568270688692043
test_same_cam_id_det_gfn_mAP: 0.8505308214692695
test_same_cam_id_det_gfn_match: 1.0
test_same_cam_id_det_gfn_recall: 0.9790137053352912
test_same_cam_id_det_gfn_top1: 0.9859017987360233 test_same_cam_id_gt_mAP: 0.8492902613364299
test_same_cam_id_gt_match: 1.0
test_same_cam_id_gt_recall: 1.0
test_same_cam_id_gt_top1: 1.0
test_same_cam_id_image_gfn_frac_filter: 0.2125033424679168
test_same_cam_id_image_gfn_mAP: 0.6889602817530124
test_same_cam_id_image_gfn_thresh_filter: 0.5215099453926086
test_same_cam_id_image_gfn_top1: 1.0
test_test_det_gfn_mAP: 0.5827848703489255
test_test_det_gfn_match: 1.0
test_test_det_gfn_recall: 0.9788257565934494
test_test_det_gfn_top1: 0.924161400097229
test_test_det_mAP: 0.5760174356415955
test_test_det_match: 1.0
test_test_det_recall: 0.9788257565934494
test_test_det_top1: 0.8945065629557608
test_test_gt_gfn_mAP: 0.5990268233035214
test_test_gt_gfn_match: 1.0
test_test_gt_gfn_recall: 1.0
test_test_gt_gfn_top1: 0.9309674282936315
test_test_gt_mAP: 0.5917157777048339
test_test_gt_match: 1.0
test_test_gt_recall: 1.0
test_test_gt_top1: 0.8998541565386485
test_test_image_gfn_frac_filter: 0.11459317284876058
test_test_image_gfn_mAP: 0.20687682098861937
test_test_image_gfn_thresh_filter: 0.4808691143989563
test_test_image_gfn_top1: 0.7841516771998055
time_since_restore: 8954.510276317596
time_this_iter_s: 8954.510276317596
time_total_s: 8954.510276317596
timestamp: 1679493556
timesteps_since_restore: 0
training_iteration: 1
trial_id: cda35_00000
warmup_time: 16.191797971725464
== Status ==
Current time: 2023-03-22 21:59:20 (running for 02:29:53.97)
Memory usage on this node: 10.7/62.8 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/12 CPUs, 0/2 GPUs, 0.0/34.04 GiB heap, 0.0/17.02 GiB objects (0.0/1.0 accelerator_type:V100)
Result logdir: /diskvdb/yyl/zqx_PS/paper/GFN-main/logging/prw_final
Number of trials: 1/1 (1 TERMINATED)
+----------------------------------------------+------------+---------------------+--------+------------------+---------------+-------------------+----------------------+
| Trial name | status | loc | iter | total time (s) | [email protected] | [email protected] | test_test_gt_match |
|----------------------------------------------+------------+---------------------+--------+------------------+---------------+-------------------+----------------------|
| WrappedDistributedTorchTrainable_cda35_00000 | TERMINATED | 10.60.150.135:18464 | 1 | 8954.51 | 0.934801 | 0.956827 | 1 |
+----------------------------------------------+------------+---------------------+--------+------------------+---------------+-------------------+----------------------+
2023-03-22 21:59:20,951 INFO tune.py:702 -- Total run time: 8994.12 seconds (8993.95 seconds for the tuning loop).
Hello, I have changed debug to True to train GFN, but I could not achieve the effect in the paper. The value of test_test_det_gfn_mAP is 0.487, and the value of test_test_det_gfn_top1 is 0.871. But the values for [email protected] and [email protected] are normal, at 0.9264 and 0.9512, respectively. What is the reason? Hoping for your reply.
I’am trying to run the web_demo.ipynb but meet an error as follow.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_11836\3154094266.py in <module>
1 # Load torchscript version of model
----> 2 model = torch.jit.load(torchscript_path)
c:\Users\SUSIE\Anaconda3\envs\osr\lib\site-packages\torch\jit\_serialization.py in load(f, map_location, _extra_files)
160 cu = torch._C.CompilationUnit()
161 if isinstance(f, str) or isinstance(f, pathlib.Path):
--> 162 cpp_module = torch._C.import_ir_module(cu, str(f), map_location, _extra_files)
163 else:
164 cpp_module = torch._C.import_ir_module_from_buffer(
RuntimeError:
Unknown builtin op: torchvision::roi_align.
Could not find any similar ops to torchvision::roi_align. This op may not exist or may not be currently supported in TorchScript.
:
File "code/__torch__/torchvision/ops/roi_align.py", line 18
else:
rois = unchecked_cast(Tensor, boxes)
_6 = ops.torchvision.roi_align(input, rois, spatial_scale, output_size[0], output_size[1], sampling_ratio, aligned)
~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
return _6
'roi_align' is being compiled since it was called from '_multiscale_roi_align'
Serialized File "code/__torch__/torchvision/ops/poolers.py", line 128
_30 = "scales and mapper should not be None"
_31 = __torch__.torchvision.ops.poolers._convert_to_roi_format
_32 = __torch__.torchvision.ops.roi_align.roi_align
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
_33 = "This Python function is annotated to be ignored and cannot be run"
_34 = uninitialized(Tensor)
'_multiscale_roi_align' is being compiled since it was called from 'MultiScaleRoIAlign.forward'
Serialized File "code/__torch__/torchvision/ops/poolers.py", line 19
_0 = __torch__.torchvision.ops.poolers._filter_input
_1 = __torch__.torchvision.ops.poolers._setup_scales
_2 = __torch__.torchvision.ops.poolers._multiscale_roi_align
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
featmap_names = self.featmap_names
x_filtered = _0(x, featmap_names, )
A similar problem (https://stackoverflow.com/questions/65705160/torch-vision-c-interface-error-unknown-builtin-op-torchvisionnms) was solved by replacing the torch version. I also read some related issue (https://github.com/pytorch/vision/issues) but I didn't find a solution.
My environment is pytorch=1.12.0 + torchvision=0.13.0 as yours in conda.yaml.
I'm glad that the code is running successfully! However, I encountered an error while converting my trained model to Torchscript:
RuntimeError: getattr's second argument must be a string literal: File "/root/miniconda3/lib/python3.8/site-packages/pytorch_metric_learning/utils/common_functions.py", line 275 def reset_stats(input_obj): for attr_list in ["_record_these_stats"]: for r in getattr(input_obj, attr_list, []): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
It seems there is an issue with the following code:
`# def reset_stats(input_obj):
`
I have been trying to resolve this for over two weeks and am not sure how to accurately and correctly convert the model to Torchscript format. Do you have any suggestions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.