Comments (3)
Ah found the line here! https://github.com/facebookresearch/VLPart/blob/main/demo/predictor.py#L23
from vlpart.
Hi @PeizeSun,
For the zero-shot weights, do you just run each part of the vocabulary with CLIP and then save the resulting matrix?
from vlpart.
Update: When I do the above I get different weights than what are saved.
form = lambda x: f"A photo of a {' '.join(x.split(':'))}"
embeds = torch.zeros(77,1024)
with torch.no_grad():
for idx,i in enumerate(PASCAL_PART_BASE_CATEGORIES):
embeds[idx,:] = text_encoder(text_encoder.tokenize(form(i['name'])))
saved_embeds = np.load("datasets/metadata/pascal_part_base_clip_RN50_a+cname.npy")
print(embeds)
tensor([[-0.0448, 0.0072, -0.1996, ..., 0.0097, -0.2787, 0.3235],
[-0.0216, 0.2441, -0.2372, ..., -0.1914, -0.1295, 0.1115],
[ 0.0405, -0.1673, -0.1807, ..., -0.2903, -0.2093, 0.0923],
...,
[ 0.0482, -0.1842, -0.0162, ..., -0.3602, 0.1764, 0.0118],
[ 0.0588, -0.1146, -0.0537, ..., -0.1661, -0.0903, 0.0582],
[ 0.4419, 0.0859, 0.1314, ..., -0.0912, -0.3575, -0.1130]])
print(saved_embeds)
array([[-0.2603 , 0.1097 , -0.2805 , ..., -0.03482 , -0.1514 ,
0.3855 ],
[-0.1748 , 0.328 , -0.3494 , ..., -0.3804 , -0.0924 ,
0.141 ],
[-0.09174 , -0.04358 , -0.2373 , ..., -0.3523 , -0.1426 ,
0.1539 ],
...,
[-0.01486 , -0.1587 , -0.11475 , ..., -0.4219 , 0.1277 ,
0.1537 ],
[-0.0338 , -0.004128, -0.2137 , ..., -0.3127 , -0.04684 ,
0.1978 ],
[ 0.2861 , 0.1469 , -0.0263 , ..., -0.2102 , -0.3286 ,
0.03644 ]], dtype=float16)
from vlpart.
Related Issues (16)
- Wrong link in the Blog post HOT 1
- Need some explanation on a data preparation code. HOT 9
- How to parse image level data into part level data HOT 2
- Dynamically changing vocabulary HOT 4
- Export alpha mask
- object detection on aerial image HOT 1
- Pascal Part Clarification HOT 1
- Unstable training when saving zero-shot weights
- Higher Evaluation Result on PACO
- how to visualize the results of paco dataset
- question about free input image
- A Failure of Easy Human Part Segmentation
- 下载链接失效 HOT 1
- demo out of memory on vocabulary 'lvis-paco'
- Where to control granularity of the segmentation?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vlpart.