Comments (10)
Hi @NielsRogge! Thanks for your suggestions! We have created a org for https://huggingface.co/videomae. I want to know how to upload our models correctly?
from videomae.
Hi @NielsRogge! Is there any update?
from videomae.
Hi @yztongzhan,
I just worked a bit further on it, I've implemented VideoMAEForPreTraining
now as well, which includes the decoder and loss computation. The PR is now ready for review and will be reviewed by my colleagues.
Also, would it be possible to create an organization on the hub for Multimedia Computing Group, Nanjing University, with a short name (rather than just VideoMAE)? Cause otherwise people will have to do:
from transformers import VideoMAEModel
model = VideoMAEForVideoClassification.from_pretrained("VideoMAE/videomae-base-finetuned-kinetics")
for instance, which means they have to type quite a lot of videomae ๐ also, if there would be newer models coming out that are also part of the research of the same organization (such as AdaMixer), it makes sense to upload them to the same organization on the hub.
Regards,
Niels
from videomae.
Hi @NielsRogge ,
Thanks for your update. We have created an organization account on the hub:
https://huggingface.co/MCG-NJU
You can use this organization for storing our model checkpoints. BTW, you could also include our other repo such as AdaMixer and MixFormer.
Best,
Limin
from videomae.
@NielsRogge Any update๏ผ
from videomae.
Hi @wanglimin,
the model will soon be added to the library. I'll transfer the weights to the MCG-NJYU organization today.
Are you interested in collaborating on a script for easy fine-tuning?
from videomae.
I've currently transferred 3 models: https://huggingface.co/models?other=videomae.
To make the model names not too long, I would use the following names:
model_names = [
# Kinetics-400 checkpoints (short = pretrained only for 800 epochs instead of 1600)
"videomae-base-short",
"videomae-base-short-finetuned-kinetics",
"videomae-base",
"videomae-base-finetuned-kinetics",
"videomae-large",
"videomae-large-finetuned-kinetics",
# Something-Something-v2 checkpoints (short = pretrained only for 800 epochs instead of 2400)
"videomae-base-short-ssv2",
"videomae-base-short-finetuned-ssv2",
"videomae-base-ssv2",
"videomae-base-finetuned-ssv2",
]
Is that ok for you? Also, are you interested in adding model cards to the repos on the hub? Each model has its own git repo, and the model card is just a README (Markdown file).
from videomae.
Hi @wanglimin,
VideoMAE has been added to the library! https://huggingface.co/docs/transformers/main/en/model_doc/videomae
Checkpoints are on the hub: https://huggingface.co/models?other=videomae
from videomae.
Hi @NielsRogge! Thanks again for your efforts! We will add these links in README.
from videomae.
@NielsRogge , Thanks a lot for your help!
from videomae.
Related Issues (20)
- What's the finetuning differences between ViT-B 80%acc and 81%acc?
- Can I fine-tine it on a video dataset of 32 frames?
- The Batch size and training epoch not metch with paper
- About the encoder layer output
- Issue Encountered When Loading the Model: "pretrain_videomae_base_patch16_224" HOT 5
- ModuleNotFoundError: No module named 'petrel_client'
- License of Kinetics-400
- video xxx not correctly load during training
- The GPU memory usage of UCF and Kinetics is different.
- TODO/videomae_pretrain_base_patch16_224_frame_16x4_tube_mask_ratio_0.9_e1600/checkpoint-1599.pth HOT 1
- HMDB checkpoint
- Rebuild Video
- the numberi of rebuild images is too small
- AttributeError: module 'torch._C' has no attribute '_get_privateuse1_backend_name'
- How to obtain the reconstructed image for inference and masked
- How good model handle the different duration of clips?
- The dataset files in the link are not available HOT 2
- BUG: Incorrect temporal indexing?
- UCF101 HOT 4
- Questions about performence on ssv2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from videomae.