Comments (14)
yea you are right, that is acceptable
only advantage to using buffers is that it would manage its device with the rest of the network parameters, if you were to move it
from coca-pytorch.
@gshaikov-paige oh yes, thanks! i'll also go make the change for a contending method here
from coca-pytorch.
@gshaikov-paige so one note is that since i open sourced this, there has been research suggesting PaLM to have some sort of inherent instability. i think i should start removing PaLM architecture from all my repositories soon, and replace it with the one from llama (which is close, just serial)
from coca-pytorch.
Thanks for the heads up! I can help with transition to LLAMA in this repo, if you're open to PRs?
from coca-pytorch.
I also added type hints on my branch, I can commit them back if you'd like.
from coca-pytorch.
@lucidrains thanks - yes, I am doing research to modify CoCa or related methods for my use case, so needed to be able to train it from scratch. I do plan to experiment with a pretrained model though!
Re Llama port: I am planning to make the change from my personal account (@gshaikov) but I can do it only next week since I am away this week, so if this work is urgent please don't wait for me :)
from coca-pytorch.
@gshaikov-paige sg! i think finetuning it for the medical domain, if that's what you are still doing, should work well
yea sg, i won't wait haha
from coca-pytorch.
@gshaikov-paige let me know if 0.0.8 takes care of it
from coca-pytorch.
Thanks! Almost - you need to change it in __init__
as well
from coca-pytorch.
happy training!
from coca-pytorch.
@gshaikov-paige yea absolutely! ๐
from coca-pytorch.
@gshaikov-paige hmm, i think this project is small enough type hints are not that crucial
from coca-pytorch.
@gshaikov-paige are you aware that Laion has already trained a CoCa? https://laion.ai/blog/coca/
from coca-pytorch.
Ok fair enough :)
from coca-pytorch.
Related Issues (18)
- Maybe don't need this rearrange HOT 2
- Contrastive loss should be applied to L2-normed embeddings instead of layer normed? HOT 2
- Image tokens dimension mismatch with the layernorm bias dimension HOT 12
- LayerNorm after attentional_pooler HOT 1
- convolution encoder better result then vit HOT 13
- Residual connection misssing?
- pad_dim_to() is not implemented HOT 3
- Reproduce Problem HOT 2
- Any plan to share the pre-trained models? HOT 24
- contrastive_labels - dist batch size isn't correct
- Generating the caption of a given image HOT 2
- why train VIT visual encoder first? HOT 1
- Reproducing the results in the paper HOT 2
- Extractor in vit_pytorch will detach the tensor. HOT 2
- Do you plan to give the code for fine-tuning๏ผ HOT 4
- attn_mask HOT 3
- How to train the model using my own dataset? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coca-pytorch.