Comments (6)
I am not sure if this is going to work with zero-shot. But I will note that in the training corpus I do see examples at the end of abstracts and articles with "Keywords: ..." ... So you could try appending "Keywords: " to the end of your example and seeing what happens. I think the key for this to work is finding a format that appears repeatedly in the Pubmed training data ... another direction is to briefly fine tune ... I think if you came up with several hundred examples and fine tuned on those it should produce reasonable results. We could help if you have trouble getting access to compute for the fine tuning ... I'll try to explore this for keyword as well and let you know what I see !
from biomedlm.
I am not sure how many training examples are needed for fine tuning, but on the MeQSum task there are only 500 examples, so it is possible with a relatively small training set the model could be fine tuned to do the right thing ...
from biomedlm.
My advice would be to look at the PubMed abstracts and articles and look at what patterns involve "Keywords" ... I see things like "Keywords used", "Keywords: ", "Keywords included " ... I think if this works zero-shot it would be because there is a common pattern in the PubMed abstracts and articles.
from biomedlm.
Here is a real PubMed example for instance:
Identification of the experimental herbaceous host range of the Apscaviroids infecting citrus species.\nCitrus viroid V (CVd-V), citrus dwarfing viroid (CDVd) and citrus bent leaf viroid (CBLVd) (the genus Apscaviroid, the family Pospiviroidae) have been reported to be restricted to citrus species naturally. The herbaceous host range of these viroids was identified using the viroids infectious clones. Several herbaceous plants from the Cucurbitaceae, Solanaceae, Fabaceae and Asteraceae families were found to be susceptible to CVd-V, CDVd and CBLVd. Also, the viroids could be transferred to these hosts through rubbing of monomeric DNA plasmids and through mechanical inoculation of infected sap. Keywords: citrus; viroid; host range; CVd-V; CBLVd; CDVd.
from biomedlm.
And remember our model only has context length of 1024, so you need to break up input into 1024 blocks ... so if you had long input it'd be better to get keywords for each section and then combine in the end ...
from biomedlm.
Hi @J38,
Many thanks for your reply, it was extremely helpful. Actually, you are right about the out-of-the-box zero-shot performance, the initial results don't look very promising. I still have to search for more prompt patterns, maybe the results get improved. Unfortunately, we only have (very) limited data and that is why we were hoping for a zero-shot setup to work out, but in the worst case, we will try to get additional data and get it annotated.
Thankfully, we have access to compute but thank you for offering your support, really appreciate it!
from biomedlm.
Related Issues (20)
- can it be fine tuned in samller GPU HOT 10
- fine tuning on seqcls task with deepspeed hit RuntimeError: a leaf Variable that requires grad is being used in an in-place operation. HOT 14
- BioMedLm for NER and sentiment analysis HOT 3
- 2pac style rap
- How to run the evaluator for MedQA-USMLE HOT 20
- Evaluate MedQA_USMLE on a saved model HOT 1
- Using a UMLS based retriever to enhance MedQA-USMLE performance HOT 1
- sentence embedding HOT 1
- How to cite BioMedLM HOT 3
- How can I try question answering ?
- Unexpected bug for generate function
- Seqcls for multi-label task HOT 2
- Finetuning BioMedLM for Medical QA HOT 10
- Running generation batch misses file HOT 4
- Max Input and Output length HOT 2
- Generation is suspiciously slow for long sequences HOT 4
- The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. HOT 2
- torch.distributed.launch on eight 40G A100, CUDA out of memory.
- Requesting for dataset used in the section "Free Response Question Answering"
- I set tokenizer.pad_token = tokenizer.eos_token and found tokenizer.pad_token_id==None, which leads to an error.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from biomedlm.