jannisborn / covid19_ultrasound

Open source lung ultrasound (LUS) data collection initiative for COVID-19.

Automatic Detection of COVID-19 from Ultrasound Data

Summary

This repo contains the code for the paper Accelerating Detection of Lung Pathologies with Explainable Ultrasound Image Analysis.

News

• April '24: We released COVID-BLUES, a new dataset of 371 videos from 63 patients collected in a prospective clinical study. Please check out the data and consider using it instead of this one since it is of much higher quality. The paper for this data will appear soon.

Dataset

Feel free to use (and cite) our dataset. We currently have >200 LUS videos labelled with a diagnostic outcome. Moreover, lung severity scores for 136 videos are made available in the dataset_metadata.csv under the column "Lung Severity Score" from Gare et al., 2022. Further clinical information (symptoms, visible LUS patterns etc) are provided for some videos. For details see data/README.md.

If you are looking for more data, please consider using the 40,000 carefully simulated LUS images and paired labels from the paper by Zhao et al. (2024, Communications Medicine). In addition, segmentation labels for a subset of the in vivo data in this repo are also available. For details see PULSE Lab Repository/README.md.

NOTE: Please make sure to create a meaningful train/test data split. Do not split the data on a frame-level, but on a video/patient-level. The task becomes trivial otherwise. See the instructions here.

Please note: The founders/authors of the repository take no responsibility or liability for the data contributed to this archive. The contributing sites have to ensure that the collection and use of the data fulfills all applicable legal and ethical requirements.

Contribution

Overview figure about current efforts. Public dataset consists of >200 LUS videos.

Motivation:

From the ML community, ultrasound has gained much less attention than CT and X-Ray in the context of COVID-19. But many voices from the medical community have advocated for a more prominent role of ultrasound in the current pandemic.

Summary

We developed methods for the automatic detection of COVID-19 from Lung Ultrasound (LUS) recordings. Our results show that one can accurately distinguish LUS samples from COVID-19 patients from healthy controls and bacterial pneumonia. Our model is validated on an external dataset (ICLUS) where we achieve promising performance. The CAMs of the model were validated in a blinded study by US experts and found to highlight relevant pulmonary biomarkers. Using model uncertainty techniques, we can further boost model performance and find samples which are likely to be incorrectly classified. Our dataset complements the current data collection initiaves that only focus on CT or X-Ray data.

Evidence for ultrasound

Ultrasound is non-invasive, cheap, portable (bedside execution), repeatable and available in almost all medical facilities. But even for trained doctors detecting patterns of COVID-19 from ultrasound data is challenging and time-consuming. Since their time is scarce, there is an urgent need to simplify, fasten & automatize the detection of COVID-19.

• LUS is more sensitive than X-Ray in diagnosing COVID-19
• COVID-19 outbreak: less stethoscope, more ultrasound
• Ultrasound can evidence the same symptoms as CT: (Point-by-point correspondance of CT and ultrasound findings through COVID-19 disease process)

Learn more about the project

• Read our manuscript
• Read our blogpost

Installation

Ultrasound data

Find all details on the current state of the database in the data folder.

Deep learning model (pocovidnet)

Find all details on how to reproduce our experiments and train your models on ultrasound data in the pocovidnet folder.

Web interface (pocovidscreen)

Find all details on how to get started in the pocovidscreen folder.

Contact

• If you experience problems with the code, please open an issue.
• If you have questions about the project, please reach out: [email protected].

Citation

An abstract of our work was published in Thorax as part of the BTS Winter Meeting 2021. The full paper is available via the COVID-19 special issue of Applied Sciences. Please cite these in favor of our deprecated POCOVID-Net preprint.

Please use the following bibtex entry to cite this dataset:

@article{born2021accelerating,
  title={Accelerating Detection of Lung Pathologies with Explainable Ultrasound Image Analysis}, 
  author={Born, Jannis and Wiedemann, Nina and Cossio, Manuel and Buhre, Charlotte and Brändle, Gabriel and Leidermann, Konstantin and      Aujayeb, Avinash and Moor, Michael and Rieck, Bastian and Borgwardt, Karsten}, 
  volume={11}, ISSN={2076-3417}, 
  url={http://dx.doi.org/10.3390/app11020672}, 
  DOI={10.3390/app11020672}, 
  number={2}, 
  journal={Applied Sciences}, 
  publisher={MDPI AG}, 
  year={2021}, 
  month={Jan}, 
  pages={672}
}

covid19_ultrasound's Issues

60 unlabled videos

From website 'https://www.disi.unitn.it/iclus', 60 videos can be obtained but they aren't labled.

CAP

There's one CAP (Community-acquired-pneumonia) image and one healthy image: https://www.karger.com/Article/Pdf/357449

Healthy should be usable, about CAP I'm unsure since CAP can be caused by both viruses and bacteria.

Data sources to check

Possible sources for data (healthy patients in particular) might be ...
Websites from:

• EFSUMB (European society for sonography) - They also did a webinar on Covid ultrasound in mid-aril, maybe that's available online as well
• DEGUM (German equivalent)
• BIKUS

Are we using this video: https://www.youtube.com/watch?v=cTZzrB_o5iI&feature=youtu.be already?

Pneumonia images from https://www.sonographiebilder.de/sonographie-bilder/thorax/pneumonie/

Healthy lung images:

• https://www.researchgate.net/figure/Ultrasound-scan-of-a-single-B-line-in-a-normal-lung-A-and-interstitial-syndrome-in_fig3_287791558
• https://cloud.clarius.me/cases/24142/pneumonia/

10 videos!

https://www.sciencedirect.com/science/article/pii/S0733862715000772#mmc1

• 10 videos in total just in one publication :)
• convex and linear probes
• class labels are not always clear but there are definitely a few healthy and a few bacterial pneumonia. Little to no artifacts.

3 Videos

This paper releases 3 videos:

https://www.medrxiv.org/content/10.1101/2020.10.13.20212258v2

• Vid 3 is clearly COVID
• Vid 2 is non-covid-ards: can we find whether it's bacterial or viral?
• Vid 1 is hydrostatic pulmonary edema (probably cant be sorted into any of our buckets)

Annotated Data

I cannot find any file which annotates the given dataset. I appreciate if you can update the data with annotations or if it is there, would you please point out the file to me?

Thanks
Mudit

wfumb 3 images

https://wfumb.info/ultrasound-the-best-cases/

2 Healthy & 5 COVID videos

https://www.stemlynsblog.org/combatting-covid19-is-lung-ultrasound-an-option/

• 2 healthy videos
• 5 COVID videos

There seems to be a publication associated, however it doesn't seem published:
https://www.researchgate.net/publication/342215033_Combatting_COVID-19_Is_Ultrasound_an_Important_Piece_in_the_Diagnostic_Puzzle_Correspondence_to_Word_count_excluding_figures_in-text_references_bibliography

Butterfly data

1. I'm having trouble downloading the butterfly data:
   I press the "download" icon and my request "is processed", but eventually I get an error

File requested not found

Is there an alternative way of getting this data?

2. Once I obtain the data, the README instruct to run sh parse_butterfly.sh. For some reason I don't find this script in the repository. Am I missing something here?

Update README

Readme should be professionalized and updated with new references.

14 COVID videos

Plenty new COVID videos from this paper.

Total of 14 COVID videos. Looks a bit heterogenous in light conditions etc.

I copy the symptoms below as listed in the paper.

Those seem to be "standard" LUS-COVID findings (S1-S6)

1. (A) B-lines (arrow)
2. (B) irregular and broken pleural lines with multiple B-lines (dotted region);
3. (C) peripheral or subpleural consolidation with
4. (D) minimal colour Doppler signal.
5. (E) Larger zone of consolidation in right lower posterior base with air bronchograms an
6. (F) reduced perfusion using colour Dopple

S7 is CT data

S8 and S9 are COVID findings with additional complications: (not sure we should use those -> ask MDs)
7. B) Transthoracic short-axis aortic valve view showing a dilated right ventricular outflow tract (RVOT) of 35 mm and pulmonary artery in a patient with pulmonary embolism. (video not available currently).
8. C) Deep venous thrombosis of the femoral vein.

S10 - S15 is a COVID patient with severe hypotoxia
9. - 11. before (A–C)
12.-14. after (D–F) prone positioning. Note the significant changes with loss of pleural fluid and consolidation with increased aeration.
Note that this is a single patient. If we take it, it should be in the same split

typo in train/result

I think we should stop pushing to master directly, this is why I open an issue about this (minor) thing.

There's a typo in TrainResults.js:

We really appreciate your donation. 
Your images are going to be checked by our ata Scientists and Medical Doctors. 
After that, the images will be added to our dataset.

I would fix this and also mention that donators are welcome to contact us.

We really appreciate your donation. 
Your data will be verified by our medical doctors and, if approved, will be added to our database.
If you would like to be recognized as donator or would to discuss further collaboration, please contact [email protected]

Add new self-recorded data

Sonosite Data

2 more videos:

• https://www.youtube.com/watch?time_continue=3&v=mtcEyaDGh8U&feature=emb_logo (COVID)
• https://www.youtube.com/watch?time_continue=7&v=5Ozz6gMmZmg&feature=emb_logo (Pneumonia)

Probably not licensed. We can ask whether we can redistribute.
More data may be here: https://de.secure.sonosite.com/COVID-19

2 videos of other viral pneumonia

https://link.springer.com/article/10.1186/cc11201

2 videos of H1N1 (viral pneumonia that is NOT covid)

3 covid videos (CC license)

http://www.emdocs.net/lung-ultrasound-in-covid-19/

database of ICLUS project - get access

Contact people from ICLUS project (university of Trento) - need to login here:
https://covid19.disi.unitn.it/iclusdb/login

Metadata Update

Hello,
Your dataset was added to CoronaWhy (https://www.coronawhy.org/) Data Lake on Dataverse as a piece of common COVID-19 data framehttp://datasets.coronawhy.org/dataset.xhtml?persistentId=doi:10.5072/FK2/46KAJH

Would you be willing to help with the maintenance of your dataset in Dataverse, e.g. adding the relevant metadata and keeping the dataset up-to-date? That will help to make the dataset findable and accessible for the medical science community.

BJA- publication: Practical approach to lung ultrasound

Great approach and publication on ultrasound findings and diagnostics. This paper highlights urgent-care and critical-care diagnostic approaches for improved diagnostic accuracy.

Lung ultrasound (LU) relies on direct visualization of structures and artifact interpretation.
A curvilinear probe is the best ‘all-rounder’.
LU has a very high diagnostic accuracy.
A comprehensive point-of-care scan can be done in <5 min.
A training structure and competencies specific to critical care in the UK have been developed.

https://academic.oup.com/bjaed/article/16/2/39/2897763

about 'data_from_butterfly.json'

I would like to ask where is this file 'data_from_butterfly.json'

Add Donated Data

• Aujayeb data
• Website donations

ToDo:

• Verify class labels with the donators.
• Get feedback from MDs.

Metadata collection

We have to start collecting metadata for all videos, wherever possible:

• patient_ID
• source_ID
• sex
• gender
• age
• severity score
• symptoms (streamline names to ease postprocessing, not Fever and fever)
• pattterns visible

Data from Como (Pub)

DOI: 10.1016/j.ejro.2020.100231

https://www.sciencedirect.com/science/article/pii/S2352047720300204#fig0005

3 images in total, all COVID

1. Convex: (A) shows the simultaneous co-existence of multiple lung B-lines and A-lines.
2. Convex: patient with Coronavirus disease-19 shows compact B-line without horizontal reverberation (white lung sign), with tiny areas of subpleural consolidation.
3. Linear: Ultrasound thoracic examination using a linear transducer of a Patient with Coronavirus disease-19 (A) shows the irregular appearance of the pleural line with multifocal B-line.

Around 5 images for US for covid

https://www.acutemedicine.org.uk/wp-content/uploads/2020/03/COVID-infographic.pdf Lotte pointed me on that, seems useful

3 COVID videos with time progression

https://www.ajemjournal.com/article/S0735-6757(20)30440-X/fulltext

Excellent data source. 3 patients with LUS performed daily from day 1 to day 7. All symptoms reported. We should save all the metadata for those videos too.

We may want to introduce a severity score too now.

6-7 videos

Labels are not super clear (seems symptom based)

https://theultrasoundjournal.springeropen.com/articles/10.1186/s13089-020-00177-4#Sec8

Permanent issue: Check US websites for new data (grepmed, Atlas, wfumb, etc.)

We need to check the websites Grepmed and Pocus Atlas whether they have any new data. We need to be very careful though to check which ones are in the dataset already, and which ones where checked by Gabriel and found to be of bad quality. Maybe check by upload data if available? Anything uploaded after April is interesting.

Model pretraining

Currently we use VGG, pretrained on Imagenet.

Instead we could use models pretrained for 2D or 3D medical image analysis, e.g. Models Genesis: https://arxiv.org/pdf/1908.06912.pdf

They have models pretrained on CT/CXR data. On ultrasound task, Imagenet still performs better than their self-supervised pretrained models (although not significantly).

But for 3D tasks, their pretraining clearly outperforms Imagenet 2D pretraining. Could be useful for the video classification setting @SaheliDe.

Data from esaote (company)

https://www.esaote.com/fr-BE/special-covid-19/our-solutions/lung-ultrasound-in-critical-care/ Several covid videos available

pneumonia educational website (>10 pneumonia videos probably)

https://litfl.com/lung-ultrasound-cases/ Seems to contain many useful videos (tutorials)

https://litfl.com/lung-ultrasound-pneumonia/ Article with three videos that can be downloaded. Only the second one contains artefacts (probably not useful)

1 patient, 1-3 images

https://www.ajog.org/article/S0002-9378(20)30468-3/fulltext

Ask for US data

http://deepultrasound.ai Maybe get in touch and ask if they have access to ultrasound data?

A few images

https://www.sciencedirect.com/science/article/pii/S0720048X16304260?casa_token=b8FWvdm_EkAAAAAA:H31to1wr9OoD-yRrZsfmFuEmRk06uoQJ_o4ig4VgtRb95U8PYuPzLc_2TzrnScnV81Gwd-bfcKE#fig0005

Not super sure on bacterial vs viral pneumonia. Linear probes
4 images in total, not sure on quality.

Publication search for bacterial pneumonia data

Update butterfly data

They also have a site with just lung ultrasound images https://www.butterflynetwork.com/de/clinical/lung

I haven't compared the videos in their Covid gallery https://butterflynetwork.getbynder.com/transfer/2cf9af2398bdc35b6b0d2100180cb69580198f41faeb4c25d9cdae61d1ca8e91 to the ones we have, but I think we should check again because they uploaded new data probably and it would be nice to use it

Add comment field in frontend

The donations website should have a commentary field:

E.g. “Tell us about the origin of that data: Did you record it yourself? If yes, where? ICU? Emergency room? If not, where did you get the data from, do you have a link? Please try to be precise, we have to avoid duplicate dataset entries. "

This field should not allowed to be empty and have a minimum amount of X characters (I suggest 10, to make sure people write at least a few words).

4 images from children with COVID-19 with artifacts

https://pediatrics.aappublications.org/content/pediatrics/early/2020/04/17/peds.2020-1157.full.pdf

Images have artifacts/arrows

Mobile version of pocovidscreen

The title ("Pocovidscreen", right before "An AI tool for early screening...") and the photo carousel of team members are partially hidden in the mobile version, at least in my Chrome for Android. Cheers :)

any explanation for the part of cross validation

-first thank for making this interesting dataset available publicly
-we are a team of 3 teachers at delhi university (india)

can any one explain this part it's not clear for us and thank you

python3 scripts/cross_val_splitter.py --splits 5
Now, your data folder should contain a new folder cross_validation with folders fold_1, fold_2. Each folder contains only the test data for that specific fold.

so if we understand that's mean there is 5 folders (the folder0, folder3 and folder4 for the training) and the folder1 and folder2 for the test part ???
cause we execute this python3 scripts/cross_val_splitter.py --splits 5 we get 5 folders

split data how its work !!!

hi i would like to thank you guys for this wonderful work , just i have a question about how can i split the dataset , after the using the explaination in read e i got a dataset of 3 classes (covid, pnemounia, regular), i start working with the dataset , and i split the dataset randomly in 80% for training and 20% for test im just asking if this is a good why to split the dataset , it seems that the test will content also the images token from the videos for the training

8 COVID-19 videos from pregnant women

I was pointed to this publication: https://onlinelibrary.wiley.com/doi/full/10.1002/jum.15367

• 8 pregnant women with COVID (confirmed by PCR)
• 8 videos available (see supplementary)
• data seems high quality but requires manual cropping
• a lot of metadata on all patients, e.g. visible patterns, availiability of CT/CXR, age, gender etc.

-> First author would like to be mentioned in our repo (with his article).
-> We could create a section with papers from which we got data :)

We could consider to collect meta-data too. If we featurize it properly, it could be valuable in the future.

4 videos plus a few images

https://theultrasoundjournal.springeropen.com/articles/10.1186/2036-7902-4-16

There is 4 videos in this publication!

• All linear probes

• 2x viral pneumonia (not COVID)

• 1x bacterial pn.

• 1x symptoms of both

• Several images (but contain a lot of artifacts)

List of POCUS for COVID resources

I downloaded the videos on this site https://www.stemlynsblog.org/combatting-covid19-is-lung-ultrasound-an-option/, but if you scroll to the very bottom of the article, there are many further resources: For example, check:

https://www.coreultrasound.com/category/ultrasound-of-the-week/

https://westernsono.ca/screencasts/lung-ultrasound/ and so on

Data Privacy Statement

Will be added.

3 images (2 covid white lung, 1 normal patterns)

https://www.ajog.org/action/showPdf?pii=S0002-9378%2820%2930468-3
Publication: The Diagnosis of Pneumonia in a Pregnant Woman with COVID-19 Using Maternal Lung Ultrasound

Meta-source for new data

https://secure.sonosite.com/covid-19-publications?elqTrackId=7d217d40c559455c9e707385d54587dc&elq=00000000000000000000000000000000&elqaid=3969&elqat=2&elqCampaignId=&elqah=650AEBA86B164C831FE7E71D87AD75F4E70BB442FD444C56ACAA16E4666C576B

A little confusion

It's said that Covid-19 belongs to one kind of viral pneumonia. So I am a little confused about the difference between class Covid-19 and class pneumonia in the classification. It seems that the paper hasn't talk about it.

1 image per class

https://link.springer.com/content/pdf/10.1007/s00247-017-3910-1.pdf

• Fig1: ealthy image
• Fig2: B-lines and interstinal patterns (indicate for viral pneumonia, but this is not COVID since it's old)
• Fig3: Seems like bacterial pneumonia (air bronchograms)

questionable to use since there is artifacts.

~10 COVID-19 images

https://link.springer.com/article/10.1007%2Fs40477-020-00458-7

Data looks rather clean, we have to check which ones to use.

~10 images from single patient

Not sure we can use this data, it's quite heterogeneous

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3544750