qiicr / tciabrowser

3D Slicer module for browsing and downloading medical imaging collections from The Cancer Imaging Archive (TCIA).

TCIABrowser

Introduction

TCIABrowser is a 3D Slicer module for browsing and downloading medical imaging collections from The Cancer Imaging Archive (TCIA). See documentation and our PPT usage guide for more information.

Acknowledgments

This work is supported in part by the following National Institutes of Health grant:

• Quantitative Image Informatics for Cancer Research QIICR (U24 CA180918, PIs Kikinis and Fedorov)
• This project has been funded in whole or in part with Federal funds from the National Cancer Institute, National Institutes of Health, under Contract No. 75N91019D00024. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

Contributors:

• Alireza Mehrtash, Brigham and Women's Hospital
• Andrey Fedorov, Brigham and Women's Hospital
• Adam Li, Georgetown University
• Justin Kirby, Frederick National Laboratory for Cancer Research

License

• Slicer License

tciabrowser's Issues

Raise browser

When the browser is open (in the background), returning to the TCIA Browser module should raise the browser window.

Add QIICR logo to the "About" section of the module

Fix selection of dataset to download to avoid blocking testing

Summary

Running the test py_TCIABrowser leads to a blocking popup preventing the extension from finishing until the test timeout.

Steps to reproduce

Locally building the extension and running the test py_TCIABrowser lead to the blocking popup:

ctest -R "py_TCIABrowser$"

Environment

• Slicer version: Slicer/Slicer@3f0e381
• Extension version: 2151dcd
• Operating system: All

Download in the background

When user requests download, do it in a separate thread. Provide a separate tab showing currently active downloads and status.

Only scalar volumes are loaded with scalar volume plugin

Instead, any series retrieved should go through the same "Load" process as done in DICOM Browser

Http errors should be reported to the user via popup

Use QUrl instead of urlib2

After Download and Load, the DICOM browser is not updated

If a dataset is selected and then the 'Download and Load' button clicked, it is downloaded and loaded alright. However if you open the DICOM browser, it will not contain the newly acquired dataset. Need to switch database and back or restart Slicer for it to show up.

Probably something like
dicomWidget.dicomBrowser.databaseDirectory = slicer.app.dicomDatabase.databaseDirectory
would do the trick.

Error message after download

I got the attached error message after every download (i.e., once for every file in a batch download):

Apparently, the download still succeeded, at least I see things in the DICOM database.

(Also, when I follow the instructions in the message and change the path to the local DB, the error message will still contain the old path, while the data obviously ends up in the right place.)

Download is broken in the current version

Reported on Slicer user list:

I am trying to download a collection from the TCIA archive using slicer. I
have successfully installed the TCIA browser module and added it the key for
my TCIA account.

I can browse the database but I can neither load images on the scene nor add
to my dicom database .

If I try to browse the database again, after trying to download something, I
get this error:

Traceback (most recent call last):
File
"C:/Users/rayox010/AppData/Roaming/NA-MIC/Extensions-25516/TCIABrowser/lib/Slicer-4.6/qt-scripted-modules/TCIABrowser.py",
line 682, in studiesTableSelectionChanged
if self.studyInstanceUIDs[n].isSelected():
ValueError: Trying to call 'isSelected' on a destroyed QTableWidgetItem
object

This is the full log:

[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Session start time .......:
2017-01-11 20:06:46
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Slicer version ...........:
4.6.2 (revision 25516) win-amd64 - installed
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Operating system .........:
Windows / 7 / Service Pack 1 (Build 7601) - 64-bit
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Memory ...................:
6066 MB physical, 12131 MB virtual
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - CPU ......................:
GenuineIntel , 2 cores, 16 logical processors
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Developer mode enabled ...:
no
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Prefer executable CLI ....:
no
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Additional module paths ..:
C:/Users/rayox010/AppData/Roaming/NA-MIC/Extensions-25516/DeveloperToolsForExtensions/lib/Slicer-4.6/qt-scripted-modules,
C:\programas\Slicer_4_6_2\XNATSlicer-master\XNATSlicer,
C:/Users/rayox010/AppData/Roaming/NA-MIC/Extensions-25516/VolumeClip/lib/Slicer-4.6/qt-scripted-modules,
C:/Users/rayox010/AppData/Roaming/NA-MIC/Extensions-25516/TCIABrowser/lib/Slicer-4.6/qt-scripted-modules
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Number of registered
modules: 136
[DEBUG][Qt] 11.01.2017 20:06:46 [] (unknown:0) - Number of instantiated
modules: 136
[INFO][Stream] 11.01.2017 20:06:47 [] (unknown:0) - Initializing terminology
mapping for map file
C:/programas/Slicer_4_6_2/share/Slicer-4.6/ColorFiles/Terminology//GenericAnatomyColors-SNOMED.csv
[INFO][Stream] 11.01.2017 20:06:47 [] (unknown:0) - 288 terms were read for
Slicer LUT GenericAnatomyColors
[DEBUG][Qt] 11.01.2017 20:06:47 [] (unknown:0) - Number of loaded modules:
136
[DEBUG][Qt] 11.01.2017 20:06:47 [] (unknown:0) - Switch to module:
"Welcome"
[WARNING][Qt] 11.01.2017 20:07:11 [] (unknown:0) - libpng warning: iCCP:
known incorrect sRGB profile
[WARNING][Qt] 11.01.2017 20:07:11 [] (unknown:0) - libpng warning: iCCP:
known incorrect sRGB profile
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) - Switch to module:
"TCIABrowser"
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::generateQuery: query is: SELECT UID as UID,
PatientsName as Name, PatientsAge as Age, PatientsBirthDate as Date,
PatientID as "Subject ID" FROM Patients ORDER BY "Name" DESC
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::updateQueries for Root: query is: SELECT UID as UID,
PatientsName as Name, PatientsAge as Age, PatientsBirthDate as Date,
PatientID as "Subject ID" FROM Patients ORDER BY "Name" DESC
[WARNING][Qt] 11.01.2017 20:07:14 [] (unknown:0) - QSqlQuery::exec: database
not open
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::generateQuery: query is: SELECT UID as UID,
PatientsName as Name, PatientsAge as Age, PatientsBirthDate as Date,
PatientID as "Subject ID" FROM Patients ORDER BY "Name" DESC
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::updateQueries for Root: query is: SELECT UID as UID,
PatientsName as Name, PatientsAge as Age, PatientsBirthDate as Date,
PatientID as "Subject ID" FROM Patients ORDER BY "Name" DESC
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::generateQuery: query is: SELECT StudyInstanceUID as
UID, StudyDescription as Name, ModalitiesInStudy as Scan, StudyDate as Date,
AccessionNumber as Number, InstitutionName as Institution,
ReferringPhysician as Referrer, PerformingPhysiciansName as Performer FROM
Studies WHERE PatientsUID='30' ORDER BY "Name" DESC
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::updateQueries for Patient: query is: SELECT
StudyInstanceUID as UID, StudyDescription as Name, ModalitiesInStudy as
Scan, StudyDate as Date, AccessionNumber as Number, InstitutionName as
Institution, ReferringPhysician as Referrer, PerformingPhysiciansName as
Performer FROM Studies WHERE PatientsUID='30' ORDER BY "Name" DESC
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::generateQuery: query is: SELECT StudyInstanceUID as
UID, StudyDescription as Name, ModalitiesInStudy as Scan, StudyDate as Date,
AccessionNumber as Number, InstitutionName as Institution,
ReferringPhysician as Referrer, PerformingPhysiciansName as Performer FROM
Studies WHERE PatientsUID='31' ORDER BY "Name" DESC
[DEBUG][Qt] 11.01.2017 20:07:14 [] (unknown:0) -
ctkDICOMModelPrivate::updateQueries for Patient: query is: SELECT
StudyInstanceUID as UID, StudyDescription as Name, ModalitiesInStudy as
Scan, StudyDate as Date, AccessionNumber as Number, InstitutionName as
Institution, ReferringPhysician as Referrer, PerformingPhysiciansName as
Performer FROM Studies WHERE PatientsUID='31' ORDER BY "Name" DESC
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) - Traceback (most
recent call last):
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -   File
"C:/Users/rayox010/AppData/Roaming/NA-MIC/Extensions-25516/TCIABrowser/lib/Slicer-4.6/qt-scripted-modules/TCIABrowser.py",
line 755, in onLoadButton
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -
self.addSelectedToDownloadQueue()
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -   File
"C:/Users/rayox010/AppData/Roaming/NA-MIC/Extensions-25516/TCIABrowser/lib/Slicer-4.6/qt-scripted-modules/TCIABrowser.py",
line 814, in addSelectedToDownloadQueue
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -     loadables =
plugin.examine([fileList])
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -   File
"C:\programas\Slicer_4_6_2\lib\Slicer-4.6\qt-scripted-modules\DICOMLib\DICOMPlugin.py",
line 119, in examine
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -     return
self.examineForImport(fileList)
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -   File
"C:/programas/Slicer_4_6_2/lib/Slicer-4.6/qt-scripted-modules/DICOMScalarVolumePlugin.py",
line 52, in examineForImport
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -
loadablesForFiles = self.examineFiles(files)
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -   File
"C:/programas/Slicer_4_6_2/lib/Slicer-4.6/qt-scripted-modules/DICOMScalarVolumePlugin.py",
line 67, in examineFiles
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) -     seriesUID =
slicer.dicomDatabase.fileValue(files[0],self.tags['seriesUID'])
[CRITICAL][Stream] 11.01.2017 20:07:40 [] (unknown:0) - IndexError: tuple
index out of range

Does any one of you know where the problem is and how to solve it?

Thanks in advance.

Félix.

Cancel Downloads

Add the functionality to cancel the active and other waiting downloads in queue.

Failure on Windows: filename too long

Add tests

Ideally, there should be a self-test associated with this extension. Let me know if you need pointers to specific examples of self-tests - there are several in Slicer trunk, and also in ChangeTrackerPy and Reporting extensions. They can be fully scripted in python.

Show TCIA browser in a popup widget

As suggested by @rkikinis, module panel does not provide optimum experience for the user. To allow for better use of screen real estate, it would be best to show it in a popup window, similar to DICOM browser. @mehrtash: can you comment what you think is the effort needed to implement this?

Create Logo for the Module

Integrate clinical data from TCGA

Need to investigate what data is available and what is the best way to integrate into the GUI.

From Justin Kirby:

TCGA:

• Our “patient id” should match their “case id” or a shortened version of their “sample id”. The sample ids correspond to a tissue sample from a patient and look something like TCGA-01-1234-01 where our image patient ID would just be TCGA-01-1234 in this imaginary example.
• http://www.cbioportal.org/public-portal/web_api.jsp describes the CBIO Portal API which is one of the GDACs I mentioned. You might want to search for the section called “Get Clinical Data”.

Remember user settings

Use qsettings to store the following settings:

• browser window size and position
• storage directory
• user specific APIs

Add test(s)

There should at least be a test that makes sure TCIA server works as expected.

Add double click support for patients and studies table

... to parallel commontk/CTK#627, as suggested by @hmeine

Allow to sort browser table content by column

This would be a nice feature. Alireza: could you add this if it is a trivial fix?

Improve download progress reporting

It appears that TCIA API now includes the capability to query the total series size (getSeriesSize endpoint, see https://wiki.cancerimagingarchive.net/display/Public/TCIA+Programmatic+Interface+%28REST+API%29+Usage+Guide). We should probably have progress bars for downloads report actual progress instead of "in progress".

Improve user experience while importing data

Two options (later add icons?):

1. import into DB only
2. import & load

Import and load:

1. index the directory:
   indexer = ctk.ctkDICOMIndexer(); indexer.addDirectory(slicer.dicomDatabase, inputStudyDir)
2. get the list of loadables and examine using each one (?)
   plugin = slicer.modules.dicomPlugins'DICOMScalarVolumePlugin'
   svs = plugin.examine([filenames])
   volume = plugin.load(svs[0])

Add a button to delete cache

Currently, when a series fails to download, it is impossible to retry, since it is marked as downloaded in the TCIA Browser.

User API selection

Adding the option for the users to be able to specify their own API keys in order to have access to the data that is not exposed to Slicer API key.
UI can be a pop up window

TCIA API limitations/imporovements summary

Describe the current limitations that do not allow to provide user experience similar to the one provided in the TCIA web interface. Send email to help desk, keep track of the responses in this issue.

Download of active and multi-selection at collection/study/series level

@rkikinis requested that multiple selection should be supported, ie, it should be possible to select multiple series and all selected should be loaded.

Ron would also like to be able to download at study level, and even at collection level. User should be warned about how many images are about to be downloaded, and ideally about how much time it will take.

Error if the browser is opened without an initialized DICOM database

• Delete NA-MIC/Slicer.ini
• Start Slicer
• Open TCIA browser
• Error

The solution is to open the DICOM browser before opening the TCIA browser, because the DICOM browser initializes the database. It would be nice if the TCIA browser could do this as well.

Histology Pathology Imaging

Hello, I am wondering if any of the studies had access to histopathology imaging. I can only seem to find CT, MRI, PET imaging.

Progress reporting is confusing: keeping popup progress modal and up until read() is done

Documentation Page in Slicer Wiki

http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Extensions/TCIABrowser

Publish the browser in Slicer Extension

Download and load button should be deactivated unless the last step for downloading series

Dummy progress windows pop up on showing browser

When the TCIA browser is opened, two progress windows appear that do nothing. If they are closed, then nothing happens either. This is not a big issue, but may be confusing for users.

token refresh doesn't work properly

If a user logs in and then tries to access data a few hours later after their login token has expired they get an error that doesn't provide any clue as to what is wrong. They should be told their session expired and then redirected to the login screen.