renato145 / fastai_scans

Fastai extension to work with 3d medical images

Fastai scans

Fastai extension to work with 3d medical images.

Work in progress...

Notes:

• Data must be stored as a bcolz array to use the data loaders (http://bcolz.blosc.org/).
• To visualize display 3d volumes the method x.show3d() uses ipyvolume (http://ipyvolume.readthedocs.io/).
• Examples in notebooks folder using Brats dataset (https://www.med.upenn.edu/sbia/brats2018/data.html).

fastai_scans's Issues

Prediction with load_learner()

Hi, thanks for a great repo. I am having trouble using the trained learner for inference. The fastai documentation suggests exporting the learner and then load it with load_learner, but that gives:

"TypeError: init() missing 3 required positional arguments: 'bcolz_array', 'bcolz_array_masks', and 'metadata'".

I think it probably is because the .export/.predict functions was made for 2d images and not bcolz-3d-arrays? I have had no luck with regular learner.save and .load either. Is this a known issue or is there some other way of doing inference?

Using windows 10, anaconda installation.

Examples for input formats

Hi Renato,

thanks for the great repo. I would like to use it as well!

Can you give a bit more information on the input formats for volumes and labels?
E.g.: I have an ndarray for each of my volumes and a single label (I am working on a classification rather than segmenation problem)
Thx!

Are you still maintaining this Project?

I liked this project, I leanred a lot
Are you still maintaining this Project?

new version fastai demands RandomTransform has attr 'use_on_y', but your transform does have this attr.

databunch API seems to be broken.

Hi Renato,

I updated my fastai repository and looking like it broke fastai_scans.
for

data = (fastai_scans.SegmentationItemList.from_paths(train_data_bolz, train_labels_bolz)
                                         .random_split_by_pct(0.2, seed=7)
                                         .label_from_bcolz()
                                         .transform(fastai_scans.get_transforms(), tfm_y=True)
                                         .databunch(bs=bs)
                                         .normalize())

I am getting

TypeError Traceback (most recent call last)
in
3 .label_from_bcolz()
4 .transform(fastai_scans.get_transforms(), tfm_y=True)
----> 5 .databunch(bs=bs)
6 .normalize())

~/fastai-fork/fastai/data_block.py in databunch(self, path, bs, val_bs, num_workers, dl_tfms, device, collate_fn, no_check, **kwargs)
494 path = Path(ifnone(path, self.path))
495 data = self.x._bunch.create(self.train, self.valid, test_ds=self.test, path=path, bs=bs, val_bs=val_bs,
--> 496 num_workers=num_workers, device=device, collate_fn=collate_fn, no_check=no_check, **kwargs)
497 if getattr(self, 'normalize', False):#In case a normalization was serialized
498 norm = self.normalize

~/tutorials/fastai_scans/fastai_scans/data.py in create(cls, train_ds, valid_ds, **kwargs)
25 @classmethod
26 def create(cls, train_ds, valid_ds, **kwargs):
---> 27 return super().create(train_ds, valid_ds, **kwargs)
28
29 def normalize(self, format3d=True, do_x=True, do_y=False):

TypeError: create() got multiple values for keyword argument 'num_workers'

databunch issue: 'from_item_lists'

Hi Renato,

Thanks again for sharing all your work. I was previously able to run your segmentation code with no issues. But recently I've been encountering this problem when I try to create a databunch:

TypeError Traceback (most recent call last)
in ()
----> 1 data = (fastai_scans.fastai_scans.SegmentationItemList.from_paths(x_path, y_path)
2 .no_split()#.random_split_by_pct(0.2, seed=7)
3 .label_from_bcolz()
4 .transform(fastai_scans.fastai_scans.get_transforms(), tfm_y=True)
5 .databunch(bs=bs)

/usr/local/lib/python3.6/dist-packages/fastai/data_block.py in _inner(*args, **kwargs)
434 assert isinstance(fv, Callable)
435 def _inner(*args, **kwargs):
--> 436 self.train = ft(*args, from_item_lists=True, **kwargs)
437 assert isinstance(self.train, LabelList)
438 kwargs['label_cls'] = self.train.y.class

/content/fastai_scans/fastai_scans/data.py in label_from_bcolz(self, **kwargs)
149 def label_from_bcolz(self, **kwargs):
150 y = self._label_cls.from_carray(self.bcolz_array_masks, self.metadata, self.tfm_params,
--> 151 items=self.items, path=self.path, **kwargs)
152 res = self._label_list(x=self, y=y)
153 return res

/content/fastai_scans/fastai_scans/data.py in from_carray(cls, bcolz_array, metadata, tfm_params, items, **kwargs)
52 def from_carray(cls, bcolz_array, metadata, tfm_params=None, items=None, **kwargs):
53 if items is None: items = range(len(bcolz_array))
---> 54 return cls(items, bcolz_array, metadata, tfm_params=tfm_params, **kwargs)
55
56 @classmethod

/content/fastai_scans/fastai_scans/data.py in init(self, items, bcolz_array, metadata, tfm_params, **kwargs)
112 _item_cls = VolumeSegment
113 def init(self, items, bcolz_array:bcolz.carray, metadata:pd.DataFrame, tfm_params=None, **kwargs):
--> 114 super().init(items, bcolz_array, metadata, tfm_params, **kwargs)
115 self.loss_func = SegCrossEntropy()
116

/content/fastai_scans/fastai_scans/data.py in init(self, items, bcolz_array, metadata, tfm_params, **kwargs)
38 self.metadata = metadata
39 self.tfm_params = tfm_params
---> 40 super().init(items, **kwargs)
41
42 def new(self, items, **kwargs):

TypeError: init() got an unexpected keyword argument 'from_item_lists'

My inexperienced self is not too sure how to correct this 'from_item_lists' argument problem. Any ideas?

Thanks for your help,

Rob

arg "num_workers" invalid

Dear renato

class BcolzDataBunch(DataBunch):
    @classmethod
    def create(cls, train_ds, valid_ds, **kwargs):
        if 'num_workers' in kwargs: kwargs.pop('num_workers')
        return super().create(train_ds, valid_ds, num_workers=0, **kwargs)

    def normalize(self, do_x=True, do_y=False):
        if getattr(self,'norm',False): raise Exception('Can not call normalize twice')
        self.norm = partial(normalize_fun, do_x=do_x, do_y=do_y)
        self.denorm = denormalize
        self.add_tfm(self.norm)
        return self

here if 'num_workers' in kwargs: kwargs.pop('num_workers') make arg "num_workers" invalid, Is there a reason?

Can I free it ?

Data augmentation

Hi Renato,

I am trying to do following..

data = (fastai_scans.SegmentationItemList.from_paths(train_data_bolz, train_labels_bolz)
                                         .random_split_by_pct(0.2, seed=7)
                                         .label_from_bcolz()
                                         .transform(fastai_scans.get_transforms(), tfm_y=True)
                                         .databunch(bs=bs)
                                         .normalize())
tfms = fastai_scans.get_transforms(); len(tfms)

volume0 =data.train_ds.x[0];volume0

So fa so good

Volume (42, 288, 288)

but when i tried...

volume0.apply_tfms(tfms=tfms)

it produced

AttributeError Traceback (most recent call last)
in
1
----> 2 volume0.apply_tfms(tfms=tfms)

~/myenv/fastai_scans/fastai_scans/volume.py in apply_tfms(self, tfms, do_resolve, **kwargs)
21
22 def apply_tfms(self, tfms, do_resolve=True, **kwargs):
---> 23 tfms = sorted(listify(tfms), key=lambda o: o.tfm.order)
24 if do_resolve:
25 for f in listify(tfms):

~/myenv/fastai_scans/fastai_scans/volume.py in (o)
21
22 def apply_tfms(self, tfms, do_resolve=True, **kwargs):
---> 23 tfms = sorted(listify(tfms), key=lambda o: o.tfm.order)
24 if do_resolve:
25 for f in listify(tfms):

AttributeError: 'list' object has no attribute 'tfm'

Can you please help to resolve or give more information about this issue?

FileNotFoundError: [Errno 2] No such file or directory: '/content/nifti/meta/sizes'

Hii,
I am new to NIFTI images, FOund your repository very useful and interesting.
and was using your repoistory, but got following error

i used following code

data = (fastai_scans.VolumeItemList.from_paths(path/'nifti', path/'nifti/labels.csv')
                                   .split_by_rand_pct(0.2, seed=7)
                                   .label_from_metadata('')
                                   .transform(fastai_scans.get_transforms())
                                   .databunch(bs=bs)
                                   .normalize())

And i am Getting error -

FileNotFoundError                         Traceback (most recent call last)

<ipython-input-91-2e1bfe6fc602> in <module>()
----> 1 data = fastai_scans.VolumeItemList.from_paths(path/'nifti', path/'nifti/labels.csv')

1 frames

/usr/local/lib/python3.6/dist-packages/bcolz/toplevel.py in open(rootdir, mode)
    138         return bcolz.ctable(rootdir=rootdir, mode=mode)
    139     else:
--> 140         return bcolz.carray(rootdir=rootdir, mode=mode)

bcolz/carray_ext.pyx in bcolz.carray_ext.carray.__cinit__()

bcolz/carray_ext.pyx in bcolz.carray_ext.carray._read_meta()

FileNotFoundError: [Errno 2] No such file or directory: '/content/nifti/meta/sizes'

Please help,
Thanks in advance.