turagalab / pygreentea Goto Github PK

If train() is called with a dataset with in-memory arrays, and the dataset includes a 'mask' attribute, then an error occurs because mask_slice is not defined.

Currently train() logs the overall iteration time. This would be the time just for solver.step(1).

File "/groups/saalfeld/home/bogovicj/unet-tut/caffe-janelia/PyGreentea/PyGreentea.py", line 562, in train
    print("[Iter %i] Loss: %f, frac_pos=%f, w_pos=%f" % (i,loss,frac_pos,w_pos))
UnboundLocalError: local variable 'frac_pos' referenced before assignment

frac_pos needs a value too, or maybe better, shouldn't be printed when scale_error is false.

2 ideas...

1. use a different driver when opening the files. e.g.

   h5py.File('myfile.hdf5', driver=<driver name>, <driver_kwds>).

   • Docs: http://docs.h5py.org/en/latest/high/file.html#file-drivers
   • 'sec2' is one of the available drivers. It's "unbuffered"... Not sure exactly what that means, but maybe it means it uses less memory.

2. Set the cache settings to use less memory: E.g.,

   import h5py
   import contextlib
   
   filename = '/tmp/foo.hdf5'
   propfaid = h5py.h5p.create(h5py.h5p.FILE_ACCESS)
   settings = list(propfaid.get_cache())
   settings[2] /= 100  # this attribute is the size of the raw data cache in bytes
   propfaid.set_cache(*settings)
   with contextlib.closing(h5py.h5f.open(filename, fapl=propfaid)) as fid:
      f = h5py.File(fid)
   

   where propfaid is a "property file access identifier", documented here