it would be nice to have a few test data sets (a la readfidTest) that can be used to test the orientation in L-R S-I A-P directions for various supine/prone and head/feet first acquisition schemes so that ParaVision and our display always agree.

This is somewhat important and urgent since ROI definitions will depend on particular orientations.

This must have crept up without us noticing. readfid can't read fids

import sarpy
scn=sarpy.Scan('readfidTest.ix1/1')
scn.fid`

...
File "/Users/stefan/sarpy/sarpy/io/BRUKERIO.py", line 453, in readfid
+ (idx % ACQ_rare_factor)]
IndexError: arrays used as indices must be of integer (or boolean) type

To reproduce type in:

import sarpy
dBloch = sarpy.Experiment('dBloch').studies[0].scans[35]
dBloch.method.BSPulse

This will print:

Out[59]: '(8, 195, 0.01, 115.8030276379, 100, 0, 100, LIB_EXCITATION, < fermi.exc>, 1560, 0.69268, 50, 0.1001, conventional)'

As opposed to the nice list that we had managed to get with before where each comma separated value was a list element.

for instance, 195 would be accessed by doing dBloch.method.BSPulse[1]

this also implies that there is only one file for a given key
-> conflict could lead to (forced?) overwriting

Expected: x,y,z,t -

data.shape

(128,64,6,100)

currently it is:

data.shape
(100, 6, 64, 128)

steps to reproduce:

import SARlabpy.io.BRUKER_classes as cls
NecS1dce = cls.Experiment('NecS1').studies[2].scans[6].pdata[0].data

NecS1dce.shape

edit - turns out I forgot to assign it to another variable and that's why I couldn't get at the the other param files.

My bad...

Can still get at data if you be stubborn and just ask for it.

< edited to describe bug conditions >

read2dseq fails for:

a) multi echo sequences are used (msme for e.g.,)
b) 3D multi echo
c) 3D flash
d) multi b-values

this might actually have to be derived from a general processed class.
in that case, data can either be BRUKER processed (2dseq) or come from the fid

if we have a class that represents processed BRUKER data, should we think about storing it?

currently annie has sent me (Firas) Matlab code that apparently works.

Noticed this when reading in some scans that annie acquired (they are NOT acquired under stefan so they will not exist in your ~/data file. I manually copied them over to my data folder. Best to do this on my machine or copy over this data set:

On warburg:
/Volumes/Data/brukerdata-rsync/stefan/nmr/DiLL.iI1/

In python:

import sarpy
phantom = sarpy.Experiment('DiLL')

Works: LLdata = phantom.find_scan_by_protocol('PRESS-waterline_modified')
Doesn't work: LLdata = phantom.find_scan_by_protocol('MOBILE2_1slice_dixon2(modified)')

I have a sneaking suspicion the special characters have something to do with it. looking into it.

Currently read2dseq takes in 'procdirname' which is the directory of the pdata file.

Namely: ~/data/study_name/scan_num/pdata/pdata_num/

This is inconsistent with readfid which takes in:

~/data/study_name/scan_num/

This is of course, sensible because fid file is two directories higher than the 2dseq file, and there can be multiple recons for each scan.

I propose that both routines take in:

~/data/study_name/scan_num/

and read2dseq adds /pdata/1 (by default) and if user inputs another number later, that number is used instead of 1 (i.e. 2)

currently if 6 slices are acquired with an NR of 25, read2dseq gives a matrix of:

[150, 100, 100]

as opposed to the more sensible: [6,100,100,25]

i.e. all 6 slices are put together for each time point

• should have some meta data
• needs uids
• should have some generic loaders and getters

import SARlabpy.io.BRUKER_classes as sar
import SARlabpy.io.SARlogger
SARlabpy.io.SARlogger.initiate_logging(sar)

NecS3Exp = sar.Experiment('NecS3Hs')

for study in NecS3Exp.studies:
print('-'*40+'\n'+study.subject.SUBJECT_id)
for scan in study.scans:
print(" "+scan.acqp.ACQ_protocol_name)

gives an error since
21:20 WARNING: BRUKER_classes - SUBJECT_study_name not found in study: /home/stefan/data/NecS3Hs12.iM2

Steps to troubleshoot:

Check whether the raising happens the same way everywhere.
Are we doing any intercepting/re-raising?
Is there a difference between the spyder console/ipython console/external console?

[x] create AData hookin for easy addition and saving of processed bits of data
[ ] create a method in PData that knows to build the object, save it and add it to the in-memory copy of that list of AData files.
[ ] check how this works prior to having fully loaded all the other ones. There might be an issue with overwriting newer AData sets

readfid does string stripping and parsing before we made changes to readjcamp

for e.g.,

determine array dimensions

ACQ_size = acqp['ACQ_size'].split() # matrix size

def readfid(fptr=None...

fptr !?!?

this needs updating to account for the various changes in readJCAMP and others

we should figure this out tomorrow.

this seems like a good option here:

http://code.google.com/p/guiqwt/

this is to test feasibility and required effort
I'm hoping it's easy

MATLAB analysis code currently exists

There were some spurious lists of lists created before. BRUKERIO.inner_value takes care of that. However, thtat's over zealous and really, readJCAMP should restotre the array structure as shown in the acqp file, e.g.

$ACQ_size=( 2 )

256 128
is a two-element array.

$ACQ_O2_list=( 1 )

0
is a one list element and should not be shortened to ACQ_02_list=0

Warning: Sometimes this is also used to describe strings

$ACQ_movie_descr=( 1, 20 )

<>
when in fact there aren't really 20 characters present ...

That's JCAMP for you. Good luck

Collect some representative BRUKER fids and their header files acqp and test BRUKERIO.readfid

This is revenge for two nights ago isn't it?

import sarpy
Traceback (most recent call last):

File "", line 1, in
import sarpy

File "/Volumes/Data/Dropboxes/PhD./Dropbox/code/python/sarpy/init.py", line 11, in
from io.BRUKER_classes import (natural_sort,

File "/Volumes/Data/Dropboxes/PhD./Dropbox/code/python/sarpy/io/BRUKER_classes.py", line 9, in
import AData_classes

File "/Volumes/Data/Dropboxes/PhD./Dropbox/code/python/sarpy/io/AData_classes.py", line 11, in
import nibabel

ImportError: No module named nibabel

You might be interested not just in a precise string match but a more complicated variety:

• find me all 04_DCE, 05-DCE, 06_DCE_improved
  would need a string like this:
  "0[456][-_]DCE.*"

powerful, no? these are regular expressions they will transform your life.
read: http://docs.python.org/2/howto/regex.html

Before we tackle this: What is the problem?
When would it be nice not to access a Scan only to find out that it's not there?

Currently (SHA 7df3cb1) we should have a working function to read JCAMP files and to typcast them appropriately. This requires more testing and documentation.
Then, maybe pull into master branch.

This seems to be due to the fact that everytime you reload BRUKERIO you will run setup_custom_logger again and this causes repeat calls to logging.addHandler.

Needs to be avoided somehow

Objects in the BRUKER_classes module that are directly exposed to the user should overload their
'str' and 'repr' and

This way a print Scan('NecS3') should give more than some weird looking object reference.

some [good information as usual on stackexchange](http://stackoverflow.com/questions/1436703/difference-between-str-and-repr-in-python

this should definitely have lazy loading implemented
we also need to think about how this should behave in a hierarchy together with a JCAMP object for header, method and such

how to properly import things?

I started importing BRUKERIO as sar, but now you seem to call BRUKER_classes as sar.

Can we make things consistent somehow and simple? What is the pythonic convention (I hope it's not to type out the full module because that's untenable - typing out SARlabpy.io.BRUKER_classes.Experiment is just too many characters)

I hesitate to propose anything (since it seems arbitrary) but I will if pushed.

is it now worth it to change the name of SARlabpy now since we're all in?

how:

import SARlabpy

SARlabpy.SARlogger.initiate_logging(SARlabpy,handler_level=10)

AttributeError Traceback (most recent call last)
/home/stefan/SARlabpy/ in ()
----> 1 SARlabpy.SARlogger.initiate_logging(SARlabpy,handler_level=10)

/home/stefan/SARlabpy/SARlabpy/io/SARlogger.pyc in initiate_logging(module, formatter, handler, handler_level, logger_level)
66 # remove all NullHandlers:

 67     handlers_to_be_removed = []

---> 68 for hdlr in module.logger.handlers:
69 if isinstance(hdlr, logging.NullHandler):
70 handlers_to_be_removed.append(hdlr)

AttributeError: 'module' object has no attribute 'logger'

Should be ordered by scan number (i.e 1,2,3...etc).

Not 1,10, 11... For obvious reasons. This was a huge pet peeve of mine in Matlab!

And also not ordered by a time stamp since I sometimes like undoing scans and repeating them out of chronological order.

I wouldn't even mind if you have to replace the scan number 5 with 05 in the header files (but knowing you, you'll want to leave the original header in place and add an additional user parameter called scan_order and use that to sort properly).

I can fix it if your rather tackle more interesting problems.

For inspiration:

I think there is a misunderstanding of the purpose of init.py

It is meant to aid the availability of various 'stable' parts of the package to the importing user.

E.g. it says
from io.BRUKER_classes import Scan

This means it loads the class Scan() into the namespace of the sarpy module making it accessible as sarpy.Scan().

I suggest you DO NOT put code like

from fmoosvi import test_suite

in there if that code happens to be in a state of flux. (Note that only in commit 4774a09
did I include the imports to the previously rather fluid Scan class).

If you do that stuff like this happens:

>>> import sarpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/stefan/SARlabpy/sarpy/__init__.py", line 14, in <module>
    from fmoosvi import test_suite
  File "/Users/stefan/SARlabpy/sarpy/fmoosvi/test_suite.py", line 57, in <module>
    NecS1dce = sarpy.Experiment('NecS1').studies[2].scans[6]
IndexError: list index out of range

Which annoys users who don't have a particular interest in fmoosvi.test_suite

If you feel that it's too much trouble to issue this line yourself (and there are ways to shorten it slightly) even in the presence of command history we can come up with a use specific loader in the init routine.

import sarpy

NecS3Exp= sarpy.Experiment('NecS3')
LL_scans = sarpy.Experiment.find_scan_in_experiment(NecS3Exp, '04_ubcLL2')

The line using find_scan_in_experiment uses up >2GB of memory...

is it storing all the scans in LL_scans!?

We don't want to do that do we?

Below works fine for structures with 2 or more elements but fails for one

xx=BRUKERIO.readJCAMP(os.path.expanduser('/data/readfidTest.ix1/14/pdata/1/visu_pars'))
xx['VisuFGOrderDesc']
[['5', ' <FG_SLICE>', ' <>', ' 0', ' 2'], ['2', ' <FG_MOVIE>', ' ', ' 2', ' 1']]
xx=BRUKERIO.readJCAMP(os.path.expanduser('/data/readfidTest.ix1/2/pdata/1/visu_pars'))
xx['VisuFGOrderDesc']
'(5, <FG_SLICE>, <>, 0, 2)'

I am expecting in the second case
[['5', ' <FG_SLICE>', ' <>', ' 0', ' 2']]

this is not crucial but is a test case for the fid / 2dseq situation