wanderine / broccoli Goto Github PK

I'm running Broccoli for permutation tests on MRI data. I'm getting an error of the type:

Run kernel error for kernel 'CalculateBetaWeightsGLM' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
Run kernel error for kernel 'CalculateStatisticalMapsGLMTTest' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
Run kernel error for kernel 'CalculateStatisticalMapsGLMTTestSecondLevelPermutation' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
Run kernel error for kernel 'TransformData' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'

It seems this is a memory issue. Are there anyways getting around this? The data I'm permuting are spatial maps of dimension 256x256x256 for several hundred samples. I'm using a mask generated from "Smoothing".

What is more is that, prior to this error, the output of RandomiseGroupLevel states

Permutation threshold for contrast 1 for a significance level of 0.050000 is 0.000000
Permutation threshold for contrast 2 for a significance level of 0.050000 is 0.000000
Permutation threshold for contrast 3 for a significance level of 0.050000 is 0.000000

Could this imply that the data is too highly correlated over voxels for "RandomiseGroupLevel" to work properly or is it most likely a memory issue? The volumes file is 2.5 GB in total with 356 subjects. My device information is:

Device info

---

Platform number: 0

Platform vendor: NVIDIA Corporation
Platform name: NVIDIA CUDA
Platform extentions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer
Platform profile: FULL_PROFILE

---

Device number: 0

Device vendor: NVIDIA Corporation
Device name: Tesla K80
Hardware version: OpenCL 1.2 CUDA
Software version: 375.66
OpenCL C version: OpenCL C 1.2
Device extensions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer
Global memory size in MB: 11439
Size of largest memory object in MB: 2859
Global memory cache size in KB: 208
Local memory size in KB: 48
Constant memory size in KB: 64
Parallel compute units: 13
Clock frequency in MHz: 823
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1024 64

It seems the hardware could theoretically handle loading 2.5 Gigs of data (but not sure if it is enough for Permutation tests).

thank you.

Best,
Chintan

Notice this is the same issue I opened for BIDS-Apps/BROCCOLI (issue #10); I don't think the problem is specific to the Docker version, but please update Docker as well after this issue is resolved:

Trying to run BROCCOLI docker on a BIDS dataset with multiple runs creates the condition *.txt files and then I assume it tries to do something (no logs during that time) and then crashed with:

/Downloads/BROCCOLI/code/bids/broccolipipeline.sh: line 4: 323 Segmentation fault FirstLevelAnalysis -runs ${num_runs} ${bold_files} ${output_dir}/${subject}/${subject}_T1w_brain.nii.gz /usr/local/fsl/data/standard/MNI152_T1_2mm_brain.nii.gz ${regressor_files} ${output_dir}/${subject}/${task_name}/contrasts.txt -output ${output_dir}/${subject}/${task_name}/${subject} -device 0 -savemnimask -saveallaligned -savedesignmatrix -saveoriginaldesignmatrix -regressmotion

I used BIDS dataset ds001 (I downloaded it from https://drive.google.com/drive/folders/0B2JWN60ZLkgkMGlUY3B4MXZIZW8)

This issue happens with both bids/broccoli:v1.0.0 and bids/broccoli:latest (which I believe should be v1.0.1)

Hello,
I would like to ask whether you plan on adding TFCE support in RandomiseGroupLevel? It is specified in the documentation and command-line arguments, but says it is not yet implemented.

Hello.
I'm trying to run first level GLM. Everything works great until I try to add dummy regressors for subjects that did not complete a task. I followed the documentation and created files with NumEvents 0 and added these to the design_file.

When I add these dummy regressors, and only when I add these, all beta's, copes, and tscores files are empty images.

I checked both original and total design matrix files and they seem fine in both cases, with or without empty regressors. Columns for empty regressors are filed with 0, as expected.

The issue seem to affect both GLM and FIRSTLEVEL analysis function.
I use the linux version of the script on Ubuntu 12.04.

--participant_label can be specified in the group analysis level to analyse only a subset of subjects in the input dataset. Currently it is ignored.

Hi Anders,

I would like to use RegisterTwoVolumes to coregister a whole 4D time series of functionals with an anatomical scan. However, the resulting images "...aligned_linear.nii.gz" and "...aligned_nonlinear.nii.gz" are just 3D.

Is there an easy way to do that in BROCCOLI? Or is it possible to output the linear/non-linear transformations from one call to RegisterTwoVolumes and then apply them to another 4D series?

Cheers,
Michael

Currently broccolipipeline.sh is relying on existence of cond files which are not part of the BIDS standard. Instead you should query the trial_type column from the _events.tsv files to obtain similar information.

Hello, I'm new to using BROCCOLI and have just installed it on my Mac. It set up and installed correctly as far as I can tell, but when I try to use the TFCE option for RandomiseGroupLevel, I get a "TFCE is currently turned off!" error. Am I missing something or is TFCE not implemented?

Here is the command I ran: RandomiseGroupLevel /Users/catcamacho/Box/FFnHK-Oddball/dataprocessing/workflows/L2workflow/_cope_cope1/mergecopes/cope1_merged.nii -design /Users/catcamacho/Box/FFnHK-Oddball/dataprocessing/misc/design.mat -contrasts /Users/catcamacho/Box/FFnHK-Oddball/dataprocessing/misc/tcon.con -inferencemode 3 -verbose

Thanks for you help!

Thank you for the great toolbox!

I'm getting different results for the t-maps outputted by BROCCOLI's RandomiseGroupLevel and fsl randomise where I'm only testing for the groupmean (-groupmean flag in Broccoli and -1 flag in randomise).

The commands I ran were:
RandomiseGroupLevel volumes.nii -groupmean
randomise -i volumes.nii -o output -1

The maps look the same but the t-values are off by a factor of ~470 for BROCCOLI. So like 0.06 instead of 28. I've also done a 1 sample t-test in spm and the results are very similar to the randomise ones.

I've attached the information about my system from getOpenCLInfo.

I can put the data on dropbox for you if you wan't to try it out yourself. Any idea what could be the problem?

CLI_info.txt

Currently the task name is hardcoded as rhymejudgement. It would be better to iterate over all tasks for each subject:

https://github.com/wanderine/BROCCOLI/blob/master/code/bids/broccolipipeline.sh#L137

I was wondering if it would be possible to allow saving the uncorrected results after running RandomiseGroupLevel, as in FSL. One could use this as input for a different multiple comparison correction. Many thanks!

Hi Anders,

Thanks for all the hard work put into BROCCOLI! I have an issue that's similar to one that I contacted you about a while ago. I have a time series dataset of the bowel (NIFTI and .mat). I'm registering all volumes to the last volume (vol. 35) using BROCCOLI, but the registration doesn't seem to work. The output appears to be unchanged from the input, and there are no error messages. I did the registration using FSL as well for comparison, and that worked perfectly. The issue last time was the small signal intensities I was using (10^-14 to 10^-7). This dataset ranges from 10^-9 to 10^-6, and I've tried multiplying all signals by 10^7 as well as stretching out the min and max values to adjust the contrast, but nothing seems to help. Do you mind taking a look to see what might be wrong?

Thank you!
Satyam

Hi, I am trying to run RandomiseGroupLevel for the first time using CPU and am getting the following error. Do you know how to resolve this? Thank you!

Authored by K.A. Eklund
Data size: 197 x 233 x 189 x 976
Number of permutations: 5000
Number of regressors: 4
Number of contrasts: 2
Performing 2 t-tests
Two sample t-test design detected for t-contrast 1, 590 subjects in group 1 and 386 subjects in group 2
Two sample t-test design detected for t-contrast 2, 590 subjects in group 1 and 386 subjects in group 2
Max number of permutations for contrast 1 is 7.84211e+282
Max number of permutations for contrast 2 is 7.84211e+282
Unable to write to binary file for kernel kernelConvolution.cpp, null file pointer!
Both operands to ICmp instruction are not of the same type!
%41 = icmp slt i32 %39, i64 0
Both operands to ICmp instruction are not of the same type!
%61 = icmp sgt i64 %53, i32 %40
Invalid operands for select instruction!
%62 = select i1 %61, i64 %53, i32 %40
Broken module found, compilation aborted!
Stack dump:
0. Running pass 'Function Pass Manager' on module 'main'.

1.  Running pass 'Module Verifier' on function '@CalculatePhaseGradientsY'
   

Aborted (core dumped)

I'm testing out first-level analysis using BROCCOLI with some data I'd previously analyzed with FSL. When I run FirstLevelAnalysis using my CPU as the OpenCL device, everything builds perfectly fine and the analysis seems to run as intended. When I try running the same analysis with my GPU, however, I get the following error:

Source build error for kernelBayesian.cpp is CL_BUILD_PROGRAM_FAILURE 
One or several kernels were not created correctly, check buildInfo* !

Looking at the build log for that file for my GPU, it contains just this single line:

ptxas error   : Program using constant pointers passed as entry function parameter cannot use cvta.const

The only reference I could turn up to this error on Google was another OpenCL developer having the same issue on a Mac, but he never posted a solution. Here's my GetOpenCLInfo output:

Device info 
 
---------------------------------------------
Platform number: 0
---------------------------------------------
Platform vendor: Apple
Platform name: Apple
Platform extentions: cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event
Platform profile: FULL_PROFILE
---------------------------------------------

---------------------------------------------
Device number: 0
---------------------------------------------
Device vendor: Intel
Device name: Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz
Hardware version: OpenCL 1.2 
Software version: 1.1
OpenCL C version: OpenCL C 1.2 
Device extensions: cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_image2d_from_buffer cl_APPLE_fp64_basic_ops cl_APPLE_fixed_alpha_channel_orders cl_APPLE_biased_fixed_point_image_formats cl_APPLE_command_queue_priority
Global memory size in MB: 8192
Size of largest memory object in MB: 2048
Global memory cache size in KB: 0
Local memory size in KB: 32
Constant memory size in KB: 64
Parallel compute units: 8
Clock frequency in MHz: 3500
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1 1

---------------------------------------------
Device number: 1
---------------------------------------------
Device vendor: NVIDIA
Device name: GeForce GTX 780M
Hardware version: OpenCL 1.2 
Software version: 10.30.25 355.11.10.10.30.120
OpenCL C version: OpenCL C 1.2 
Device extensions: cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_APPLE_fp64_basic_ops cl_khr_fp64 cl_khr_3d_image_writes cl_khr_depth_images cl_khr_gl_depth_images cl_khr_gl_msaa_sharing cl_khr_image2d_from_buffer cl_APPLE_ycbcr_422 cl_APPLE_rgb_422 
Global memory size in MB: 4096
Size of largest memory object in MB: 1024
Global memory cache size in KB: 0
Local memory size in KB: 48
Constant memory size in KB: 64
Parallel compute units: 8
Clock frequency in MHz: 648
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1024 64

I've tried installing the NVIDIA Web Drivers and CUDA drivers for my GPU to see if that would make a difference, but without any luck. I'm running macOS 10.13.4 (High Sierra). Any idea what's going on?

Thanks in advance!

Hi, guys!

I try to compile python wrapper, but unfortunately there are error appears:

gcc: error: broccoli_lib_wrap.cxx: No such file or directory
gcc: fatal error: no input files
compilation terminated.
g++: error: broccoli_lib_wrap.o: No such file or directory

and there are no any file with name broccoli_lib_wrap* in BROCCOLI folder. What am I doing wrong?

Is anyone allowed to copy and use this code? If so, under the terms of which license?

Thank you for the fantastic work on BROCCOLI. I am still new at this toolbox and migrating from fsl's Feat.

I am able to run FirstLevelAnalysis but I cannot get the permutations to work. Every time I use the -permute switch I get the error that permutation testing is currently turned off. If I do not use the -pemute switch the command runs and dhe resulting copes and betas look fine. Perhaps I am not using the correct syntax or completely misunderstanding the use of -permute in the scope of First level analyses.

Thanks in advance for your help.

FYI, my command is:

FirstLevelAnalysis /misc/mansfield/lconcha/TMP/circe/forBroccoli/frs404.nii.gz \
/misc/mansfield/lconcha/TMP/circe/forBroccoli/brs404.nii.gz \
/home/inb/lconcha/fmrilab_software/fsl_5.0.6/data/standard/MNI152_T1_2mm_brain.nii.gz \
outbase_regressors.txt \
outbase_contrasts.txt \
-smoothing 5 \
-saveall \
-output outbase_brocproc \
-verbose \
-savedesignmatrix \
-saveoriginaldesignmatrix \
-regressmotion \
-permute \
-permutations 1000 \
-inferencemode 1 \
-cdt 2.3

Permutation testing is currently turned off!

Cheers,

Luis

Device info  
---------------------------------------------
Platform number: 0
---------------------------------------------
Platform vendor: NVIDIA Corporation
Platform name: NVIDIA CUDA
Platform extentions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll 
Platform profile: FULL_PROFILE
---------------------------------------------

---------------------------------------------
Device number: 0
---------------------------------------------
Device vendor: NVIDIA Corporation
Device name: Quadro K2000
Hardware version: OpenCL 1.1 CUDA
Software version: 304.135
OpenCL C version: OpenCL C 1.1 
Device extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll  cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 
Global memory size in MB: 2047
Size of largest memory object in MB: 511
Global memory cache size in KB: 32
Local memory size in KB: 48
Constant memory size in KB: 64
Parallel compute units: 2
Clock frequency in MHz: 954
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1024 64

Just discovered this project through the frontiers article and am very impressed with the speed you've managed to achieve with OpenCL. However, given the lack of recent activity in the repo, I'm concerned that the funding or interest for maintaining/refining the project has died out, making it potentially a bad time investment to learn. If there is still a will and intent to keep working on BROCCOLI (which I really hope there is, being able to do first-level analyses in minutes instead of days would be amazing), is there a development roadmap that could be made public to reassure us that the project has a future?

If funding is an issue, I'd imagine a little publicity on some neuro blogs and a proper website with some simple tutorials would help get more people interested in the project, including people who would happily contribute code/donations.

Cheers,

• Austin

I have not find anywhere in the documentation information about estimated inverse transformations for non-linear fields.

I would like to perform template-to-image transforms based on the fields estimated with RegisterTwoVolumes during image-to-template transform. Is there a currently working solution to that, or could it be implemented?

I assume the affine matrix can be inverted using fsl's convert_xfm function

I tried to run GetOpenCLInfo
but it is not able to find the clblas library. I installed AMD APP SDK 3.0, Boost, ACML

I want to know how to install from this ppa
https://launchpad.net/ubuntu/+source/clblas

I tried the .deb file but it said dependency not satisfied in synaptics package manager. I am not very familiar with ubuntu so I would rather not go through the cmake route. Something like apt-get would be great for me.
Thank you

I am trying to use BROCCOLI to speed up permutations testing for TBSS on skeletonized WM data. I was able to run permutations testing successfully on my platform and device (AMD GPU) with your sample data:

BROCCOLI/code/testing_scripts/randomise/betas49.nii.gz

However, when I now try with my own 4D NIFTI file consisting of FA data registered to a group-derived template, I get an error stating that there are 0 possible permutations, even when just attempting -groupmean (which is meaningless for FA data). It does read in the correct dimensions of my file, and does find the correct # of subjects (n = 84) in the 4th dimension.

Command:
RandomiseGroupLevel all_FA_skeletonised.nii.gz -groupmean -platform 0 -device 0 -permutations 10000 -output tbss_parallel -verbose

Output:
Warning: No mask being used, doing permutations for all voxels.
It took 64.409843 seconds to read the nifti file(s)
Warning: Number of possible sign flips for group mean is 0, but 10000 permutations were requested. Lowering number of permutations to number of possible sign flips.
Authored by K.A. Eklund
Data size: 176 x 512 x 512 x 84
Number of permutations: 0
Number of regressors: 1
Number of contrasts: 1

Here also are info output on my dataset:

fslinfo all_FA_skeletonised.nii.gz:
data_type FLOAT32
dim1 176
dim2 512
dim3 512
dim4 84
datatype 16
pixdim1 1.000000
pixdim2 0.500000
pixdim3 0.500000
pixdim4 1.000000
cal_max 1.5417
cal_min 0.0000
file_type NIFTI-1+

3dinfo all_FA_skeletonised.nii.gz:
Dataset File: all_FA_skeletonised.nii.gz
Identifier Code: NII__AL_eWytYUadLGoiKqBqBQ Creation Date: Wed Apr 22 09:02:21 2015
Template Space: ORIG
Dataset Type: Echo Planar (-epan)
Byte Order: LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode: NIFTI
Storage Space: 15,502,147,584 (16 billion [giga]) bytes
Geometry String: "MATRIX(1,0,0,-91.02654,0,-0.5,0,94.37949,0,0,0.5,-146.8557):176,512,512"
Data Axes Tilt: Plumb
Data Axes Orientation:
first (x) = Right-to-Left
second (y) = Posterior-to-Anterior
third (z) = Inferior-to-Superior [-orient RPI]
R-to-L extent: -91.027 [R] -to- 83.973 [L] -step- 1.000 mm [176 voxels]
A-to-P extent: -161.121 [A] -to- 94.379 [P] -step- 0.500 mm [512 voxels]
I-to-S extent: -146.856 [I] -to- 108.644 [S] -step- 0.500 mm [512 voxels]
Number of time steps = 84 Time step = 1.00000s Origin = 0.00000s
-- At sub-brick #0 '?' datum type is float
-- At sub-brick #1 '?' datum type is float
-- At sub-brick #2 '?' datum type is float
** For info on all 84 sub-bricks, use '3dinfo -verb' **

Any ideas why this might be happening? Many thanks!

Hi,
I was trying to analyze high-res fMRI using 'GLM' but received the error message below:

Authored by K.A. Eklund
Data size for run 1: 256 x 256 x 256 x 147
Data size for run 2: 256 x 256 x 256 x 147
Data size for run 3: 256 x 256 x 256 x 147
Data size for run 4: 256 x 256 x 256 x 147
Data size for run 5: 256 x 256 x 256 x 147
Data size for run 6: 256 x 256 x 256 x 147
Data size for run 7: 256 x 256 x 256 x 147
Data size for run 8: 256 x 256 x 256 x 147
Total data size: 256 x 256 x 256 x 1176
Number of original regressors: 7
Number of total regressors: 15
Number of contrasts: 13
Performing 13 t-tests
GLM -runs 8 data_dir/fmri_run1.nii.gz data_dir/fmri_run2.nii.gz data_dir/fmri_run3.nii.gz data_dir/fmri_run4.nii.gz data_dir/fmri_run5.nii.gz data_dir/fmri_run6.nii.gz data_dir/fmri_run7.nii.gz data_dir/fmri_run8.nii.gz -designfiles regress_run1.txt regress_run2.txt regress_run3.txt regress_run4.txt regress_run5.txt regress_run6.txt regress_run7.txt regress_run8.txt -mask data_dir/brainmask_crop.nii -contrasts contrast.txt -firstlevel -detrendingregressors 1 -output brc_: Segmentation fault

The size of each input file is ~850MB. Someone recommended me to put all the input & output files in the directory. I don't know why though. Any help will be highly appreciated.

Hi,

1: In NITRC BROCCOLI is specified as ' Operating System: Windows, POSIX/UNIX-like'.
How I can get it run in Windows?

2: Running RandomiseGroupLevel I got the message 'TFCE is currently disabled'. Will it be enabled again? If so, when?

Best,
Bertram

Hi all,

so I recently installed an NVIDIA driver using (NVIDIA-Linux-x86_64-375.66.run). Now if I run clinfo I get the following message attached to this message. However, when I cd into the appropriate folder and run ./GetOpenCLInfo then I get an error message:
./GetOpenCLInfo: error while loading shared libraries: libclBLAS.so.2: cannot open shared object file: No such file or directory
I'm not sure exactly why this is happening and I'm very very new to linux so I'm having some trouble figuring it out. I was hoping somebody on here might be able to help. Thank you in advance!

Best,

Sri

clinfo_output.txt

on ds000009 R2.0.1 - sub-29

docker run -ti -v D:\:/d bids/broccoli:v1.0.1 /d/data/ds000009_R2.0.1/ /d/scratch/broccoli participant --participant_label 29
Running the BIDS validator for the dataset
        1: Not all subjects/sessions/runs have the same scanning parameters. (code: 39 - INCONSISTENT_PARAMETERS)
                /sub-01/func/sub-01_task-balloonanalogrisktask_bold.nii.gz
                /sub-02/func/sub-02_task-balloonanalogrisktask_bold.nii.gz
                /sub-03/func/sub-03_task-balloonanalogrisktask_bold.nii.gz
                /sub-04/func/sub-04_task-balloonanalogrisktask_bold.nii.gz
                /sub-05/dwi/sub-05_dwi.nii.gz
                /sub-06/func/sub-06_task-balloonanalogrisktask_bold.nii.gz
                /sub-07/func/sub-07_task-balloonanalogrisktask_bold.nii.gz
                /sub-09/func/sub-09_task-balloonanalogrisktask_bold.nii.gz
                /sub-10/func/sub-10_task-balloonanalogrisktask_bold.nii.gz
                /sub-11/func/sub-11_task-balloonanalogrisktask_bold.nii.gz
                ... and 14 more files having this issue (Use --verbose to see them all).

        Summary:                 Available Tasks:                Available Modalities:
        855 Files, 6.07GB        discounting                     T1w
        24 - Subjects            balloon analog risk task        inplaneT2
        1 - Session              emotional regulation            dwi
                                 stop signal                     bold


Doing first level analysis


bids_dir is /d/data/ds000009_R2.0.1/, output_dir is /d/scratch/broccoli, analysis type is participant, participants are 29


Task names are

balloonanalogrisktask
discounting
emotionalregulation
stopsignal




Analyzing subject sub-29




Analyzing subject sub-29 task balloonanalogrisktask


Number of runs is 1

Creating '/d/scratch/broccoli/sub-29/balloonanalogrisktask/cond_run1_BALOON.txt'...

BROCOLLI exist with a non-zero exit code without producing the expected analysis outputs.

Hi, I was wondering if it is possible to control for permutations within groups/subjects. This is possible in FSL randomise by adding -e design.grp file, which basically restrict randomisation within a group (so for repeated measures you need to set a different group identifier (e.g. number) for each subject).
many thanks

Ido

Dear Anders,

I was wondering if it would be feasible to implement elastic net regularization in GLM. It would be cool to estimate weights for very large design matrices with regularization at the speed of light :) Is there hope for this to come into existence?

Best,
Michael

Add support for slice timings file, with arbitrary slice timings

I am performing permutation tests on results and want to use RandomiseGroupLevel. To test it out, I placed placed spatial volumes (beta-maps) for 10 subjects into a single nifti file and then created files for design/contrast matrices for 10 subjects. I ran the following to test the program.

RandomiseGroupLevel a.nii -design design.mat -contrasts design.con

The output was:

Warning: No mask being used, doing permutations for all voxels.
Input data is a single volume, nothing to permute!

I am not using a mask in this test. But, there are definitely 10 volumes in the "a.nii" file. Does this commonly occur?

I created the "a.nii" file by using AFNI's 3dbucket to compile them and then 3dAFNItoNIFTI to convert to nifti format. Could this be a source of error?

Thank you.

To improve user experience the broccolipipeline.sh script should check if bids-validator is available and call it (bids-validator $bids_dir) to check input data compatibility before running any analysis.

I'll send a PR to BIDS-App/BROCCOLI updating the Dockerfile to include the validator.

Hello! I have tried to install Broccoli under Ubuntu 16.04 with an AMDgpu driver, but it fails to compile the following error:

Authored by K.A. Eklund
Data size: 91 x 109 x 91 x 23
Number of permutations: 5000
Number of regressors: 1
Number of contrasts: 1
Performing 1 t-tests
Mean design detected for t-contrast 1
Max number of sign flips is 8.38861e+06
Initialization error is "Unable to create a command queue for the selected device and platform."
OpenCL error is "CL_OUT_OF_HOST_MEMORY"
OpenCL initialization failed, aborting!
See buildInfo* for output of OpenCL compilation!

My OpenCL seems to be installed on platform 0 and device 0:

Device info

Platform number: 0

Platform vendor: Advanced Micro Devices, Inc.
Platform name: AMD Accelerated Parallel Processing
Platform extentions: cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
Platform profile: FULL_PROFILE

Device number: 0

Device vendor: Advanced Micro Devices, Inc.
Device name: Oland
Hardware version: OpenCL 1.2 AMD-APP (2527.3)
Software version: 2527.3
OpenCL C version: OpenCL C 1.2
Device extensions: cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_khr_spir cl_khr_gl_event
Global memory size in MB: 1619
Size of largest memory object in MB: 1169
Global memory cache size in KB: 16
Local memory size in KB: 32
Constant memory size in KB: 64
Parallel compute units: 6
Clock frequency in MHz: 1050
Max number of threads per block: 256
Max number of threads in each dimension: 1024 1024 1024

Then I tried Docker and it failed to download the dropbox files.

I will be happy if you can help me running the Broccoli on either Ubuntu 16 or under the dropbox container.

I have spent many hours trying to make Broccoli run on Microsoft Surface Pro 7+ i7, inside WSL2. I have installed all sorts of packages and GetOpenCLInfo shows that OpenCL is available. Yet, the Kernel code cannot be compiled:

$ /opt/brp/depend/Broccoli/compiled/Bash/Linux/Release/RegisterTwoVolumes ~/SampleData/mprage_reorg.nii ~/SampleData/atlas.nii -output ~/SampleData/OUTPUT.nii -saveinterpolated
Authored by K.A. Eklund
Volume 1 size: 176 x 224 x 256
Volume 1 voxel size: 1.000000 x 1.000000 x 1.000000 mm
Volume 2 size: 182 x 218 x 182
Volume 2 voxel size: 1.000000 x 1.000000 x 1.000000 mm
Source build error for kernelMisc.cpp is CL_BUILD_PROGRAM_FAILURE
Source build error for kernelBayesian.cpp is CL_BUILD_PROGRAM_FAILURE
One or several kernels were not created correctly, check buildInfo* !
ll
Create kernel error for kernel 'SliceTimingCorrection' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateMagnitudes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateColumnSums' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateRowSums' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateColumnMaxs' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateRowMaxs' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateMaxAtomic' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'ThresholdVolume' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'Memset' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MemsetDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MemsetInt' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MemsetFloat2' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'IdentityMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'IdentityMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'GetSubMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'GetSubMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'PermuteMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'PermuteMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'LogitMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'LogitMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolume' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolumes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolumesOverwrite' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolumesOverwriteDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'AddVolume' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'AddVolumes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'AddVolumesOverwrite' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'SubtractVolumes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'SubtractVolumesOverwrite' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'SubtractVolumesOverwriteDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'RemoveMean' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateStatisticalMapsGLMBayesian' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Run kernel error for kernel 'CalculateColumnMaxs' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'CalculateRowMaxs' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'Memset' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'MemsetFloat2' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'MultiplyVolumes' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'AddVolumesOverwrite' is 'CL_INVALID_KERNEL'

Affine registration parameters
 0.977015 -0.016522 -0.000435 0.274039
 0.011848 0.928244 -0.168395 -3.405550
 0.011702 0.173381 0.876269 5.861003
 0.000000 0.000000 0.000000 1.000000

The output of the above is two files, one for affine registration and one non-linear registration. Both are identical though, looks like Broccoli did not manage to do any nonlinear registration.

This is the output of GetOpenCLInfo:

$ GetOpenCLInfo

Device info

---------------------------------------------
Platform number: 0
---------------------------------------------
Platform vendor: Intel(R) Corporation
Platform name: Intel(R) OpenCL Graphics
Platform extentions: cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_command_queue_families cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_mem_force_host_memory cl_khr_subgroup_extended_types cl_khr_subgroup_non_uniform_vote cl_khr_subgroup_ballot cl_khr_subgroup_non_uniform_arithmetic cl_khr_subgroup_shuffle cl_khr_subgroup_shuffle_relative cl_khr_subgroup_clustered_reduce cl_intel_device_attribute_query cl_khr_suggested_local_work_size cl_intel_split_work_group_barrier cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_linkonce_odr cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_ext_float_atomics cl_khr_external_memory cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_3d_image_writes cl_intel_media_block_io cl_intel_subgroup_local_block_io cl_khr_integer_dot_product cl_khr_gl_sharing cl_khr_gl_depth_images cl_khr_gl_event cl_khr_gl_msaa_sharing cl_intel_sharing_format_query cl_khr_pci_bus_info
Platform profile: FULL_PROFILE
---------------------------------------------

---------------------------------------------
Device number: 0
---------------------------------------------
Device vendor: Intel(R) Corporation
Device name: Intel(R) Graphics [0x9a49]
Hardware version: OpenCL 3.0 NEO
Software version: 23.33.0
OpenCL C version: OpenCL C 1.2
Device extensions: cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_command_queue_families cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_mem_force_host_memory cl_khr_subgroup_extended_types cl_khr_subgroup_non_uniform_vote cl_khr_subgroup_ballot cl_khr_subgroup_non_uniform_arithmetic cl_khr_subgroup_shuffle cl_khr_subgroup_shuffle_relative cl_khr_subgroup_clustered_reduce cl_intel_device_attribute_query cl_khr_suggested_local_work_size cl_intel_split_work_group_barrier cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_linkonce_odr cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_ext_float_atomics cl_khr_external_memory cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_3d_image_writes cl_intel_media_block_io cl_intel_subgroup_local_block_io cl_khr_integer_dot_product cl_khr_gl_sharing cl_khr_gl_depth_images cl_khr_gl_event cl_khr_gl_msaa_sharing cl_intel_sharing_format_query cl_khr_pci_bus_info
Global memory size in MB: 6490
Size of largest memory object in MB: 1024
Global memory cache size in KB: 3840
Local memory size in KB: 64
Constant memory size in KB: 1048576
Parallel compute units: 96
Clock frequency in MHz: 1300
Max number of threads per block: 512
Max number of threads in each dimension: 512 512 512

Trying to compile libraries also fails to find opencl.h, although I installed the neo driver from source:

$ ./compile_broccoli_library.sh
broccoli_lib.cpp:48:10: fatal error: opencl.h: No such file or directory
   48 | #include <opencl.h>
      |          ^~~~~~~~~~
compilation terminated.
ar: broccoli_lib.o: No such file or directory
mv: cannot stat 'libBROCCOLI_LIB.a': No such file or directory

There are many examples showing that OpenCL works on WSL2, and I can see it is there, but I can't find what is Broccoli's problem. Any help?

Warning: You are using gcc version "4.6.3-1ubuntu5)". The version
currently supported with MEX is "4.4.6".
For a list of currently supported compilers see:
http://www.mathworks.com/support/compilers/current_release/

Do you use the gcc-4.4 and g++-4.4?
I tried gcc-4.4 and g++-4.4 and got the following:

In file included from ../code/Matlab_Wrapper/FirstLevelAnalysis.cpp:20:
../code/Matlab_Wrapper/help_functions.cpp:1: fatal error: broccoli_lib.h: No such file or directory
compilation terminated.

mex: compile of ' "../code/Matlab_Wrapper/FirstLevelAnalysis.cpp"' failed.

Error using mex (line 206)
Unable to complete successfully.

Error in FirstLevelAnalysis_MixedGamblesTask (line 44)
mex ../code/Matlab_Wrapper/FirstLevelAnalysis.cpp -lOpenCL -lBROCCOLI_LIB
-I/usr/local/cuda-5.0/include/ -I/usr/local/cuda-5.0/include/CL -L/usr/lib
-I/home/andek/Research_projects/BROCCOLI/BROC

And how do you update the addPath and basePath?

(sponsor is sense of uploads to Debian ;) )
just wanted to reach out. unfortunately don't have time bandwidth to dive into it myself ATM

Cheers,

Dear Anders,

The nipype nodes provided with BROCCOLI work fine when executed on their own. Ideally, however, one would want to include them along-side with other nodes (e.g. FSL nodes) in nipype's workflow objects. Unfortunately, running such workflows fails because any downstream nodes do not receive any input from earlier BROCCOLI nodes. For example smoothing a functional image and then performing maths on the results as described here would fail:

`
from nipype.interfaces import broccoli, fsl
from nipype import Node, Workflow
import os

nii_path = '/path/to/some/functional_data.nii'

sm_node = Node(broccoli.Smoothing(fwhm=8, platform=0, device=2), name='sm_node')
fslmaths_node = Node(fsl.ImageMaths(op_string='-sub .2'), name='fslmaths_node')

wf = Workflow(name='broccoli_smoothing_test')
wf.connect([(sm_node, fslmaths_node, [('smoothed_file', 'in_file')])])

wf.inputs.sm_node.in_file = nii_path
wf.base_dir = os.path.expanduser('~/broccoli_smoothing_test')
wf.run()
`

This will produce this error:

"ValueError: ImageMaths requires a value for input 'in_file'. For a list of required inputs, see ImageMaths.help()"

because the smoothed image, which the Smoothing node was supposed to create, does not exist although BROCCOLI does not report any errors.

This error exists both on macOS and Ubuntu.

What can be done about this?

Thanks and best,
Michael

Check and update functions for permutation testing at the single subject level

Hi,
I have the issue that when I use RegisterTwoVolumes and apply the transformation matrix to the moving object in using TransformVolume, the MyImage_allign_linear and the MyImage_warpped don't overlap (when I apply linear transform).

This was the case on linux, up until I downloaded BROCCOLI's latest version (22Feb2017). when it seemed to have been fixed. However the issue remain on my Mac, despite having downloaded BROCCOLI's latest.

Code I use is as follow:
RegisterTwoVolumes MyImage standard -savematrix
TransformVolume MyImage standard -matrix MyImage_affinematrix.txt
fslview MyImage_aligned_linear MyImage_warpped

Hi Anders,

Thanks for developing this cool package. I am trying to compile the Matlab wrapper on Mac and I notice that you guys never got to do it. I suspect there is a reason that I am missing right now? For one thing, I don't seem to find the OPENCL_INCLUDE_DIRECTORY1 and 2.
Any help would be much appreciated.
Thanks!

v