Hello,
I am currently running the cell detection on a lot of different brain regions using multiple parameters. I keep running into an issue were an error stating: "ZMQError: Too many open files." I notice in my system monitor that this creates a bunch of python processes but they are never getting deleted. I am not really experienced with this type of stuff so I am unsure if there is a work around. I tried looking through the code but I don't see where a file isn't being closed and I am unsure what's being opened and saved to the RAM.

Thank you!

Hello,

I'm trying to run the analysis script to compare two groups -- i've already run the process script for each of the samples that I'm analyzing and have organized outputs from each sample into separate folders. In one group, I have 7 samples and in the other there are 6 samples.

After adding in the folder and file paths into the analysis script, I keep getting an error saying:
"ValueError: all the input array dimensions except for the concatenation axis must match exactly". I get this error after running the line: "g1 = stat.readDataGroup(group1);"

It looks like the script is getting caught up in the vstack module in the shape_base.py script.

I get this error too when i'm comparing groups of equal sample sizes. Has anyone else had a similar issue / have any suggestions on how to fix this issue?

Hi guys!

I'm hitting the following issue in our end and not sure how to deal with this when running with Elastix 4.9

I'm using a Singularity image
with the following defs

Bootstrap:docker
From:ubuntu:16.04

%post
# Create a directory for working with ClearMap
mkdir clearmap
# Set this as working directory
cd clearmap

# Install the neccessary Ubuntu packages
apt-get update
apt-get install -y python2.7
apt-get install -y python-pip
apt-get install -y python-dev
apt-get install -y build-essential
apt-get install -y wget
apt-get install -y bzip2
apt-get install -y unzip
apt-get install -y libgomp1
apt-get install -y python-opencv
apt-get install -y cython
apt-get install -y python-tifffile
apt-get install -y python-h5py
apt-get install -y python-natsort
apt-get install -y python-scipy
apt-get install -y python-networkx
apt-get install -y python-numpy

# Installing neccessary python packages
#pip install --upgrade pip
#pip install Cython
pip install numpy==1.13
#pip install matplotlib
pip install scikit-image==0.12.3

# Copy additional ClearMap resources.
# Included in the docker repo to combat any future
# changes to the iDisco website.
wget https://idiscodotinfo.files.wordpress.com/2016/05/clearmap_ressources_mouse_brain.zip
unzip clearmap_ressources_mouse_brain.zip
rm clearmap_ressources_mouse_brain.zip
rm -r __MACOSX
mv ClearMap_ressources resources

# Programmatically set the version of Elastix to pull.
export ElastixVersion="4.9.0"

# Download a release of Elastix.
# Development deemed stable (July, 2018).
#wget https://github.com/SuperElastix/elastix/releases/download/${ElastixVersion}/elastix-${ElastixVersion}-linux.tar.bz2
#tar xvjf elastix-4.9.0-linux.tar.bz2
#rm elastix-4.9.0-linux.tar.bz2

wget https://github.com/SuperElastix/elastix/archive/elastix_04_8.tar.gz
tar xzvf elastix_04_8.tar.gz
rm elastix_04_8.tar.gz


# Cleaning / reorganizing binaries and libraries.
mv bin/* /bin
mv lib/* /lib
rm -r bin && rm -r lib

# Keeping the original license (good practice).
mkdir elastix_metadata
mv LICENSE elastix_metadata && mv NOTICE elastix_metadata

# Updating the bashrc
#echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib" >> ~/.bashrc

# Install our custom ClearMap
pip install https://github.com/LernerLab/ClearMap/zipball/master

And the issue I'm seeing is this one

Current time: Tue Sep 24 12:01:52 2019.
Reading the elastix parameters from file ...

TIFFReadDirectory: Warning, Unknown field with tag 50838 (0xc696) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50839 (0xc697) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50838 (0xc696) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50839 (0xc697) encountered.
ELASTIX version: 4.900
Command line options from ElastixBase:
-f        /groups/keays/Pascal/Mole-rat/Clearing/Mole_rat_MagStim2018/Caudal/Clearmaptest/autofluo_resampled.tif
-m        /groups/keays/Lukas/Clearedbrains2017/MRIBrainsI/ClearMap_ressources/25umAutofluoReference/molerat_brain_template/c13_back_488_50_25um_testcrop.tif
-fMask    unspecified, so no fixed mask used
-mMask    unspecified, so no moving mask used
-out      /groups/keays/Pascal/Mole-rat/Clearing/Mole_rat_MagStim2018/Caudal/Clearmaptest/elastix_auto_to_atlas/
-p        /groups/keays/Lukas/Clearedbrains2017/MRIBrainsI/ClearMap_ressources/Par0000affine.txt
-p        /groups/keays/Lukas/Clearedbrains2017/MRIBrainsI/ClearMap_ressources/Par0000bspline.txt
-threads  16
WARNING: The parameter "UseDirectionCosines", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.

WARNING: The option "UseDirectionCosines" was not found in your parameter file.
  From elastix 4.8 it defaults to true!
This may change the behavior of your registrations considerably.

Command line options from TransformBase:
-t0       unspecified, so no initial transform used
WARNING: The parameter "BSplineTransformSplineOrder", requested at entry number 0, does not exist at all.
  The default value "3" is used instead.
WARNING: The parameter "UseCyclicTransform", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.

Reading images...
Reading images took 0 ms.

WARNING: the fixed pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: the moving pyramid schedule is not fully specified!
  A default pyramid schedule is used.
Initialization of all components (before registration) took: 11 ms.
Preparation of the image pyramids took: 268 ms.

Resolution: 0
WARNING: The parameter "UseMultiThreadingForMetrics", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
Setting the fixed masks took: 0 ms.
Setting the moving masks took: 0 ms.

itk::ExceptionObject (0x534d000)
Location: "ElastixTemplate - Run()"
File: /home/mstaring/tmp_elastix_release_4.9.0/elastix/Components/ImageSamplers/RandomCoordinate/elxRandomCoordinateSampler.hxx
Line: 113
Description: itk::ERROR: RandomCoordinateSampler(0x535edd0): ERROR: in your parameter file you selected
  SampleRegionSize[ 2 ] = 50 mm,
  while the fixed image size at dim = 2 is 40 voxels or 40 mm.
  Please select a smaller SampleRegionSize!
  It is recommended to be not larger than 1/3 of the image size in mm.

Error occurred during actual registration.


Errors occurred!
Traceback (most recent call last):

Any idea what to tweak/change?

Hi,

I am having a similar problem as #3 however, Python crashes instead of hangs. #3 was resolved by using an older version of open.cv (2.x) but when I try to use pip install opencv-python==2.4.9, I get the following error message
ERROR: Could not find a version that satisfies the requirement opencv-python==2.4.9 (from versions: 3.1.0.0, 3.1.0.1, 3.1.0.2, 3.1.0.3, 3.1.0.4, 3.1.0.5, 3.2.0.6, 3.2.0.7, 3.2.0.8, 3.3.0.9, 3.3.0.10, 3.3.1.11, 3.4.0.12, 3.4.0.14, 3.4.1.15, 3.4.2.16, 3.4.2.17, 3.4.3.18, 3.4.4.19, 3.4.5.20, 4.0.0.21, 4.0.1.24, 4.1.0.25)

Has anyone else run into this issue or have any suggestions?

I ran the code on the haloperidol/saline hemispheres from the website, I consistently get this wall thing at the most anterior part. Has anyone seen something like this and would know what it is/how to get rid of it? I tried to remove anterior slices from the atlas template but it didnt help.

Hi,

I'm hitting an issue (probably with Python multiprocessing) within ClearMap in that when it starts to execute the Process 0: resampleData: resampling in XY step. I tried setting processes to 1 and also passing in "sequential" as the processMethod. Any ideas? The only syscalls being executed are wait4() and futex() within the processes of the pool. Which would be normal for the way multiprocessing works, I'm just surprised that no work happens within the workers of the pool ( 0% CPU usage) and hanging for hours.

[...]
resampleData: processing 0/549
resampleData: processing 25/549
resampleData: processing 50/549
resampleData: processing 75/549
resampleData: processing 100/549
resampleData: processing 125/549
resampleData: processing 150/549
resampleData: processing 175/549
resampleData: processing 200/549
resampleData: processing 225/549
resampleData: processing 250/549
resampleData: processing 275/549
resampleData: processing 300/549
resampleData: processing 325/549
resampleData: processing 350/549
resampleData: processing 375/549
resampleData: processing 400/549
resampleData: processing 425/549
resampleData: processing 450/549
resampleData: processing 475/549
resampleData: processing 500/549
resampleData: processing 525/549
resampleData: resampled data size: (549, 650, 287)
Process 0: resampleData: resampling in XY: image 0 / 1727
Process 0: resampleData: Imagesize: 2160, 2560 Resampled Imagesize: 549, 650

Hi guys!

I'm tripping over a non existing iDISCO python module that needs to be imported in one of the files of the test suite. Can't find anything python related to iDISCO. Any hints? I'm running master:

>>> print ClearMap.__version__
0.9.2

It can be found here in format .nrrd:

http://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/ccf_2017/

So far the resampling and aligning works well, but the heatmaps are empty...

Hello I am trying to run this code
#%% Aignment - resampled to autofluorescence

align the two channels

align_channels_parameter = {
#moving and reference images
"moving_image" : ws.filename('resampled', postfix='autofluorescence'),
"fixed_image" : ws.filename('resampled'),

  #elastix parameter files for alignment
  "affine_parameter_file"  : align_channels_affine_file,
  "bspline_parameter_file" : None,
  
  #directory of the alig'/home/nicolas.renier/Documents/ClearMap_Ressources/Par0000affine.txt',nment result
  "result_directory" :  ws.filename('resampled_to_auto')
  }; 

elx.align(**align_channels_parameter);

and i am getting this error:
runcell('Aignment - resampled to autofluorescence', '/home/hesham/ClearMap2/ClearMap/Scripts/CellMap.py')
Traceback (most recent call last):

File "/home/hesham/ClearMap2/ClearMap/Scripts/CellMap.py", line 143, in
elx.align(**align_channels_parameter);

File "/home/hesham/ClearMap2/ClearMap/Alignment/Elastix.py", line 552, in align
raise RuntimeError('align: failed executing: ' + cmd);

RuntimeError: align: failed executing: /home/hesham/ClearMap2/ClearMap/External/elastix/build/bin/elastix -threads 16 -m /media/hesham/Extra/1.1 final/resampled_autofluorescence.tif -f /media/hesham/Extra/1.1 final/resampled.tif -p /home/hesham/ClearMap2/ClearMap/Resources/Alignment/align_affine.txt -out /media/hesham/Extra/1.1 final/elastix_resampled_to_auto

does anyone have an idea how to fix?

I am receiving this error when I try to run the detectCells function on my full image. I understand that this has to do with the version of numpy and python I am using but am hesitant to update since the program was tested with numpy 1.11 and python 2.7. If anyone has encountered this error and gotten around it, it would be extremely helpful. Thanks!

Hi All,
Has anyone used the Ilastik interface? I'm having some trouble understanding which filename I'm supposed to right in the input and output of Ilastik in imageprocessing.ilastik.ilastik.py code. Does the 'source' and 'sink' needs to be the file paths for the raw images?
Best,
Nimrod

ignore

Following the tutorial

File "/home/vzickus/anaconda3/envs/Python27/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7-linux-x86_64.egg/ClearMap/Visualization/Plot.py", line 250, in plotOverlayPoints cimg = overlayPoints(dataSource, pointSource, pointColor = pointColor, x = x, y = y, z = z); File "/home/vzickus/anaconda3/envs/Python27/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7-linux-x86_64.egg/ClearMap/Visualization/Plot.py", line 220, in overlayPoints cimage[p[0], p[1], p[2], :] = pointColor; IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

Dear Christoph and users,

I have succesfully run the full code on the provided Saline and Haloperidol samples.
However, something is odd on the pvalues stack generated:
-It has double the amount of slices compared to the pvals variable (if written to disk as a .mhd stack pvals has 528 slices while pvalues has 1056)
-The apparent duplication visually looks as if another brain stack had been added sequentially to the pvals stack.
-However, it is not a simple duplicacion of pvals since slices of pvals appear intercalated with other slices of unknown origin.
In the attached image you can appreciate how pvalues.mhd looks like in the FIJI 3D viewer:

Any ideas?

ImportError: cannot import name 'Texture3D' from 'vispy.visuals.volume'

import ClearMap.Alignment.Stitching.StitchingRigid as stg

Getting this error when I try and import StitchingRigid and run the file in Spyder. Anyone else have this problem or fix? On Linux Ubuntu.

Hi,

I am trying to run the setup.py file however, I keep getting the error:

le "setup.py", line 5, in
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'

However, I have this module installed in my base anaconda environment. Any ideas on what may be causing this error.

I'm getting a set of warnings through Elastix:

TIFFReadDirectory: Warning, Unknown field with tag 50838 (0xc696) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50839 (0xc697) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50838 (0xc696) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50839 (0xc697) encountered.

Additionally, while the ClearMap seems to run, it's not outputting any heatmap or points_voxelized tif file. The program stops like this:

Reading the elastix parameters from file ...

Calling all ReadFromFile()'s took 0.012353 s
Transforming points ...
The transform is evaluated on some points, specified in the input point file.
Reading input point file: /tmp/elastix_input.txt
Input points are specified as image indices.
Number of specified input points: 1159
The input points are transformed.
The transformed points are saved in: /tmp/elastix_output/outputpoints.txt
Transforming points done, it took 0.01s
Compute determinant of spatial Jacobian ...
The command-line option "-jac" is not used, so no det(dT/dx) computed.
Computing determinant of spatial Jacobian done, it took 0.00s
Compute spatial Jacobian (full matrix) ...
The command-line option "-jacmat" is not used, so no dT/dx computed.
Computing spatial Jacobian done, it took 0.00s

transformix has finished at Fri Aug 2 12:29:58 2019.
Total time elapsed: 0.1s.

Processed 0/1159

Processed 0/1159

Process finished with exit code 0

I'm not sure if the TIFFReadDirectory warnings are leading to the problem above that no data is being processed in transformix - any insight would be helpful! Thank you

In the Regions IDs.csv file we download in the https://idisco.info/clearmap-2/ there are 6 regions simply labelled as Unlabelled, acronym NoL, under CTX as parent.

after playing with the region-specific heatmaps I observed that they were actually well defined regions from the cortex, as excluding these regions would leave 5 remarking blank spots in the cortex. Below I summarize what I identified in the Allen Brain Atlas for these regions. In the list we have the regions ID, the name, the acronym and their parent region's acronym

182305696 - Primary Somatomotor area, unassigned, layers1/3, SSP-un1/3, SSP
182305712 - Primary Somatomotor area, unassigned, layers4/6, SSP-un4/6, SSP
312782560 - Anterior area, VISa, PTLp
312782592 - Rostrolateral visual area, layers1/3, VISrl1/3, PTLp + Laterointermediate area, layer1/3, VISli1/3, VIS
312782624 - Laterointermediate area, layer4/6, VISli4/6, VIS + Postrhinal area, layer1/3, VISpor1/3, VIS
312782656 - Postrhinal area, layer4/6, VISpor4/6, VIS

The simplest way to fix this would be to change the indexing of 312782592 to get the whole VISrl and 312782624 to get the whole VISli and 312782656 to get the whole VISpor.
Is there an easy/practical way to do this? I am pretty sure that these mixed up indexes are somewhere in the reference brain. So I guess it would be a matter of indexing it correctly in there. Any Ideas??

Can someone reproduce this error?

TypeError: writeData() got an unexpected keyword argument 'startIndex'

For instance, use DoG and set

filterDoGParameter = {
    "size": (3,3,3),

   "sigma": None, 

    "sigma2": None,

    "save": 'DoG_image.tif',

    "verbose": True

}

in their parameter file?

Hi Christoph,

Next hiccup on our end can be seen here:

[...]
Transformix has finished at Wed Jul 13 23:13:36 2016.
Elapsed time: 4 Seconds.

Traceback (most recent call last):
  File "process_template_1.py", line 76, in <module>
    vox = voxelize(points, AtlasFile, **voxelizeParameter);
  File "/groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py", line 45, in voxelize
    data = vox.voxelizeSphere(points.astype('float'), dataSize[0], dataSize[1], dataSize[2], size[0], size[1], size[2]);
IndexError: tuple index out of range

Any idea what to try to debug this?

The voxelization params are

voxelizeParameter = {
     "method" : 'Spherical',
     "size" : (15,15,15),
     "weights" : None,
}

Hi,
I'm trying to use ilastik interfaced within ClearMap, but when trying to run detectCells on the ilastik_for some reason ClearMap says the .ilp trained in ilastik does not exist. Did it happened to anyone else?

This is the error message I get:
detectCells(**ImageProcessingParameter);
ChunkSize: Estimated chunk size 98.94444444444444 in 36 chunks!
[(0, 98), (66, 165), (133, 232), (200, 299), (267, 366), (334, 433), (401, 500), (468, 567), (535, 634), (602, 701), (669, 768), (736, 835), (803, 902), (870, 969), (937, 1036), (1004, 1103), (1071, 1170), (1138, 1236), (1204, 1303), (1271, 1370), (1338, 1437), (1405, 1504), (1472, 1571), (1539, 1638), (1606, 1705), (1673, 1772), (1740, 1839), (1807, 1906), (1874, 1973), (1941, 2040), (2008, 2107), (2075, 2174), (2142, 2241), (2209, 2308), (2276, 2375), (2343, 2442)]
ChunkSize: final chunks : [(0, 98), (66, 165), (133, 232), (200, 299), (267, 366), (334, 433), (401, 500), (468, 567), (535, 634), (602, 701), (669, 768), (736, 835), (803, 902), (870, 969), (937, 1036), (1004, 1103), (1071, 1170), (1138, 1236), (1204, 1303), (1271, 1370), (1338, 1437), (1405, 1504), (1472, 1571), (1539, 1638), (1606, 1705), (1673, 1772), (1740, 1839), (1807, 1906), (1874, 1973), (1941, 2040), (2008, 2107), (2075, 2174), (2142, 2241), (2209, 2308), (2276, 2375), (2343, 2442)]
ChunkSize: final centers: [0, 82.0, 149.0, 216.0, 283.0, 350.0, 417.0, 484.0, 551.0, 618.0, 685.0, 752.0, 819.0, 886.0, 953.0, 1020.0, 1087.0, 1154.0, 1220.0, 1287.0, 1354.0, 1421.0, 1488.0, 1555.0, 1622.0, 1689.0, 1756.0, 1823.0, 1890.0, 1957.0, 2024.0, 2091.0, 2158.0, 2225.0, 2292.0, 2359.0, 2442]
Process 0: processing substack 0/36
Process 0: file = /media/kupchiklab/4TBsata/TRIO/822-00-F4/r_hemi_010320/C02/For_BigStitcher/vertical/verC02Z\d{4}.tif
Process 0: segmentation = <function classifyCells at 0x7f2c088c9d30>
Process 0: ranges: x,y,z = ,,(0, 98)
Process 0: Reading data of size (1306, 2048, 98): elapsed time: 0:00:02
Process 0: Ilastik cell detection: classifier: classifier
Process 0: Ilastik cell detection: classindex: classifier
Process 0: Ilastik cell detection: save : classifier
Process 0:
Ilastik: running: /home/kupchiklab/CM/ilastik-1.3.3post3-Linux/run_ilastik.sh --headless --project=" classifier " --output_format="hdf5" --output_filename_format="/tmp/tmphppuysc7.h5" "/tmp/tmp31g3zwl6/ilastik.npy"
Warning: Ignoring your non-empty LD_LIBRARY_PATH
Warning: Ignoring your non-empty PYTHONPATH/PYTHONHOME
/home/kupchiklab/CM/ilastik-1.3.3post3-Linux/ilastik-meta/lazyflow/lazyflow/classifiers/init.py:20: UserWarning: init: Could not import tiktorch classifier
warnings.warn("init: Could not import tiktorch classifier")
Traceback (most recent call last):
File "/home/kupchiklab/CM/ilastik-1.3.3post3-Linux/ilastik-meta/ilastik/ilastik.py", line 134, in
main()
File "/home/kupchiklab/CM/ilastik-1.3.3post3-Linux/ilastik-meta/ilastik/ilastik.py", line 128, in main
hShell = ilastik_main.main(parsed_args, workflow_cmdline_args)
File "/home/kupchiklab/CM/ilastik-1.3.3post3-Linux/ilastik-meta/ilastik/ilastik_main.py", line 137, in main
load_fn = _prepare_auto_open_project(parsed_args)
File "/home/kupchiklab/CM/ilastik-1.3.3post3-Linux/ilastik-meta/ilastik/ilastik_main.py", line 336, in _prepare_auto_open_project
raise RuntimeError("Project file '" + parsed_args.project + "' does not exist.")
RuntimeError: Project file ' classifier ' does not exist.
Traceback (most recent call last):

File "", line 1, in
detectCells(**ImageProcessingParameter);

File "/home/kupchiklab/CM/ClearMap/ImageProcessing/CellDetection.py", line 85, in detectCells
result = sequentiallyProcessStack(source, sink = sink, function = detectCells, verbose = verbose, **parameter);

File "/home/kupchiklab/CM/ClearMap/ImageProcessing/StackProcessing.py", line 468, in sequentiallyProcessStack
results.append(_processSubStack(argdata[i]));

File "/home/kupchiklab/CM/ClearMap/ImageProcessing/StackProcessing.py", line 89, in _processSubStack
seg = sf(img, subStack = sub, out = pw, **pp);

File "/home/kupchiklab/CM/ClearMap/ImageProcessing/IlastikClassification.py", line 173, in classifyCells
imgmax = ilastik.classifyPixel(classifier, img);

File "/home/kupchiklab/CM/ClearMap/ImageProcessing/Ilastik/Ilastik.py", line 314, in classifyPixel
run(args);

File "/home/kupchiklab/CM/ClearMap/ImageProcessing/Ilastik/Ilastik.py", line 160, in run
raise RuntimeError('runIlastik: failed executing: ' + cmd);

RuntimeError: runIlastik: failed executing: /home/kupchiklab/CM/ilastik-1.3.3post3-Linux/run_ilastik.sh --headless --project=" classifier " --output_format="hdf5" --output_filename_format="/tmp/tmphppuysc7.h5" "/tmp/tmp31g3zwl6/ilastik.npy"

Thanks!

Hi,

The IO.writeData() function returns a string of the file name that is has written data to. I think I'm hitting an issue in the voxelization code where the tutorial and paper supplemental material use code as this:

vox = voxelize(points, AtlasFile, **voxelizationParameter)
io.writeData('/tmp/foo.tif', vox.astype('int32')) 

Since voxelize itself returns IO.writeData() I hit an exception about the .astype() method of str not being implemented.

Traceback (most recent call last):
  File "process_template_1.py", line 91, in <module>
    ids, counts = countPointsInRegions(points, labeledImage = AnnotationFile, intensities = intensities, intensityRow = 0);
  File "/groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Label.py", line 234, in countPointsInRegions
    pointLabels = labelPoints(points, labeledImage, level = level, collapse = collapse);
  File "/groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Label.py", line 218, in labelPoints
    pointLabels[i] = labelImage[int(x[i]), int(y[i]), int(z[i])];
ValueError: setting an array element with a sequence.

Any idea what's wrong here?

Need to structure order as one of the fields in the CSV file

I am having the problem that is similar to #13. However, I am using the .nrrd file as my annotation file. When I come to this bit of code :
vox = voxelize(points, AtlasFile, **voxelizeParameter); if not isinstance(vox, basestring): io.writeData(os.path.join(BaseDirectory, 'cells_heatmap.tif'), vox.astype('int32'));

I get "0/n processed" without any error or counting up to n. This happens even though cell counts look good when I run the test. Has any one else run into this issue?

Hey all,

I am relatively new to python, and I have having some trouble setting the correct path to Elastix. When I run the parameter file, it returns to me:

raise RuntimeError("Cannot find elastix binary %s, set path in Settings.py accordingly!" % elastixbin);

RuntimeError: Cannot find elastix binary C:\Users\USER1\Desktop\Elastix\bin/elastix, set path in Settings.py accordingly!

I am really not sure what I am doing wrong and all help would be greatly appreciated.

Best,
Edward

For several datasets the cells_heatmap appears to be of the wrong size - it does not match the size of any of the resampled images. My guess for now is that voxelization requires the voxel size to divide the raw data without a remainder.

Hello, I'm stuck at an error that says there is no module named VoxelizationCode.

The full output:

Elastix sucessfully initialized from path: /om/user/mathiasg/elastix
Traceback (most recent call last):
  File "parameter_file_template.py", line 15, in <module>
    from ClearMap.ImageProcessing.CellDetection import detectCells
  File "/om/user/mathiasg/envs/idisco_env/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/ImageProcessing/CellDetection.py", line 38, in <module>
    import ClearMap.ImageProcessing.SpotDetection
  File "/om/user/mathiasg/envs/idisco_env/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/ImageProcessing/SpotDetection.py", line 52, in <module>
    from ClearMap.ImageProcessing.IlluminationCorrection import correctIllumination
  File "/om/user/mathiasg/envs/idisco_env/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/ImageProcessing/IlluminationCorrection.py", line 40, in <module>
    from ClearMap.Visualization.Plot import plotTiling
  File "/om/user/mathiasg/envs/idisco_env/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Visualization/Plot.py", line 24, in <module>
    import ClearMap.Analysis.Voxelization as vox
  File "/om/user/mathiasg/envs/idisco_env/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py", line 16, in <module>
    import ClearMap.Analysis.VoxelizationCode as vox
ImportError: No module named VoxelizationCode

I thought it was an error with pyximport - but even after simplifying Voxelization.py:

import numpy
import math

import ClearMap.IO as io
import pyximport; pyximport.install()
#pyximport.install(setup_args={"include_dirs":numpy.get_include()}, reload_support=True)

#import ClearMap.IO as io
import ClearMap.Analysis.VoxelizationCode as vox

the .pyx module was not found. I'm running this on a freshly made environment, including up to date Cython and Python 2.7. Any ideas why this could be?

Hi Chris,
Where can I download the annotation files for your test data(half brain tif)?
I only found the full brain tif from https://idisco.info/clearmap-2/

AtlasFile      = os.path.join(PathReg, 'half_template_25_right_fullWD.tif');
AnnotationFile = os.path.join(PathReg, 'annotation_25_right_fullWD.tif');

Thanks!

Reproduce:

1. Set hmax to an integer in parameter file
2. Run process script

Issue:
findCenterOfMaxima returns non integer center coordinates. This causes an error in detectCellShape in the voxelizePixel function, where these coordinates are used as array indices

Potential fix:
In ClearMap/ImageProcessing/MaximaDetection.py, I added a line in findCenterOfMaxima after creating the centers array (line 231)
centers = numpy.rint(centers).astype(int)

Alternatively, you could keep the non integer center coordinates and convert them to integers only when using as array indices

I think I am having issues caused by this bit of code in the process_template.py script.

#Table generation:
##################
#With integrated weigths from the intensity file (here raw intensity):
ids, counts = countPointsInRegions(points, labeledImage = AnnotationFile, intensities = intensities, intensityRow = 0);
table = numpy.zeros(ids.shape, dtype=[('id','int64'),('counts','f8'),('name', 'a256')])
table["id"] = ids;
table["counts"] = counts;
table["name"] = labelToName(ids);
io.writeTable(os.path.join(BaseDirectory, 'Annotated_counts_intensities.csv'), table);

#Without weigths (pure cell number):
ids, counts = countPointsInRegions(points, labeledImage = AnnotationFile, intensities = None);
table = numpy.zeros(ids.shape, dtype=[('id','int64'),('counts','f8'),('name', 'a256')])
table["id"] = ids;
table["counts"] = counts;
table["name"] = labelToName(ids);
io.writeTable(os.path.join(BaseDirectory, 'Annotated_counts.csv'), table);

The first red flag is that the terminal outputs "Processed 0/6059", but never increases and the script ends without any error. However, the output of "Annotated_counts.csv" and "Annotated_counts_intensities.csv" only contain one row with three columns. I ran the "Check Cell detection" bit of the code, and can clearly see a lot more cells detected. Any ideas what could be the cause of this? I initially ran into issue #6, but after hard-coding the correct path, I am still getting only a single row input in the aforementioned files.

Hi Christoph,
I'm getting this error message even though all of my directories are _ or no spaces (I looked at the other closed issues). Any idea why the os isn't properly accessing the files? I've looked and they're all created. Is there a specific folder structure required for my base directory or alignment files? Thanks

File "/home/users/kmurph/ClearMap/ClearMap/Scripts/process_template.py", line 26, in
resultDirectory = alignData(**RegistrationAlignmentParameter);
File "/home/users/kmurph/ClearMap/ClearMap/Alignment/Elastix.py", line 466, in alignData
raise RuntimeError('alignData: failed executing: ' + cmd);
RuntimeError: alignData: failed executing: /home/users/kmurph/elastix/bin/elastix -threads 16 -m /home/users/kmurph/brain_resources/template_25.tif -f /scratch/users/kmurph/base/autofluo_resampled.tif -p /home/users/kmurph/brain_resources/Par0000affine.txt -p /home/users/kmurph/brain_resources/Par0000bspline.txt -out /scratch/users/kmurph/base/elastix_auto_to_atlas

Python can't find "VoxelizationCode.so" shared library in ClearMap/Analysis.

Installing ClearMap by running python setup.py install returns this warning:
Warning: Extension name 'ClearMap/Analysis/VoxelizationCode' does not match fully qualified name 'ClearMap.Analysis.VoxelizationCode' of 'ClearMap/Analysis/VoxelizationCode.pyx'

Installing older version of Cython (0.24) solved the problem in my case.

L92-93 in filterKernel2D

Was it meant to be this:

x = g[0,:,:] + add[0];
y = g[1,:,:] + add[1];

?

I tried tracking this down, but I cannot seem to find the exact location where the FileList is messed up. This is following the tutorial here: https://rawgit.com/ChristophKirst/ClearMap/master/docs/_build/html/imageanalysis.html

settings.ClearMapPath /home/vzickus/anaconda3/envs/Python27/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7-linux-x86_64.egg/ClearMap Traceback (most recent call last): File "clearmap_test.py", line 19, in <module> print stat.calculateStatistics(filename, method = 'mean') File "/home/vzickus/anaconda3/envs/Python27/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7-linux-x86_64.egg/ClearMap/ImageProcessing/ImageStatistics.py", line 61, in calculateStatistics result = parallelProcessStack(source, sink = sink, function = calculateStatisticsOnStack, join = joinStatistics, method = method, remove= remove, verbose = verbose, **parameter); File "/home/vzickus/anaconda3/envs/Python27/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7-linux-x86_64.egg/ClearMap/ImageProcessing/StackProcessing.py", line 422, in parallelProcessStack results = pool.map(_processSubStack, argdata); File "/home/vzickus/anaconda3/envs/Python27/lib/python2.7/multiprocessing/pool.py", line 253, in map return self.map_async(func, iterable, chunksize).get() File "/home/vzickus/anaconda3/envs/Python27/lib/python2.7/multiprocessing/pool.py", line 572, in get raise self._value AttributeError: 'module' object has no attribute 'FileList'

I get the following error when I try to run alignData. Do you have any suggestions?

Thanks.

resultDirectory = alignData(**RegistrationAlignmentParameter);
Traceback (most recent call last):

File "", line 1, in
resultDirectory = alignData(**RegistrationAlignmentParameter);

File "ClearMap/Alignment/Elastix.py", line 466, in alignData
raise RuntimeError('alignData: failed executing: ' + cmd);

RuntimeError: alignData: failed executing: /home/kandellabalien/ClearMap/ClearMap/Elastix/bin/elastix -threads 16 -m /home/kandellabalien/ClearMap/ClearMap/clearmap_ressources_mouse_brain/ClearMap_ressources/25umAutofluoReference/template_25_half.tif -f /home/kandellabalien/ClearMap/Eleanortesthemifromimage450/autofluo_resampled.tif -p /home/kandellabalien/ClearMap/ClearMap/clearmap_ressources_mouse_brain/ClearMap_ressources/Parameter_files/Par0000affine.txt -p /home/kandellabalien/ClearMap/ClearMap/clearmap_ressources_mouse_brain/ClearMap_ressources/Parameter_files/Par0000bspline.txt -out /home/kandellabalien/ClearMap/Eleanortesthemifromimage450/elastix_auto_to_atlas

Hello,

I am trying to confirm my setup of Clearmap, I am following instructions on http://christophkirst.github.io/ClearMap/build/html/installation.html , running on Windows 10. Setting up the setting.py file I get this issue when I try to run the code:

print 'Settings: elastix path %s does not exists, cf. Settings.py or type help(Settings) for details.' % ElastixPath;
^
SyntaxError: invalid syntax

This is after I gave location to ilastik and elastix in lines 50-55.

Any help on this issue would be appreciated.

Has anyone had any success making a region specific heatmap? I don't understand the region id hierarchy but am trying to use the lbl.labelAtLevel function to isolate the dorsomedial nucleus of the hypothalamus. Right now I'm using this bit of code

for l in labelids:
if not lbl.labelAtLevel(l, 3) == 688:
outside = numpy.logical_or(outside, label == l);
Where 688 was the id in the region count csv output, but I don't think thats correct
Any help would be much appreciated.

Hello I am facing an issue when conducting transformation of point coordinates.
I am working with a new environment on a cluster computer.
The error I am facing is below, it seems like when I run the transformPoints function, it is not able to refer to 'elastix_input.txt' file.
I am not finding where this file is or what it is.
Can anyone help me out here?

>>> points = transformPoints(points, transformDirectory = CorrectionAlignmentParameter["resultDirectory"], indices = False, resultDirectory = None);
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../ClearMap/ClearMap/Alignment/Elastix.py", line 709, in transformPoints
    tmpFile = os.path.join(tempfile.tempdir, 'elastix_input.txt');
  File "/.../miniconda3/envs/ClearMap1/lib/python2.7/posixpath.py", line 70, in join
    elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

Get this error (in the title) before RuntimeError('alignData: failed executing: ' + cmd) is raised (however, I cannot track the reason down). Any ideas? For some reason, the

res = os.system(cmd)

returns res = os.system(cmd)

Traceback:
File "process_template.py", line 26, in
resultDirectory = alignData(**RegistrationAlignmentParameter)

I wonder if this could have anything to do with the "bleed" in atlas. However, I believe I just used the default settings provided in the tutorial, and by eye the atlas and the LSFM data look of similar size.

I'm been trying to get clearmap to work for a while but haven't any success. When I type python setup.py install I get this error message:

Settings: elastix path /usr/local/elastix does not exists, cf. Settings.py or type help(Settings) for details.
Settings: ilastik path /usr/local/ilastik-1.1.9-Linux does not exists, cf. Settings.py or type help(Settings) for details.

When I cd to Clearmap/Clearmap and open Settings.py, I don't know which one to edit? Also, am I suppost to pointed it the software or the directory?

thanks

Hello,
I am trying to run elastix v4.8 with Clearmap. I have added the folder to the path as suggested How to install "elastix: a toolbox for rigid and nonrigid registration of images" in Mac OS X 10.10 (Yosemite). My .bash_profile reads as such:
export PATH=/Users/tony/Desktop/elastix_macosx64_v4/bin:$PATH export DYLD_LIBRARY_PATH=/Users/tony/Desktop/elastix_macosx64_v4/lib:$DYLD_LIBRARY_PATH
However I am getting the following error
Elastix sucessfully initialized from path: /Users/tony/Desktop/elastix_macosx64_v4 dyld: Library not loaded: libANNlib.dylib Referenced from: /Users/tony/Desktop/elastix_macosx64_v4/bin/elastix Reason: image not found
This occurs when I run the script either from terminal or an IDE
Any suggestions?