arbor-sim / arbor Goto Github PK
View Code? Open in Web Editor NEWThe Arbor multi-compartment neural network simulation library.
Home Page: https://arbor-sim.org
License: BSD 3-Clause "New" or "Revised" License
The Arbor multi-compartment neural network simulation library.
Home Page: https://arbor-sim.org
License: BSD 3-Clause "New" or "Revised" License
point estimate: 10 points
feature: #67
The GPU branch is compiling and generating numerically stable results, however the miniapp is not generating spikes. This task will aim to reproduce correct results on the GPU.
The following steps are proposed:
This will require running and debugging tests and miniapp, creating and updating existing validation and unit tests for the GPU along the way.
Like it says on the tin!
Step 1: extend modcc
to parse them.
modcc segfaults on a roughly fixed mod file produces by the C.Elegans software.
The mod files produce correct results when run in NEURON
The errors in the original file are fixed by rewriting or deleting lines. These issues will be reported in seperate issues.
./modcc -t cpu -o file.txt generic_neuron_iaf_cell_segfault.mod
Segmentation fault
generic_neuron_iaf_cell.txt
generic_neuron_iaf_cell_segfault.txt
Better to use an analytic (series) solution for a tapered cable-only model in validation testing, than asking NEURON to generate data for this case.
When compiling on Julia (PCP machine) I get a warning with g++ 6.1.0
Warning: '...' may be used uninitialized in this function
It is in the file nestmc-proto/tests/validation/trace_analysis.hpp line 63
I had a look and cannot solve this easily.
In file included from /gpfs/homeb/pcp0/pcp0016/code/nestmc/trunk/nestmc-proto/tests/validation/validate_synapses.cpp:19:0:
/gpfs/homeb/pcp0/pcp0016/code/nestmc/trunk/nestmc-proto/tests/validation/trace_analysis.hpp: In function ‘void nest::mc::assert_convergence(nest::mc::conv_data<Param>&) [with Param = int]’:
/gpfs/homeb/pcp0/pcp0016/code/nestmc/trunk/nestmc-proto/tests/validation/trace_analysis.hpp:63:68: warning: ‘*((void*)& smallest_pd +24)’ may be used uninitialized in this function [-Wmaybe-uninitialized]
auto tbound = [](trace_peak p) { return std::abs(p.t)+p.t_err; };
Point estimate: 4
Feature: #65
Until we have a more sophisticated system in place with better coverage, we should use e.g. Travis to check our PRs.
Implement this for at least one compiler/configuration.
When attempting to declare a state variable with (nA) as the unit an error occurs:
./modcc -t cpu -o file.txt generic_neuron_iaf_cell.txt
error: generic_neuron_iaf_cell.txt:(line 64,col 8)
'(' is not a valid name for a state variable
63 STATE {
64 vI (nA) :
65 }
Problem also occurs with (mA).
Removing the unit results 'fixes' the issue
An interface for constructing the nest::mc::memory::array<>
containers from a range is needed, because we currently have to construct the array, then use std::copy
to copy the range in.
// currently
memory::array<int> a(range.size);
std::copy(range.begin(), range.end(), a.begin());
// preferable range aware interface
memory::array<int> a(range);
// and an STL-style interface
memory::array<int> a(range.begin(), range.end());
Nonstandard LaTeX fonts should be avoided in the docs. Looking at you passive_cable
docs...
https://github.com/eth-cscs/nestmc-proto/blob/master/docs/passive_cable/cable_computation.tex#L11
Point estimate: 2
Feature: #65
The tests/
subdirectories have expanded in number and some of the CMake script that has been inherited is more complex that it needs to be or needs some review. We should review the scripts in tests/
and across the source more broadly, simplifying where feasible.
net_send(delay, flag)
Allows a NetCon object to send an event to self with delay and flag (Neuron book 294 - 295.)
This function is used primarily in the NET_RECEIVE block (neuron book 288), which implements how a NetCon object reacts to events
delay : unit time
flag : unit ?int?
Flag is called by value (unlike the the explicit arguments declared inside the parenthesis, which are "call by reference")
From the NEURON documentation it not clear: but it appears that flag is an int and multiple flag values are valid
Typicall use case (iaf neuron refract period) pseudo code:
PARAMETER {
refract_period = 5 (ms)
refractionary = 0
}
NET_RECEIVE (w) {
if (refractionary == 0) {
<update neuron state>
if (state == spike) {
refractionary = 1
net_send(refract_period, refractionary) : send self event with delay
}
}
else if (flag == 1) { : self event received
refractionary = 0
<reset neuron state>
} : else ignore external events
}
Found at least in the following blocks:
INITIAL
NET_RECEIVE
Example usage:
http://neuralensemble.org/trac/PyNN/browser/trunk/src/neuron/nmodl/netstim2.mod?rev=888
When attempting to load a file containing the a COMMENT block modcc throws an error:
./modcc -t cpu -o file.txt generic_neuron_iaf_cell.mod
error: generic_neuron_iaf_cell.mod:(line 3,col 1)
Where our results differ from e.g. NEURON, we need to be able to verify that our solutions are valid approximations. Some of the groundwork for this is in place with our numeric
reference solutions. Further, we should be able to produce experimental validation of any claims we make about orders of convergence and the like.
For some amenable test cases:
gold standard
reference solutions which focus on numerical accuracy.nestmc
!Test cases such as the soma-only H–H model are a good start; simpler ODEs are also a possibility; anything with an analytic solution is great.
A large number of neuro scientific models have a source of random spikes: We do not model the whole brain and the activity of the non-modeled neurons is typically injected as a uncorrelated poisson spike train on each modeled neuron. The typical rate of spike generation for this use-case can be in the order of 20000Hz or higher.
In NEST this feature is implemented as a node that generates independent spike trains to each synapse connected. Optimizing by generating the spikes on the target node it self.
http://www.nest-simulator.org/py_sample/sinusoidal-poisson-generator-example/
Introduction:
D. Heeger. "Poisson Model of Spike Generation" 2000 Doi: 10.1.1.37.6580
Possible features of the spike generator, (cheap compared to the implementation of the basic version)
Observations based on 'current' version of code:
add_artificial_spike()
class model {}
Demo: replace BREAKPOINT
block in pas.mod
with the following:
BREAKPOINT {
i = identity(g)*(v - e)
}
FUNCTION identity(x) {
identity = x
}
Running modcc -t cpu
on this gives an internal compiler error:
I don't know how to variable inlining for this statement : x @ (line 28,col 16)
Goal is to support having all the state evolution numerical work able to be performed on the GPU for the models currently implemented in the miniapp.
Specifically:
Tasks
#61 - clean up GPU branch [10]
Point estimate: 6
Feature: #60
modcc
that rewrites a parsed kinetic block as a derivative block.CONSERVE
semantics is not in scope for this task.Expect a system such as
~ a + b <-> c (x, y)
~ 2c <-> a (u, v)
to be transformed to a representation equivalent to that produced from the derivative expressions
a' = -a*b*x + c*y + c*c*u - a*v
b' = -a*b*x + c*y
c' = a*b*x - c*y - 2*c*c*u + 2*a*v
Point estimate: 10
Feature: #60
Investigate representations as below; document conclusions on wiki or demonstrate proof of concept of choice in code.
The internal representation of an ODE system as specified by a DERIVATIVE block or KINETIC block needs to be such that the following operations are practical:
The reduction operation is necessary to support the semantics of the mechanism as described in the feature. The linearity and diagonal tests are necessary to determine the choice of solver in the mechanism instantiation.
Consider the use of third party packages such as symengine.
These keep popping up in different examples, so they have to be added in some form.
Two questions
net_send(d, w)
will send an event to "self" at t+d
.net_event(t)
sends an event over any netcon that is attached to the point process that calls it.net_send
should be straightforward: just push an event into the local event queue, taking care to avoid race conditions on the queue (possible complications on GPU)net_event
is a pain, because we made the naive assumption that synapses are targets, not sources of spikes/source events. We might work around this in most cases, because it might be a category mistake to allow a point process to generate "spikes" in our network model.ELECTRODE_CURRENT
is currently not recognized by modcc.
The documentation regarding this feature is sparse. It appears to be a direct injection of current in the cell. Not passing the cell membrane.
The NEURON book does have an entry, but does not describe anything about it.
The mention I found online:
intracellular stimulating electrode
An intracellular stimulating electrode is similar to a shunt in the sense that both are localized
sources of current that are modeled as point processes. However, the current from a stimulating
electrode is not generated by an opening in the cell membrane but instead is injected directly into
the cell. This particular model of a stimulating electrode has the additional difference that the
current changes discontinuously, i.e. it is a pulse with distinct start and stop times.
: Current clamp
NEURON {
POINT_PROCESS IClamp1
RANGE del, dur, amp, i
ELECTRODE_CURRENT i
}
UNITS { (nA) = (nanoamp) }
Figure 4
Hines and Carnevale: Expanding NEURON with NMODL
Revised 4/6/2000 Page 12
PARAMETER {
del (ms)
dur (ms) < 0, 1e9 >
amp (nA)
}
ASSIGNED { i (nA) }
INITIAL { i = 0 }
BREAKPOINT {
at_time(del)
at_time(del+dur)
if (t < del + dur && t > del) {
i = amp
} else {
i = 0
}
}
Listing 3. iclamp1.mod
The NEURON block
This mechanism is identical to the built-in IClamp model. Calling it IClamp1 allows the
reader to test and modify it without conflict with the existing IClamp point process.
This model of a current clamp generates a rectangular current pulse whose amplitude amp in
nanoamperes, start time del in milliseconds, and duration dur in milliseconds are all adjustable
by the user. Furthermore, these parameters are individually adjustable for each separate instance
of this mechanism. Therefore they are declared as RANGE variables in the NEURON block.
The current i delivered by IClamp1 is declared in the NEURON block to make it available for
examination. The ELECTRODE_CURRENT statement has two important consequences: positive
values of i will depolarize the cell (in contrast to the hyperpolarizing effect of positive
transmembrane currents), and when the extracellular mechanism is present there will be a
change in the extracellular potential vext. Further discussion of extracellular fields is beyond the
scope of this paper.
There are currently some discrepancies between our results, the results of NEURON and the results of our numeric validation models.
The nature of these discrepancies needs to be determined, and if necessary, the discrepancies addressed, so that we can have confidence in our implementations.
Concretely:
nestmc
.nestmc
, as we have only a first-order in time solver implemented, and time discretization errors dominate.Tasks may include:
nestmc
.Let's try meson! Dependencies are Python3 and Ninja, and in principle we can install it locally in our environments with Pip.
In the first instance, let's just trial it for a serial (no MPI) build.
Until (if ever) it becomes a robust alternative to the existing CMake scripts, CMake will be the preferred build system and the one which needs to be in a working state.
Add and triage features related to code quality and organization below, with the prototype deadline in mind.
CMakeLists.txt
files and organisationSteps to reproduce:
validate.exe
after configuring CMake to use g++ as a C++ compiler, e.g.mkdir b; cd b
CXX=g++ CC=gcc cmake -DWITH_ASSERTIONS=ON -DWITH_TRACE=OFF -DWITH_TBB=ON -DWITH_PROFILING=ON -DWITH_MPI=ON -DCMAKE_BUILD_TYPE=debug
make -j validate.exe
./tests/validate.exe --gtest_filter=ball_and_taper.neuron_ref
Observed output:
[ RUN ] ball_and_taper.neuron_ref
unknown file: Failure
C++ exception with description "cannot use operator[] with array" thrown in the test body.
[ FAILED ] ball_and_taper.neuron_ref (209 ms)
Point estimate: 6
Feature: #60
KINETIC
keyword; CONSERVE
keyword; ~
reaction introduction symbol; <->
reaction arrows symbol; distinguish real and integral numbers.Expression
sub-type for reactions and add any further sub-types for sub-components of a reaction.Example kinetic block for validation:
PROCEDURE rates {
: compute f1(v), r1(v) etc.
...
}
KINETIC kin {
rates(v)
~ s1 <-> s2 (f1, r1)
~ s2 <-> s3 (f2, r2)
~ s2 <-> s4 (f3, r3)
CONSERVE s1 + s3 + s4 - s2 = 1
}
Partially done already, but address:
assert_convergence
code.If modcc is given an invalid file name, e.g.
modcc -t cpu -o kernels/hh.hpp hh.mod
where the path kernels
doesn't exist, it prints a message to the effect that it successfully compiled the file, when in fact the file kernels/hh.hpp
was never created.
Modcc: exits with a segmentation fault if a function is called which is not defined.
It should fail gracefully with an error.
Minimal code to reproduce the error:
TITLE Mod file for component: Component(id=exc_syn type=expTwoSynapse)
NEURON {
}
UNITS {
}
PARAMETER {
}
ASSIGNED {
}
STATE {
}
INITIAL {
rates()
}
Somewhere along the line a merge went awry, and we're no longer validating the simple ball plus linear taper cell model. This should be re-introduced.
The current CUDA implementation to solve multiples Hines systems does not include the use of shared memory and atomic operations
We want to analyze and implement this optimization (shared memory).
Also, due to the particular characteristics of the Hines system, it is necessary to use atomic operation on those points where the branches take place to avoid race conditions.
end of line comment prepended with a ? leads to an error:
./modcc -t cpu -o file.txt generic_neuron_iaf_cell.txt
error: unexpected character '?' at (line 70,col 13)
generic_neuron_iaf_cell.txt:(line 70,col 13)
expected a new line after call expression, found '?'
line 70: rates() ? To ensure correct initialisation.
The line WATCH (v > thresh) 1000
leads to an error:
-bash-4.2$ ./modcc -t cpu -o file.txt generic_neuron_iaf_cell.txt
error: generic_neuron_iaf_cell.txt:(line 87,col 29)
expected a new line after call expression, found '1000'
I know we do the connecting of these manually, reporting the issue
Setting parameters on a mechanism added to a cell segment does not result in those values being propagated to mechanism instances (setting parameters on the pseudo-mechanism "membrane"
however works.)
Addressing this will require changes to modcc
and the core nestmc
.
At the point this is implemented, it would be a good opportunity to move to a thinner parameter_list
representation that splits the data from the schemata.
How easy to incorporate or robust is symengine?
Let's trial it for the computation of the derivative in the cnexp
processing in modcc
.
The GPU backend is has been validated against the multicore backend for all of the validation tests and the miniapp. However, these tests take much longer to run on the GPU than the CPU.
The main reason for this is many small copies, typically of a single floating point value, between host and device memory whenever
These issues should be addressed before we start to benchmark and optimize the core GPU algorithms
This task has two steps
This task will not involve implementation of the design changes, which will be done in a later task after this analysis is complete.
NEST-MC implements currently a Thomas algorithm to solve tridiagonal systems.
Although this method is the optimal method in terms of number of operations, there are other methods which can solve the system in parallel, such as Parallel Cyclic Reduction (PCR), Cyclic Reduction, among others.
In particular, there is a subrutine called gtsvStridedBatch() in cuSPARSE (http://docs.nvidia.com/cuda/cusparse/#axzz4PL9ndAze) which can solve multiple tridiagonal systems in parallel using GPUs.
Moreover, we have the experience of implementing some of these methods in CUDA, see
*Pedro Valero-Lara, Alfredo Pinelli, Julien Favier, Manuel Prieto-Matías:
Block Tridiagonal Solvers on Heterogeneous Architectures. ISPA 2012: 609-616
*Pedro Valero-Lara, Alfredo Pinelli, Manuel Prieto-Matías:
Fast finite difference Poisson solvers on heterogeneous architectures. Computer Physics Communications 185(4): 1265-1272 (2014)
There is a problem regarding to the specific features of NEST-MC, as each of the tridiagonal systems can have different sizes, this makes difficult to use the cuSPARSE routine.
So, we are going to implement a cuda kernel which is able to deal with multiples systems with different sizes in parallel.
Validate numerically the application by using a simple test case for systems with the same size and using cuSparse routine
Implement new CUDA kernels for multiple tridiagonal systems with different sizes
Performance evaluation of the different approaches
The initial feature subset comprises:
Implementation requires the validated evolution of a mechanism described by a KINETIC block of the above form.
This is an extension of the previous issue "Analysis and implementation of Hines solvers on GPUs I".
Numerical validation of the new GPU solvers
Performance evaluation of each of the GPU implementations
Point estimate: 4
Feature: #65
The functionality and semantics of our range-like utilities are under-documented and thus hard to use for anyone who isn't the author of the range code (ahem). We also have some inconsistencies in the names of some of these utilities and classes which should be resolved.
Consider the pas.mod
file with the following replacing the BREAKPOINT
block:
BREAKPOINT {
if (1 == 0) {
dummy1()
}
else if (2 == 0) {
dummy2()
}
else {
dummy3()
}
i = g*(v - e)
}
PROCEDURE dummy1 { }
PROCEDURE dummy2 { }
PROCEDURE dummy3 { }
The corresponding generated (cpu) code in nrn_current()
references dummy1
, but not dummy2
or dummy3
:
void nrn_current() override {
[…]
for(int i_=0; i_<n_; ++i_) {
value_type v = vec_v[i_];
value_type current_, i;
if( 1== 0) {
dummy1(i_);
};
i = g[i_]*(v-e[i_]);
current_ = i;
vec_i[i_] += current_;
}
}
Similarly for gpu target.
NEST-MC implements currently a Hines algorithm similar to Thomas algorithm to solve tridiagonal systems.
Although this method is the optimal method in terms of number of operations, it is sequential. There are other methods which can solve the system in parallel, such as Parallel Cyclic Reduction (PCR), Cyclic Reduction, among others.
In particular, there is a subrutine called gtsvStridedBatch() in cuSPARSE (http://docs.nvidia.com/cuda/cusparse/#axzz4PL9ndAze) which can solve multiple tridiagonal systems in parallel using GPUs.
Moreover, we have the experience of implementing some of these methods in CUDA, see:
*Pedro Valero-Lara, Alfredo Pinelli, Julien Favier, Manuel Prieto-Matías:
Block Tridiagonal Solvers on Heterogeneous Architectures. ISPA 2012: 609-616
*Pedro Valero-Lara, Alfredo Pinelli, Manuel Prieto-Matías:
Fast finite difference Poisson solvers on heterogeneous architectures. Computer Physics Communications 185(4): 1265-1272 (2014)
There is a problem regarding to the specific features of NEST-MC, as each of the tridiagonal systems can have different sizes and the use of one additional vector "p". This makes difficult to use the cuSPARSE routine.
So, we are going to implement a cuda kernel which is able to deal with multiples systems with different sizes in parallel.
The FVM formulation does not correctly map mechanism types to control volumes at branching points (including the soma).
At such CVs, there should be contributions from the density channels that are defined on each branch, however we currently only consider contributions from density channels defined on the "root" segment.
The solution will require that each density channel instance has an aditional state variable corresponding to the surface area of the CV that the channel covers.
We should hold off implementing this until @halfflat has finished merging his validation branch. We have this ticket, and the fvm_multi.mechanism_indexes
unit test that fails.
Note that the solution still converges to the correct solution as we refine the mesh, however for course discretisations the errors are large.
point estimate: 10 points
feature: #67
The GPU branch github.com/bcumming/nestmc-proto/tree/gpu needs to be merged with master. There are issues that make this difficult
This task aims to target steps 1 and 2 in a gpu branch of the main repository. Once the state of the branch is satisfactory, the validation in step 3 will be addressed in a separate project task.
Acceptance criteria
Steps
When attempting to parse a line with a scientific notation for a small number an error is thrown:
./modcc -t cpu -o file.txt generic_neuron_iaf_cell.mod
error: generic_neuron_iaf_cell.mod:(line 43,col 34)
PARAMETER block unexpected symbol '-'
line 42: leakConductance = 1.00000005E-4 (uS)
generic_neuron_iaf_cell.txt
Demo: replace BREAKPOINT
block in pas.mod
with the following:
BREAKPOINT {
if (1 == 1) {
i = identity(g)*(v - e)
}
}
FUNCTION identity(x) {
identity = x + 0
}
Generated (cpu) code from modcc
fails to compile:
.../mechanisms/multicore/pas.hpp:95:21: error: use of undeclared identifier 'identity'
i = identity(i_, g[i_])*(v-e[i_]);
This task will set up a test bed for experimentation with the solution of Hines matrices.
On completion, we will have a test bed for further experimentation and an understanding of Hines matrices.
Steps involved
l
, u
, p
and rhs
vectors)v
from the previous time step, which will be used as the starting point for iterative methods.PR#71 changes introduced warnings at compile time that do not reflect erroneous code or behaviour, but which can be addressed easily:
nestmc-proto/modcc/lexer.cpp:266:42: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
is_plusminus(current_[1]) && is_numeric(current_[2]))
~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
and
nestmc-proto/tests/gtest.h:19992:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (expected == actual) {
~~~~~~~~~^~~~~~~~~
Using the && to chain two boolean statement leads to an error:
error: unexpected character '&' at (line 55,col 22)
./offset_current.mod:(line 55,col 22)
unexpected token &
Codeblock:
if (t >= delay && t < duration + delay) {
i = amplitude : standard OnCondition
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.