|
Spiking Neuron Network Simulator
1.0
Simulation and training of spiking neuron networks, primarily theta neurons
|
|
Spiking Neuron Network Simulator
1.0
Simulation and training of spiking neuron networks, primarily theta neurons
|
The spiking neuron network simulator is designed to be a .NET C# based tool for simulating the and training networks of spiking neurons. The tool consists of a library containing the simulation and training code, a simple console app that demonstrates some of the library functionality and a set of tests that go into more detail in library usage. The tool source code described in this documentation is available below:
This tool library, though built to be expanded as a general spiking neuron network tool, primarily is focused on the theta neuron presently. Theta neurons are dynamic neuron models, that is to say their state is a function of time as well as a function of input spikes. Theta neurons represent an excellent trade-off between model complexity and analytical tractability. Other simpler spiking models such as the Linear Integrate and Fire model do not capture many important dynamic properties while other more complicated neuron models such as the Izhikevich or Hodgkins-Huxley Model have multiple state variables and are generally too complicated to enable the development of simple gradient-based learning equations. Much more information about theta neurons can be found here.
In order to understand the functionality of the theta neuron class it is helpful to have at least an overview understanding of the theta neuron dynamics. The trajectory of the state in a theta neuron is described by:
![\[ \frac{{d\theta }}{{dt}} = \left( {1 - \cos \theta } \right) + \alpha I\left( t \right)\left( {1 + \cos \theta } \right) \]](form_0.png)
where 
is the neuron phase (state); 
is a scaling constant; and 
is the input current that drives the dynamics. The input current is composed of the synaptic currents (generated by input spikes) and a baseline line current that determines the neuron dynamic properties. Thus the neuron state changes as a function of time, input spikes and neuron properties. It is important to note that in traditional neural networks used by engineers there is no concept of state, rather the output is instaneously generated from the input, making it difficult to accomdate time-based learning applications.
Many machine learning applications are ideal for theta neurons because of their simple yet dynamic nature. However, in order to simulate the theta neuron dynamic equation shown above, an inefficient process called numerical integration is typically used. The inefficiency of this process makes it difficult for theta neurons to be used in embedded applications such as robotics where the real-time response of the system is important. Thus, the theta neuron class also contains two ways to simulate theta neuron dynamics. Besides the numerical method, the Theta Neuon class contains event-driven simulation methods, which through some mathematical tricks only needs to perform calculations when an input spike is received rather than at every time iteration, thus enabling robotic applications such as this one here.
Many extensions are possible:
Add Visualization and Web Interface (e.g. http://www.codeproject.com/Articles/5603/QuickGraph-A-100-C-graph-library-with-Graphviz-Sup)
Add Implementations for Training Algorithms RProp and QuickProp
Add Tests for rerun training after a reset and verfiy results are the same
Add Tests for Two consequtive trainings with different learning rates. This test will involve making TrainingParameters tracked per epoch interval and specifying the max number of epochs to train for (Cumulative) and the stopping criteria in the input
Add a Cosine type test
Add parallel computations where possible
Add constructors for multiple types of neurons in SpikingNeuronNetwork
Add neuron implementations for:
* QIF - http://hal.inria.fr/docs/00/10/35/00/PDF/eventdrivenQIF.pdf
* LIF - http://icwww.epfl.ch/~gerstner/SPNM/node26.html
* Izhikevich - http://www.abrg.group.shef.ac.uk/!DATA/attachment/0085.Humphries_simple_model_solutions_preprint.pdf, http://www.abrg.group.shef.ac.uk/code/othercode/simple_model_solutions_code.zip and http://faculty.washington.edu/shlizee/publications/rqif-det.pdf
* Fitz-Hugh Nagumo - http://eqworld.ipmnet.ru/en/solutions/npde/npde1103.pdf
Add Exponential synaptic current, see QIF ref, Bessel Functions
Add 2D models that can be solved analytically, perhaps is possible with dirac synaptic currents
1.8.7 
