numerical_gradient

NUMERICAL_GRADIENT calculates part of the numerical gradient

Description:
Function to calculate the numerical gradient for a single weight or delay 
on the synapse between Neuron1 and Neuron2.

Syntax:
DEDParam=NUMERICAL_GRADIENT(ThNN,Neuron1,Neuron2,ti,td);
DEDParam=NUMERICAL_GRADIENT(...,[ParamType],[GradientDelta]);

Input Parameters:
o ThNN: An object of the theta neuron network class
o Neuron1: Index to one Neuron connected to synapse of interest
o Neuron2: Index to the other Neuron connected to synapse of interest
o ti: A qx2 array that contains the neuron indices (globally indexed to
    the network) for each spike time (column 1) and the input spike times
    (column 2). ti may be empty, which in most cases will result in a
    non-firing network.
o td: An rx2 array that contains the neuron indices (globally indexed to
    the network) for each spike time (column 1) and the desired output
    spike times (column 2).
o ParamType: Optional string with possible values 'Weights' or 'Delays' to
    indicate the parameter that the error should be numerically calculated
    with respect to. The default value is 'Weights'.
o GradientDelta: Optional scalar indicating the delta by which to adjust
    the parameter in order to numerical calculate the gradient.  Smaller
    values are generally more accurate, but if the value is too small, 
    then issues arise with floating point accuracy. The default value is
    1e-5.

Output Parameters:
o DEDParam: A scalar indicating the change in error with respect to change
    in the parameter.

Examples:
>> ThNN=theta_neuron_network;
>> ts=run_network(ThNN, [2 3.5])
>> DEDW=numerical_gradient(ThNN,2,3,[2 3.5],[3 20])
>> DEDW=numerical_gradient(ThNN,2,3,[2 3.5],[3 ts{3}])
>> DEDW=numerical_gradient(ThNN,2,3,[2 3.5],[3 ts{3}],1e-10)

See also theta_neuron_network

0001 function DEDParam=numerical_gradient(ThNN,Neuron1,Neuron2,ti,td,varargin)
0002 %NUMERICAL_GRADIENT calculates part of the numerical gradient
0003 %
0004 %Description:
0005 %Function to calculate the numerical gradient for a single weight or delay
0006 %on the synapse between Neuron1 and Neuron2.
0007 %
0008 %Syntax:
0009 %DEDParam=NUMERICAL_GRADIENT(ThNN,Neuron1,Neuron2,ti,td);
0010 %DEDParam=NUMERICAL_GRADIENT(...,[ParamType],[GradientDelta]);
0011 %
0012 %Input Parameters:
0013 %o ThNN: An object of the theta neuron network class
0014 %o Neuron1: Index to one Neuron connected to synapse of interest
0015 %o Neuron2: Index to the other Neuron connected to synapse of interest
0016 %o ti: A qx2 array that contains the neuron indices (globally indexed to
0017 %    the network) for each spike time (column 1) and the input spike times
0018 %    (column 2). ti may be empty, which in most cases will result in a
0019 %    non-firing network.
0020 %o td: An rx2 array that contains the neuron indices (globally indexed to
0021 %    the network) for each spike time (column 1) and the desired output
0022 %    spike times (column 2).
0023 %o ParamType: Optional string with possible values 'Weights' or 'Delays' to
0024 %    indicate the parameter that the error should be numerically calculated
0025 %    with respect to. The default value is 'Weights'.
0026 %o GradientDelta: Optional scalar indicating the delta by which to adjust
0027 %    the parameter in order to numerical calculate the gradient.  Smaller
0028 %    values are generally more accurate, but if the value is too small,
0029 %    then issues arise with floating point accuracy. The default value is
0030 %    1e-5.
0031 %
0032 %Output Parameters:
0033 %o DEDParam: A scalar indicating the change in error with respect to change
0034 %    in the parameter.
0035 %
0036 %Examples:
0037 %>> ThNN=theta_neuron_network;
0038 %>> ts=run_network(ThNN, [2 3.5])
0039 %>> DEDW=numerical_gradient(ThNN,2,3,[2 3.5],[3 20])
0040 %>> DEDW=numerical_gradient(ThNN,2,3,[2 3.5],[3 ts{3}])
0041 %>> DEDW=numerical_gradient(ThNN,2,3,[2 3.5],[3 ts{3}],1e-10)
0042 %
0043 %See also theta_neuron_network
0044 
0045 %Copyright (C) 2008 Sam McKennoch <Samuel.McKennoch@loria.fr>
0046 
0047 DEDParam=-1;
0048 if nargin<5
0049     disp('Error in numerical_gradient: Wrong number of input arguements');
0050     disp(['Needed between 5 and 7 inputs but got ' num2str(nargin)]);
0051     return;
0052 end
0053 if nargin==7
0054     if ischar(varargin{1})
0055         ParamType=varargin{1};
0056         GradientDelta=varargin{2};
0057     else
0058         ParamType=varargin{2};
0059         GradientDelta=varargin{1};
0060     end
0061 elseif nargin==6 && ischar(varargin{1})
0062     ParamType=varargin{1};
0063     GradientDelta=1e-5;
0064 elseif nargin==6
0065     ParamType='Weights';
0066     GradientDelta=varargin{1};
0067 else
0068     ParamType='Weights';
0069     GradientDelta=1e-5;
0070 end
0071 
0072 
0073 WeightsIni=ThNN.Weights;
0074 DelaysIni=ThNN.Delays;
0075 for m=1:3
0076     ThNN.Weights=WeightsIni;
0077     ThNN.Delays=DelaysIni;
0078     if strcmp(ParamType,'Weights')
0079         ThNN.Weights(Neuron1,Neuron2)=ThNN.Weights(Neuron1,Neuron2)+GradientDelta*(m-2);
0080     else
0081         ThNN.Delays(Neuron1,Neuron2)=ThNN.Delays(Neuron1,Neuron2)+GradientDelta*(m-2);
0082     end
0083     ts = run_network(ThNN,ti,1000,0,0);
0084     SSE(m)=get_error(ThNN,ts,td,0);
0085 end
0086 DEDParam=mean(gradient(SSE,GradientDelta));
0087 
0088