<?xml version="1.0" encoding="UTF-8"?>
<!--
- * Scilab ( http://www.scilab.org/ ) - This file is part of Scilab
- * Copyright (C) 2008 - INRIA
- *
- * This file must be used under the terms of the CeCILL.
- * This source file is licensed as described in the file COPYING, which
- * you should have received as part of this distribution. The terms
- * are also available at
- * http://www.cecill.info/licences/Licence_CeCILL_V2.1-en.txt
- *
- -->
+* Scilab ( http://www.scilab.org/ ) - This file is part of Scilab
+* Copyright (C) 2008 - INRIA
+*
+* This file must be used under the terms of the CeCILL.
+* This source file is licensed as described in the file COPYING, which
+* you should have received as part of this distribution. The terms
+* are also available at
+* http://www.cecill.info/licences/Licence_CeCILL_V2.1-en.txt
+*
+-->
<refentry xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svg="http://www.w3.org/2000/svg" xmlns:ns5="http://www.w3.org/1999/xhtml" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:db="http://docbook.org/ns/docbook" xmlns:scilab="http://www.scilab.org" xml:id="numdiff" xml:lang="en">
- <refnamediv>
- <refname>numdiff</refname>
- <refpurpose>numerical gradient estimation at one point</refpurpose>
- </refnamediv>
- <refsynopsisdiv>
- <title>Calling Sequence</title>
- <synopsis>g = numdiff(fun, x [,dx])</synopsis>
- </refsynopsisdiv>
- <refsection>
- <title>Arguments</title>
- <variablelist>
- <varlistentry>
- <term>fun</term>
- <listitem>
- <para>
- an external, Scilab function or list. See below for calling
- sequence, see also <link linkend="external">external</link> for
- details about external functions.
- f: R<superscript>n</superscript> --> R<superscript>p</superscript>
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>x</term>
- <listitem>
- <para>
- a vector of the <code>n</code> coordinates of the single point at which the gradient is sought.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>dx</term>
- <listitem>
- <para>
- a vector, the finite difference step. Default value is
- <code>dx = sqrt(%eps)*(1+1d-3*abs(x))</code>.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>g</term>
- <listitem>
- <para>
- a matrix, the estimated gradient at the locus <varname>x</varname>.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </refsection>
- <refsection>
- <title>Description</title>
- <para>
- Given a function <code>fun(x)</code> from
- R<superscript>n</superscript> to R<superscript>p</superscript> computes the <code>p x n</code> matrix
- <varname>g</varname> such that
- </para>
- <programlisting role="no-scilab-exec">
- <
- ]]>
- </programlisting>
- <para>
- using finite difference methods.
- Uses an order 1 formula.
- </para>
- <para>
- Without parameters, the function <varname>fun</varname> calling sequence is
- <code>y = fun(x)</code>, with <varname>x</varname> ∈ R<superscript>n</superscript> and <varname>y</varname> ∈ R<superscript>p</superscript>, and <function>numdiff</function> can be called as
- <code>g = numdiff(fun, x)</code>. Else the function <varname>fun</varname> calling
- sequence must be <literal>y = fun(x, param_1, pararm_2, ..., param_q)</literal>.
- If parameters <literal>param_1, param_2, ..., param_q</literal> exist then
- <function>numdiff</function> can be called as follow
- <literal>g = numdiff(list(fun, param_1, param_2, ..., param_q), x)</literal>.
- </para>
- <para>
- See the
- <link linkend="derivative">derivative</link> with respect to numerical accuracy
- issues and comparison between the two algorithms.
- </para>
- </refsection>
- <refsection>
- <title>Examples</title>
- <programlisting role="example">
- <![CDATA[
-// Example 1 (without parameters)
-// myfun is a function from R^2 to R: (x(1), x(2)) |--> myfun(x)
-function f = myfun(x)
- f = x(1)*x(1) + x(1)*x(2)
-endfunction
+ <refnamediv>
+ <refname>numdiff</refname>
+ <refpurpose>numerical gradient estimation at one point. <emphasis role="bold"> This function is obsolete. Please use the <link linkend="numderivative">numderivative</link> function instead.</emphasis></refpurpose>
+ </refnamediv>
+ <refsynopsisdiv>
+ <title>Calling Sequence</title>
+ <synopsis>g = numdiff(fun, x [,dx])</synopsis>
+ </refsynopsisdiv>
+ <refsection>
+ <title>Arguments</title>
+ <variablelist>
+ <varlistentry>
+ <term>fun</term>
+ <listitem>
+ <para>
+ an external, Scilab function or list. See below for calling
+ sequence, see also <link linkend="external">external</link> for
+ details about external functions.
+ f: R<superscript>n</superscript> --> R<superscript>p</superscript>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>x</term>
+ <listitem>
+ <para>
+ a vector of the <code>n</code> coordinates of the single point at which the gradient is sought.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>dx</term>
+ <listitem>
+ <para>
+ a vector, the finite difference step. Default value is
+ <code>dx = sqrt(%eps)*(1+1d-3*abs(x))</code>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>g</term>
+ <listitem>
+ <para>
+ a matrix, the estimated gradient at the locus <varname>x</varname>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsection>
+ <refsection>
+ <title>Description</title>
+ <para>
+ Given a function <code>fun(x)</code> from
+ R<superscript>n</superscript> to R<superscript>p</superscript> computes the <code>p x n</code> matrix
+ <varname>g</varname> such that
+ </para>
+ <programlisting role="no-scilab-exec">
+ <
+ ]]>
+ </programlisting>
+ <para>
+ using finite difference methods.
+ Uses an order 1 formula.
+ </para>
+ <para>
+ Without parameters, the function <varname>fun</varname> calling sequence is
+ <code>y = fun(x)</code>, with <varname>x</varname> ∈ R<superscript>n</superscript> and <varname>y</varname> ∈ R<superscript>p</superscript>, and <function>numdiff</function> can be called as
+ <code>g = numdiff(fun, x)</code>. Else the function <varname>fun</varname> calling
+ sequence must be <literal>y = fun(x, param_1, pararm_2, ..., param_q)</literal>.
+ If parameters <literal>param_1, param_2, ..., param_q</literal> exist then
+ <function>numdiff</function> can be called as follow
+ <literal>g = numdiff(list(fun, param_1, param_2, ..., param_q), x)</literal>.
+ </para>
+ <para>
+ See the
+ <link linkend="derivative">derivative</link> with respect to numerical accuracy
+ issues and comparison between the two algorithms.
+ </para>
+ </refsection>
+ <refsection>
+ <title>Examples</title>
+ <programlisting role="example">
+ <![CDATA[
+ // Example 1 (without parameters)
+ // myfun is a function from R^2 to R: (x(1), x(2)) |--> myfun(x)
+ function f = myfun(x)
+ f = x(1)*x(1) + x(1)*x(2)
+ endfunction
-x = [5 8];
-g = numdiff(myfun, x)
+ x = [5 8];
+ g = numdiff(myfun, x)
-// The exact gradient (i.e first component = derivate with respect to x(1)
-// and second component = derivate with respect to x(2)) is:
-exact = [2*x(1)+x(2) x(1)]
+ // The exact gradient (i.e first component = derivate with respect to x(1)
+ // and second component = derivate with respect to x(2)) is:
+ exact = [2*x(1)+x(2) x(1)]
-// Example 2 (with parameters)
-// myfun is a function from R to R: x |--> myfun(x)
-// myfun contains 3 parameters: a, b and c
-function f = myfun(x, a, b, c)
- f = (x+a)^c + b
-endfunction
+ // Example 2 (with parameters)
+ // myfun is a function from R to R: x |--> myfun(x)
+ // myfun contains 3 parameters: a, b and c
+ function f = myfun(x, a, b, c)
+ f = (x+a)^c + b
+ endfunction
-a = 3; b = 4; c = 2;
-x = 1;
-g2 = numdiff(list(myfun, a, b, c), x)
+ a = 3; b = 4; c = 2;
+ x = 1;
+ g2 = numdiff(list(myfun, a, b, c), x)
-// The exact gradient, i.e derivate with respiect to x, is:
-exact2 = c*(x+a)^(c-1)
+ // The exact gradient, i.e derivate with respiect to x, is:
+ exact2 = c*(x+a)^(c-1)
-// Example 3 (f: R^3 --> R^3)
-// myfun is a function from R^2 to R^2: (x(1), x(2), x(3)) |--> (myfun(x)(1), myfun(x)(2), mfun(x)(3))
-function f = myfun(x)
- f(1) = x(1) * x(1);
- f(2) = x(1) * x(2) * x(3);
- f(3) = 2*x(1) + 2*x(2) + 2*x(3);
-endfunction
+ // Example 3 (f: R^3 --> R^3)
+ // myfun is a function from R^2 to R^2: (x(1), x(2), x(3)) |--> (myfun(x)(1), myfun(x)(2), mfun(x)(3))
+ function f = myfun(x)
+ f(1) = x(1) * x(1);
+ f(2) = x(1) * x(2) * x(3);
+ f(3) = 2*x(1) + 2*x(2) + 2*x(3);
+ endfunction
-x = [5 8 10];
-g = numdiff(myfun, x)
+ x = [5 8 10];
+ g = numdiff(myfun, x)
-// The exact gradient is:
-// [ df_1/dx_1 df_1/dx_2 df_1/dx_3 ;
-// df_2/dx_1 df_2/dx_2 df_2/dx_3 ;
-// df_3/dx_1 df_3/dx_2 df_3/dx_3 ; ]
-exact3 = [2*x(1) 0 0 ; x(2)*x(3) x(1)*x(3) x(1)*x(2) ; 2 2 2]
+ // The exact gradient is:
+ // [ df_1/dx_1 df_1/dx_2 df_1/dx_3 ;
+ // df_2/dx_1 df_2/dx_2 df_2/dx_3 ;
+ // df_3/dx_1 df_3/dx_2 df_3/dx_3 ; ]
+ exact3 = [2*x(1) 0 0 ; x(2)*x(3) x(1)*x(3) x(1)*x(2) ; 2 2 2]
+
+ ]]>
+ </programlisting>
+ </refsection>
+ <refsection role="see also">
+ <title>See Also</title>
+ <simplelist type="inline">
+ <member>
+ <link linkend="interp">interp</link>
+ </member>
+ <member>
+ <link linkend="interp2d">interp2d</link>
+ </member>
+ <member>
+ <link linkend="splin">splin</link>
+ </member>
+ <member>
+ <link linkend="eval_cshep2d">eval_cshep2d</link>
+ </member>
+ <member>
+ <link linkend="optim">optim</link>
+ </member>
+ <member>
+ <link linkend="diff">diff</link>
+ </member>
+ <member>
+ <link linkend="derivative">derivative</link>
+ </member>
+ <member>
+ <link linkend="numderivative">numderivative</link>
+ </member>
+ <member>
+ <link linkend="external">external</link>
+ </member>
+ </simplelist>
+ </refsection>
+ <refsection>
+ <title>History</title>
+ <revhistory>
+ <revision>
+ <revnumber>5.5.0</revnumber>
+ <revremark>Tagged as obsolete. Will be removed in Scilab 6.0.0.</revremark>
+ </revision>
+ </revhistory>
+ </refsection>
+ <refsection>
+ <title>Appendix</title>
+ <para>
+ We now discuss how a script using the <literal>numdiff</literal> function can be
+ updated to use the <literal>numderivative</literal> function.
+ </para>
+ <para>
+ Consider the function:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ function f = myfun(x)
+ f = x(1)*x(1)+x(1)*x(2)
+ endfunction
+ ]]>
+ </programlisting>
+ <para>
+ and the point:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ x = [5 8]
+ ]]>
+ </programlisting>
+ <para>
+ Therefore, the statement:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ g1 = numdiff(myfun, x)
+ ]]>
+ </programlisting>
+ <para>
+ can be replaced with
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ g2 = numderivative(myfun, x)
+ ]]>
+ </programlisting>
+ <para>
+ If having exactly the same step is important, we force the step to the same value as in
+ <literal>numdiff</literal>:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ x = [5 8];
+ h = sqrt(%eps)*(1+1d-3*abs(x))
+ g1 = numdiff(myfun, x)
+ g2 = numderivative(myfun, x, h)
+ g1 == g2
+ ]]>
+ </programlisting>
+ </refsection>
- ]]>
- </programlisting>
- </refsection>
- <refsection>
- <title>History</title>
- <para>
- Starting from Scilab 5.5.0, the <literal>numdiff</literal> function is
- obsolete. It will be removed in Scilab 6.0.0.
- Please use the <literal>numderivative</literal> function instead.
- </para>
- <para>
- We now discuss how a script using the <literal>numdiff</literal> function can be
- updated to use the <literal>numderivative</literal> function.
- </para>
- <para>
- Consider the function:
- </para>
- <programlisting role="example">
- <![CDATA[
-function f = myfun(x)
- f = x(1)*x(1)+x(1)*x(2)
-endfunction
-]]>
- </programlisting>
- <para>
- and the point:
- </para>
- <programlisting role="example">
- <![CDATA[
-x = [5 8]
-]]>
- </programlisting>
- <para>
- Therefore, the statement:
- </para>
- <programlisting role="example">
- <![CDATA[
-g1 = numdiff(myfun, x)
-]]>
- </programlisting>
- <para>
- can be replaced with
- </para>
- <programlisting role="example">
- <![CDATA[
-g2 = numderivative(myfun, x)
-]]>
- </programlisting>
- <para>
- If having exactly the same step is important, we force the step to the same value as in
- <literal>numdiff</literal>:
- </para>
- <programlisting role="example">
- <![CDATA[
-x = [5 8];
-h = sqrt(%eps)*(1+1d-3*abs(x))
-g1 = numdiff(myfun, x)
-g2 = numderivative(myfun, x, h)
-g1 == g2
-]]>
- </programlisting>
- </refsection>
- <refsection role="see also">
- <title>See Also</title>
- <simplelist type="inline">
- <member>
- <link linkend="interp">interp</link>
- </member>
- <member>
- <link linkend="interp2d">interp2d</link>
- </member>
- <member>
- <link linkend="splin">splin</link>
- </member>
- <member>
- <link linkend="eval_cshep2d">eval_cshep2d</link>
- </member>
- <member>
- <link linkend="optim">optim</link>
- </member>
- <member>
- <link linkend="diff">diff</link>
- </member>
- <member>
- <link linkend="derivative">derivative</link>
- </member>
- <member>
- <link linkend="numderivative">numderivative</link>
- </member>
- <member>
- <link linkend="external">external</link>
- </member>
- </simplelist>
- </refsection>
</refentry>
<?xml version="1.0" encoding="UTF-8"?>
<refentry xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svg="http://www.w3.org/2000/svg" xmlns:ns4="http://www.w3.org/1999/xhtml" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:db="http://docbook.org/ns/docbook" xmlns:scilab="http://www.scilab.org" xml:id="derivative" xml:lang="en">
- <refnamediv>
- <refname>derivative</refname>
- <refpurpose>approximate derivatives of a function</refpurpose>
- </refnamediv>
- <refsynopsisdiv>
- <title>Calling Sequence</title>
- <synopsis>
- derivative(F,x)
- [J [,H]] = derivative(F,x [,h ,order ,H_form ,Q])
- </synopsis>
- </refsynopsisdiv>
- <refsection>
- <title>Arguments</title>
- <variablelist>
- <varlistentry>
- <term>F</term>
- <listitem>
- <para>
- a Scilab function F: <literal>R^n --> R^m</literal> or a
- <literal>list(F,p1,...,pk)</literal>, where F is a scilab function
- in the form <literal>y=F(x,p1,...,pk)</literal>, p1, ..., pk being
- any scilab objects (matrices, lists,...).
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>x</term>
- <listitem>
- <para>real column vector of dimension n.</para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>h</term>
- <listitem>
- <para>
- (optional) real, the stepsize used in the finite difference
- approximations.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>order</term>
- <listitem>
- <para>
- (optional) integer, the order of the finite difference formula
- used to approximate the derivatives (order = 1,2 or 4, default is
- order=2 ).
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>H_form</term>
- <listitem>
- <para>
- (optional) string, the form in which the Hessean will be
- returned. Possible forms are:
- </para>
- <variablelist>
- <varlistentry>
- <term>H_form='default'</term>
- <listitem>
- <para>
- H is a m x (<literal>n^2</literal>) matrix ; in this
- form, the k-th row of H corresponds to the Hessean of the k-th
- component of F, given as the following row vector :
- </para>
- <informalequation>
- <mediaobject>
- <imageobject>
- <imagedata align="center" fileref="../mml/derivative_equation_1.mml"/>
- </imageobject>
- </mediaobject>
- </informalequation>
- <para>((grad(F_k) being a row vector).</para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>H_form='blockmat' :</term>
- <listitem>
- <para>
- H is a (mxn) x n block matrix : the classic Hessean
- matrices (of each component of F) are stacked by row (H = [H1
- ; H2 ; ... ; Hm] in scilab syntax).
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>H_form='hypermat' :</term>
- <listitem>
- <para>
- H is a n x n matrix for m=1, and a n x n x m hypermatrix
- otherwise. H(:,:,k) is the classic Hessean matrix of the k-th
- component of F.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>Q</term>
- <listitem>
- <para>
- (optional) real matrix, orthogonal (default is eye(n,n)). Q is added to have the possibility to remove
- the arbitrariness of using the canonical basis to approximate the derivatives of a function and it should be an
- orthogonal matrix. It is not mandatory but better to recover the derivative as you need the inverse matrix (and
- so simply Q' instead of inv(Q)).
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>J</term>
- <listitem>
- <para>approximated Jacobian</para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>H</term>
- <listitem>
- <para>approximated Hessian</para>
- </listitem>
- </varlistentry>
- </variablelist>
- </refsection>
- <refsection>
- <title>Description</title>
- <para>
- Numerical approximation of the first and second derivatives of a
- function F: <literal> R^n --> R^m</literal> at the point x. The
- Jacobian is computed by approximating the directional derivatives of the
- components of F in the direction of the columns of Q. (For m=1, v=Q(:,k) :
- grad(F(x))*v = Dv(F(x)).) The second derivatives are computed by
- composition of first order derivatives. If H is given in its default form
- the Taylor series of F(x) up to terms of second order is given by :
- </para>
- <informalequation>
- <mediaobject>
- <imageobject>
- <imagedata align="center" fileref="../mml/derivative_equation_2.mml"/>
- </imageobject>
- </mediaobject>
- </informalequation>
- <para>(([J,H]=derivative(F,x,H_form='default'), J=J(x), H=H(x).)</para>
- </refsection>
- <refsection>
- <title>Performances</title>
- <para>
- If the problem is correctly scaled, increasing the accuracy reduces
- the total error but requires more function evaluations.
- The following list presents the number of function evaluations required to
- compute the Jacobian depending on the order of the formula and the dimension of <literal>x</literal>,
- denoted by <literal>n</literal>:
- </para>
- <itemizedlist>
+ <refnamediv>
+ <refname>derivative</refname>
+ <refpurpose>approximate derivatives of a function. <emphasis role="bold"> This function is obsolete. Please use the <link linkend="numderivative">numderivative</link> function instead.</emphasis></refpurpose>
+ </refnamediv>
+ <refsynopsisdiv>
+ <title>Calling Sequence</title>
+ <synopsis>
+ derivative(F,x)
+ [J [,H]] = derivative(F,x [,h ,order ,H_form ,Q])
+ </synopsis>
+ </refsynopsisdiv>
+ <refsection>
+ <title>Arguments</title>
+ <variablelist>
+ <varlistentry>
+ <term>F</term>
<listitem>
- <para>
- <literal>order=1</literal>, the number of function evaluations is <literal>n+1</literal>,
- </para>
+ <para>
+ a Scilab function F: <literal>R^n --> R^m</literal> or a
+ <literal>list(F,p1,...,pk)</literal>, where F is a scilab function
+ in the form <literal>y=F(x,p1,...,pk)</literal>, p1, ..., pk being
+ any scilab objects (matrices, lists,...).
+ </para>
</listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>x</term>
<listitem>
- <para>
- <literal>order=2</literal>, the number of function evaluations is <literal>2n</literal>,
- </para>
+ <para>real column vector of dimension n.</para>
</listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>h</term>
<listitem>
- <para>
- <literal>order=4</literal>, the number of function evaluations is <literal>4n</literal>.
- </para>
+ <para>
+ (optional) real, the stepsize used in the finite difference
+ approximations.
+ </para>
</listitem>
- </itemizedlist>
- <para>
- Computing the Hessian matrix requires square the number of function evaluations,
- as detailed in the following list.
- </para>
- <itemizedlist>
+ </varlistentry>
+ <varlistentry>
+ <term>order</term>
<listitem>
- <para>
- <literal>order=1</literal>, the number of function evaluations is <literal>(n+1)^2</literal>,
- </para>
+ <para>
+ (optional) integer, the order of the finite difference formula
+ used to approximate the derivatives (order = 1,2 or 4, default is
+ order=2 ).
+ </para>
</listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>H_form</term>
<listitem>
- <para>
- <literal>order=2</literal>, the number of function evaluations is <literal>4n^2</literal>,
- </para>
+ <para>
+ (optional) string, the form in which the Hessean will be
+ returned. Possible forms are:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term>H_form='default'</term>
+ <listitem>
+ <para>
+ H is a m x (<literal>n^2</literal>) matrix ; in this
+ form, the k-th row of H corresponds to the Hessean of the k-th
+ component of F, given as the following row vector :
+ </para>
+ <informalequation>
+ <mediaobject>
+ <imageobject>
+ <imagedata align="center" fileref="../mml/derivative_equation_1.mml"/>
+ </imageobject>
+ </mediaobject>
+ </informalequation>
+ <para>((grad(F_k) being a row vector).</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>H_form='blockmat' :</term>
+ <listitem>
+ <para>
+ H is a (mxn) x n block matrix : the classic Hessean
+ matrices (of each component of F) are stacked by row (H = [H1
+ ; H2 ; ... ; Hm] in scilab syntax).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>H_form='hypermat' :</term>
+ <listitem>
+ <para>
+ H is a n x n matrix for m=1, and a n x n x m hypermatrix
+ otherwise. H(:,:,k) is the classic Hessean matrix of the k-th
+ component of F.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
</listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>Q</term>
<listitem>
- <para>
- <literal>order=4</literal>, the number of function evaluations is <literal>16n^2</literal>.
- </para>
+ <para>
+ (optional) real matrix, orthogonal (default is eye(n,n)). Q is added to have the possibility to remove
+ the arbitrariness of using the canonical basis to approximate the derivatives of a function and it should be an
+ orthogonal matrix. It is not mandatory but better to recover the derivative as you need the inverse matrix (and
+ so simply Q' instead of inv(Q)).
+ </para>
</listitem>
- </itemizedlist>
- </refsection>
- <refsection>
- <title>Remarks</title>
- <para>
- The step size h must be small to get a low error but if it is too
- small floating point errors will dominate by cancellation. As a rule of
- thumb, do not change the default step size. To work around numerical
- difficulties one may also change the order and/or choose different
- orthogonal matrices Q (the default is eye(n,n)), especially if the
- approximate derivatives are used in optimization routines. All the
- optional arguments may also be passed as named arguments, so that one can
- use calls in the form :
- </para>
- <programlisting>
- <![CDATA[
-derivative(F, x, H_form = "hypermat")
-derivative(F, x, order = 4) etc.
- ]]>
- </programlisting>
- </refsection>
- <refsection>
- <title>Examples</title>
- <programlisting role="example">
- <![CDATA[
-function y=F(x)
- y=[sin(x(1)*x(2))+exp(x(2)*x(3)+x(1)) ; sum(x.^3)];
-endfunction
+ </varlistentry>
+ <varlistentry>
+ <term>J</term>
+ <listitem>
+ <para>approximated Jacobian</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>H</term>
+ <listitem>
+ <para>approximated Hessian</para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsection>
+ <refsection>
+ <title>Description</title>
+ <para>
+ Numerical approximation of the first and second derivatives of a
+ function F: <literal> R^n --> R^m</literal> at the point x. The
+ Jacobian is computed by approximating the directional derivatives of the
+ components of F in the direction of the columns of Q. (For m=1, v=Q(:,k) :
+ grad(F(x))*v = Dv(F(x)).) The second derivatives are computed by
+ composition of first order derivatives. If H is given in its default form
+ the Taylor series of F(x) up to terms of second order is given by :
+ </para>
+ <informalequation>
+ <mediaobject>
+ <imageobject>
+ <imagedata align="center" fileref="../mml/derivative_equation_2.mml"/>
+ </imageobject>
+ </mediaobject>
+ </informalequation>
+ <para>(([J,H]=derivative(F,x,H_form='default'), J=J(x), H=H(x).)</para>
+ </refsection>
+ <refsection>
+ <title>Performances</title>
+ <para>
+ If the problem is correctly scaled, increasing the accuracy reduces
+ the total error but requires more function evaluations.
+ The following list presents the number of function evaluations required to
+ compute the Jacobian depending on the order of the formula and the dimension of <literal>x</literal>,
+ denoted by <literal>n</literal>:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>order=1</literal>, the number of function evaluations is <literal>n+1</literal>,
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>order=2</literal>, the number of function evaluations is <literal>2n</literal>,
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>order=4</literal>, the number of function evaluations is <literal>4n</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ Computing the Hessian matrix requires square the number of function evaluations,
+ as detailed in the following list.
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>order=1</literal>, the number of function evaluations is <literal>(n+1)^2</literal>,
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>order=2</literal>, the number of function evaluations is <literal>4n^2</literal>,
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>order=4</literal>, the number of function evaluations is <literal>16n^2</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsection>
+ <refsection>
+ <title>Remarks</title>
+ <para>
+ The step size h must be small to get a low error but if it is too
+ small floating point errors will dominate by cancellation. As a rule of
+ thumb, do not change the default step size. To work around numerical
+ difficulties one may also change the order and/or choose different
+ orthogonal matrices Q (the default is eye(n,n)), especially if the
+ approximate derivatives are used in optimization routines. All the
+ optional arguments may also be passed as named arguments, so that one can
+ use calls in the form :
+ </para>
+ <programlisting>
+ <![CDATA[
+ derivative(F, x, H_form = "hypermat")
+ derivative(F, x, order = 4) etc.
+ ]]>
+ </programlisting>
+ </refsection>
+ <refsection>
+ <title>Examples</title>
+ <programlisting role="example">
+ <![CDATA[
+ function y=F(x)
+ y=[sin(x(1)*x(2))+exp(x(2)*x(3)+x(1)) ; sum(x.^3)];
+ endfunction
-function y=G(x,p)
- y=[sin(x(1)*x(2)*p)+exp(x(2)*x(3)+x(1)) ; sum(x.^3)];
-endfunction
+ function y=G(x,p)
+ y=[sin(x(1)*x(2)*p)+exp(x(2)*x(3)+x(1)) ; sum(x.^3)];
+ endfunction
-x=[1;2;3];
-[J,H]=derivative(F,x,H_form='blockmat')
+ x=[1;2;3];
+ [J,H]=derivative(F,x,H_form='blockmat')
-n=3;
-// form an orthogonal matrix :
-Q = qr(rand(n,n))
-// Test order 1, 2 and 4 formulas.
-for i=[1,2,4]
- [J,H]=derivative(F,x,order=i,H_form='blockmat',Q=Q);
- mprintf("order= %d \n",i);
- H,
-end
+ n=3;
+ // form an orthogonal matrix :
+ Q = qr(rand(n,n))
+ // Test order 1, 2 and 4 formulas.
+ for i=[1,2,4]
+ [J,H]=derivative(F,x,order=i,H_form='blockmat',Q=Q);
+ mprintf("order= %d \n",i);
+ H,
+ end
-p=1;
-h=1e-3;
-[J,H]=derivative(list(G,p),x,h,2,H_form='hypermat');
-H
-[J,H]=derivative(list(G,p),x,h,4,Q=Q);
-H
+ p=1;
+ h=1e-3;
+ [J,H]=derivative(list(G,p),x,h,2,H_form='hypermat');
+ H
+ [J,H]=derivative(list(G,p),x,h,4,Q=Q);
+ H
-// Taylor series example:
-dx=1e-3*[1;1;-1];
-[J,H]=derivative(F,x);
-F(x+dx)
-F(x+dx)-F(x)
-F(x+dx)-F(x)-J*dx
-F(x+dx)-F(x)-J*dx-1/2*H*(dx .*. dx)
+ // Taylor series example:
+ dx=1e-3*[1;1;-1];
+ [J,H]=derivative(F,x);
+ F(x+dx)
+ F(x+dx)-F(x)
+ F(x+dx)-F(x)-J*dx
+ F(x+dx)-F(x)-J*dx-1/2*H*(dx .*. dx)
-// A trivial example
-function y=f(x,A,p,w)
- y=x'*A*x+p'*x+w;
-endfunction
-// with Jacobian and Hessean given by J(x)=x'*(A+A')+p', and H(x)=A+A'.
-A = rand(3,3);
-p = rand(3,1);
-w = 1;
-x = rand(3,1);
-[J,H]=derivative(list(f,A,p,w),x,h=1,H_form='blockmat')
+ // A trivial example
+ function y=f(x,A,p,w)
+ y=x'*A*x+p'*x+w;
+ endfunction
+ // with Jacobian and Hessean given by J(x)=x'*(A+A')+p', and H(x)=A+A'.
+ A = rand(3,3);
+ p = rand(3,1);
+ w = 1;
+ x = rand(3,1);
+ [J,H]=derivative(list(f,A,p,w),x,h=1,H_form='blockmat')
-// Since f(x) is quadratic in x, approximate derivatives of order=2 or 4 by finite
-// differences should be exact for all h~=0. The apparent errors are caused by
-// cancellation in the floating point operations, so a "big" h is choosen.
-// Comparison with the exact matrices:
-Je = x'*(A+A')+p'
-He = A+A'
-clean(Je - J)
-clean(He - H)
- ]]>
- </programlisting>
- </refsection>
- <refsection>
- <title>Accuracy issues</title>
- <para>
- The <literal>derivative</literal> function uses the same step <literal>h</literal>
- whatever the direction and whatever the norm of <literal>x</literal>.
- This may lead to a poor scaling with respect to <literal>x</literal>.
- An accurate scaling of the step is not possible without many evaluations
- of the function. Still, the user has the possibility to compare the results
- produced by the <literal>derivative</literal> and the <literal>numdiff</literal>
- functions. Indeed, the <literal>numdiff</literal> function scales the
- step depending on the absolute value of <literal>x</literal>.
- This scaling may produce more accurate results, especially if
- the magnitude of <literal>x</literal> is large.
- </para>
- <para>
- In the following Scilab script, we compute the derivative of an
- univariate quadratic function. The exact derivative can be
- computed analytically and the relative error is computed.
- In this rather extreme case, the <literal>derivative</literal> function
- produces no significant digits, while the <literal>numdiff</literal>
- function produces 6 significant digits.
- </para>
- <programlisting role="example">
- <![CDATA[
- // Difference between derivative and numdiff when x is large
-function y = myfunction (x)
- y = x*x;
-endfunction
-x = 1.e100;
-fe = 2.0 * x;
-fp = derivative(myfunction,x);
-e = abs(fp-fe)/fe;
-mprintf("Relative error with derivative: %e\n",e)
-fp = numdiff(myfunction,x);
-e = abs(fp-fe)/fe;
-mprintf("Relative error with numdiff: %e\n",e)
-]]>
- </programlisting>
- <para>
- The previous script produces the following output.
- </para>
- <programlisting role="example">
- <![CDATA[
-Relative error with derivative: 1.000000e+000
-Relative error with numdiff: 7.140672e-006
-]]>
- </programlisting>
- <para>
- In a practical situation, we may not know what is the correct numerical
- derivative. Still, we are warned that the numerical derivatives
- should be used with caution in this specific case.
- </para>
- </refsection>
- <refsection>
- <title>History</title>
- <para>
- Starting from Scilab 5.5.0, the <literal>derivative</literal> function is
- obsolete. It will be removed in Scilab 6.0.0.
- Please use the <literal>numderivative</literal> function instead.
- </para>
- <para>
- We now discuss how a script using the <literal>derivative</literal> function can be
- updated to use the <literal>numderivative</literal> function.
- </para>
- <para>
- Consider the function:
- </para>
- <programlisting role="example">
- <![CDATA[
-function y = F(x)
- f1 = sin(x(1)*x(2)) + exp(x(2)*x(3)+x(1))
- f2 = sum(x.^3)
- y = [f1 ; f2];
-endfunction
-]]>
- </programlisting>
- <para>
- and the point:
- </para>
- <programlisting role="example">
- <![CDATA[
-x = [1 ; 2 ; 3];
-]]>
- </programlisting>
- <para>
- Therefore, the statement:
- </para>
- <programlisting role="example">
- <![CDATA[
-[J, H] = derivative(F, x)
-]]>
- </programlisting>
- <para>
- can be replaced with
- </para>
- <programlisting role="example">
- <![CDATA[
-[J, H] = numderivative(F, x)
-]]>
- </programlisting>
- <para>
- The statement:
- </para>
- <programlisting role="example">
- <![CDATA[
-[J, H] = derivative(F, x, order=4)
-]]>
- </programlisting>
- <para>
- can be replaced by:
- </para>
- <programlisting role="example">
- <![CDATA[
-[J, H] = numderivative(F, x, [], 4)
-]]>
- </programlisting>
- <para>
- The statement:
- </para>
- <programlisting role="example">
- <![CDATA[
-[J, H] = derivative(F, x, H_form="blockmat")
-]]>
- </programlisting>
- <para>
- can be replaced by:
- </para>
- <programlisting role="example">
- <![CDATA[
-[J, H] = numderivative(F, x, [], [], "blockmat")
-]]>
- </programlisting>
- <para>
- We emphasize that <literal>numderivative</literal> and <literal>derivative</literal> do not
- use the same strategy for the choice of the default step <literal>h</literal>.
- Hence, in general, the functions <literal>numderivative</literal> and <literal>derivative</literal>
- will not produce exactly the same outputs.
- Still, in general, we expect that <literal>numderivative</literal> is more accurate.
- It is not easy to get the same step in <literal>numderivative</literal> as in <literal>derivative</literal>,
- because the choice of the step depends on the degree of
- differenciation (Jacobian or Hessian) and the order of the formula.
- </para>
- </refsection>
- <refsection role="see also">
- <title>See Also</title>
- <simplelist type="inline">
- <member>
- <link linkend="interp">interp</link>
- </member>
- <member>
- <link linkend="interp2d">interp2d</link>
- </member>
- <member>
- <link linkend="splin">splin</link>
- </member>
- <member>
- <link linkend="eval_cshep2d">eval_cshep2d</link>
- </member>
- <member>
- <link linkend="diff">diff</link>
- </member>
- <member>
- <link linkend="numdiff">numdiff</link>
- </member>
- <member>
- <link linkend="derivat">derivat</link>
- </member>
- <member>
- <link linkend="numderivative">numderivative</link>
- </member>
- </simplelist>
- </refsection>
+ // Since f(x) is quadratic in x, approximate derivatives of order=2 or 4 by finite
+ // differences should be exact for all h~=0. The apparent errors are caused by
+ // cancellation in the floating point operations, so a "big" h is choosen.
+ // Comparison with the exact matrices:
+ Je = x'*(A+A')+p'
+ He = A+A'
+ clean(Je - J)
+ clean(He - H)
+ ]]>
+ </programlisting>
+ </refsection>
+ <refsection>
+ <title>Accuracy issues</title>
+ <para>
+ The <literal>derivative</literal> function uses the same step <literal>h</literal>
+ whatever the direction and whatever the norm of <literal>x</literal>.
+ This may lead to a poor scaling with respect to <literal>x</literal>.
+ An accurate scaling of the step is not possible without many evaluations
+ of the function. Still, the user has the possibility to compare the results
+ produced by the <literal>derivative</literal> and the <literal>numdiff</literal>
+ functions. Indeed, the <literal>numdiff</literal> function scales the
+ step depending on the absolute value of <literal>x</literal>.
+ This scaling may produce more accurate results, especially if
+ the magnitude of <literal>x</literal> is large.
+ </para>
+ <para>
+ In the following Scilab script, we compute the derivative of an
+ univariate quadratic function. The exact derivative can be
+ computed analytically and the relative error is computed.
+ In this rather extreme case, the <literal>derivative</literal> function
+ produces no significant digits, while the <literal>numdiff</literal>
+ function produces 6 significant digits.
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ // Difference between derivative and numdiff when x is large
+ function y = myfunction (x)
+ y = x*x;
+ endfunction
+ x = 1.e100;
+ fe = 2.0 * x;
+ fp = derivative(myfunction,x);
+ e = abs(fp-fe)/fe;
+ mprintf("Relative error with derivative: %e\n",e)
+ fp = numdiff(myfunction,x);
+ e = abs(fp-fe)/fe;
+ mprintf("Relative error with numdiff: %e\n",e)
+ ]]>
+ </programlisting>
+ <para>
+ The previous script produces the following output.
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ Relative error with derivative: 1.000000e+000
+ Relative error with numdiff: 7.140672e-006
+ ]]>
+ </programlisting>
+ <para>
+ In a practical situation, we may not know what is the correct numerical
+ derivative. Still, we are warned that the numerical derivatives
+ should be used with caution in this specific case.
+ </para>
+ </refsection>
+ <refsection role="see also">
+ <title>See Also</title>
+ <simplelist type="inline">
+ <member>
+ <link linkend="interp">interp</link>
+ </member>
+ <member>
+ <link linkend="interp2d">interp2d</link>
+ </member>
+ <member>
+ <link linkend="splin">splin</link>
+ </member>
+ <member>
+ <link linkend="eval_cshep2d">eval_cshep2d</link>
+ </member>
+ <member>
+ <link linkend="diff">diff</link>
+ </member>
+ <member>
+ <link linkend="numdiff">numdiff</link>
+ </member>
+ <member>
+ <link linkend="derivat">derivat</link>
+ </member>
+ <member>
+ <link linkend="numderivative">numderivative</link>
+ </member>
+ </simplelist>
+ </refsection> <refsection>
+ <title>History</title>
+ <revhistory>
+ <revision>
+ <revnumber>5.5.0</revnumber>
+ <revremark>Tagged as obsolete. Will be removed in Scilab 6.0.0.</revremark>
+ </revision>
+ </revhistory>
+ </refsection>
+ <refsection>
+ <title>Appendix</title>
+ <para>
+ We now discuss how a script using the <literal>derivative</literal> function can be
+ updated to use the <literal>numderivative</literal> function.
+ </para>
+ <para>
+ Consider the function:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ function y = F(x)
+ f1 = sin(x(1)*x(2)) + exp(x(2)*x(3)+x(1))
+ f2 = sum(x.^3)
+ y = [f1 ; f2];
+ endfunction
+ ]]>
+ </programlisting>
+ <para>
+ and the point:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ x = [1 ; 2 ; 3];
+ ]]>
+ </programlisting>
+ <para>
+ Therefore, the statement:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ [J, H] = derivative(F, x)
+ ]]>
+ </programlisting>
+ <para>
+ can be replaced with
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ [J, H] = numderivative(F, x)
+ ]]>
+ </programlisting>
+ <para>
+ The statement:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ [J, H] = derivative(F, x, order=4)
+ ]]>
+ </programlisting>
+ <para>
+ can be replaced by:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ [J, H] = numderivative(F, x, [], 4)
+ ]]>
+ </programlisting>
+ <para>
+ The statement:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ [J, H] = derivative(F, x, H_form="blockmat")
+ ]]>
+ </programlisting>
+ <para>
+ can be replaced by:
+ </para>
+ <programlisting role="example">
+ <![CDATA[
+ [J, H] = numderivative(F, x, [], [], "blockmat")
+ ]]>
+ </programlisting>
+ <para>
+ We emphasize that <literal>numderivative</literal> and <literal>derivative</literal> do not
+ use the same strategy for the choice of the default step <literal>h</literal>.
+ Hence, in general, the functions <literal>numderivative</literal> and <literal>derivative</literal>
+ will not produce exactly the same outputs.
+ Still, in general, we expect that <literal>numderivative</literal> is more accurate.
+ It is not easy to get the same step in <literal>numderivative</literal> as in <literal>derivative</literal>,
+ because the choice of the step depends on the degree of
+ differenciation (Jacobian or Hessian) and the order of the formula.
+ </para>
+ </refsection>
</refentry>
projects / scilab.git / commitdiff