<refentry xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svg="http://www.w3.org/2000/svg" xmlns:ns5="http://www.w3.org/1999/xhtml" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:db="http://docbook.org/ns/docbook" xmlns:scilab="http://www.scilab.org" xml:id="bench_run" xml:lang="en">
<refnamediv>
<refname>bench_run</refname>
- <refpurpose>Launch benchmark tests</refpurpose>
+ <refpurpose>Launches benchmark tests</refpurpose>
</refnamediv>
<refsynopsisdiv>
<title>Calling Sequence</title>
<synopsis>
- bench_run()
- bench_run(module[,test_name[,options]])
+ [modutests_names, elapsed_time, nb_iterations] = bench_run()
+ [modutests_names, elapsed_time, nb_iterations] = bench_run(module[, test_name[, options, [exportToFile]])
+ [modutests_names, elapsed_time, nb_iterations] = bench_run(path_to_module[, test_name[, options, [exportToFile]])
</synopsis>
</refsynopsisdiv>
<refsection>
<varlistentry>
<term>module</term>
<listitem>
- <para>a vector of string. It can be the name of a module or the absolute path of a toolbox.</para>
+ <para>a vector of string. Contains the names of a Scilab modules to benchmark.</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>path_to_module</term>
+ <listitem>
+ <para>
+ a vector of string. Contains the paths to directories of modules to test. If <literal>"/path/to/directory"</literal> is given as input parameter, tests are retrieved in the subdirectory
+ <literal>
+ /path/to/directory/<emphasis role="bold">tests/benchmarks</emphasis>
+ </literal>
+ .Used for homemade benchmarks.
+ </para>
</listitem>
</varlistentry>
<varlistentry>
<term>test_name</term>
<listitem>
- <para>a vector of string</para>
+ <para>a vector of string. Contains the names of the tests to perform.</para>
+ <para>
+ The name of a test is its filename without <literal>.tst</literal>. If several modules or directory are given as first input parameter, scans for tests in each of these modules or directory.
+ </para>
</listitem>
</varlistentry>
<varlistentry>
<para>a vector of string</para>
<itemizedlist>
<listitem>
- <para>list : list of the benchmark tests available in a module</para>
+ <para>
+ <literal>"list"</literal>: list of the benchmark tests (<literal>test_name</literal>) available in a module
+ </para>
</listitem>
<listitem>
- <para>help : displays some examples of use in the Scilab console</para>
+ <para>
+ <literal>"help"</literal>: displays some examples of use in the Scilab console
+ </para>
</listitem>
<listitem>
- <para>nb_run=value : repeat the benchmark test value times</para>
+ <para>
+ <literal>"nb_run=value"</literal>: runs each benchmark <literal>value</literal> times ; by default <function>bench_run</function> runs 10000 times the code between BENCH START and BENCH END tags (see below). Overrides any <literal>BENCH NB RUN</literal> specified in the benchmark test files.
+ </para>
</listitem>
</itemizedlist>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>exportToFile</term>
+ <listitem>
+ <para>a single string</para>
+ <para>
+ File path to the result of the <function>bench_run</function> in xml format. By default, or if <literal>"", "[]"</literal> or <literal>[]</literal> is given, the output directory is <literal>TMPDIR/benchmarks/</literal>.
+ </para>
+ <para>
+ If <literal>exportToFile</literal> is a directory, creates a timestamped output file is the directory, otherwize creates the file <literal>exportToFile</literal>. If the file could not be created a warning is issued and the file is created under <literal>TMPDIR/benchmarks/</literal> instead.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>modutests_names</term>
+ <listitem>
+ <para>a N-by-2 matrix of strings</para>
+ <para>
+ the first column lists the modules tested by <function>bench_run</function>, the second column lists the names of the benchmarks
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>elapsed_time</term>
+ <listitem>
+ <para>a vector of doubles</para>
+ <para>the execution time for each benchmark</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>nb_iterations</term>
+ <listitem>
+ <para>a vector of doubles of size N</para>
+ <para>the number of iterations of respective test</para>
+ </listitem>
+ </varlistentry>
</variablelist>
</refsection>
<refsection>
<title>Description</title>
<para>
- Search for .tst files in benchmark test library
- execute them, and display a report about execution time.
- The .tst files are searched in directories SCI+"/modules/*/tests/benchmark".
+ Performs benchmark tests, measures execution time and produces a report about benchmark tests.
+ </para>
+ <para>
+ Searches for .tst files in benchmark test library or input parameter path under <literal>tests/benchmark</literal> subdirectory,
+ executes them 10000 times and displays a report about execution time.
</para>
<para>
Special tags may be inserted in the .tst file, which help to
<itemizedlist>
<listitem>
<para>
- <-- BENCH NB RUN : 10 -->
- This test will be repeated 10 times.
+ <literal><-- BENCH NB RUN : 10 --></literal>
+ </para>
+ <para>
+ By default, this test will be repeated 10 times, unless the "nb_run=###"<literal>bench_run(..)</literal> option is used. The value given for the flag can be set to any integer value.
</para>
</listitem>
<listitem>
+ <programlisting role="no-scilab-exec"><![CDATA[
+// <-- BENCH START -->
+[code to be executed]
+// <-- BENCH END -->
+]]></programlisting>
<para>
- <-- BENCH START -->
- <-- BENCH END -->
- The interesting part of the benchmark must be enclosed by these
- tags.
+ Code between these tags will be repeated. Code before will be executed before the repetition, code after will be executed after the repetition.
+ If these are not present, the entire code will be repeated.
</para>
</listitem>
</itemizedlist>
<para>Some simple examples of invocation of bench_run</para>
<programlisting role="example"><![CDATA[
// Launch all tests
-bench_run();
-bench_run([]);
-bench_run([],[]);
+// This may take some time...
+// bench_run();
+// bench_run([]);
+// bench_run([],[]);
// Test one or several module
bench_run('core');
// With options
bench_run([],[],'list');
bench_run([],[],'help');
-bench_run([],[],'nb_run=2000');
- ]]></programlisting>
+bench_run("string", [], 'nb_run=100');
+// results in an output file in the current directory
+bench_run("string", [], 'nb_run=100', 'my_output_file.xml');
+// results in an output directory, TMPDIR/benchmarks is the default
+bench_run("string", [], 'nb_run=100', TMPDIR);
+]]></programlisting>
<para>An example of a benchmark file. This file corresponds to the
file
SCI/modules/linear_algebra/tests/benchmarks/bench_chol.tst.
// <-- BENCH START -->
b = chol(a);
// <-- BENCH END -->
- ]]></programlisting>
+]]></programlisting>
<para>The result of the test</para>
- <programlisting role="example"><![CDATA[
+ <screen><![CDATA[
-->bench_run('linear_algebra','bench_chol')
- For Loop (as reference) ........................... 143.00 ms [ 1000000 x]
+For Loop (as reference) ........................... 33.20 ms [ 1000000 x]
- 001/001 - [linear_algebra] bench_chol ...................... 130.60 ms [ 10 x]
- ]]></programlisting>
+001/001 - [linear_algebra] bench_chol ...................... 1233.93 ms [ 10 x]
+ ]]></screen>
</refsection>
<refsection role="see also">
<title>See Also</title>
</member>
</simplelist>
</refsection>
+ <refsection role="history">
+ <title>History</title>
+ <revhistory>
+ <revision>
+ <revnumber>6.0</revnumber>
+ <revdescription>
+ <itemizedlist>
+ <listitem>
+ <literal>bench_run()</literal> can now return its results through the new
+ <literal>modutests_names</literal>, <literal>elapsed_time</literal>
+ and <literal>nb_iterations</literal> output parameters.
+ </listitem>
+ <listitem>
+ Exportation of results in XML is now possible
+ </listitem>
+ <listitem>
+ Global configuration settings mode(),
+ format(), ieee(), warning() and funcprot()
+ are now protected against tests.
+ </listitem>
+ </itemizedlist>
+ </revdescription>
+ </revision>
+ </revhistory>
+ </refsection>
</refentry>
projects / scilab.git / blobdiff