Testbed » History » Revision 4
Revision 3 (Anonymous, 01/25/2016 11:50 AM) → Revision 4/6 (Anonymous, 01/25/2016 11:50 AM)
h1. Testbed
h2. The S/PHI/nX testbed
The S/PHI/nX testbed allows to check for basic functionality of the compiled programs after building. It can be found in the sphinx/testbed folder. Running the testbed requires GNU make, which is the default make on most Unix systems.
Setup
<pre>
cd sphinx/testbed
./setup
./configure --with-sxdir=BUILD/PATH/TO/BE/CHECKED
</pre>
The configure line can be omitted if the build to be checked is ../src (standard S/PHI/nX user build). When debug and release mode have been compiled, typical choices for BUILD/PATH/TO/BE/CHECKED will be ../debug or ../release .
h2. Running the testbed
The testbed can be run with
<pre>
make level1 level2
</pre>
On multi-processor machines, parallel make ( make -j N ) is possible; where N is the number of processors. Level 1 checks are typically quick to run. Level 2 checks may be slow, in particular by the debug mode. The two levels can be run independently.
It is also possible to run individual tests via
<pre>
make -C TO_BE_TESTED run
</pre>
Reports
Reports are generated as a set of html pages in the report/ folder via
<pre>
make reports
</pre>
Parallel make is not possible.
The report can then be viewed with your favorite web browser, e.g.
<pre>
firefox report/index.html
</pre>
Individual test reports are generated via
<pre>
make -C TO_BE_TESTED report
</pre>
The test report will be attached to the end of the report/index.html file. Note that for technical reasons, graphs from previous test failures will be included in the new report if the report/ folder has not been cleaned.
h2. Failures?
p(. If some tests fail, this need not mean that the code is broken. We have not spent particular efforts in developing a reliable test metrics. Typically, one should look at the detailed analysis. Differences in the number of steps and small differences in energy may result from hard-ware and compilation-dependent numerical noise. Dramatic deterioration of the convergence behavior, however, is a strong hint for a serious problem.
p(. Some tests fail also if the necessary utility programs are not in the search path (e.g. perl, ncdump, etc.).
p(. If in doubt, contact the SPHInX user mailing list sxusers at mpie de after registration (sxusers-subscribe@…).
h2. Cleaning
The testbed can be cleaned with
<pre>
make cleanall
</pre>
Individual tests can be cleaned via
<pre>
make -C TO_BE_TESTED clean
</pre>
This does not clean the corresponding report.