(gccinstall.info.gz) Testing

(gccinstall.info.gz) Installing GCC
(gccinstall.info.gz) Final install
 
 6 Installing GCC: Testing
 *************************
 
    Before you install GCC, we encourage you to run the testsuites and to
 compare your results with results from a similar configuration that have
 been submitted to the gcc-testresults mailing list.  Some of these
 archived results are linked from the build status lists at
 `http://gcc.gnu.org/buildstat.html', although not everyone who reports
 a successful build runs the testsuites and submits the results.  This
 step is optional and may require you to download additional software,
 but it can give you confidence in your new GCC installation or point out
 problems before you install and start using your new GCC.
 
    First, you must have downloaded the testsuites.  These are part of
 the full distribution, but if you downloaded the "core" compiler plus
 any front ends, you must download the testsuites separately.
 
    Second, you must have the testing tools installed.  This includes
 DejaGnu, Tcl, and Expect; the DejaGnu site has links to these.
 
    If the directories where `runtest' and `expect' were installed are
 not in the `PATH', you may need to set the following environment
 variables appropriately, as in the following example (which assumes
 that DejaGnu has been installed under `/usr/local'):
 
           TCL_LIBRARY = /usr/local/share/tcl8.0
           DEJAGNULIBS = /usr/local/share/dejagnu
 
    (On systems such as Cygwin, these paths are required to be actual
 paths, not mounts or links; presumably this is due to some lack of
 portability in the DejaGnu code.)
 
    Finally, you can run the testsuite (which may take a long time):
           cd OBJDIR; make -k check
 
    This will test various components of GCC, such as compiler front
 ends and runtime libraries.  While running the testsuite, DejaGnu might
 emit some harmless messages resembling `WARNING: Couldn't find the
 global config file.' or `WARNING: Couldn't find tool init file' that
 can be ignored.
 
    If you are testing a cross-compiler, you may want to run the
 testsuite on a simulator as described at
 `http://gcc.gnu.org/simtest-howto.html'.
 
 6.1 How can you run the testsuite on selected tests?
 ====================================================
 
 In order to run sets of tests selectively, there are targets `make
 check-gcc' and `make check-g++' in the `gcc' subdirectory of the object
 directory.  You can also just run `make check' in a subdirectory of the
 object directory.
 
    A more selective way to just run all `gcc' execute tests in the
 testsuite is to use
 
          make check-gcc RUNTESTFLAGS="execute.exp OTHER-OPTIONS"
 
    Likewise, in order to run only the `g++' "old-deja" tests in the
 testsuite with filenames matching `9805*', you would use
 
          make check-g++ RUNTESTFLAGS="old-deja.exp=9805* OTHER-OPTIONS"
 
    The `*.exp' files are located in the testsuite directories of the GCC
 source, the most important ones being `compile.exp', `execute.exp',
 `dg.exp' and `old-deja.exp'.  To get a list of the possible `*.exp'
 files, pipe the output of `make check' into a file and look at the
 `Running ...  .exp' lines.
 
 6.2 Passing options and running multiple testsuites
 ===================================================
 
 You can pass multiple options to the testsuite using the
 `--target_board' option of DejaGNU, either passed as part of
 `RUNTESTFLAGS', or directly to `runtest' if you prefer to work outside
 the makefiles.  For example,
 
          make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants"
 
    will run the standard `g++' testsuites ("unix" is the target name
 for a standard native testsuite situation), passing `-O3
 -fmerge-constants' to the compiler on every test, i.e., slashes
 separate options.
 
    You can run the testsuites multiple times using combinations of
 options with a syntax similar to the brace expansion of popular shells:
 
          ..."--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}"
 
    (Note the empty option caused by the trailing comma in the final
 group.)  The following will run each testsuite eight times using the
 `arm-sim' target, as if you had specified all possible combinations
 yourself:
 
          --target_board=arm-sim/-mhard-float/-O1
          --target_board=arm-sim/-mhard-float/-O2
          --target_board=arm-sim/-mhard-float/-O3
          --target_board=arm-sim/-mhard-float
          --target_board=arm-sim/-msoft-float/-O1
          --target_board=arm-sim/-msoft-float/-O2
          --target_board=arm-sim/-msoft-float/-O3
          --target_board=arm-sim/-msoft-float
 
    They can be combined as many times as you wish, in arbitrary ways.
 This list:
 
          ..."--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}"
 
    will generate four combinations, all involving `-Wextra'.
 
    The disadvantage to this method is that the testsuites are run in
 serial, which is a waste on multiprocessor systems.  For users with GNU
 Make and a shell which performs brace expansion, you can run the
 testsuites in parallel by having the shell perform the combinations and
 `make' do the parallel runs.  Instead of using `--target_board', use a
 special makefile target:
 
          make -jN check-TESTSUITE//TEST-TARGET/OPTION1/OPTION2/...
 
    For example,
 
          make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu}
 
    will run three concurrent "make-gcc" testsuites, eventually testing
 all ten combinations as described above.  Note that this is currently
 only supported in the `gcc' subdirectory.  (To see how this works, try
 typing `echo' before the example given here.)
 
 6.3 Additional testing for Java Class Libraries
 ===============================================
 
 The Java runtime tests can be executed via `make check' in the
 `TARGET/libjava/testsuite' directory in the build tree.
 
    The Mauve Project provides a suite of tests for the Java Class
 Libraries.  This suite can be run as part of libgcj testing by placing
 the Mauve tree within the libjava testsuite at
 `libjava/testsuite/libjava.mauve/mauve', or by specifying the location
 of that tree when invoking `make', as in `make MAUVEDIR=~/mauve check'.
 
 6.4 How to interpret test results
 =================================
 
 The result of running the testsuite are various `*.sum' and `*.log'
 files in the testsuite subdirectories.  The `*.log' files contain a
 detailed log of the compiler invocations and the corresponding results,
 the `*.sum' files summarize the results.  These summaries contain
 status codes for all tests:
 
    * PASS: the test passed as expected
 
    * XPASS: the test unexpectedly passed
 
    * FAIL: the test unexpectedly failed
 
    * XFAIL: the test failed as expected
 
    * UNSUPPORTED: the test is not supported on this platform
 
    * ERROR: the testsuite detected an error
 
    * WARNING: the testsuite detected a possible problem
 
    It is normal for some tests to report unexpected failures.  At the
 current time the testing harness does not allow fine grained control
 over whether or not a test is expected to fail.  This problem should be
 fixed in future releases.
 
 6.5 Submitting test results
 ===========================
 
 If you want to report the results to the GCC project, use the
 `contrib/test_summary' shell script.  Start it in the OBJDIR with
 
          SRCDIR/contrib/test_summary -p your_commentary.txt \
              -m gcc-testresults@gcc.gnu.org |sh
 
    This script uses the `Mail' program to send the results, so make
 sure it is in your `PATH'.  The file `your_commentary.txt' is prepended
 to the testsuite summary and should contain any special remarks you
 have on your results or your build environment.  Please do not edit the
 testsuite result block or the subject line, as these messages may be
 automatically processed.
Info Catalog
(gccinstall.info.gz) Building
(gccinstall.info.gz) Installing GCC
(gccinstall.info.gz) Final install
automatically generated by info2html