From 094400332afe10f3820c91aa72f11762ffe73c41 Mon Sep 17 00:00:00 2001 From: apcraig Date: Tue, 14 Apr 2020 19:00:15 +0000 Subject: [PATCH] Update documenation --- doc/source/user_guide/ug_testing.rst | 54 +++++++++++++++++----------- 1 file changed, 33 insertions(+), 21 deletions(-) diff --git a/doc/source/user_guide/ug_testing.rst b/doc/source/user_guide/ug_testing.rst index 332b8b56f..5deb40042 100755 --- a/doc/source/user_guide/ug_testing.rst +++ b/doc/source/user_guide/ug_testing.rst @@ -496,6 +496,39 @@ The reporting can also be automated in a test suite by adding ``--report`` to `` With ``--report``, the suite will create all the tests, build and submit them, wait for all runs to be complete, and run the results and report_results scripts. +.. _codecoverage: + +Code Coverage Testing +------------------------ + +The ``--codecov`` feature in **icepack.setup** provides a method to diagnose code coverage. +This argument turns on special compiler flags including reduced optimization and then +invokes the gcov tool. +This option is currently only available with the gnu compiler and on a few systems. +To use, submit a full test suite using a version of Icepack on the Consortium master +and the gnu compiler with the ``--codecov`` argument. +The test suite will run and then a report will be generated and uploaded to +the `codecov.io site `_ by the +**report_codecov.csh** script. + +This is a special diagnostic test and does not constitute proper model testing. +General use is not recommended, this is mainly used as a diagnostic to periodically +assess test coverage. The interaction with codecov.io is not always robust and +can be tricky to manage. Some constraints are that the output generated at runtime +is copied into the directory where compilation took place. That means each +test should be compiled separately. Tests that invoke multiple runs +(such as exact restart) will only save coverage information +for the last run, so some coverage information may be lost. The gcov tool can +be a little slow to run on large test suites, and the codecov.io bash uploader +(that runs gcov and uploads the data to codecov.io) is constantly evolving. +Finally, gcov requires that the diagnostic output be copied into the git sandbox for +analysis. These constraints are handled by the current scripts, but may change +in the future. + +A sample job submission would look like :: + +$ ./icepack.setup -m conrad -e gnu --suite base_suite,travis_suite,quick_suite --testid cc01 --codecov + .. _testplotting: Test Plotting @@ -550,24 +583,3 @@ This plotting script can be used to plot the following variables: - snow-ice (m) - initial energy change (:math:`W/m^2`) -.. _codecoverage: - -Code Coverage Testing ------------------------- - -The ``--codecov`` feature in **icepack.setup** provides a method to diagnose code coverage. -This argument turns on special compiler flags including reduced optimization and then -invokes the gcov tool. -This option is currently only available with the gnu compiler and on a few systems. -To use, submit a full test suite using a version of Icepack on the Consortium master -and the gnu compiler with the ``--codecov`` argument to **icepack.setup**. -The test suite will run and then a report will be generated and uploaded to -the `codecov.io site `_ by the -**report_codecov.csh** script. -This is a special diagnostic test and does not constitute proper model testing. -General use is not recommended, this is mainly used as a diagnostic to periodically -assess test coverage. In addition, the interaction with codecov.io is not always robust. -A sample job submission would look like :: - -$ ./icepack.setup -m conrad -e gnu --suite base_suite,travis_suite,quick_suite --testid cc01 --codecov -