From ae3d59f7b62051922fc273d17b0cbe7d0e5f1278 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Thu, 9 May 2024 15:58:00 +0000 Subject: [PATCH] build based on 9aeb0f6 --- previews/PR109/.documenter-siteinfo.json | 2 +- previews/PR109/contributing/index.html | 2 +- previews/PR109/index.html | 2 +- previews/PR109/library/index.html | 26 ++++++++++++------------ previews/PR109/manual/index.html | 2 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --git a/previews/PR109/.documenter-siteinfo.json b/previews/PR109/.documenter-siteinfo.json index a85e9e9..24836f6 100644 --- a/previews/PR109/.documenter-siteinfo.json +++ b/previews/PR109/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.3","generation_timestamp":"2024-05-09T15:54:28","documenter_version":"1.4.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.3","generation_timestamp":"2024-05-09T15:57:56","documenter_version":"1.4.0"}} \ No newline at end of file diff --git a/previews/PR109/contributing/index.html b/previews/PR109/contributing/index.html index 4ccde48..bd26cc9 100644 --- a/previews/PR109/contributing/index.html +++ b/previews/PR109/contributing/index.html @@ -1,2 +1,2 @@ -Contributing · TestReports.jl
GitHub

Contributing

`TestReports.jl follows the ColPrac guide for collaborative practices. New contributors should make sure to read that guide

+Contributing · TestReports.jl
GitHub

Contributing

`TestReports.jl follows the ColPrac guide for collaborative practices. New contributors should make sure to read that guide

diff --git a/previews/PR109/index.html b/previews/PR109/index.html index fe333b2..5c91336 100644 --- a/previews/PR109/index.html +++ b/previews/PR109/index.html @@ -30,4 +30,4 @@ </testcase> <testcase name="pass (info lost)" id="_testcase_id_"/> </testsuite> -</testsuites> +</testsuites> diff --git a/previews/PR109/library/index.html b/previews/PR109/library/index.html index e428158..a4ae1bb 100644 --- a/previews/PR109/library/index.html +++ b/previews/PR109/library/index.html @@ -1,6 +1,6 @@ Library · TestReports.jl
GitHub

Library

Contents

Public

Documentation for TestReports.jl's public interface.

TestReports.test — Function
TestReports.test(; kwargs...)
-TestReports.test(pkg::Union{AbstractString, Vector{AbstractString}; kwargs...)

Keyword arguments:

  • coverage::Bool=false: enable or disable generation of coverage statistics.
  • julia_args::Union{Cmd, Vector{String}}: options to be passed to the test process.
  • test_args::Union{Cmd, Vector{String}}: test arguments (ARGS) available in the test process.
  • logfilepath::AbstractString=pwd(): file path where test reports are saved.
  • logfilename::Union{AbstractString, Vector{AbstractString}}: name(s) of test report file(s).

Generates a JUnit XML for the tests of package pkg, or for the current project (which thus needs to be a package) if no positional argument is given to TestReports.test. The test report is saved in the current working directory and called testlog.xml if both logfilepath and logfilename are not supplied. If pkg is of type Vector{String}, the report filenames are prepended with the package name, for example Example_testlog.xml.

If logfilename is supplied, it must match the type (and length, if a vector) of pkg.

The tests are run in the same way as Pkg.test.

source
TestReports.recordproperty — Function
recordproperty(name::String, value)

Adds a property to a testset with name and value that will in turn be added to the <properties> node of the corresponding testsuite in the JUnit XML.

Multiple properties can be added to one testset, but if the same property is set on both parent and child testsets, the value in the child testset takes precedence over that in the parent.

The suggested use of this function is to place it inside a testset with unspecified type (see Examples). This will ensure that Pkg.test is unnaffected, but that the properties are added to the report when TestReports.test is used. This is because properties are only added when the Testset type has a TestReports.properties method defined, as does the ReportingTestSet used by TestReports. TestReports.properties can be extended for custom TestSets.

If a child testset has this method defined but its parent doesn't, the property should be in the report when TestReport.test is used, assuming that the parent testset type doesn't do anything to affect the reporting behaviour. However this is not tested functionality.

value must be serializable by EzXML, which gives quite a lot of freedom.

Examples

using TestReports
+TestReports.test(pkg::Union{AbstractString, Vector{AbstractString}; kwargs...)

Keyword arguments:

  • coverage::Bool=false: enable or disable generation of coverage statistics.
  • julia_args::Union{Cmd, Vector{String}}: options to be passed to the test process.
  • test_args::Union{Cmd, Vector{String}}: test arguments (ARGS) available in the test process.
  • logfilepath::AbstractString=pwd(): file path where test reports are saved.
  • logfilename::Union{AbstractString, Vector{AbstractString}}: name(s) of test report file(s).

Generates a JUnit XML for the tests of package pkg, or for the current project (which thus needs to be a package) if no positional argument is given to TestReports.test. The test report is saved in the current working directory and called testlog.xml if both logfilepath and logfilename are not supplied. If pkg is of type Vector{String}, the report filenames are prepended with the package name, for example Example_testlog.xml.

If logfilename is supplied, it must match the type (and length, if a vector) of pkg.

The tests are run in the same way as Pkg.test.

source
TestReports.recordproperty — Function
recordproperty(name::String, value)

Adds a property to a testset with name and value that will in turn be added to the <properties> node of the corresponding testsuite in the JUnit XML.

Multiple properties can be added to one testset, but if the same property is set on both parent and child testsets, the value in the child testset takes precedence over that in the parent.

The suggested use of this function is to place it inside a testset with unspecified type (see Examples). This will ensure that Pkg.test is unnaffected, but that the properties are added to the report when TestReports.test is used. This is because properties are only added when the Testset type has a TestReports.properties method defined, as does the ReportingTestSet used by TestReports. TestReports.properties can be extended for custom TestSets.

If a child testset has this method defined but its parent doesn't, the property should be in the report when TestReport.test is used, assuming that the parent testset type doesn't do anything to affect the reporting behaviour. However this is not tested functionality.

value must be serializable by EzXML, which gives quite a lot of freedom.

Examples

using TestReports
 
 # Default testset used, so function will not affect Pkg.test but will be used when
 # generating JUnit XML.
@@ -10,26 +10,26 @@
     recordproperty("Bool", true)
     @test 1==1
     @test 2==2
-end

See also: properties

source
TestReports.ReportingTestSet — Type
ReportingTestSet

Custom AbstractTestSet type designed to be used by TestReports.jl for creation of JUnit XMLs.

Does not throw an error when a test fails or has an error. Upon finishing, a ReportingTestSet will display the default test output, and then flatten to a structure that is suitable for report generation.

It is designed to be wrapped around a package's runtests.jl file and this is assumed when both the test results are displayed and when the TestSet is flatted upon finish. See bin/reporttests.jl for an example of this use. ReportingTestSets are not designed to be used directly in a package's tests, and this is not recommended or supported.

A ReportingTestSet has the description and results fields as per a DefaultTestSet, and has the following additional fields:

  • properties: a dictionary which is used to record properties to be inserted into the report.
  • start_time::DateTime: the start date and time of the testing (local system time).
  • time_taken::Millisecond: the time taken in milliseconds to run the TestSet.
  • last_record_time::DateTime: the time when record was last called.
  • hostname::String: the name of host on which the testset was executed.

See also: flatten_results!, recordproperty, report

source
TestReports.any_problems — Function
any_problems(ts)

Checks a testset to see if there were any problems (Errors or Fails). Note that unlike the DefaultTestSet, the ReportingTestSet does not throw an exception on a failure. Thus to set the exit code from the runner code, we check it using exit(any_problems(top_level_testset)).

source
TestReports.report — Function
report(ts::AbstractTestSet) -> XMLDocument

Produce an JUnit XML report details about the contained TestSets and Results. As the JUnit XML schema does not allow nested testsuite elements the report will flatten the hierarchical TestSet structure. Each TestSet will become a testsuite element and each Result will become a testcase element.

A Result will only be reported once within its parent TestSet to avoid having duplicate entries within the report and avoid problems with total test counts not matching Julia output.

All AbstractTestSets contained within ts must have a description::AbstractString field and an iterable results field.

source

Private

Package internals documentation.

Report Generation

TestReports.checkexitcode! — Method
checkexitcode!(errs, proc, pkg, logfilename)

Checks proc.exitcode and acts as follows:

  • If 0, displays tests passed info message
  • If equal to TESTS_FAILED const, warning is displayed and pkg added to errs
  • If anything else, throws a PkgTestError
source
TestReports.gen_runner_code — Method
gen_runner_code(testfilename, logfilename, test_args)

Returns runner code that will run the tests and generate the report in a new Julia instance.

source
TestReports.get_deps — Method
get_deps(manifest, pkg) = get_deps!(String[], manifest, pkg)

Get list of dependencies for pkg found in manifest

source
TestReports.gettestfilepath — Method
gettestfilepath(ctx::Context, pkgspec::Pkg.Types.PackageSpec)

Gets the testfile path of the package. Code for each Julia version mirrors that found in Pkg/src/Operations.jl.

source
TestReports.isinstalled! — Method
isinstalled!(ctx::Context, pkgspec::Pkg.Types.PackageSpec)

Checks if the package is installed by using ensure_resolved from Pkg/src/Types.jl. This function fails if the package is not installed, but here we wrap it in a try-catch as we may want to test another package after the one that isn't installed.

For Julia versions V1.4 and later, the first arguments of the Pkg functions used is of type Pkg.Types.Context. For earlier versions, they are of type Pkg.Types.EnvCache.

source
TestReports.runtests! — Method
runtests!(errs::Vector, pkg, cmd, logfilename)

Runs cmd which will run the tests of pkg. The exit code of the process is then checked.

source
TestReports.ReportingTestSet — Type
ReportingTestSet

Custom AbstractTestSet type designed to be used by TestReports.jl for creation of JUnit XMLs.

Does not throw an error when a test fails or has an error. Upon finishing, a ReportingTestSet will display the default test output, and then flatten to a structure that is suitable for report generation.

It is designed to be wrapped around a package's runtests.jl file and this is assumed when both the test results are displayed and when the TestSet is flatted upon finish. See bin/reporttests.jl for an example of this use. ReportingTestSets are not designed to be used directly in a package's tests, and this is not recommended or supported.

A ReportingTestSet has the description and results fields as per a DefaultTestSet, and has the following additional fields:

  • properties: a dictionary which is used to record properties to be inserted into the report.
  • start_time::DateTime: the start date and time of the testing (local system time).
  • time_taken::Millisecond: the time taken in milliseconds to run the TestSet.
  • last_record_time::DateTime: the time when record was last called.
  • hostname::String: the name of host on which the testset was executed.

See also: flatten_results!, recordproperty, report

source
TestReports.any_problems — Function
any_problems(ts)

Checks a testset to see if there were any problems (Errors or Fails). Note that unlike the DefaultTestSet, the ReportingTestSet does not throw an exception on a failure. Thus to set the exit code from the runner code, we check it using exit(any_problems(top_level_testset)).

source
TestReports.report — Function
report(ts::AbstractTestSet) -> XMLDocument

Produce an JUnit XML report details about the contained TestSets and Results. As the JUnit XML schema does not allow nested testsuite elements the report will flatten the hierarchical TestSet structure. Each TestSet will become a testsuite element and each Result will become a testcase element.

A Result will only be reported once within its parent TestSet to avoid having duplicate entries within the report and avoid problems with total test counts not matching Julia output.

All AbstractTestSets contained within ts must have a description::AbstractString field and an iterable results field.

source

Private

Package internals documentation.

Report Generation

TestReports.checkexitcode! — Method
checkexitcode!(errs, proc, pkg, logfilename)

Checks proc.exitcode and acts as follows:

  • If 0, displays tests passed info message
  • If equal to TESTS_FAILED const, warning is displayed and pkg added to errs
  • If anything else, throws a PkgTestError
source
TestReports.gen_runner_code — Method
gen_runner_code(testfilename, logfilename, test_args)

Returns runner code that will run the tests and generate the report in a new Julia instance.

source
TestReports.get_deps — Method
get_deps(manifest, pkg) = get_deps!(String[], manifest, pkg)

Get list of dependencies for pkg found in manifest

source
TestReports.gettestfilepath — Method
gettestfilepath(ctx::Context, pkgspec::Pkg.Types.PackageSpec)

Gets the testfile path of the package. Code for each Julia version mirrors that found in Pkg/src/Operations.jl.

source
TestReports.isinstalled! — Method
isinstalled!(ctx::Context, pkgspec::Pkg.Types.PackageSpec)

Checks if the package is installed by using ensure_resolved from Pkg/src/Types.jl. This function fails if the package is not installed, but here we wrap it in a try-catch as we may want to test another package after the one that isn't installed.

For Julia versions V1.4 and later, the first arguments of the Pkg functions used is of type Pkg.Types.Context. For earlier versions, they are of type Pkg.Types.EnvCache.

source
TestReports.runtests! — Method
runtests!(errs::Vector, pkg, cmd, logfilename)

Runs cmd which will run the tests of pkg. The exit code of the process is then checked.

source
TestReports.test! — Method
test!(pkg::AbstractString,
       errs::Vector{AbstractString},
       nopkgs::Vector{AbstractString},
       notests::Vector{AbstractString},
       logfilename::AbstractString;
       coverage::Bool=false,
       julia_args::Union{Cmd, AbstractVector{<:AbstractString}}=``,
-      test_args::Union{Cmd, AbstractVector{<:AbstractString}}=``)

Tests pkg and save report to logfilename. Tests are run in the same way as Pkg.test.

If tests error pkg is added to nopkgs. If pkg has no testfile it is added to notests. If pkg is not installed it is added to nopkgs.

source

TestSets

TestReports.ReportingResult — Type
ReportingResult{T}

Wraps a Result of type T so that additional metadata can be saved.

fields

  • result::T: Result that is being wrapped.
  • time_taken::Millisecond: the time taken (milliseconds) for this test to be run.
source
TestReports.add_to_ts_default! — Method
add_to_ts_default!(ts_default::DefaultTestSet, result::Result)
+      test_args::Union{Cmd, AbstractVector{<:AbstractString}}=``)

Tests pkg and save report to logfilename. Tests are run in the same way as Pkg.test.

If tests error pkg is added to nopkgs. If pkg has no testfile it is added to notests. If pkg is not installed it is added to nopkgs.

source

TestSets

TestReports.ReportingResult — Type
ReportingResult{T}

Wraps a Result of type T so that additional metadata can be saved.

fields

  • result::T: Result that is being wrapped.
  • time_taken::Millisecond: the time taken (milliseconds) for this test to be run.
source
TestReports.add_to_ts_default! — Method
add_to_ts_default!(ts_default::DefaultTestSet, result::Result)
 add_to_ts_default!(ts_default::DefaultTestSet, result::ReportingResult)
 add_to_ts_default!(ts_default::DefaultTestSet, ts::AbstractTestSet)
-add_to_ts_default!(ts_default::DefaultTestSet, ts::ReportingTestSet)

Populate ts_default with the supplied variable. If result is a Result or an AbstractTestSet (but not a ReportingTestSet) then it is recorded. If it is a ReportingTestSet then a new DefaultTestSet with matching description is created, populated by recursively calling this function and then added to the results of ts_default. If result is a ReportingResult, the Result contained by the ReportingResult is added to ts_default.

source
TestReports.flatten_results! — Method
flatten_results!(ts::AbstractTestSet)

Returns a flat vector of TestSets which only contain Results. This is necessary for writing a JUnit XML report the schema does not allow nested XML testsuite elements.

source
TestReports.hostname — Method
hostname(ts::ReportingTestSet)
-hostname(ts::AbstractTestSet)

Get the hostname of a ReportingTestSet, returns gethostname() for an AbstractTestSet. Can be extended for custom TestSets, must return a string.

source
TestReports.ispass — Method
ispass(result)

Return true if result is a Pass or a ReportingResult{Pass}, otherwise return false.

source
TestReports.properties — Method
properties(ts::ReportingTestSet)
-properties(ts::AbstractTestSet)

Get the properties dictionary of a ReportingTestSet, returns nothing for an AbstractTestSet. Can be extended for custom TestSets, and must return either a Dict or nothing.

source
TestReports.set_start_time! — Method
set_start_time!(ts::ReportingTestSet, start_time)
-set_start_time!(ts::AbstractTestSet, start_time)

Sets the start time field of a ReportingTestSet. This is used when flattening ReportingTestSets for report generation and an be extended for custom TestSets.

source
TestReports.set_time_taken! — Method
set_time_taken!(ts::ReportingTestSet, time_taken)
-set_time_taken!(ts::AbstractTestSet, time_taken)

Sets the time taken field of a ReportingTestSet. This is used when flattening ReportingTestSets for report generation and an be extended for custom TestSets.

source
TestReports.start_time — Method
start_time(ts::ReportingTestSet)
-start_time(ts::AbstractTestSet)

Get the start time of a ReportingTestSet, returns Dates.now() for an AbstractTestSet. Can be extended for custom TestSets, must return a DateTime.

source
TestReports.time_taken — Method
time_taken(ts::ReportingTestSet)
-time_taken(ts::AbstractTestSet)

Get the time taken of a ReportingTestSet, returns Dates.Millisecond(0) for an AbstractTestSet. Can be extended for custom TestSets, must return a Dates.Millisecond.

source
TestReports.time_taken — Method
time_taken(result::ReportingResult)
-time_taken(result::Result)

For a ReportingResult, return the time taken for the test to run. For a Result, return Dates.Millisecond(0).

source
TestReports.update_testset_properties! — Method
update_testset_properties!(childts::AbstractTestSet, ts::AbstractTestSet)

Adds properties of ts to childts. If any properties being added already exist in childts, a warning is displayed and the value in ts is overwritten.

If the types of ts and\or childts do not a method defined for TestReports.properties, this is handled as follows:

  • If method not defined for typeof(ts), it has no properties to add to childts and therefore nothing happens.
  • If method not defined for typeof(chidlts) and ts has properties, then a warning is shown.

See also: properties

source

XML Writing

TestReports.add_testsuite_properties! — Method
add_testsuite_properties!(x_testsuite, ts::AbstractTestSet)

Add all key value pairs in the properties field of a AbstractTestSet to the corresponding testsuite xml element. This function assumes that the type of ts has a TestReports.properties method defined.

See also: properties

source
TestReports.check_ts — Method
check_ts(ts::AbstractTestSet)

Throws an exception if ts does not have the right structure for report or if the results of ts do not have both description or results fields.

See also: report

source
TestReports.error_xml — Method
failure_xml(message, test_type, content)

Create an error node (which will be the child of a testcase).

source
TestReports.format_period_for_xml — Method

formatperiodfor_xml(time::Millisecond)

Formats a millisecond value into a string in seconds with 3 decimal places.

@sprintf is used to ensure that value does not use scientific notification, which is not allowed in an XML decimal.

Methods for other Periods have not been written as are not currently required.

source
TestReports.get_error_info — Method
get_error_info(v::Error)

Return message and type of error for testcase attribute, and number of tests (either 1 or 0). Uses test_type field to determine what caused the original error.

source
TestReports.get_failure_message — Method

getfailuremessage(v::Fail)

Return message for failed test testcase attribute. Uses test_type field to determine what caused the original failure.

source
TestReports.set_attribute! — Method
set_attribute!(node, attr, val)
-set_attribute!(node, attr, val::Period)

Add the attritube with name attr and value val to node.

If val is of type Period, format with format_period_for_xml.

source
TestReports.testcase_xml — Method
testcase_xml(name, id, x_children)

Create a testcase element of a JUnit XML.

This is the generic form (with name, id and children) that is used by other methods.

source
TestReports.testcase_xml — Method
testcase_xml(v::Result, childs)

Create a testcase element of a JUnit XML for the result v.

The original expression of the test is used as the name, whilst the id is defaulted to testcaseid_.

source
TestReports.to_xml — Method
to_xml(ts::AbstractTestSet)

Create a testsuite node from a AbstractTestSet, by creating nodes for each result in ts.results. For creating a JUnit XML, all results must be AbstractResults, that is they cannot be AbstractTestSets, as the XML cannot have one testsuite nested inside another.

source
TestReports.to_xml — Method
to_xml(res::Pass)
+add_to_ts_default!(ts_default::DefaultTestSet, ts::ReportingTestSet)

Populate ts_default with the supplied variable. If result is a Result or an AbstractTestSet (but not a ReportingTestSet) then it is recorded. If it is a ReportingTestSet then a new DefaultTestSet with matching description is created, populated by recursively calling this function and then added to the results of ts_default. If result is a ReportingResult, the Result contained by the ReportingResult is added to ts_default.

source
TestReports.flatten_results! — Method
flatten_results!(ts::AbstractTestSet)

Returns a flat vector of TestSets which only contain Results. This is necessary for writing a JUnit XML report the schema does not allow nested XML testsuite elements.

source
TestReports.hostname — Method
hostname(ts::ReportingTestSet)
+hostname(ts::AbstractTestSet)

Get the hostname of a ReportingTestSet, returns gethostname() for an AbstractTestSet. Can be extended for custom TestSets, must return a string.

source
TestReports.ispass — Method
ispass(result)

Return true if result is a Pass or a ReportingResult{Pass}, otherwise return false.

source
TestReports.properties — Method
properties(ts::ReportingTestSet)
+properties(ts::AbstractTestSet)

Get the properties dictionary of a ReportingTestSet, returns nothing for an AbstractTestSet. Can be extended for custom TestSets, and must return either a Dict or nothing.

source
TestReports.set_start_time! — Method
set_start_time!(ts::ReportingTestSet, start_time)
+set_start_time!(ts::AbstractTestSet, start_time)

Sets the start time field of a ReportingTestSet. This is used when flattening ReportingTestSets for report generation and an be extended for custom TestSets.

source
TestReports.set_time_taken! — Method
set_time_taken!(ts::ReportingTestSet, time_taken)
+set_time_taken!(ts::AbstractTestSet, time_taken)

Sets the time taken field of a ReportingTestSet. This is used when flattening ReportingTestSets for report generation and an be extended for custom TestSets.

source
TestReports.start_time — Method
start_time(ts::ReportingTestSet)
+start_time(ts::AbstractTestSet)

Get the start time of a ReportingTestSet, returns Dates.now() for an AbstractTestSet. Can be extended for custom TestSets, must return a DateTime.

source
TestReports.time_taken — Method
time_taken(ts::ReportingTestSet)
+time_taken(ts::AbstractTestSet)

Get the time taken of a ReportingTestSet, returns Dates.Millisecond(0) for an AbstractTestSet. Can be extended for custom TestSets, must return a Dates.Millisecond.

source
TestReports.time_taken — Method
time_taken(result::ReportingResult)
+time_taken(result::Result)

For a ReportingResult, return the time taken for the test to run. For a Result, return Dates.Millisecond(0).

source
TestReports.update_testset_properties! — Method
update_testset_properties!(childts::AbstractTestSet, ts::AbstractTestSet)

Adds properties of ts to childts. If any properties being added already exist in childts, a warning is displayed and the value in ts is overwritten.

If the types of ts and\or childts do not a method defined for TestReports.properties, this is handled as follows:

  • If method not defined for typeof(ts), it has no properties to add to childts and therefore nothing happens.
  • If method not defined for typeof(chidlts) and ts has properties, then a warning is shown.

See also: properties

source

XML Writing

TestReports.add_testsuite_properties! — Method
add_testsuite_properties!(x_testsuite, ts::AbstractTestSet)

Add all key value pairs in the properties field of a AbstractTestSet to the corresponding testsuite xml element. This function assumes that the type of ts has a TestReports.properties method defined.

See also: properties

source
TestReports.check_ts — Method
check_ts(ts::AbstractTestSet)

Throws an exception if ts does not have the right structure for report or if the results of ts do not have both description or results fields.

See also: report

source
TestReports.error_xml — Method
failure_xml(message, test_type, content)

Create an error node (which will be the child of a testcase).

source
TestReports.format_period_for_xml — Method

formatperiodfor_xml(time::Millisecond)

Formats a millisecond value into a string in seconds with 3 decimal places.

@sprintf is used to ensure that value does not use scientific notification, which is not allowed in an XML decimal.

Methods for other Periods have not been written as are not currently required.

source
TestReports.get_error_info — Method
get_error_info(v::Error)

Return message and type of error for testcase attribute, and number of tests (either 1 or 0). Uses test_type field to determine what caused the original error.

source
TestReports.get_failure_message — Method

getfailuremessage(v::Fail)

Return message for failed test testcase attribute. Uses test_type field to determine what caused the original failure.

source
TestReports.set_attribute! — Method
set_attribute!(node, attr, val)
+set_attribute!(node, attr, val::Period)

Add the attritube with name attr and value val to node.

If val is of type Period, format with format_period_for_xml.

source
TestReports.testcase_xml — Method
testcase_xml(name, id, x_children)

Create a testcase element of a JUnit XML.

This is the generic form (with name, id and children) that is used by other methods.

source
TestReports.testcase_xml — Method
testcase_xml(v::Result, childs)

Create a testcase element of a JUnit XML for the result v.

The original expression of the test is used as the name, whilst the id is defaulted to testcaseid_.

source
TestReports.to_xml — Method
to_xml(ts::AbstractTestSet)

Create a testsuite node from a AbstractTestSet, by creating nodes for each result in ts.results. For creating a JUnit XML, all results must be AbstractResults, that is they cannot be AbstractTestSets, as the XML cannot have one testsuite nested inside another.

source
TestReports.to_xml — Method
to_xml(res::Pass)
 to_xml(res::Fail)
 to_xml(res::Broken)
 to_xml(res::Error)
-to_xml(res::ReportingResult)

Create a testcase node from the result and return it along with information on number of tests.

source

Index

+to_xml(res::ReportingResult)

Create a testcase node from the result and return it along with information on number of tests.

source

Index

diff --git a/previews/PR109/manual/index.html b/previews/PR109/manual/index.html index 4afca11..d7b096c 100644 --- a/previews/PR109/manual/index.html +++ b/previews/PR109/manual/index.html @@ -59,4 +59,4 @@ <testsuite name="TopLevel" tests="1" failures="0" errors="0" time="0.156" timestamp="2022-03-30T07:55:12.173" hostname="hostname" id="2"> <testcase name="1 == 1" id="1" classname="TopLevel" time="0.000"/> </testsuite> -</testsuites>

Each test is recorded in a separate testsuite with the name showing the original nesting.

Custom TestSet Types

TestReports.jl has not been tested significantly with custom TestSet types, please raise an issue if you find any problems/have a request.

However at a minimum, for a custom TestSet type to work with TestReports it must:

The following information in a JUnit XML relies on the functionality of ReportingTestSets but can be added to your own custom TestSet as described in the table.

InformationDescription
testcase timeThis is extracted from a ReportingResult by the TestReports.time_taken function. For standard Results, rather than ReportingResults, this function returns Dates.Millisecond(0). This function can be extended for other custom Result types.
testsuite timeThis is extracted from a TestSet by the TestReports.time_taken function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to Dates.Millisecond(0).
testsuite timestampThis is extracted from a TestSet by the TestReports.start_time function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to Dates.now().
testsuite hostnameThis is extracted from a TestSet by the TestReports.hostname function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to gethostname().
testsuite propertiesThis is extracted from a TestSet by the TestReports.properties function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to nothing.

For further details on extending these fuctions, see the docstrings in TestSets.

The source code of TestReports can be used as a starting point for including this behaviour in your custom TestSets.

If no TestSet types are specified (as per the standard Test approach), TestSet functionality will ensure that all child TestSets inherit the ReportingTestSet type.

+</testsuites>

Each test is recorded in a separate testsuite with the name showing the original nesting.

Custom TestSet Types

TestReports.jl has not been tested significantly with custom TestSet types, please raise an issue if you find any problems/have a request.

However at a minimum, for a custom TestSet type to work with TestReports it must:

The following information in a JUnit XML relies on the functionality of ReportingTestSets but can be added to your own custom TestSet as described in the table.

InformationDescription
testcase timeThis is extracted from a ReportingResult by the TestReports.time_taken function. For standard Results, rather than ReportingResults, this function returns Dates.Millisecond(0). This function can be extended for other custom Result types.
testsuite timeThis is extracted from a TestSet by the TestReports.time_taken function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to Dates.Millisecond(0).
testsuite timestampThis is extracted from a TestSet by the TestReports.start_time function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to Dates.now().
testsuite hostnameThis is extracted from a TestSet by the TestReports.hostname function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to gethostname().
testsuite propertiesThis is extracted from a TestSet by the TestReports.properties function, which can be extended for custom TestSets. If not extended, the AbstractTestSet method will be used and the value defaults to nothing.

For further details on extending these fuctions, see the docstrings in TestSets.

The source code of TestReports can be used as a starting point for including this behaviour in your custom TestSets.

If no TestSet types are specified (as per the standard Test approach), TestSet functionality will ensure that all child TestSets inherit the ReportingTestSet type.