Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pretty formatted XML #111

Merged
merged 8 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name = "TestReports"
uuid = "dcd651b4-b50a-5b6b-8f22-87e9f253a252"
version = "0.7.3"
version = "1.0.0"

[deps]
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
Expand Down
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,15 @@ To add to CI:
$ julia -e 'using Pkg; Pkg.add("TestReports"); using TestReports; TestReports.test("MyPackage")'
```

Additionally, properties can be added to your `TestSet`s. To do this, use the `recordproperty`
function like so:
Additionally, properties can be associated with your testsets or tests. To do this, use the `record_testset_property` or `record_test_property` functions within a testset:

```julia
using Test
using TestReports

@testset "MyTests" begin
recordproperty("ID", 1)
@test 1==1
record_testset_property("ID", 1)
@test 1 == 1
end
```

Expand Down
7 changes: 3 additions & 4 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,15 @@ To add to CI:
$ julia -e 'using Pkg; Pkg.add("TestReports"); using TestReports; TestReports.test("MyPackage")'
```

Additionally, properties can be added to your `TestSet`s. To do this, use the `recordproperty`
function like so:
Additionally, properties can be associated with your testsets or tests. To do this, use the `record_testset_property` or `record_test_property` functions within a testset:

```julia
using Test
using TestReports

@testset "MyTests" begin
recordproperty("ID", 1)
@test 1==1
record_testset_property("ID", 1)
@test 1 == 1
end
```

Expand Down
3 changes: 2 additions & 1 deletion docs/src/library.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ Documentation for `TestReports.jl`'s public interface.

```@docs
TestReports.test
recordproperty
record_testset_property
record_test_property
ReportingTestSet
any_problems
report
Expand Down
34 changes: 16 additions & 18 deletions docs/src/manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,30 +77,26 @@ julia> TestReports.test(["MyPackage1", "MyPackage2"], logfilename=["1.xml", "2.x
# File paths are joinpath("path", "1.xml") and joinpath("path", "2.xml")
```

## Adding Properties
## Associating Properties

Properties can be added to a `TestSet` using the exported function `recordproperty`
within `runtests.jl` and other `include`d scripts:
Properties can be associated to a testsets and/or tests by using the respective
`record_testset_property` and `record_test_property` functions within a testset:

```julia
using Test
using TestReports

@testset "MyTests" begin
recordproperty("ID", 1)
@test 1==1
record_testset_property("ID", 1)
@test 1 == 1
end
```

`recordproperty` will have no affect during normal unit testing.

The added properties will be added to the corresponding testsuite in the generated report.
Multiple properties can be added, and a property added to a parent `TestSet` will be applied
to all child `TestSet`s.
The `record_*_property` functions will have no affect during normal unit testing.

Properties with the same name are allowed to be set multiple times within the same
`TestSet`. If both a parent and a child set the same named property both properties will
appear in the child when generating the report.
The associated properties will be added to the corresponding testsuite or testcase in the
generated report. Multiple properties can be added, and a properties added to a parent
testset will be applied to all child testsets. Duplicate properties are allowed to be set.

The property name must always be a `String`, but the value can be anything that is serializable
by `EzXML.jl`. In practice this means that `String`s, `Number`s, `Expr`s and `Symbols` can be used,
Expand All @@ -113,19 +109,21 @@ using Test
using TestReports

@testset "TopLevelTests" begin
recordproperty("TestFile" @__FILE__) # This will be added to all child testsets in report
# This will be added to all child testsets in report. May not be appropriate when using
# `include`.
record_testset_property("TestFile", @__FILE__)

@testset "MiddleLevelTests" begin
recordproperty("Testsuite", 100)
recordproperty("TestSubject", "Example")
record_testset_property("Testsuite", 100)
record_test_property("TestSubject", "Example")

@testset "Inner" begin
recordproperty("Testsuite", 101) # Both testsuite 100 and 101 apply to this testset
record_testset_property("Testsuite", 101) # Associate with both testsuite 100 and 101
@test 1 == 1
end

@testset "Types" begin
recordproperty("Prop1", :Val1)
record_test_property("Prop1", :Val1)
@test 1 == 1
end
end
Expand Down
4 changes: 2 additions & 2 deletions src/TestReports.jl
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ else
using Pkg.Types: SHA1
end

export ReportingTestSet, any_problems, report, recordproperty
export ReportingTestSet, any_problems, report, record_test_property, record_testset_property

const TESTREPORTS_VERSION = let # Copied from Documenter.jl
project = joinpath(dirname(dirname(pathof(TestReports))), "Project.toml")
Expand All @@ -50,6 +50,6 @@ include("./testsets.jl")
include("to_xml.jl")
include("compat_check.jl")
include("runner.jl")
include("recordproperty.jl")
include("properties.jl")

end # module
81 changes: 81 additions & 0 deletions src/properties.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
"""
record_testset_property(name::AbstractString, value)

Associates a property with a testset. The `name` and `value` will be turned into a
`<property>` element within the corresponding `<testsuite>` element within the JUnit XML
report.

Multiple properties can be added to one testset and child testsets inherit the properties
defined by their parents. If a child testset records a property which is already set both
will be present in the resulting report.

The suggested use of this function is to place it inside a testset with unspecified type
(see Examples). This will ensure that `Pkg.test` is unnaffected, but that the properties
are added to the report when `TestReports.test` is used. This is because properties are
only added when the `Testset` type has a `TestReports.testset_properties` method defined, as
does the `ReportingTestSet` used by `TestReports`. `TestReports.testset_properties` can be
extended for custom `TestSet`s.

If a child testset has this method defined but its parent doesn't, the property should
be in the report when `TestReport.test` is used, assuming that the parent testset
type doesn't do anything to affect the reporting behaviour. However this is not tested
functionality.

The `value` argument must be serializable by EzXML, which gives quite a lot of freedom.

# Examples
```julia
using TestReports

# Default testset used, so function will not affect Pkg.test but will be used when
# generating JUnit XML.
@testset "MyTestSet" begin
record_testset_property("ID", 42)
record_testset_property("File", @__FILE__)
record_testset_property("Bool", true)
@test 1 == 1
@test 2 == 2
end
```

See also: [`testset_properties`](@ref), [`record_test_property`](@ref).
"""
function record_testset_property(name::AbstractString, value)
record_testset_property!(get_testset(), name, value)
return value
end

"""
record_test_property(name::AbstractString, value)

Associates a property with the tests contained with the testset. The `name` and `value` will
be turned into a `<property>` element with the corresponding `<testcase>` element within the
JUnit XML report.

Multiple test properties can be assigned within a testset and child testsets will inherit
the test properties defined by their parents. If a child testset records a test property
with an already used name both properties will be present in the resulting report.

For more details see the documentation for [`record_testset_property`](@ref).

# Examples
```julia
using TestReports

# Default testset used, record property call is ignored by `Pkg.test` but will be used when
# generating JUnit XML.
@testset "MyTestSet" begin
record_test_property("ID", 42)
record_test_property("File", @__FILE__)
record_test_property("Bool", true)
@test 1 == 1
@test 2 == 2
end
```

See also: [`test_properties`](@ref), [`record_testset_property`](@ref).
"""
function record_test_property(name::AbstractString, value)
record_test_property!(get_testset(), name, value)
return value
end
46 changes: 0 additions & 46 deletions src/recordproperty.jl

This file was deleted.

Loading
Loading