Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect JUnit XML for error outside of test case #97

Closed
gbruer15 opened this issue Mar 10, 2024 · 1 comment · Fixed by #98
Closed

Incorrect JUnit XML for error outside of test case #97

gbruer15 opened this issue Mar 10, 2024 · 1 comment · Fixed by #98

Comments

@gbruer15
Copy link
Contributor

I encountered an issue trying to convert JUnit XML output from TestReports.jl to html using any package. None of them were displayed the result from an exception being thrown.

I couldn't find a clear definition for what the XML format is supposed to be, but I believe the element needs to be wrapped in a element for other packages to parse it correctly.

Below, I have example tests that throw errors and my proposed fix.

using Test
using TestReports

# Current behavior
ts = @testset ReportingTestSet "" begin
    @testset "Normal tests" begin
        @test true
        @test false
    end
    @testset "Error inside test" begin
        @test true
        @test false
        @test doesntexist == 1
    end
    @testset "Error outside test" begin
        @test true
        @test false
        a = doesntexist
    end
end
println("This XML puts <error> as direct children of <testsuite>")
TestReports.EzXML.prettyprint(report(ts))


# Update to put error result inside a test case.
function TestReports.to_xml(v::TestReports.Error)
    message, type, ntest = TestReports.get_error_info(v)
    x_error = TestReports.error_xml(message, type, v.backtrace)
    x_testcase = TestReports.testcase_xml(v, [x_error])
    x_testcase, ntest, 0, 1  # Increment number of errors by 1
end

ts = @testset ReportingTestSet "" begin
    @testset "Normal tests" begin
        @test true
        @test false
    end
    @testset "Error inside test" begin
        @test true
        @test false
        @test doesntexist == 1
    end
    @testset "Error outside test" begin
        @test true
        @test false
        a = doesntexist
    end
end
println("This XML puts <error> as direct children of <testcase>")
TestReports.EzXML.prettyprint(report(ts))
@oxinabox
Copy link
Member

Seems reasonable. Open a PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants