I have a Spock @Unroll
test like this:
@Unroll
def 'A session uses the TLS protocol #enabledProtocolsOfSocket #protocolOfSession'() {
given:
// ...
expect:
// ...
where:
enabledProtocolsOfSocket | protocolOfSession
["TLSv1.3"] | "TLSv1.3"
["TLSv1.2", "TLSv1.3"] | "TLSv1.3"
["TLSv1.2"] | "TLSv1.2"
["TLSv1.2", "TLSv1.1"] | "TLSv1.2"
["TLSv1.1", "TLSv1.2"] | "TLSv1.2"
["TLSv1.1"] | "NONE"
["TLSv1"] | "NONE"
["TLSv1", "TLSv1.1"] | "NONE"
["TLSv1.1", "TLSv1"] | "NONE"
}
which leads to this JUnit result XML:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="com.acme.TlsProtocolSpec" tests="9" skipped="0" failures="0" errors="0" timestamp="2024-07-19T17:23:47" hostname="acme-build5eb650" time="0.403">
<properties/>
<testcase name="A session uses the TLS protocol [TLSv1.3] TLSv1.3" classname="com.acme.TlsProtocolSpec" time="0.081"/>
<testcase name="A session uses the TLS protocol [TLSv1.2, TLSv1.3] TLSv1.3" classname="com.acme.TlsProtocolSpec" time="0.037"/>
<testcase name="A session uses the TLS protocol [TLSv1.2] TLSv1.2" classname="com.acme.TlsProtocolSpec" time="0.084"/>
<testcase name="A session uses the TLS protocol [TLSv1.2, TLSv1.1] TLSv1.2" classname="com.acme.TlsProtocolSpec" time="0.07"/>
<testcase name="A session uses the TLS protocol [TLSv1.1, TLSv1.2] TLSv1.2" classname="com.acme.TlsProtocolSpec" time="0.071"/>
<testcase name="A session uses the TLS protocol [TLSv1.1] NONE" classname="com.acme.TlsProtocolSpec" time="0.014"/>
<testcase name="A session uses the TLS protocol [TLSv1] NONE" classname="com.acme.TlsProtocolSpec" time="0.013"/>
<testcase name="A session uses the TLS protocol [TLSv1, TLSv1.1] NONE" classname="com.acme.TlsProtocolSpec" time="0.013"/>
<testcase name="A session uses the TLS protocol [TLSv1.1, TLSv1] NONE" classname="com.acme.TlsProtocolSpec" time="0.013"/>
<system-out><![CDATA[]]></system-out>
<system-err><![CDATA[]]></system-err>
</testsuite>
The problem is: I cannot map these results to the test in post-processing the result file – there is no result named A session uses the TLS protocol #enabledProtocolsOfSocket #protocolOfSession
(due to the placeholders) and there are no tests named A session uses the TLS protocol [TLSv1.2, TLSv1.3] TLSv1.3
or … or … (due to the placeholders being replaced).
Is there some feature to “group” or “merge” these test runs into one result, named as the test? I’m thinking of something like xUnit’s <collection>
Element, which contains one <test>
child node for each test run, like so:
<collection total="9" passed="9" failed="0" skipped="0" name="Test collection for Com.Acme.SomeClass.Tests.SomeTestName" time="0.484">
<test name="Com.Acme.SomeClass.Tests.SomeTestName(someParam: "10")" ...>
</test>
<test name="Com.Acme.SomeClass.Tests.SomeTestName(someParam: "11")" ...>
</test>
...
</collection>
But any solution to preserve the relation of the generically named test to all test runs (i.e. if just one test run fails, the test is reported failing) is welcome, be it by configuring the JUnit output or by some Spock feature.