mirrors/llvm-project
0
0
Fork 0
mirror of https://github.com/llvm/llvm-project.git synced 2025-05-03 22:26:05 +00:00
Code Issues Projects Releases Wiki Activity
llvm-project/llvm/utils/lit/tests/xunit-output.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

28 lines
1.5 KiB
Python
Raw Normal View History

Mark test with "REQUIRES: shell" since it directly invokes "sh" and was failing on Windows. llvm-svn: 332563
2018-05-17 01:36:25 +00:00
# REQUIRES: shell
Add a test-case for lit xunit output llvm-svn: 223307
2014-12-03 23:01:35 +00:00
# Check xunit output
Use not to catch unexpected pass as well as remove old test results As per review feedback, make sure we rm temp files, and make the return code checking for lit more specific. llvm-svn: 332423
2018-05-16 00:33:29 +00:00
# RUN: rm -rf %t.xunit.xml
# RUN: not %{lit} --xunit-xml-output %t.xunit.xml %{inputs}/xunit-output
Escape ]]> in xunit xml output Summary: This sequence ends the CDATA block so any characters after that are no longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>". Reviewers: cmatthews Reviewed By: cmatthews Differential Revision: https://reviews.llvm.org/D46886 llvm-svn: 332440
2018-05-16 09:00:28 +00:00
# If xmllint is installed verify that the generated xml is well-formed
# RUN: sh -c 'if command -v xmllint 2>/dev/null; then xmllint --noout %t.xunit.xml; fi'
Add a test-case for lit xunit output llvm-svn: 223307
2014-12-03 23:01:35 +00:00
# RUN: FileCheck < %t.xunit.xml %s
[lit] Include unexecuted tests in xUnit report Pass in all discovered tests to report generators. The XunitReport generator now creates testcase items for unexecuted tests and documents why they have been skipped. This makes it easier to compare test runs with different filters or configurations, or across platforms. I don't know who is using the JsonReport generator and what the expectations there are (it doesn't have tests), so decided to preserve the old behavior by filtering out the unexecuted tests. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D81316
2020-06-05 15:45:43 -07:00
# CHECK: <?xml version="1.0" encoding="UTF-8"?>
[lit] Include total elapsed time in xunit output The time= attribute can also be used for the testsuites tag. While this attribute appears to be ignored by Jenkins (https://github.com/jenkinsci/junit-plugin/blob/bab34bcc96154a494f8c371953efe06d45813f67/src/main/java/hudson/tasks/junit/SuiteResult.java#L202), it is still useful if you manually inspect the xml file. Reviewed By: yln Differential Revision: https://reviews.llvm.org/D84230
2020-07-23 10:45:55 +01:00
# CHECK-NEXT: <testsuites time="{{[0-9.]+}}">
[lit] Include unexecuted tests in xUnit report Pass in all discovered tests to report generators. The XunitReport generator now creates testcase items for unexecuted tests and documents why they have been skipped. This makes it easier to compare test runs with different filters or configurations, or across platforms. I don't know who is using the JsonReport generator and what the expectations there are (it doesn't have tests), so decided to preserve the old behavior by filtering out the unexecuted tests. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D81316
2020-06-05 15:45:43 -07:00
# CHECK-NEXT: <testsuite name="test-data" tests="5" failures="1" skipped="3">
# CHECK-NEXT: <testcase classname="test-data.test-data" name="bad&amp;name.ini" time="{{[0-1]\.[0-9]+}}">
# CHECK-NEXT: <failure><![CDATA[& < > ]]]]><![CDATA[> &"]]></failure>
# CHECK-NEXT: </testcase>
# CHECK-NEXT: <testcase classname="test-data.test-data" name="excluded.ini" time="{{[0-1]\.[0-9]+}}">
[lit] Don't include tests skipped due to sharding in reports When running multiple shards, don't include skipped tests in the xunit output since merging the files will result in duplicates. In our CHERI Jenkins CI, I configured the libc++ tests to run using sharding (since we are testing using a single-CPU QEMU). We then merge the generated XUnit xml files to produce a final result, but if the individual XMLs report tests excluded due to sharding each test is included N times in the final result. This also makes it difficult to find the tests that were skipped due to missing REQUIRES: etc. Reviewed By: yln Differential Revision: https://reviews.llvm.org/D84235
2020-07-26 11:39:22 +01:00
# CHECK-NEXT: <skipped message="Test not selected (--filter, --max-tests)"/>
[lit] Include unexecuted tests in xUnit report Pass in all discovered tests to report generators. The XunitReport generator now creates testcase items for unexecuted tests and documents why they have been skipped. This makes it easier to compare test runs with different filters or configurations, or across platforms. I don't know who is using the JsonReport generator and what the expectations there are (it doesn't have tests), so decided to preserve the old behavior by filtering out the unexecuted tests. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D81316
2020-06-05 15:45:43 -07:00
# CHECK-NEXT: </testcase>
# CHECK-NEXT: <testcase classname="test-data.test-data" name="missing_feature.ini" time="{{[0-1]\.[0-9]+}}">
# CHECK-NEXT: <skipped message="Missing required feature(s): dummy_feature"/>
# CHECK-NEXT: </testcase>
# CHECK-NEXT: <testcase classname="test-data.test-data" name="pass.ini" time="{{[0-1]\.[0-9]+}}"/>
# CHECK-NEXT: <testcase classname="test-data.test-data" name="unsupported.ini" time="{{[0-1]\.[0-9]+}}">
# CHECK-NEXT: <skipped message="Unsupported configuration"/>
# CHECK-NEXT: </testcase>
# CHECK-NEXT: </testsuite>
# CHECK-NEXT: </testsuites>
Reference in New Issue Copy Permalink