mirrors/llvm-project
0
0
Fork 0
mirror of https://github.com/llvm/llvm-project.git synced 2025-05-04 04:46:07 +00:00
Code Issues Projects Releases Wiki Activity
llvm-project/llvm/utils/lit/tests/googletest-format.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

43 lines
1.4 KiB
Python
Raw Normal View History

[lit] Add test coverage of gtest format. llvm-svn: 188417
2013-08-14 22:21:11 +00:00
# Check the various features of the GoogleTest format.
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# RUN: not %{lit} -v --order=random %{inputs}/googletest-format > %t.out
[lit] Dump some FileCheck inputs to try to debug some failing tests llvm-svn: 309400
2017-07-28 16:24:18 +00:00
# FIXME: Temporarily dump test output so we can debug failing tests on
# buildbots.
# RUN: cat %t.out
[lit] Add test coverage of gtest format. llvm-svn: 188417
2013-08-14 22:21:11 +00:00
# RUN: FileCheck < %t.out %s
#
# END.
# CHECK: -- Testing:
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# CHECK: FAIL: googletest-format :: [[PATH:[Dd]ummy[Ss]ub[Dd]ir/]][[FILE:OneTest\.py]]/0
# CHECK: *** TEST 'googletest-format :: [[PATH]][[FILE]]/0{{.*}} FAILED ***
# CHECK-NEXT: Script(shard):
# CHECK-NEXT: --
[lit][test] relaxed GTEST_TOTAL_SHARDS checking for some googletests For machines with a small number of cores, GTEST_TOTAL_SHARDS may be lower than 6.
2022-06-27 16:12:45 -07:00
# CHECK-NEXT: GTEST_OUTPUT=json:{{[^[:space:]]*}} GTEST_SHUFFLE=1 GTEST_TOTAL_SHARDS={{[1-6]}} GTEST_SHARD_INDEX=0 GTEST_RANDOM_SEED=123 {{.*}}[[FILE]]
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# CHECK-NEXT: --
# CHECK-EMPTY:
[lit] Print full googletest commad line Similar to regular output of LIT tests: https://github.com/llvm/llvm-project/blob/c162f086ba632ffaedfe92d63bf21571bc8ae4da/llvm/utils/lit/lit/TestRunner.py#L1569 Differential Revision: https://reviews.llvm.org/D102899
2021-05-21 11:27:44 -07:00
# CHECK-NEXT: Script:
# CHECK-NEXT: --
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# CHECK-NEXT: [[FILE]] --gtest_filter=FirstTest.subTestB
[lit] Print full googletest commad line Similar to regular output of LIT tests: https://github.com/llvm/llvm-project/blob/c162f086ba632ffaedfe92d63bf21571bc8ae4da/llvm/utils/lit/lit/TestRunner.py#L1569 Differential Revision: https://reviews.llvm.org/D102899
2021-05-21 11:27:44 -07:00
# CHECK-NEXT: --
[lit] Keep stdout/stderr when using GoogleTest format When a unit test crashes or timeout, print the shard's stdout and stderr. When a unit test fails, attaches the test's output to the LIT output to help debugging. While at it, concatenating shard's environment variables using space instead of newline to make the reproducer script user friendly. Based on D123797. (Thanks to @lenary)
2022-04-23 19:30:00 -07:00
# CHECK-NEXT: I am subTest B output
# CHECK-EMPTY:
[lit] Add test coverage of gtest format. llvm-svn: 188417
2013-08-14 22:21:11 +00:00
# CHECK-NEXT: I am subTest B, I FAIL
# CHECK-NEXT: And I have two lines of output
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# CHECK-EMPTY:
# CHECK: Script:
[lit] Print full googletest commad line Similar to regular output of LIT tests: https://github.com/llvm/llvm-project/blob/c162f086ba632ffaedfe92d63bf21571bc8ae4da/llvm/utils/lit/lit/TestRunner.py#L1569 Differential Revision: https://reviews.llvm.org/D102899
2021-05-21 11:27:44 -07:00
# CHECK-NEXT: --
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# CHECK-NEXT: [[FILE]] --gtest_filter=FirstTest.subTestD
[lit] Print full googletest commad line Similar to regular output of LIT tests: https://github.com/llvm/llvm-project/blob/c162f086ba632ffaedfe92d63bf21571bc8ae4da/llvm/utils/lit/lit/TestRunner.py#L1569 Differential Revision: https://reviews.llvm.org/D102899
2021-05-21 11:27:44 -07:00
# CHECK-NEXT: --
[Reland][lit] Use sharding for GoogleTest format This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation). The shards are executed by the test runner and the results are stored in the json format supported by the GoogleTest. Later in the test reporting stage, all test results in the json file are retrieved to continue the test results summary etc. On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s. On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s. Reviewed By: yln, rnk, abrachet Differential Revision: https://reviews.llvm.org/D122251
2022-04-12 12:09:34 -07:00
# CHECK-NEXT: unresolved test result
# CHECK: ***
# CHECK: Unresolved Tests (1):
# CHECK-NEXT: googletest-format :: [[PATH]][[FILE]]/FirstTest/subTestD
# CHECK: ***
# CHECK-NEXT: Failed Tests (1):
# CHECK-NEXT: googletest-format :: [[PATH]][[FILE]]/FirstTest/subTestB
[NFC][lit] Add skipped test into upstream format Missing from D102694
2021-05-21 11:22:31 -07:00
# CHECK: Skipped{{ *}}: 1
[nfc][lit] Relax spacing check
2021-05-21 11:20:34 -07:00
# CHECK: Passed{{ *}}: 3
[NFC][lit] Add missing UNRESOLVED test D102899 will change it behavour.
2021-05-21 11:27:44 -07:00
# CHECK: Unresolved{{ *}}: 1
[nfc][lit] Relax spacing check
2021-05-21 11:20:34 -07:00
# CHECK: Failed{{ *}}: 1
Reference in New Issue Copy Permalink