6 Commits

Author SHA1 Message Date
Stella Stamenova
4f780ea126 [lit] Disable shtest-timeout on Windows
Summary: This is the only test that is still failing on Windows - or rather, it is expected to fail on the bots, but passes on the new bot that we're preparing causing a failure, so I'm going to disable it. Since the test has rarely, if ever, passed on the bots, this should have the same effect and it will unblock the creation of the new bot.

Reviewers: asmith, delcypher, zturner

Subscribers: stella.stamenova, llvm-commits

Differential Revision: https://reviews.llvm.org/D51871

llvm-svn: 341856
2018-09-10 20:24:05 +00:00
Stella Stamenova
8f8832d261 [lit, shtest-timeout] Always use an internal shell for the shtest-timeout to diagnose buildbot failures
Summary:
Right now this test is failing on the builtbots on Windows but we have a very similar setup where the test passes. The test is meant to test that specifying a timeout works correctly by running an infnite loop and having it timeout - on the buildbot, the infinite loop doesn't actually execute. This change runs all of the tests in the set using an internal shell rather than an external shell. I expect this will make the test pass which means that either the way the external shell is invoked or the external shell setup on the buildbots is not correct. Regardless of whether the test passes with this change, we'll need to undo this change and have a real fix.

@gkistanova was able to get logs from the buildbot to rule out a number of theories as to why this test is failing, but they didn't have enough information to confirm exactly what the issue is. The purpose of this change is to narrow it down, but if someone has a local repro and can aid in debugging, that would make it much speedier (and less prone to making the bots fail).

Reviewers: gkistanova, asmith, zturner, modocache, rnk, delcypher

Reviewed By: rnk

Subscribers: delcypher, llvm-commits, gkistanova

Differential Revision: https://reviews.llvm.org/D51326

llvm-svn: 340840
2018-08-28 16:24:55 +00:00
Stella Stamenova
8efc95ac2f [lit, python3] Update lit error logging to work correctly in python3 and other test fixes
Summary:
In Python2 'unicode' is a distinct type from 'str', but in Python3 'unicode' does not exist and instead all 'str' objects are Unicode string. This change updates the logic in the test logging for lit to correctly process each of the types, and more importantly, to not just fail in Python3.

This change also reverses the use of quotes in several of the cfg files. By using '""' we are guaranteeing that the resulting path will work correctly on Windows while "''" only works correctly sometimes. This also fixes one of the failing tests.

Reviewers: asmith, zturner

Subscribers: stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D50397

llvm-svn: 339179
2018-08-07 20:54:38 +00:00
Stella Stamenova
cc2404c01d [lit, python] Always add quotes around the python path in lit
Summary:
The issue with the python path is that the path to python on Windows can contain spaces. To make the tests always work, the path to python needs to be surrounded by quotes.

This change updates several configuration files which specify the path to python as a substitution and also remove quotes from existing tests.

Reviewers: asmith, zturner, alexshap, jakehehrlich

Reviewed By: zturner, alexshap, jakehehrlich

Subscribers: mehdi_amini, nemanjai, eraman, kbarton, jakehehrlich, steven_wu, dexonsmith, stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D50206

llvm-svn: 339073
2018-08-06 22:37:44 +00:00
Dan Liew
6dfcc78364 [lit] Try to make shtest-timeout.py test more reliable by using a
larger timeout value. This really isn't very good because it will
still be susceptible to machine performance.

While we are here also fix a bug in validation of
`maxIndividualTestTime` where previously it wasn't checked if the
type was an int.

rdar://problem/40221572

llvm-svn: 332987
2018-05-22 15:06:29 +00:00
Dan Liew
7574241053 [lit] Implement support of per test timeout in lit.
This should work with ShTest (executed externally or internally) and GTest
test formats.

To set the timeout a new option ``--timeout=`` has
been added which specifies the maximum run time of an individual test
in seconds. By default this 0 which causes no timeout to be enforced.

The timeout can also be set from a lit configuration file by modifying
the ``lit_config.maxIndividualTestTime`` property.

To implement a timeout we now require the psutil Python module if a
 timeout is requested. This dependency is confined to the newly added
 ``lit.util.killProcessAndChildren()``. A note has been added into the
 TODO document describing how we can remove the dependency on the
 ``pustil`` module in the future. It would be nice to remove this
 immediately but that is a lot more work and Daniel Dunbar believes it is
better that we get a working implementation first and then improve it.

To avoid breaking the existing behaviour the psutil module will not be
imported if no timeout is requested.

The included testcases are derived from test cases provided by
 Jonathan Roelofs which were in an previous attempt to add a per test
 timeout to lit (http://reviews.llvm.org/D6584). Thanks Jonathan!

Reviewers: ddunbar, jroelofs, cmatthews, MatzeB

Subscribers: cmatthews, llvm-commits

Differential Revision: http://reviews.llvm.org/D14706

llvm-svn: 256471
2015-12-27 14:03:49 +00:00