217 Commits

Author SHA1 Message Date
Todd Fiala
3a7fcbb441 test infra: support test reruns in xunit formatter.
llvm-svn: 255705
2015-12-15 23:25:56 +00:00
Enrico Granata
f85b008de8 Enable the 'type X list' formatters commands to list formatters in language categories
llvm-svn: 255687
2015-12-15 22:20:19 +00:00
Todd Fiala
6d3de12315 test-infra: refactored new summary results into base ResultsFormatter class
This allows more specialized formatters to still reuse the results
summarization display from the base class.

llvm-svn: 255676
2015-12-15 21:33:38 +00:00
Tamas Berghammer
0ecdae1bdc Merge ENABLE_THREADS and ENABLE_STD_THREADS markers
Both of these markers are used in the test suit for annotating when a
test needs multi threaded support. Previously they had slightly
different meening but they converged to the point where they are used
interchangably. This CL removes the ENABLE_STD_THREADS one to simplify
the test suite and avoid some confusion.

Differential revision: http://reviews.llvm.org/D15498

llvm-svn: 255641
2015-12-15 12:11:00 +00:00
Siva Chandra
a386358c36 Make few adjustments after r255542.
Reviewers: zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15511

llvm-svn: 255584
2015-12-15 00:26:52 +00:00
Todd Fiala
101ed12712 test infra: catch and print exception info on test runner socket listener
This is the listener's spawned connection, not the listener itself.
(i.e. this is the test runner's receiving side of test event sockets).
A standard socket.error will just issue an INFO statement and continue.
Something other than a socket.error will get an ERROR: printed (and
also continue).

Hopefully this gets us more info and also handles the completely
to-be-expected scenario that the test inferior might go down at
any point.

llvm-svn: 255581
2015-12-14 23:45:38 +00:00
Kate Stone
2a83045ab1 [Test] Addresses failing test when path to make contains spaces
llvm-svn: 255568
2015-12-14 22:59:26 +00:00
Zachary Turner
8a927c4c10 Remove the multiplier loop.
This is leading to some kind of subtle issue related to local
functions and closures, so let's just go back to the old way for
now.

llvm-svn: 255567
2015-12-14 22:58:16 +00:00
Todd Fiala
f801290a91 Revert "Temporarily skip TestWithLimitDebugInfo on Darwin and OS X"
This reverts commit 30ed0826a1bb800454088ea1ae16c113a69b92b1.

llvm-svn: 255557
2015-12-14 22:04:20 +00:00
Todd Fiala
6ea44cd89b Temporarily skip TestWithLimitDebugInfo on Darwin and OS X
This test is erroring out on a sequence call to a function.

llvm-svn: 255549
2015-12-14 21:49:39 +00:00
Omair Javaid
798ea5b6ff Correction in TestFrames.py test for arm targets in thumb mode
Differential revision: http://reviews.llvm.org/D15061

llvm-svn: 255547
2015-12-14 21:41:18 +00:00
Todd Fiala
685a7570ec test infra: enable single-worker rerun phase for flakey tests.
Use of --rerun-all-issues will enable any test method failure, not just
test methods marked with the flakey decorator, to rerun.

Currently this does not change the flakey logic's immediate rerun
attempt.  I want to make sure this doesn't cause any significant issues
before changing that part.

The rerun reporting is only known to work properly with the
default (new) BasicResultsFormatter reporting.  Once we work out
any issues, I'll go back and make sure the curses output handles
it properly as well.

llvm-svn: 255543
2015-12-14 21:28:46 +00:00
Zachary Turner
e1eb5e39f5 Make skipIf decorator support not_in() functor.
llvm-svn: 255542
2015-12-14 21:26:49 +00:00
Zachary Turner
f098e4fb19 Make debug info specification use categories system.
Reviewed By: Tamas Berghammer, Pavel Labath
Differential Revision: http://reviews.llvm.org/D15428

llvm-svn: 255525
2015-12-14 18:49:16 +00:00
Pavel Labath
ffbf9e86b2 Make test categories composable
Summary:
Previously the add_test_categories would simply overwrite the current set of categories for a
method. This change makes the decorator truly "add" categories, by extending the current set of
categories instead of replacing it.

To do this, I have:
- replaced the getCategories() property on a method (which was itself a method), with a simple
  list property "categories". This makes add_test_categories easier to implement, and test
  categories isn't something which should change between calls anyway.
- rewritten the getCategoriesForTest function to merge method categories with the categories of
  the test case. Previously, it would just use the method categories if they were present. I have
  also greatly simplified this method. Originally, it would use a lot of introspection to enable
  it being called on various types of objects. Based on my tests, it was only ever being called
  on a test case. The new function uses much less introspection then the preivous one, so we
  should easily catch any stray uses, if there are any, as they will generate exceptions now.

Reviewers: zturner, tfiala, tberghammer

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15451

llvm-svn: 255493
2015-12-14 13:17:18 +00:00
Pavel Labath
b8ea7a0ed3 Extend XFAIL on TestNamespaceLookup on linux
one of the tests seems to (occasionally) fail with clang as well.

llvm-svn: 255492
2015-12-14 12:09:28 +00:00
Pavel Labath
25cf6727d1 XFAIL TestNamespaceLookup for linux
llvm-svn: 255490
2015-12-14 11:05:44 +00:00
Mohit K. Bhakkad
1951f719df [LLDB][MIPS] Mark TestConcurrentEvents.py expected failure, as MIPS atomic sequences are yet to be supported in LLDB
Reviewers: jaydeep.
Subscribers: lldb-commits.
Differential Revision: http://reviews.llvm.org/D15488

llvm-svn: 255488
2015-12-14 10:26:18 +00:00
Dawn Perchik
b6d737c9b8 Test case for "Fix scope-based lookup when more than one function is found."
Missed commit in r255439.
Differential Revision: http://reviews.llvm.org/D15312

llvm-svn: 255440
2015-12-12 20:44:08 +00:00
Dawn Perchik
b59257841e Fix scope-based lookup when more than one function is found.
When multiple functions are found by name, lldb removes duplicate entries of
functions with the same type, so the first function in the symbol context list
is chosen, even if it isn't in scope. This patch uses the declaration context
of the execution context to select the function which is in scope.

This fixes cases like the following:

    int func();
    namespace ns {
	int func();
	void here() {
	    // Run to BP here and eval 'p func()';
	    // lldb used to find ::func(), now finds ns::func().
	}
    }

Reviewed by: clayborg
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D15312

llvm-svn: 255439
2015-12-12 19:31:41 +00:00
Todd Fiala
93153922db test infra: adds book-keeping for rerunnable tests
Also adds full path info for exceptional exits and timeouts when
no test method is currently running.

Adds --rerun-all-issues command line arg.  If specified, all
test issues are eligible for rerun.  If not specified, only tests
marked flakey are eligible for rerun.

The actual rerunning will occur in an upcoming change.  This
change just handles tha accounting of what should be rerun.

llvm-svn: 255438
2015-12-12 19:26:56 +00:00
Todd Fiala
d06a9c9f61 Decouple test execution and test finder logic in parallel test runner.
llvm-svn: 255400
2015-12-12 00:34:57 +00:00
Todd Fiala
9a58133698 Use new set when checking if a test event type matches the job/test result types
llvm-svn: 255385
2015-12-11 22:45:52 +00:00
Todd Fiala
b68dbfa227 Revert "Revert "Turn on new test summary results by default.""
This reverts commit f994b46a2028c8a8b9b55fe010a95122bca07540.

llvm-svn: 255381
2015-12-11 22:29:34 +00:00
Adrian McCarthy
cf6177131a Add some tests for stack and local variable inspection for mini dumps.
Differential Revision: http://reviews.llvm.org/D15435

llvm-svn: 255379
2015-12-11 22:27:57 +00:00
Todd Fiala
a8fee7f981 Add expected timeout support to test event architecture.
llvm-svn: 255363
2015-12-11 19:44:23 +00:00
Zachary Turner
742afdb3d2 Remove -S option from dotest.py.
llvm-svn: 255361
2015-12-11 19:21:49 +00:00
Zachary Turner
6a188e6c1e Remove -r and -R options from dotest.py.
llvm-svn: 255360
2015-12-11 19:21:34 +00:00
Todd Fiala
9187f27e32 Add test event marking a test as explicitly eligible for rerun if it is marked flakey.
This will be used in a future change to support rerunning flakey tests
that hit a test result isue in a low-load, single worker test runner phase.

This is implemented as an additive-style event rather than being
evaluated and added to the start_test event because the decorator code
only runs after the start_test event is created and sent.  i.e.
LLDBTestResult.startTest() runs before the test method decorators run.

llvm-svn: 255351
2015-12-11 18:06:47 +00:00
Tamas Berghammer
e43482b626 Create test for llvm.org/pr25806
LLDB don't detect the loading of a shared object file linked against the
main executable before the static initializers are executed for the
given module. Because of this it is not possible to get breakpoint hits
in these static initializers and to display proper debug info in case of
a crash in these codes.

llvm-svn: 255342
2015-12-11 16:24:14 +00:00
Pavel Labath
734a0b3f81 XFAIL TestLoadUnload for Windows->Android
llvm-svn: 255341
2015-12-11 16:16:51 +00:00
Pavel Labath
027158fad7 Revert "Turn on new test summary results by default."
The new test summary formatter does not honor the "expected timeout" markings, which makes our
buildbots all red. I'm switching it off by default until we figure out a way to make this work.

llvm-svn: 255335
2015-12-11 11:05:24 +00:00
Todd Fiala
07206ea19d test result details now print module.class.test_name in verbose mode.
And, turns off verbose mode by default.  This must have been switched
on as the default when somebody was testing.

llvm-svn: 255310
2015-12-10 23:14:24 +00:00
Kamil Rytarowski
f5d34b7b9d Add NetBSD support in the buildDriver and buildLibrary routines
Summary: NetBSD is like FreeBSD and Linux in these routines.

Reviewers: clay.chang, tfiala, emaste, joerg

Subscribers: lldb-commits, emaste

Differential Revision: http://reviews.llvm.org/D15374

llvm-svn: 255308
2015-12-10 22:56:56 +00:00
Zachary Turner
923fdbadae Remove the -y option from dotest.py.
llvm-svn: 255280
2015-12-10 18:52:09 +00:00
Zachary Turner
70ed57389a Remove the -x option from dotest.py.
llvm-svn: 255279
2015-12-10 18:51:40 +00:00
Zachary Turner
21accab1a7 Remove deprecated command line options from dotest.py
llvm-svn: 255278
2015-12-10 18:51:21 +00:00
Zachary Turner
80310c29fb Remove the --output-on-success command line argument from dotest.
llvm-svn: 255277
2015-12-10 18:51:02 +00:00
Zachary Turner
3cfa31492c Remove the -T option from dotest.py.
llvm-svn: 255276
2015-12-10 18:50:49 +00:00
Zachary Turner
b08ab72427 Remove -w option from dotest.py.
llvm-svn: 255275
2015-12-10 18:50:32 +00:00
Todd Fiala
5d96dc5629 enable timeout/exceptional exit support for xUnit formatter
Also adds enable.py/disable.py script to simplify turning on and off
the issue_verification tests helpful for testing a results formatter.

llvm-svn: 255161
2015-12-09 22:02:31 +00:00
Zachary Turner
94ea56bc8b Remove the -P option from dotest.py
This was an option to display a graphical progress bar.  Nobody
is using this, and it doesn't work correctly anyway with the new
result formatter.

llvm-svn: 255153
2015-12-09 20:48:59 +00:00
Zachary Turner
35a7610a05 Delete the -n command line option from dotest.py.
This removes the option to not print some one time version and
diagnostic information when running the test suite.

llvm-svn: 255152
2015-12-09 20:48:42 +00:00
Zachary Turner
6c9e44413f Remove the -i command line option from dotest.py.
This is part of a larger effort to remove unused command line
options from dotest.py.

llvm-svn: 255151
2015-12-09 20:48:31 +00:00
Zachary Turner
d2a93fd6ca Delete the -F command line option from dotest.py.
This removes the failfast command line option as part of an effort
to simplify dotest and remove unused command line options.  You can
still Ctrl+C any time you want to exit early.

llvm-svn: 255150
2015-12-09 20:48:19 +00:00
Zachary Turner
76972031fa Remove -k command line option from dotest.py.
This is part of an effort to remove unused command line options.

llvm-svn: 255143
2015-12-09 19:45:51 +00:00
Zachary Turner
6c8733346a Remove -e option from dotest.py.
This is part of an effort to clean up dotest command line options
that are no longer used.

llvm-svn: 255142
2015-12-09 19:45:36 +00:00
Zachary Turner
c816013641 Remove -libcxx option from dotest.py
Nobody was using this, and plus it can be achieved just as well
by using -E to set an environment variable.

llvm-svn: 255141
2015-12-09 19:45:16 +00:00
Todd Fiala
d58476da79 Move XunitFormatter into its own xunit_formatter.py file.
llvm-svn: 255139
2015-12-09 19:32:14 +00:00
Todd Fiala
0a7c32b38e Fix new summary to include exceptional exit count in determining exit value
The main dotest.py should exit with a system return code of 1 on any
issue.  This change fixes a place where I omitted counting the
exceptional exit value to determine if we should return 1 when using the
new summary results.

This change also puts a banner around the Issue Details section that comes
before the Test Result Summary.

llvm-svn: 255138
2015-12-09 19:05:44 +00:00