Summary:
LLDBTestResult.hardMarkAsSkipped marked the whole class as skipped when the first class in the
test failed the category check. This meant that subsequent tests in the same class did not run
even if they were passing the category filter. Fix that.
Reviewers: tfiala
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D22213
llvm-svn: 275173
Summary:
print build errors nicely in test output
This test infrastructure change adds a new Python exception
for test subject builds that fail. The output of the build
command is captured and propagated to both the textual test
output display code and to the test event system.
The ResultsFormatter objects have been modified to do something
more useful with this information. The xUnit formatter
now replaces the non-informative Python build error stacktrace
with the build error content. The curses ResultsFormatter
prints a 'B' for build errors rather than 'E'.
The xUnit output, in particular, makes it much easier for
developers to track down test subject build errors that cause
test failures when reports come in from CI.
Reviewers: granata.enrico
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D20252
llvm-svn: 269525
This change moves all the test event handling and its related
ResultsFormatter classes out of the packages/Python/lldbsuite/test dir
into a packages/Python/lldbsuite/test_event package. Formatters are
moved into a sub-package under that.
I am limiting the scope of this change to just the motion and a few
minor issues caught by a static Python checker (e.g. removing unused
import statements).
This is a pre-step for adding package-level tests to the test event
system. I also intend to simplify test event results formatter selection
after I make sure this doesn't break anybody.
See:
http://reviews.llvm.org/D19288
Reviewed by:
Pavel Labath
llvm-svn: 266885
Summary:
Previously the add_test_categories would simply overwrite the current set of categories for a
method. This change makes the decorator truly "add" categories, by extending the current set of
categories instead of replacing it.
To do this, I have:
- replaced the getCategories() property on a method (which was itself a method), with a simple
list property "categories". This makes add_test_categories easier to implement, and test
categories isn't something which should change between calls anyway.
- rewritten the getCategoriesForTest function to merge method categories with the categories of
the test case. Previously, it would just use the method categories if they were present. I have
also greatly simplified this method. Originally, it would use a lot of introspection to enable
it being called on various types of objects. Based on my tests, it was only ever being called
on a test case. The new function uses much less introspection then the preivous one, so we
should easily catch any stray uses, if there are any, as they will generate exceptions now.
Reviewers: zturner, tfiala, tberghammer
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D15451
llvm-svn: 255493
This will be used in a future change to support rerunning flakey tests
that hit a test result isue in a low-load, single worker test runner phase.
This is implemented as an additive-style event rather than being
evaluated and added to the start_test event because the decorator code
only runs after the start_test event is created and sent. i.e.
LLDBTestResult.startTest() runs before the test method decorators run.
llvm-svn: 255351
This was an option to display a graphical progress bar. Nobody
is using this, and it doesn't work correctly anyway with the new
result formatter.
llvm-svn: 255153
