599 Commits

Author SHA1 Message Date
Todd Fiala
9a58133698 Use new set when checking if a test event type matches the job/test result types
llvm-svn: 255385
2015-12-11 22:45:52 +00:00
Todd Fiala
b68dbfa227 Revert "Revert "Turn on new test summary results by default.""
This reverts commit f994b46a2028c8a8b9b55fe010a95122bca07540.

llvm-svn: 255381
2015-12-11 22:29:34 +00:00
Adrian McCarthy
cf6177131a Add some tests for stack and local variable inspection for mini dumps.
Differential Revision: http://reviews.llvm.org/D15435

llvm-svn: 255379
2015-12-11 22:27:57 +00:00
Todd Fiala
a8fee7f981 Add expected timeout support to test event architecture.
llvm-svn: 255363
2015-12-11 19:44:23 +00:00
Zachary Turner
742afdb3d2 Remove -S option from dotest.py.
llvm-svn: 255361
2015-12-11 19:21:49 +00:00
Zachary Turner
6a188e6c1e Remove -r and -R options from dotest.py.
llvm-svn: 255360
2015-12-11 19:21:34 +00:00
Todd Fiala
9187f27e32 Add test event marking a test as explicitly eligible for rerun if it is marked flakey.
This will be used in a future change to support rerunning flakey tests
that hit a test result isue in a low-load, single worker test runner phase.

This is implemented as an additive-style event rather than being
evaluated and added to the start_test event because the decorator code
only runs after the start_test event is created and sent.  i.e.
LLDBTestResult.startTest() runs before the test method decorators run.

llvm-svn: 255351
2015-12-11 18:06:47 +00:00
Tamas Berghammer
e43482b626 Create test for llvm.org/pr25806
LLDB don't detect the loading of a shared object file linked against the
main executable before the static initializers are executed for the
given module. Because of this it is not possible to get breakpoint hits
in these static initializers and to display proper debug info in case of
a crash in these codes.

llvm-svn: 255342
2015-12-11 16:24:14 +00:00
Pavel Labath
734a0b3f81 XFAIL TestLoadUnload for Windows->Android
llvm-svn: 255341
2015-12-11 16:16:51 +00:00
Pavel Labath
027158fad7 Revert "Turn on new test summary results by default."
The new test summary formatter does not honor the "expected timeout" markings, which makes our
buildbots all red. I'm switching it off by default until we figure out a way to make this work.

llvm-svn: 255335
2015-12-11 11:05:24 +00:00
Todd Fiala
07206ea19d test result details now print module.class.test_name in verbose mode.
And, turns off verbose mode by default.  This must have been switched
on as the default when somebody was testing.

llvm-svn: 255310
2015-12-10 23:14:24 +00:00
Kamil Rytarowski
f5d34b7b9d Add NetBSD support in the buildDriver and buildLibrary routines
Summary: NetBSD is like FreeBSD and Linux in these routines.

Reviewers: clay.chang, tfiala, emaste, joerg

Subscribers: lldb-commits, emaste

Differential Revision: http://reviews.llvm.org/D15374

llvm-svn: 255308
2015-12-10 22:56:56 +00:00
Zachary Turner
923fdbadae Remove the -y option from dotest.py.
llvm-svn: 255280
2015-12-10 18:52:09 +00:00
Zachary Turner
70ed57389a Remove the -x option from dotest.py.
llvm-svn: 255279
2015-12-10 18:51:40 +00:00
Zachary Turner
21accab1a7 Remove deprecated command line options from dotest.py
llvm-svn: 255278
2015-12-10 18:51:21 +00:00
Zachary Turner
80310c29fb Remove the --output-on-success command line argument from dotest.
llvm-svn: 255277
2015-12-10 18:51:02 +00:00
Zachary Turner
3cfa31492c Remove the -T option from dotest.py.
llvm-svn: 255276
2015-12-10 18:50:49 +00:00
Zachary Turner
b08ab72427 Remove -w option from dotest.py.
llvm-svn: 255275
2015-12-10 18:50:32 +00:00
Todd Fiala
5d96dc5629 enable timeout/exceptional exit support for xUnit formatter
Also adds enable.py/disable.py script to simplify turning on and off
the issue_verification tests helpful for testing a results formatter.

llvm-svn: 255161
2015-12-09 22:02:31 +00:00
Zachary Turner
94ea56bc8b Remove the -P option from dotest.py
This was an option to display a graphical progress bar.  Nobody
is using this, and it doesn't work correctly anyway with the new
result formatter.

llvm-svn: 255153
2015-12-09 20:48:59 +00:00
Zachary Turner
35a7610a05 Delete the -n command line option from dotest.py.
This removes the option to not print some one time version and
diagnostic information when running the test suite.

llvm-svn: 255152
2015-12-09 20:48:42 +00:00
Zachary Turner
6c9e44413f Remove the -i command line option from dotest.py.
This is part of a larger effort to remove unused command line
options from dotest.py.

llvm-svn: 255151
2015-12-09 20:48:31 +00:00
Zachary Turner
d2a93fd6ca Delete the -F command line option from dotest.py.
This removes the failfast command line option as part of an effort
to simplify dotest and remove unused command line options.  You can
still Ctrl+C any time you want to exit early.

llvm-svn: 255150
2015-12-09 20:48:19 +00:00
Zachary Turner
76972031fa Remove -k command line option from dotest.py.
This is part of an effort to remove unused command line options.

llvm-svn: 255143
2015-12-09 19:45:51 +00:00
Zachary Turner
6c8733346a Remove -e option from dotest.py.
This is part of an effort to clean up dotest command line options
that are no longer used.

llvm-svn: 255142
2015-12-09 19:45:36 +00:00
Zachary Turner
c816013641 Remove -libcxx option from dotest.py
Nobody was using this, and plus it can be achieved just as well
by using -E to set an environment variable.

llvm-svn: 255141
2015-12-09 19:45:16 +00:00
Todd Fiala
d58476da79 Move XunitFormatter into its own xunit_formatter.py file.
llvm-svn: 255139
2015-12-09 19:32:14 +00:00
Todd Fiala
0a7c32b38e Fix new summary to include exceptional exit count in determining exit value
The main dotest.py should exit with a system return code of 1 on any
issue.  This change fixes a place where I omitted counting the
exceptional exit value to determine if we should return 1 when using the
new summary results.

This change also puts a banner around the Issue Details section that comes
before the Test Result Summary.

llvm-svn: 255138
2015-12-09 19:05:44 +00:00
Todd Fiala
29508491e5 Disable the issue verification tests.
llvm-svn: 255134
2015-12-09 18:24:47 +00:00
Todd Fiala
f0cccb3189 create 3 issues for testbots: FAIL, ERROR (exceptional), and TIMEOUT
This change is a trial balloon to verify that the default test summary
output sends the right output for the buildbot issue detection script.

The effect of this change will be reverted after verifying the testbot
behavior.  This change will not stay in as is and will knowingly create
noise, see this thread:

http://lists.llvm.org/pipermail/lldb-dev/2015-December/009048.html

llvm-svn: 255131
2015-12-09 18:01:19 +00:00
Todd Fiala
1f99176fe5 Turn on new test summary results by default.
llvm-svn: 255130
2015-12-09 18:01:14 +00:00
Pavel Labath
35b952ac3e Correctly XFAIL TestReturnValue
android is not an "os", use the target triple to match it.

llvm-svn: 255118
2015-12-09 15:49:40 +00:00
Pavel Labath
7ead0b937c XFAIL TestReturnValue for remote Windows->Android tests
this also adds the ability to match the host platform to the expectedFailureAll decorator.

llvm-svn: 255105
2015-12-09 10:54:18 +00:00
Tamas Berghammer
31fef1e273 Fix a cleanup error in TestPlatformProcessConnect.py
llvm-svn: 255104
2015-12-09 10:16:05 +00:00
Todd Fiala
5183147e2d wire timeouts and exceptional inferior process exits through the test event system
The results formatter system is now fed timeouts and exceptional process
exits (i.e. inferior dotest.py process that exited by signal on POSIX
systems).

If a timeout or exceptional exit happens while a test method is running
on the worker queue, the timeout or exceptional exit is charged and
reported against that test method.  Otherwise, if no test method was
running at the time of the timeout or exceptional exit, only the test
filename will be reported as the TIMEOUT or ERROR.

Implements:
https://llvm.org/bugs/show_bug.cgi?id=24830
https://llvm.org/bugs/show_bug.cgi?id=25703

In support of:
https://llvm.org/bugs/show_bug.cgi?id=25450

llvm-svn: 255097
2015-12-09 06:45:43 +00:00
Zachary Turner
d865c6b707 Remove the -c option from dotest.py.
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today.  git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.

llvm-svn: 255060
2015-12-08 22:15:48 +00:00
Zachary Turner
5067158381 Remove the -X option from dotest.py
This removes the option to exclude a single directory.  This is
part of an effort to remove unused options and cleanup the interface
to the test suite.

llvm-svn: 255048
2015-12-08 20:36:22 +00:00
Zachary Turner
7544602d41 Remove the -g option from dotest.py
This removes the non-exclusive filterspec option as part of an
effort to remove unused / deprecated command line options from
dotest.

llvm-svn: 255041
2015-12-08 18:48:53 +00:00
Zachary Turner
2155d5d301 Remove the -b option from dotest.py
This removes the blacklist option as part of an effort to remove
unused / unmaintained command line options from the test suite.

llvm-svn: 255040
2015-12-08 18:43:16 +00:00
Zachary Turner
aad25fb9a5 Remove +b option from dotest.py
llvm-svn: 255037
2015-12-08 18:36:05 +00:00
Zachary Turner
b0dcbd461e Remove the -D option from dotest.py.
This removes the option to dump Python sys.path variable as part
of an effort to remove unused options.

llvm-svn: 255035
2015-12-08 18:25:38 +00:00
Todd Fiala
be011d6c45 flip on executable bit on test runner tests
llvm-svn: 255025
2015-12-08 16:22:27 +00:00
Tamas Berghammer
ccd6cffba3 Modify "platform connect" to connect to processes as well
The standard remote debugging workflow with gdb is to start the
application on the remote host under gdbserver (e.g.: gdbserver :5039
a.out) and then connect to it with gdb.

The same workflow is supported by debugserver/lldb-gdbserver with a very
similar syntax but to access all features of lldb we need to be
connected also to an lldb-platform instance running on the target.

Before this change this had to be done manually with starting a separate
lldb-platform on the target machine and then connecting to it with lldb
before connecting to the process.

This change modifies the behavior of "platform connect" with
automatically connecting to the process instance if it was started by
the remote platform. With this command replacing gdbserver in a gdb
based worflow is usually as simple as replacing the command to execute
gdbserver with executing lldb-platform.

Differential revision: http://reviews.llvm.org/D14952

llvm-svn: 255016
2015-12-08 14:08:19 +00:00
Tamas Berghammer
4fbd67ac11 Add a new option to Platform::LoadImage to install the image
This change introduce 3 different working mode for Platform::LoadImage
depending on the file specs specified.
* If only a remote file is specified then the remote file is loaded on
  the target (same behavior as before)
* If only a local file is specified then the local file is installed to
  the current working directory and then loaded from there.
* If both local and remote file is specified then the local file is
  installed to the specified location and then loaded from there.

The same options are exposed on the SB API with a new method LoadImage
method while the old signature presers its meaning.

On the command line the installation of the shared library can be specified
with the "--install" option of "process load".

Differential revision: http://reviews.llvm.org/D15152

llvm-svn: 255014
2015-12-08 13:43:59 +00:00
Pavel Labath
9cbf7dde88 Fixup dotest.py on mac for the configuration package
llvm-svn: 255013
2015-12-08 13:32:07 +00:00
Pavel Labath
bddf18009f Fixup dotest.py after the configuration package introduction
llvm-svn: 255009
2015-12-08 12:09:56 +00:00
Zachary Turner
b4733e6fd2 Move LLDBTestResult class to its own module.
llvm-svn: 254983
2015-12-08 01:15:44 +00:00
Zachary Turner
606e3a5221 Get rid of global variables in dotest.py
This moves all the global variables into a separate module called
`configuration`.  This has a number of advantages:

1. Configuration data is centrally maintained so it's easy to get
   a high level overview of what configuration data the test suite
   makes use of.
2. The method of sharing configuration data among different parts
   of the test suite becomes standardized.  Previously we would
   put some things into the `lldb` module, some things into the
   `lldbtest_config` module, and some things would not get shared.
   Now everything is shared through one module and is available to
   the entire test suite.
3. It opens the door to moving some of the initialization code into
   the `configuration` module, simplifying the implementation of
   `dotest.py`.

There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily.  This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable.  We can address these in the future.

llvm-svn: 254982
2015-12-08 01:15:30 +00:00
Greg Clayton
e6b36cdd4d Trying to submit 254476 one more time. This implement -gmodule debugging support.
It was previously reverted due to issues that showed up only on linux. I was able to reproduce these issues and fix the underlying cause.

So this is the same patch as 254476 with the following two fixes:
- Fix not trying to complete classes that don't have external sources
- Fix ClangASTSource::CompleteType() to check the decl context of types that it finds by basename to ensure we don't complete a type "S" with a type like "std::S". Before this fix ClangASTSource::CompleteType() would accept _any_ type that had a matching basename and copy it into the other type.

<rdar://problem/22992457>

llvm-svn: 254980
2015-12-08 01:02:08 +00:00
Todd Fiala
de02939823 Refactor ResultsFormatter creation into result_formatter.
This cleans up dotest.py and is a pre-step for getting
the test inferior runner to send post-inferior run events
to the events collector, as this code needs to be accessed
from within dosep.py.

llvm-svn: 254979
2015-12-08 00:53:56 +00:00