This change is a trial balloon to verify that the default test summary
output sends the right output for the buildbot issue detection script.
The effect of this change will be reverted after verifying the testbot
behavior. This change will not stay in as is and will knowingly create
noise, see this thread:
http://lists.llvm.org/pipermail/lldb-dev/2015-December/009048.html
llvm-svn: 255131
The results formatter system is now fed timeouts and exceptional process
exits (i.e. inferior dotest.py process that exited by signal on POSIX
systems).
If a timeout or exceptional exit happens while a test method is running
on the worker queue, the timeout or exceptional exit is charged and
reported against that test method. Otherwise, if no test method was
running at the time of the timeout or exceptional exit, only the test
filename will be reported as the TIMEOUT or ERROR.
Implements:
https://llvm.org/bugs/show_bug.cgi?id=24830https://llvm.org/bugs/show_bug.cgi?id=25703
In support of:
https://llvm.org/bugs/show_bug.cgi?id=25450
llvm-svn: 255097
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today. git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.
llvm-svn: 255060
This removes the option to exclude a single directory. This is
part of an effort to remove unused options and cleanup the interface
to the test suite.
llvm-svn: 255048
The standard remote debugging workflow with gdb is to start the
application on the remote host under gdbserver (e.g.: gdbserver :5039
a.out) and then connect to it with gdb.
The same workflow is supported by debugserver/lldb-gdbserver with a very
similar syntax but to access all features of lldb we need to be
connected also to an lldb-platform instance running on the target.
Before this change this had to be done manually with starting a separate
lldb-platform on the target machine and then connecting to it with lldb
before connecting to the process.
This change modifies the behavior of "platform connect" with
automatically connecting to the process instance if it was started by
the remote platform. With this command replacing gdbserver in a gdb
based worflow is usually as simple as replacing the command to execute
gdbserver with executing lldb-platform.
Differential revision: http://reviews.llvm.org/D14952
llvm-svn: 255016
This change introduce 3 different working mode for Platform::LoadImage
depending on the file specs specified.
* If only a remote file is specified then the remote file is loaded on
the target (same behavior as before)
* If only a local file is specified then the local file is installed to
the current working directory and then loaded from there.
* If both local and remote file is specified then the local file is
installed to the specified location and then loaded from there.
The same options are exposed on the SB API with a new method LoadImage
method while the old signature presers its meaning.
On the command line the installation of the shared library can be specified
with the "--install" option of "process load".
Differential revision: http://reviews.llvm.org/D15152
llvm-svn: 255014
This moves all the global variables into a separate module called
`configuration`. This has a number of advantages:
1. Configuration data is centrally maintained so it's easy to get
a high level overview of what configuration data the test suite
makes use of.
2. The method of sharing configuration data among different parts
of the test suite becomes standardized. Previously we would
put some things into the `lldb` module, some things into the
`lldbtest_config` module, and some things would not get shared.
Now everything is shared through one module and is available to
the entire test suite.
3. It opens the door to moving some of the initialization code into
the `configuration` module, simplifying the implementation of
`dotest.py`.
There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily. This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable. We can address these in the future.
llvm-svn: 254982
It was previously reverted due to issues that showed up only on linux. I was able to reproduce these issues and fix the underlying cause.
So this is the same patch as 254476 with the following two fixes:
- Fix not trying to complete classes that don't have external sources
- Fix ClangASTSource::CompleteType() to check the decl context of types that it finds by basename to ensure we don't complete a type "S" with a type like "std::S". Before this fix ClangASTSource::CompleteType() would accept _any_ type that had a matching basename and copy it into the other type.
<rdar://problem/22992457>
llvm-svn: 254980
This cleans up dotest.py and is a pre-step for getting
the test inferior runner to send post-inferior run events
to the events collector, as this code needs to be accessed
from within dosep.py.
llvm-svn: 254979
Summary:
The base make(1) on NetBSD is BSD make.
In the default installation of NetBSD GNU make comes via pkgsrc under the gmake name.
Reviewers: emaste, tfiala, clayborg
Subscribers: joerg, lldb-commits
Differential Revision: http://reviews.llvm.org/D15261
llvm-svn: 254947
There is already a class called LLDBTestResults which I would like
to move into a separate file, but the most appropriate filename
was taken.
llvm-svn: 254946
Summary:
The getwd() and getcwd() functions conform to IEEE Std 1003.1-1990
(POSIX.1). The IEEE Std 1003.1-2004 (POSIX.1) revision marked
getwd() as legacy and recommended the use of getcwd() instead. The IEEE
Std 1003.1-2008 (``POSIX.1'') revision removed getwd() from the
specification.
The ability to specify a NULL pointer and have getcwd() allocate memory
as necessary is an extension.
The getwd() function appeared in 4.0BSD.
Reviewers: emaste, tfiala, clayborg
Subscribers: lldb-commits, joerg
Differential Revision: http://reviews.llvm.org/D15260
llvm-svn: 254944
Allow both '-var-list-children var0' and '-var-list-children "var0"' to be used with the -var-list-children command. GDB MI allows for this and it is necessary if the variable name contains spaces, such as var5.std::_Vector_base<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<cahr> > > >.
Differential Revision: http://reviews.llvm.org/D15168
llvm-svn: 254941
This is a resubmit of r254403, see that commit's message for context. This fixes an issue in the
original commit, where we would incorrectly interrupt the process if the interrupt request came
just as we were about to send the stopped event to the public.
llvm-svn: 254902
Summary:
Because of the large number of XFAILs TestThreadStates has decayed quite a bit. This commit does
the following:
- removes the "breakpoint list" expectations. Most tests have been failing on this, because the
command output changed quite a while back. I remove it, because run_break_set_by_file_and_line
already does a decent amount of checking
- fixup test_state_after_expression: this was calling the wrong function by mistake. As now the
function actually tests something (which we know is broken), I needed to XFAIL it as well.
- replaces the sleep() with a proper wait-for-event functionality in parts which use async mode,
to stabilize the one function that actually tests something.
Reviewers: clayborg
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D15233
llvm-svn: 254901
Test summary counts now show at the end, with issue details
above.
Issue details now print "ISSUE_TYPE: test method (relative path)".
Relative paths are relative to the
packages/Python/lldbsuite/test directory.
Sample output:
test/dotest.py --executable `pwd`/build/Debug/lldb --results-formatter lldbsuite.test.basic_results_formatter.BasicResultsFormatter --threads 12
Testing: 415 test suites, 12 threads
415 out of 415 test suites processed - TestLldbGdbServer.py
Test Results
Total Test Methods Run (excluding reruns): 2470
Test Method rerun count: 0
Details:
UNEXPECTED SUCCESS: test_symbol_name_dsym (functionalities/completion/TestCompletion.py)
UNEXPECTED SUCCESS: test_symbol_name_dwarf (functionalities/completion/TestCompletion.py)
===================
Test Result Summary
===================
Success: 1329
Expected Failure: 79
Failure: 0
Error: 0
Unexpected Success: 2
Skip: 1060
llvm-svn: 254890
Summary: This is used in tests.
Reviewers: emaste, tfiala, clayborg
Subscribers: zturner, lldb-commits, joerg
Differential Revision: http://reviews.llvm.org/D15065
llvm-svn: 254853
Patch by Nitesh Jain.
Summary: There is no debug information generated for variable index with –O3 optimization flag. The DW_AT_location tag in DWARF debug_info section is empty.
Reviewers: ovyalov, clayborg
Subscribers: lldb-commits, mohit.bhakkad, sagar, bhushan, jaydeep
Differential: http://reviews.llvm.org/D15224
llvm-svn: 254850
This test would fail before if conditional breakpoints weren't
working correctly, and the nature of the test (spinning up 10
threads, etc) opens the door to raciness.
This patch vastly simplifies the test, removes the need for relying
on conditional expression evaluation, and as a result makes the
correctness of the test vastly easier to reason about and reduces
flakiness.
Differential Revision: http://reviews.llvm.org/D15241
Reviewed By: Jim Ingham
llvm-svn: 254792
SUMMARY:
Marked TestCrashDuringStep.py as XFAIL for MIPS. The test generates IO error due to breakpoint at invalid address.
Reviewers: clayborg, labath
Subscribers: nitesh.jain, mohit.bhakkad, sagar, bhushan and lldb-commits
Differential Revision: http://reviews.llvm.org/D15182
llvm-svn: 254710
Previously we used sys.os.path for appending target pathes what failed
when cased dlopen to fail on the target because of the '\'.
The fix won't work for local Windows tests but dlopen don't available
on Windows anyway so the test don't make sense in that context.
llvm-svn: 254602
Also cleans up some usages of strings where symbolic names
were safer and made more sense.
Try a test run with something like this to check out the new
basic results formatter (not used by default):
time test/dotest.py --executable `pwd`/build/Debug/lldb --results-formatter lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file stdout
This will yield something like:
Testing: 1 test suites, 8 threads
1 out of 1 test suites processed - TestHelp.py
Test Results
Total Test Methods Run (excluding reruns): 13
Test Method rerun count: 0
===================
Test Result Summary
===================
Success: 13
Expected Failure: 0
Failure: 0
Error: 0
Unexpected Success: 0
Skip: 0
Whereas something with a bit of error will look more like this:
42 out of 42 test suites processed - TestSymbolTable.py
Test Results
Total Test Methods Run (excluding reruns): 166
Test Method rerun count: 0
===================
Test Result Summary
===================
Success: 93
Expected Failure: 10
Failure: 2
Error: 2
Unexpected Success: 0
Skip: 59
Details:
FAIL:
TestModulesInlineFunctions.ModulesInlineFunctionsTestCase.test_expr_dsym
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/modules-inline-functions/TestModulesInlineFunctions.py)
FAIL:
TestModulesInlineFunctions.ModulesInlineFunctionsTestCase.test_expr_dwarf
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/modules-inline-functions/TestModulesInlineFunctions.py)
ERROR: TestObjCCheckers.ObjCCheckerTestCase.test_objc_checker_dsym
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/objc-checker/TestObjCCheckers.py)
ERROR: TestObjCCheckers.ObjCCheckerTestCase.test_objc_checker_dwarf
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/objc-checker/TestObjCCheckers.py)
The Details header only prints if there are any issues to report. The
Details section has tags that should get picked up using the normal
issue text scrapers (e.g. buildbot).
Test numbers reported are strictly test method runs.
The rerun bit at the top is in support of the multi-pass test
runner code (to run the low-load, single worker test pass for
tests that failed the first run), which I'll be able to put up
for review after this.
ResultsFormatters now have the ability to indicate they replace
the legacy summary, as this one does.
Once we come to agreement on the exact format, I will switch
us over to using this by default.
llvm-svn: 254530
