153 Commits

Author SHA1 Message Date
Zachary Turner
b4733e6fd2 Move LLDBTestResult class to its own module.
llvm-svn: 254983
2015-12-08 01:15:44 +00:00
Zachary Turner
606e3a5221 Get rid of global variables in dotest.py
This moves all the global variables into a separate module called
`configuration`.  This has a number of advantages:

1. Configuration data is centrally maintained so it's easy to get
   a high level overview of what configuration data the test suite
   makes use of.
2. The method of sharing configuration data among different parts
   of the test suite becomes standardized.  Previously we would
   put some things into the `lldb` module, some things into the
   `lldbtest_config` module, and some things would not get shared.
   Now everything is shared through one module and is available to
   the entire test suite.
3. It opens the door to moving some of the initialization code into
   the `configuration` module, simplifying the implementation of
   `dotest.py`.

There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily.  This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable.  We can address these in the future.

llvm-svn: 254982
2015-12-08 01:15:30 +00:00
Greg Clayton
e6b36cdd4d Trying to submit 254476 one more time. This implement -gmodule debugging support.
It was previously reverted due to issues that showed up only on linux. I was able to reproduce these issues and fix the underlying cause.

So this is the same patch as 254476 with the following two fixes:
- Fix not trying to complete classes that don't have external sources
- Fix ClangASTSource::CompleteType() to check the decl context of types that it finds by basename to ensure we don't complete a type "S" with a type like "std::S". Before this fix ClangASTSource::CompleteType() would accept _any_ type that had a matching basename and copy it into the other type.

<rdar://problem/22992457>

llvm-svn: 254980
2015-12-08 01:02:08 +00:00
Todd Fiala
de02939823 Refactor ResultsFormatter creation into result_formatter.
This cleans up dotest.py and is a pre-step for getting
the test inferior runner to send post-inferior run events
to the events collector, as this code needs to be accessed
from within dosep.py.

llvm-svn: 254979
2015-12-08 00:53:56 +00:00
Kamil Rytarowski
faca93e407 Add NetBSD in platform specific logging of the specified category in RegisterCommandsTestCase()
Summary: NetBSD soon will reuse this feature while running tests.

Reviewers: emaste, tfiala, clayborg

Subscribers: lldb-commits, joerg

Differential Revision: http://reviews.llvm.org/D15263

llvm-svn: 254949
2015-12-07 21:26:56 +00:00
Kamil Rytarowski
49f9fb8d26 Add initial NetBSD support in lldbsuite/test/lldbtest.py
Summary:
Add new functions:

  - expectedFailureNetBSD()
  - expectedFlakeyNetBSD()
  - skipIfNetBSD()

Add new NetBSD entry in:

  - getPlatform()
  - getHostPlatform()

Assume that libc++ is installed and use the GNU toolchain

Reviewers: joerg, emaste, tfiala, clayborg

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15262

llvm-svn: 254948
2015-12-07 21:25:57 +00:00
Kamil Rytarowski
5b76e726b3 Return gmake as the default name for GNU make on NetBSD
Summary:
The base make(1) on NetBSD is BSD make.

In the default installation of NetBSD GNU make comes via pkgsrc under the gmake name.

Reviewers: emaste, tfiala, clayborg

Subscribers: joerg, lldb-commits

Differential Revision: http://reviews.llvm.org/D15261

llvm-svn: 254947
2015-12-07 21:24:25 +00:00
Zachary Turner
905a98881b Rename test_results.py to result_formatter.py.
There is already a class called LLDBTestResults which I would like
to move into a separate file, but the most appropriate filename
was taken.

llvm-svn: 254946
2015-12-07 21:23:41 +00:00
Kamil Rytarowski
1f1a15332b getwd(3) with NULL pointer extension is supported on NetBSD
Summary:
The getwd() and getcwd() functions conform to IEEE Std 1003.1-1990
(POSIX.1).  The IEEE Std 1003.1-2004 (POSIX.1) revision marked
getwd() as legacy and recommended the use of getcwd() instead.  The IEEE
Std 1003.1-2008 (``POSIX.1'') revision removed getwd() from the
specification.

The ability to specify a NULL pointer and have getcwd() allocate memory
as necessary is an extension.

The getwd() function appeared in 4.0BSD.

Reviewers: emaste, tfiala, clayborg

Subscribers: lldb-commits, joerg

Differential Revision: http://reviews.llvm.org/D15260

llvm-svn: 254944
2015-12-07 21:21:12 +00:00
Chuck Ries
1ffd4f5093 Allow variable names to be quoted with -var-list-children
Allow both '-var-list-children var0' and '-var-list-children "var0"' to be used with the -var-list-children command. GDB MI allows for this and it is necessary if the variable name contains spaces, such as var5.std::_Vector_base<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<cahr> > > >.

Differential Revision: http://reviews.llvm.org/D15168

llvm-svn: 254941
2015-12-07 20:43:52 +00:00
Pavel Labath
19da1f16c2 Recommit "Fix race during process interruption"
This is a resubmit of r254403, see that commit's message for context. This fixes an issue in the
original commit, where we would incorrectly interrupt the process if the interrupt request came
just as we were about to send the stopped event to the public.

llvm-svn: 254902
2015-12-07 12:36:52 +00:00
Pavel Labath
cb8ea4bd23 Make TestThreadStates more stable
Summary:
Because of the large number of XFAILs TestThreadStates has decayed quite a bit. This commit does
the following:
- removes the "breakpoint list" expectations. Most tests have been failing on this, because the
  command output changed quite a while back. I remove it, because run_break_set_by_file_and_line
  already does a decent amount of checking
- fixup test_state_after_expression: this was calling the wrong function by mistake. As now the
  function actually tests something (which we know is broken), I needed to XFAIL it as well.
- replaces the sleep() with a proper wait-for-event functionality in parts which use async mode,
  to stabilize the one function that actually tests something.

Reviewers: clayborg

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15233

llvm-svn: 254901
2015-12-07 11:09:14 +00:00
Todd Fiala
1d3b5a3ac2 Added lldbsuite.lldb_test_root, switched formatter to use it.
llvm-svn: 254894
2015-12-07 06:19:48 +00:00
Mohit K. Bhakkad
f083b9fa97 [LLDB][MIPS] Marking some expected failures
Reviewers: clayborg, zturner.
Subscribers: jaydeep, bhushan, sagar, nitesh.jain, tberghammer,lldb-commits.
Differential Revision: http://reviews.llvm.org/D14944

llvm-svn: 254892
2015-12-07 05:47:35 +00:00
Todd Fiala
9dadac457e Moved total test count and rerun count into summary counts.
llvm-svn: 254891
2015-12-07 04:55:07 +00:00
Todd Fiala
35d604bfdd Move test summary counts to end; simplify issue detail line
Test summary counts now show at the end, with issue details
above.

Issue details now print "ISSUE_TYPE: test method (relative path)".
Relative paths are relative to the
packages/Python/lldbsuite/test directory.

Sample output:

test/dotest.py --executable `pwd`/build/Debug/lldb --results-formatter lldbsuite.test.basic_results_formatter.BasicResultsFormatter --threads 12
Testing: 415 test suites, 12 threads
415 out of 415 test suites processed - TestLldbGdbServer.py
Test Results
Total Test Methods Run (excluding reruns): 2470
Test Method rerun count: 0

Details:
UNEXPECTED SUCCESS: test_symbol_name_dsym (functionalities/completion/TestCompletion.py)
UNEXPECTED SUCCESS: test_symbol_name_dwarf (functionalities/completion/TestCompletion.py)

===================
Test Result Summary
===================
Success:            1329
Expected Failure:     79
Failure:               0
Error:                 0
Unexpected Success:    2
Skip:               1060

llvm-svn: 254890
2015-12-07 03:10:02 +00:00
Kamil Rytarowski
0b655da7e4 Define new builder_netbsd
Summary: This is used in tests.

Reviewers: emaste, tfiala, clayborg

Subscribers: zturner, lldb-commits, joerg

Differential Revision: http://reviews.llvm.org/D15065

llvm-svn: 254853
2015-12-05 18:46:56 +00:00
Sagar Thakur
44375e4226 [LLDB][MIPS] Fix TestConstVariables.py
Patch by Nitesh Jain.

Summary: There is no debug information generated for variable index with –O3 optimization flag. The DW_AT_location tag in DWARF debug_info section is empty.

Reviewers: ovyalov, clayborg
Subscribers: lldb-commits, mohit.bhakkad, sagar, bhushan, jaydeep
Differential: http://reviews.llvm.org/D15224
llvm-svn: 254850
2015-12-05 12:01:48 +00:00
Zachary Turner
3e9e7d2822 Make TestThreadSpecificBreakpoint.py more focused.
This test would fail before if conditional breakpoints weren't
working correctly, and the nature of the test (spinning up 10
threads, etc) opens the door to raciness.

This patch vastly simplifies the test, removes the need for relying
on conditional expression evaluation, and as a result makes the
correctness of the test vastly easier to reason about and reduces
flakiness.

Differential Revision: http://reviews.llvm.org/D15241
Reviewed By: Jim Ingham

llvm-svn: 254792
2015-12-04 22:59:41 +00:00
Enrico Granata
072bff8036 Add a similar benchmark for our std::map performance
llvm-svn: 254779
2015-12-04 22:16:14 +00:00
Enrico Granata
ef4fa44ab8 Fix an issue where all tests marked with skip_if_callable would be skipped regardless of the actual callable
llvm-svn: 254758
2015-12-04 19:50:05 +00:00
Enrico Granata
61d8c13b67 Add a benchmark that validates how much time LLDB spends trying to fully print a fairly large std::list<T>
This is meant to help me track optimizations to the libc++ std::list data formatter

llvm-svn: 254755
2015-12-04 19:40:26 +00:00
Todd Fiala
c5011a8a1a Marked TestModulesInlineFunctions.py XFAIL
Tracked here:
https://llvm.org/bugs/show_bug.cgi?id=25743

llvm-svn: 254746
2015-12-04 18:52:02 +00:00
Todd Fiala
12777837a4 Fix test error in TestObjCCheckers.py
llvm-svn: 254744
2015-12-04 18:40:34 +00:00
Mohit K. Bhakkad
1e577e66e9 [LLDB][MIPS] Fix gdbremote_testcase.py
Patch by Nitesh Jain

Reviewers: clayborg, ovyalov.
Subscribers: jaydeep, bhushan, mohit.bhakkad, sagar.
Differential Revision: http://reviews.llvm.org/D15103

llvm-svn: 254711
2015-12-04 09:58:07 +00:00
Jaydeep Patil
56cb5ebae8 [LLDB][MIPS] XFAIL TestCrashDuringStep.py for MIPS
SUMMARY:
    Marked TestCrashDuringStep.py as XFAIL for MIPS. The test generates IO error due to breakpoint at invalid address.
    
    Reviewers: clayborg, labath
    Subscribers: nitesh.jain, mohit.bhakkad, sagar, bhushan and lldb-commits
    Differential Revision: http://reviews.llvm.org/D15182

llvm-svn: 254710
2015-12-04 09:56:36 +00:00
Zachary Turner
abb4420f57 Un XFAIL some tests that are now passing on Windows.
llvm-svn: 254633
2015-12-03 19:20:05 +00:00
Tamas Berghammer
f9a52f044c Fix TestLoadUnload for Windows -> POSIX remote debugging
Previously we used sys.os.path for appending target pathes what failed
when cased dlopen to fail on the target because of the '\'.

The fix won't work for local Windows tests but dlopen don't available
on Windows anyway so the test don't make sense in that context.

llvm-svn: 254602
2015-12-03 11:02:10 +00:00
Zachary Turner
fe868acafa Fix test result serialization to use bytes.
llvm-svn: 254563
2015-12-02 23:07:33 +00:00
Todd Fiala
c84311163a Candidate fixes for python2/3 compatible string handling in pickling support.
llvm-svn: 254550
2015-12-02 21:45:15 +00:00
Todd Fiala
194913ffbe Make --results-file stdout implied if unspecified when using a results formatter.
Also cleans up pylint warnings (stock settings) in the modified function.

llvm-svn: 254546
2015-12-02 21:12:17 +00:00
Todd Fiala
46a4e34dcc Adds candidate formatter for replacing legacy summary results.
Also cleans up some usages of strings where symbolic names
were safer and made more sense.

Try a test run with something like this to check out the new
basic results formatter (not used by default):

time test/dotest.py --executable `pwd`/build/Debug/lldb --results-formatter lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file stdout

This will yield something like:

Testing: 1 test suites, 8 threads
1 out of 1 test suites processed - TestHelp.py
Test Results
Total Test Methods Run (excluding reruns): 13
Test Method rerun count: 0

===================
Test Result Summary
===================
Success:              13
Expected Failure:      0
Failure:               0
Error:                 0
Unexpected Success:    0
Skip:                  0

Whereas something with a bit of error will look more like this:

42 out of 42 test suites processed - TestSymbolTable.py
Test Results
Total Test Methods Run (excluding reruns): 166
Test Method rerun count: 0

===================
Test Result Summary
===================
Success:              93
Expected Failure:     10
Failure:               2
Error:                 2
Unexpected Success:    0
Skip:                 59

Details:
FAIL:
TestModulesInlineFunctions.ModulesInlineFunctionsTestCase.test_expr_dsym
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/modules-inline-functions/TestModulesInlineFunctions.py)
FAIL:
TestModulesInlineFunctions.ModulesInlineFunctionsTestCase.test_expr_dwarf
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/modules-inline-functions/TestModulesInlineFunctions.py)
ERROR: TestObjCCheckers.ObjCCheckerTestCase.test_objc_checker_dsym
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/objc-checker/TestObjCCheckers.py)
ERROR: TestObjCCheckers.ObjCCheckerTestCase.test_objc_checker_dwarf
(/Users/tfiala/work/lldb-tot/git-svn/lldb/packages/Python/lldbsuite/test/lang/objc/objc-checker/TestObjCCheckers.py)

The Details header only prints if there are any issues to report.  The
Details section has tags that should get picked up using the normal
issue text scrapers (e.g. buildbot).

Test numbers reported are strictly test method runs.

The rerun bit at the top is in support of the multi-pass test
runner code (to run the low-load, single worker test pass for
tests that failed the first run), which I'll be able to put up
for review after this.

ResultsFormatters now have the ability to indicate they replace
the legacy summary, as this one does.

Once we come to agreement on the exact format, I will switch
us over to using this by default.

llvm-svn: 254530
2015-12-02 18:48:38 +00:00
Tamas Berghammer
2a7043cd70 XFAIL TestReturnValue for linux i386 clang 3.5-3.6
The test failing most likely because clang 3.5 and 3.6 uses an incorrect
ABI for returning small structs from  a function.

llvm-svn: 254507
2015-12-02 13:36:17 +00:00
Tamas Berghammer
3cb132a0f4 Fix "process load/unload" on android
On android the symbols exposed by libdl (dlopen, dlclose, dlerror)
prefixed by "__dl_". This change moves the handling of process
load/unload to the platform object and override it for android to
handle the special prefix.

Differential revision: http://reviews.llvm.org/D11465

llvm-svn: 254504
2015-12-02 11:58:51 +00:00
Tamas Berghammer
fcf334b824 Revert "Added support for -gmodule debugging when debug info is left in the .o files on Darwin."
The commit caused a test failure on the linux buildbot in
TestDataFormatterSynthVal.

llvm-svn: 254502
2015-12-02 11:35:54 +00:00
Abhishek Aggarwal
296e063d8f PTRACE ABI to read FXSAVE area for 32-bit inferior
Summary:
 - Problem occurs when:
    -- 32-bit inferiors run on x86_32 machine and
       the architecture doesn't have AVX feature

    -- This causes FPRType to be set to eFPRTypeFXSAVE

    -- PTRACE_GETFPREGS was being used to read FXSAVE area

    -- For 32-bit inferiors running on x86_32 machine,
       PTRACE_GETFPREGS reads FSAVE area and not FXSAVE area

 - Changed ptrace API to PTRACE_GETREGSET for 32-bit inferiors
    -- This reads FPR data in FXSAVE format.
    -- For 64-bit inferiors, no change has been made.

 - Modified XFAIL for TestReturnValue.py
    -- Earlier, this test was passing for Linux OS
    -- Now, it passes for Android OS as well

Change-Id: Ieed72bc969b79516fc7b263b32493aa1e7a1a2ac
Signed-off-by: Abhishek Aggarwal <abhishek.a.aggarwal@intel.com>

Reviewers: ovyalov, jingham, lldb-commits, tberghammer, labath

Subscribers: jevinskie, labath, tberghammer, danalbert

Differential Revision: http://reviews.llvm.org/D15042

llvm-svn: 254499
2015-12-02 09:40:17 +00:00
Greg Clayton
5dfc4a4d02 Added support for -gmodule debugging when debug info is left in the .o files on Darwin.
This is done by finding the types that are forward declarations that come from a module, and loading that module's debug info in a separate lldb_private::Module, and copying the type over into the current module using a ClangASTImporter object. ClangASTImporter objects are already used to copy types from on clang::ASTContext to another for expressions so the type copying code has been around for a while.

A new FindTypes variant was added to SymbolVendor and SymbolFile:

size_t
SymbolVendor::FindTypes (const std::vector<CompilerContext> &context, bool append, TypeMap& types);

size_t
SymbolVendor::FindTypes (const std::vector<CompilerContext> &context, bool append, TypeMap& types);

The CompilerContext is a way to represent the exact context of a type and pass it through an agnostic API boundary so that we can find that exact context elsewhere in another file. This was required here because we can have a module that has submodules, both of which have a "foo" type.

I am not able to add tests for this yet as we currently don't build our C/C++/ObjC binaries with the clang binary that we build. There are some driver issues where it can't find the header files for the C and C++ standard library which makes compiling these tests hard. We can't also guarantee that if we are building with clang that it supporst the exact format of -gmodule debugging that we are trying to test. We have had other versions of clang that had a different implementation of -gmodule debugging that we are no longer supporting, so we can't enable tests if we are building with clang without compiling something and looking at the structure of the DWARF that was generated to ensure that it is the format we can actually use.

llvm-svn: 254476
2015-12-02 00:43:32 +00:00
Pavel Labath
00fea63627 Revert "Fix race during process interruption"
The android buildbot gets quite flaky after this change. I'm reverting it while I investigate.

llvm-svn: 254430
2015-12-01 17:59:56 +00:00
Pavel Labath
df55f522bd Fix race during process interruption
Summary:
The following situation was occuring in TestAttachResume:
- we did a "continue" from a breakpoint (which involves a private start-stop to step over the
  breakpoint)
- after receiving the stop-reply from the step-over, we issue a "detach" (which requires a
  process interrupt)
- at this moment, the public state is "running", private state is "about-to-be-stopped" (the
  stopped event was broadcast, but it was not received yet)
- StopForDestroyOrDetach (public thread) notes the public state is running, sends an interrupt
  request to the private thread
- private thread gets the eBroadcastBitInterrupt (before the eStateStopped message), and asks the
  process plugin to stop (via Halt())
- process plugin says it has nothing to do as the process is already stopped
- private thread shrugs and carries on. receives the stop event, restores the breakpoint and
  resumes the process.
- after a while, the public thread times out and says it failed to stop the process

This patch does the following:
- splits Halt() into two functions, private and public, their usage depends on the context
  - public Halt(): sends eBroadcastBitInterrupt to the private thread and waits for the Stop
    event
  - HaltPrivate(): asks the plugin to stop and makes a note that the halt was requested. When the
    next stop event comes it sets the interrupt flag on it.
- removes HijackPrivateProcessEvents(), as the only user (old Halt()) has gone away
- removes the m_currently_handling_event hack, as the new Halt() does not need it
- adds a use_run_lock parameter to public Halt() and WaitForProcessToStop(). This was needed
  because RunThreadPlan uses Halt() while holding the run lock and we don't want Halt() to take
  it away from him.

Reviewers: clayborg, jingham

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14989

llvm-svn: 254403
2015-12-01 11:28:47 +00:00
Jaydeep Patil
27c25b1591 [LLDB][MIPS] Change ARCHFLAG for MIPS
SUMMARY:
    For MIPS, ARCH is specified without m.

    Reviewers: clayborg
    Subscribers: nitesh.jain, mohit.bhakkad, sagar, bhushan and lldb-commits
    Differential Revision: http://reviews.llvm.org/D14978

llvm-svn: 254376
2015-12-01 05:24:17 +00:00
Tamas Berghammer
a88453b069 Re-add an xfail removed by r254163
The test is flakey but it fails too often with gcc 4.9.2 on x86_64 to
be marked only as expected flakey.

llvm-svn: 254194
2015-11-27 10:50:33 +00:00
Tamas Berghammer
fd29986946 Fix TestRegisters.py on arm
Previously it tried to write a bit in the FPSCR register marked as
do not modify what failed on some device.

llvm-svn: 254166
2015-11-26 17:43:24 +00:00
Tamas Berghammer
e4fbec4a37 Remove some xfail-s fixed by r253026
These tests were fixed by r253026 but they was failing on the linux
build bot because of a system setup problem. Remove xfail from them
after we fixed the build bot.

llvm-svn: 254163
2015-11-26 16:11:57 +00:00
Tamas Berghammer
54b0a1f8eb Make some of the tests in TestRegisters.py arm compatible
Differential revision: http://reviews.llvm.org/D15010

llvm-svn: 254153
2015-11-26 15:02:35 +00:00
Pavel Labath
02559bbfd7 Enable TestMiBreak on Linux
test reliably passed last 100 buildbot runs. Enable it and see what happens.

llvm-svn: 254072
2015-11-25 16:29:04 +00:00
Zachary Turner
0a2899ca82 swig_bot remote path connection / preliminary implementation.
With this patch, the client will package up all the required
inputs into a compressed zip file, establish a connection to the
server, send the input to the server, and wait for the server to
send a response (in this case the response is just echoed back to
the client).

This gets the network communication in place, and in a subsequent
patch I will follow up with the code that actually runs swig on
the server and sends back the output instead of echoing back the
input.

llvm-svn: 254023
2015-11-24 21:35:32 +00:00
Ed Maste
860e29e0c3 Temporarily add expectedFailureFreeBSD to tests that fail to detect Clang on FreeBSD
On FreeBSD we may get Clang via CC=cc or CC=/usr/bin/cc.

llvm.org/pr25626

llvm-svn: 254006
2015-11-24 18:59:51 +00:00
Ed Maste
43bd918475 Add expectedFlakeyFreeBSD to TestBatchMode failing on buildbot
This test was already expectedFlakeyLinux for occasional failures on the
Linux buildbot. It seems the new FreeBSD buildbot fails the same way on
occasion.

llvm.org/pr25172

llvm-svn: 254002
2015-11-24 18:34:17 +00:00
Ed Maste
ae82bf7947 Remove expectedFailureFreeBSD decorator from now-passing tests
These pass on my FreeBSD stable/10 desktop and my (not-yet-connected)
FreeBSD 11-CURRENT buildbot

llvm.org/pr20548

llvm-svn: 254001
2015-11-24 18:30:19 +00:00
Ed Maste
99ab4f42b7 Remove expectedFailureFreeBSD from passing ThreadExitTestCase::test_with_dwarf
This passes on my FreeBSD stable/10 desktop and my new FreeBSD
11-current buildbot (which is not yet hooked up to the buildmaster).

llvm.org/pr18190

llvm-svn: 254000
2015-11-24 18:22:37 +00:00