163 Commits

Author SHA1 Message Date
Jan Kratochvil
9a33a15766 refactor testsuite spawnLldbMi args->exe+args
Currently spawnLldbMi accepts both lldb-mi options and executable to debug as
a single parameter. Split them.

As in D55859 we will need to execute one lldb-mi command before loading the
exe. Therefore we can no longer use the exe as lldb-mi command-line parameter
as then there is no way to execute a command before loading exe specified as
lldb-mi command-line parameter.

LocateExecutableSymbolFileDsym should be static, that is also a little
refactorization.

Differential Revision: https://reviews.llvm.org/D55858

llvm-svn: 349607
2018-12-19 08:57:10 +00:00
Jason Molenda
f47c734e49 A few small updates to the testsuite for running against an iOS device.
Remove the expected-fails for 34538611; using an alternate platform
implementation handles these correctly.

llvm-svn: 349417
2018-12-17 23:33:40 +00:00
Frederic Riss
f7c101b653 Make TestAppleSimulatorOSType.py more flexible
Different versions of Xcode have different outputs for the simctl command

llvm-svn: 347117
2018-11-17 00:40:31 +00:00
Jason Molenda
0128e7f540 Tiny testsuite tweaks. Don't run the apple simulator
tests when targetting a device.  Add an include to 
safe-to-call-func to work around a modules issue with
a certain combination of header files.  Add rules for
Darwin systems to ad-hoc codesign binaries that the
testsuite builds.

llvm-svn: 344635
2018-10-16 18:14:30 +00:00
Bruce Mitchener
173946dca6 Fix typos.
Reviewers: lldb-commits

Subscribers: srhines, ki.stfu

Differential Revision: https://reviews.llvm.org/D52884

llvm-svn: 343825
2018-10-04 22:33:39 +00:00
Pavel Labath
8b0853a6da XFail one more VSCode test which fails under heavy load
llvm-svn: 341186
2018-08-31 08:31:27 +00:00
Stella Stamenova
04acc99db8 [vscode] Skip some of the vscode tests on Linux and fix one
Summary: These are already skipped on Darwin because they cause build bot failures. Both on the build bots as well as in our testing we have seen a number of these tests fail and hang. This change skips the failing/hanging tests on Linux and also fixes one of the test - the test needs the thread library to build.

Reviewers: asmith, clayborg, aprantl

Subscribers: teemperor, lldb-commits

Differential Revision: https://reviews.llvm.org/D51227

llvm-svn: 340658
2018-08-24 21:42:53 +00:00
Greg Clayton
c1f10e34ed Skip tests on Darwin for now. The build bots are not passing due to heavy load and poor machines.
llvm-svn: 340112
2018-08-18 00:33:15 +00:00
Greg Clayton
f6e36ee83e Skip the lldb-vscode attach with waitFor test for now, some build bots are having trouble.
llvm-svn: 340077
2018-08-17 20:42:06 +00:00
Greg Clayton
a75fad4fa4 Change the attach test case to use self.assertEqual so we can get more info on what is going wrong on test bots.
Also add a cleanup function to remove the copied binary.

llvm-svn: 340050
2018-08-17 17:27:53 +00:00
Vedant Kumar
642de8a088 Disable two flaky pexpect-backed tests on Darwin
These tests are sporadically timing out on our bots, e.g here:

  https://ci.swift.org/job/swift-PR-Linux/6841

llvm-svn: 339914
2018-08-16 18:18:16 +00:00
Greg Clayton
2f5cf8511a Add a new tool named "lldb-vscode" that implements the Visual Studio Code Debug Adaptor Protocol
This patch adds a new lldb-vscode tool that speaks the Microsoft Visual Studio Code debug adaptor protocol. It has full unit tests that test all packets.

This tool can be easily packaged up into a native extension and used with Visual Studio Code, and it can also be used by Nuclide

Differential Revision: https://reviews.llvm.org/D50365

llvm-svn: 339911
2018-08-16 17:59:38 +00:00
Raphael Isemann
23d7a9ebbe Fix whitespace in the python test suite.
Summary:
The test suite has often unnecessary trailing whitespace, and sometimes
unnecessary trailing lines or a missing final new line. This patch just strips
trailing whitespace/lines and adds missing newlines at the end.

Subscribers: ki.stfu, JDevlieghere, christof, lldb-commits

Differential Revision: https://reviews.llvm.org/D49943

llvm-svn: 338171
2018-07-27 22:20:59 +00:00
Alexander Polyakov
e2183d508a [lldb-mi] Re-implement data-info-line command.
Summary: Now data-info-line command uses SB API instead of HandleCommand.

Reviewers: aprantl, clayborg, jingham

Reviewed By: aprantl

Subscribers: ki.stfu, lldb-commits

Differential Revision: https://reviews.llvm.org/D49062

llvm-svn: 337689
2018-07-23 13:02:41 +00:00
Alexander Polyakov
8c670ec73f [lldb-mi] Re-implement symbol-list-lines command.
Summary: Now this command uses SB API instead of HandleCommand.

Reviewers: aprantl, clayborg

Reviewed By: aprantl, clayborg

Subscribers: ki.stfu, eraman, lldb-commits

Differential Revision: https://reviews.llvm.org/D48802

llvm-svn: 336206
2018-07-03 15:40:20 +00:00
Alexander Polyakov
fc018b071b [lldb-mi] Re-implement a few MI commands.
Summary: This patch updates exec-next-instruction, exec-step-instruction,
exec-finish, exec-interrupt commands to use SB API instead of HandleCommand.

Reviewers: aprantl, clayborg

Reviewed By: aprantl

Subscribers: ki.stfu, lldb-commits

Differential Revision: https://reviews.llvm.org/D48520

llvm-svn: 336155
2018-07-02 23:54:06 +00:00
Alexander Polyakov
4a60320a20 [lldb-mi] Re-implement MI -exec-step command.
Summary: Now -exec-step uses SB API instead of HandleCommand hack.

Reviewers: aprantl, clayborg, labath, stella.stamenova

Reviewed By: aprantl

Subscribers: ki.stfu, lldb-commits

Differential Revision: https://reviews.llvm.org/D47838

llvm-svn: 334364
2018-06-10 14:58:29 +00:00
Pavel Labath
982241097c Fix TestMiExec.py
r334215 changed the error message the tool prints for invalid thread
arguments to -exec-next command. This adjust the test to match that.

llvm-svn: 334279
2018-06-08 10:39:55 +00:00
Pavel Labath
410c5acf27 Fixup r331049 (FileSpec auto-normalization)
A typo in the patch (using syntax instead of m_syntax) resulted in the
normalization not working properly for windows filespecs when the syntax
was passed as host-native. This did not affect the unit tests, as all of
those pass an explicity syntax, but failed gloriously when running the
full test suite.

I also fix an expectation in an lldb-mi test, which was now failing
because it was expecting a path to be echoed verbatim, but we were now
normalizing it.

As a drive-by, this also fixes the default-in-fully-covered-switch
warning and removes an unused argument from the NeedsNormalization
function.

llvm-svn: 331172
2018-04-30 12:59:14 +00:00
Frederic Riss
60b94c296e [debugserver] Fix the G packet handling.
Of course r331004 needed a counterpart on the write side.

llvm-svn: 331073
2018-04-27 19:59:42 +00:00
Frederic Riss
ffb8d8711c Mark test as @skipIfOutOfTreeDebugserver
This test will currently fail for people using the system debugserver.

llvm-svn: 331043
2018-04-27 15:30:09 +00:00
Frederic Riss
34ec0bfb5e [debugserver] Fix handling of the 'g' packet
LLDB doesn't use this packet so we never hit this, but it looks like
some other projects talk to debugserver and are hitting an assert
(https://github.com/derekparker/delve/issues/1015).

We had an off by 1 in the accounting of the FPU structure sizes.
I added a test that basically just check that 'g' doesn't return
an error (currently it assert in debug builds). I didn't make
it an lldb-server test because it looks like lldb-server doesn't
implement the g packet.

llvm-svn: 331004
2018-04-27 00:09:04 +00:00
Frederic Riss
05e8bc85e6 [debugserver] Return 'ios' instead of 'iphoneos' for the ostype.
When I merged the 2 codepaths that return an OS type, I hade
checked that the places accepting 'iphoneos' would also accept
'ios', but then I got it backwards and return 'iphoneos'.

We use this value to build triples, and there 'iphoneos' is
invalid.

This also makes the test slightly simpler.

llvm-svn: 330877
2018-04-25 22:12:12 +00:00
Davide Italiano
fccc58a99d [lldb-server] Set a more generous timeout when testing gdbremote.
One of our downstream bot is struggling under load,  but this
value should be enough for everyone.

llvm-svn: 329426
2018-04-06 17:17:20 +00:00
Frederic Riss
cd90f878d4 [debugserver] Fix LC_BUILD_VERSION load command handling.
Summary:
In one of the 2 places the LC_BUILD_VERSION load command is handled, there
is a bug preventing us from actually handling them (the address where to
read the load command was not updated). This patch factors reading the
deployment target load commands into a helper and adds testing for the 2
code paths calling the helper.

The testing is a little bit complicated because the only times those load
commands matter is when debugging a simulator process. I added a new
decorator to check that a specific SDK is available. The actual testing was
fairly easy once I knew how to run a simulated process.

Reviewers: jasonmolenda, labath

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D45298

llvm-svn: 329374
2018-04-06 04:28:12 +00:00
Pavel Labath
5af3fb2b94 [LLDB][PPC64] Fix TestGdbRemoteAuxvSupport
Summary: PPC64's auxvec has a special key that must be ignored.

Reviewers: clayborg, labath

Reviewed By: clayborg, labath

Subscribers: alexandreyy, lbianc

Differential Revision: https://reviews.llvm.org/D43771
Patch by Leandro Lupori <leandro.lupori@gmail.com>.

llvm-svn: 328486
2018-03-26 12:00:52 +00:00
Pavel Labath
e92ecd3196 Fix some tests for PPC64le architecture
Summary:
- Fix test jump for powerpc64le
  Jumping directly to the return line on power architecture dos not means
  returning the value that is seen on the code. The last test fails, because
  it needs the execution of some assembly in the beginning of the function.
  Avoiding this test for this architecture.

- Avoid evaluate environ variable name on Linux
  On Linux the Symbol environ conflicts with another variable, then in
  order to avoid it, this test was moved into a specific test, which is not
  supported if the OS is Linux.

- Added PPC64le as MIPS behavior
  Checking the disassembler output, on PPC64le machines behaves as MPIS.
  Added method to identify PPC64le architecture and checking it when
  disassembling instructions in the test case.

Reviewers: labath

Reviewed By: labath

Subscribers: clayborg, labath, luporl, alexandreyy, sdardis, ki.stfu, arichardson

Differential Revision: https://reviews.llvm.org/D44101
Patch by Leonardo Bianconi <leonardo.bianconi@eldorado.org.br>.

llvm-svn: 327977
2018-03-20 12:46:33 +00:00
Vedant Kumar
1d2bd15821 [test] Skip flaky tests in TestMiSyntax on Darwin
These tests tend to time out locally and on our bots.

llvm-svn: 327906
2018-03-19 20:50:50 +00:00
Pavel Labath
63047d05c0 [dotest] Clean up test folder clean-up
Summary:
This patch implements a unified way of cleaning the build folder of each
test. This is done by completely removing the build folder before each
test, in the respective setUp() method. Previously, we were using a
combination of several methods, each with it's own drawbacks:
- nuking the entire build tree before running dotest: the issue here is
  that this did not take place if you ran dotest manually
- running "make clean" before the main "make" target: this relied on the
  clean command being correctly implemented. This was usually true, but
  not always.
- for files which were not produced by make, each python file was
  responsible for ensuring their deleting, using a variety of methods.

With this approach, the previous methods become redundant. I remove the
first two, since they are centralized. For the other various bits of
clean-up code in python files, I indend to delete it when I come
across it.

Reviewers: aprantl

Subscribers: emaste, ki.stfu, mgorny, eraman, lldb-commits

Differential Revision: https://reviews.llvm.org/D44526

llvm-svn: 327703
2018-03-16 12:04:46 +00:00
Vedant Kumar
80a0f020fa Skip more lldb-mi tests which time out on Darwin
Bot failure: https://ci.swift.org/job/oss-lldb-incremental-osx/1097/testReport/junit/TestMiTarget/MiTargetTestCase/test_lldbmi_target_attach_wait_for/

llvm-svn: 327692
2018-03-16 04:11:03 +00:00
Pavel Labath
107052ff9d Next batch of test-tree-cleaning changes
Summary:
The changes here fall into several categories.

- some tests were redirecting inferior stdout/err to a file. For these I
  make sure we use an absolute path for the file. I also create a
  lldbutil.read_file_on_target helper function to encapsulate the
  differences between reading a file locally and remotely.
- some tests were redirecting the pexpect I/O into a file. For these I
  use a python StringIO object to avoid creating a file altogether.
- the TestSettings inferior was creating a file. Here, I make sure the
  inferior is launched with pwd=build-dir so that the files end up
  created there.
- lldb-mi --log (used by some tests) creates a log file in PWD without
  the ability say differently. To make this work I make sure to run
  lldb-mi with PWD=build_dir. This in turn necessitated a couple of
  changes in other lldb-mi tests, which were using relative paths to
  access the source tree.

Reviewers: aprantl

Subscribers: ki.stfu, mehdi_amini, lldb-commits

Differential Revision: https://reviews.llvm.org/D44159

llvm-svn: 327625
2018-03-15 13:47:09 +00:00
Vedant Kumar
5df2db1216 [test] Skip more lldb-mi tests which occasionally time out on Darwin
llvm-svn: 327586
2018-03-14 22:52:32 +00:00
Vedant Kumar
ffc71c0a19 [test] Skip some lldb-mi tests which time out on Darwin
These don't always timeout, but it's inconvenient when they do.

llvm-svn: 327568
2018-03-14 20:32:10 +00:00
Vedant Kumar
bee500becb [test] Delete some xfailed lldb-mi tests
This is a first pass at removing some lldb-mi tests which have been
xfailed and unmaintained for a while. We have open PRs for most of these
tests already. I've opened up the following additional PRs:

  llvm.org/PR36739 - lldb-mi driver exits properly
  llvm.org/PR36740 - lldb-mi -gdb-set and -gdb-show
  llvm.org/PR36741 - lldb-mi -symbol-xxx

The motivation here is to address timeout and pexpect-related issues in
the test suite. This was discussed on lldb-dev in the thread: "increase
timeout for tests?".

After this change, the lldb-mi tests seem to be in better health (on
Darwin at least). I consistently get:

$ ./bin/llvm-dotest -p TestMi
===================
Test Result Summary
===================
Test Methods:        101
Reruns:                0
Success:              88
Expected Failure:      0
Failure:               0
Error:                 0
Exceptional Exit:      0
Unexpected Success:    0
Skip:                 13
Timeout:               0
Expected Timeout:      0

llvm-svn: 327552
2018-03-14 18:37:13 +00:00
Pavel Labath
b9923f049e [LLDB][PPC64] Fix single step and LldbGdbServer tests
Summary:
On PPC64, the tested functions were being entered through their local entry point, while the tests expected the program to stop at the function start address, that, for PPC64, corresponds to the global entry point.

To fix the issue, the test program was modified to call the functions to be tested through function pointers, which, on PPC64, force the calls through the global entry point, while not affecting the test on other platforms.

Reviewers: clayborg, labath

Reviewed By: labath

Subscribers: alexandreyy, lbianc

Differential Revision: https://reviews.llvm.org/D43768
Patch by Leandro Lupori <leandro.lupori@gmail.com>.

llvm-svn: 327013
2018-03-08 15:41:13 +00:00
Adrian Prantl
bff272faa9 Add test for lldb-mi interpreter
Test that "lldb-mi --interpreter" can interpret "target list" CLI command.

Patch by Alex Polyakov!

Differential Revision: https://reviews.llvm.org/D44040

llvm-svn: 326847
2018-03-06 23:25:04 +00:00
Pavel Labath
5e5dd70674 [LLDB][PPC64] Fixed issues with expedited registers
Summary:
- reg_nums were missing the end marker entry
- marked FP test to be skipped for ppc64

Reviewers: labath, clayborg

Reviewed By: labath, clayborg

Subscribers: alexandreyy, lbianc, nemanjai, kbarton

Differential Revision: https://reviews.llvm.org/D43767
Patch by Leandro Lupori <leandro.lupori@gmail.com>

llvm-svn: 326775
2018-03-06 11:54:41 +00:00
Vedant Kumar
2ddfe5c9c3 [test] Skip pexpect-based lldb-mi tests on Darwin
These tests fail with a relatively frequently on Darwin machines with
errors such as:

  File ".../lldb/third_party/Python/module/pexpect-2.4/pexpect.py", line 1444, in expect_loop
    raise EOF(str(e) + '\n' + str(self))
EOF: End Of File (EOF) in read_nonblocking(). Empty string style platform.

The unpredictable failures make these tests noisy.

rdar://37046976

llvm-svn: 326739
2018-03-05 20:16:52 +00:00
Pavel Labath
2b2d728c7b Adapt some tests to work with PPC64le architecture
Summary: Merge branch 'master' into adaptPPC64tests

Reviewers: clayborg, alexandreyy, labath

Reviewed By: clayborg, alexandreyy

Subscribers: luporl, lbianc, alexandreyy, lldb-commits

Differential Revision: https://reviews.llvm.org/D42917
Patch by Ana Julia Caetano <ana.caetano@eldorado.org.br>.

llvm-svn: 326369
2018-02-28 20:57:26 +00:00
Pavel Labath
b2f9bb9e54 Move TestGdbRemoteExitCode next to the other llgs tests
This test contained a copy of the inferior used by most of llgs test.
This was done to enable better paralelization, but now it's irrelevant.

llvm-svn: 326218
2018-02-27 18:07:53 +00:00
Pavel Labath
f3a9ab07aa Fix a couple of more tests to not create files in the source tree
Summary:
These were not being flaky, but they're still making the tree dirty.

These tests were using lldbutil.append_to_process_working_directory to
derive the file path so I fix them by modifying the function to return
the build directory for local tests.

Technically, now the path returned by this function does not point to
the process working directory for local tests, but I think it makes
sense to keep the function name, as I think we should move towards
launching the process in the build directory (and I intend to change
this for the handful of inferiors that actually care about their PWD,
for example because they need to create files there).

Reviewers: davide, aprantl

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D43506

llvm-svn: 325690
2018-02-21 15:33:53 +00:00
Pavel Labath
1c7211d754 Shorten socket names in TestPlatformProcessConnect
The test was generating long unix socket names, and the addition of a
new folder in the previous patch pushed it over the limit (although
linux has a fairly generous limit for path names, this does not apply to
unix sockets).

Modify the test to use a shorter name instead.

llvm-svn: 325340
2018-02-16 12:57:35 +00:00
Davide Italiano
fc58346bf0 [lldb-mi] This test now passes consistently, unXFAIL it.
llvm-svn: 324768
2018-02-09 19:35:07 +00:00
Adrian Prantl
5ec76fe720 Compile the LLDB tests out-of-tree.
This patch is the result of a discussion on lldb-dev, see
http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for
background.

For each test (should be eventually: each test configuration) a
separate build directory is created and we execute

  make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test

In order to make this work all LLDB tests need to be updated to find
the executable in the test build directory, since CWD still points at
the test's source directory, which is a requirement for unittest2.

Although we have done extensive testing, I'm expecting that this first
attempt will break a few bots. Please DO NOT HESITATE TO REVERT this
patch in order to get the bots green again. We will likely have to
iterate on this some more.

Differential Revision: https://reviews.llvm.org/D42281

llvm-svn: 323803
2018-01-30 18:29:16 +00:00
Davide Italiano
b0edeb90ab [test-suite] UNXfail several tests that now pass locally.
Another step towards enabling unexpected successes as failures
by default.

llvm-svn: 323707
2018-01-29 23:24:50 +00:00
Adrian Prantl
eac1c02d2a Move getBuildArtifact() from TestBase to Base and derive MiTestCaseBase from it
Thanks to Pavel Labath for pointing this out!

llvm-svn: 323219
2018-01-23 16:43:01 +00:00
Adrian Prantl
595048f3ec Wrap all references to build artifacts in the LLDB testsuite (NFC)
in TestBase::getBuildArtifact(). This NFC commit is in preparation for
https://reviews.llvm.org/D42281 (compile the LLDB tests out-of-tree).

Differential Revision: https://reviews.llvm.org/D42280

llvm-svn: 323007
2018-01-19 23:24:35 +00:00
Pavel Labath
64b6e5af13 Remove getCategories mechanism of specifying test categories
Summary:
This mechanism was mostly redundant with the file-based .categories
mechanism, and it was interfering with it, as any test which implemented
a getCategories method would not inherit the filesystem categories.

This patch removes it. The existing categories are preserved either by
adding a .categories file, or using the @add_test_categories decorator.

Reviewers: jingham, clayborg, zturner

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D39515

llvm-svn: 317277
2017-11-02 22:13:09 +00:00
Pavel Labath
f877a0820c Use ipv4 localhost address in lldb-server tests
Since the ipv6 patch, we've experienced occasional flakyness in
lldb-server tests. This was due to the fact that lldb-server was trying
to listen on both v4 and v6 localhost sockets (and consider it a success
if at least one of them succeeded), while the test framework was only
trying to connect to the v4 one.

This change makes sure lldb-server only listens on the v4 socket.

llvm-svn: 316391
2017-10-23 23:15:37 +00:00
Jason Molenda
8652b249e6 Initial patchset to get the testsuite running against armv7 and arm64 iOS devices.
Normal customer devices won't be able to run these tests, we're hoping to get
a public facing bot set up at some point.  Both devices pass the testsuite without
any errors or failures.

I have seen some instability with the armv7 test runs, I may submit additional patches
to address this.  arm64 looks good.

I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same.  I know it's a rather large patch.

One change I had to make specifically for iOS devices was that debugserver can't 
create files.  There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file.  They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.

llvm-svn: 314132
2017-09-25 18:19:39 +00:00