54 Commits

Author SHA1 Message Date
Zachary Turner
95b533fe4b Revert "Resubmit r258759 with proper unicode handling."
This reverts commit 2c79d60214e146b13b233392a859b4f79340e90e.

llvm-svn: 258978
2016-01-27 19:47:28 +00:00
Zachary Turner
671e6340da Resubmit r258759 with proper unicode handling.
Instead of opening the file in unicode mode, we need only encode
data which potentially has non-ASCII characters as UTF8 before
writing.  This should work across both Python versions, and is
also far simpler than anything else discussed.

llvm-svn: 258969
2016-01-27 19:00:51 +00:00
Zachary Turner
e89a66bd4a Fix some python 3 incompatibilities that went in overnight.
* basestring is not a thing anymore.  Must use `six.string_types`.
* Must use from __future__ import print_function in every new test
  file.

llvm-svn: 258967
2016-01-27 18:49:35 +00:00
Zachary Turner
7289e43831 Refactor some of the xfail / skip decorators to share logic.
Previously the logic of skipIf and expectedFailure were 99%
the same, but they took different sets of arguments since they
were maintained separately, and had slightly differences in
their behavior.  This makes everything consistent, there is now
only one real implementation, and the previous ones are changed
to use the single master implementation.

llvm-svn: 258966
2016-01-27 18:49:31 +00:00
Bhushan D. Attarde
df5f0b448c [LLDB][MIPS] A small fix in GetBreakableLoadAddress() for MIPS
SUMMARY:
    Get the load address for the address given by symbol and function.
    Earlier, this was done for function only, this patch does it for symbol too.
    This patch also adds TestAvoidBreakpointInDelaySlot.py to test this change.
    
    Reviewers: clayborg
    Subscribers: labath, zturner, mohit.bhakkad, sagar, jaydeep, lldb-commits
    Differential Revision: http://reviews.llvm.org/D16049

llvm-svn: 258919
2016-01-27 10:16:30 +00:00
Enrico Granata
dd54a3a887 Reverting r258759 as it is breaking the OSX build
llvm-svn: 258791
2016-01-26 04:53:10 +00:00
Zachary Turner
00b87282cd Write the session log file in UTF-8.
Previously we were writing in the default encoding, which depends
on the operating system and is not guaranteed to be unicode aware.
On Python 3, this would lead to a situation where writing unicode
text to the log file generates an exception.  The fix here is to
write session logs using the proper encoding, which incidentally
fixes another test, so xfail is removed from that.

llvm-svn: 258759
2016-01-26 00:59:42 +00:00
Zachary Turner
8012496a91 Decode files with UTF-8 in lldbutil.line_number.
Since Unicode support is different in Py2 and Py3, Py3 was throwing
exceptions about being unable to decode the file with the default
encoding.

llvm-svn: 258588
2016-01-22 23:54:49 +00:00
Pavel Labath
4b70eb7b2b Revert "Enable test log collection from remote debug servers"
Unfortunately, this turns out not to be working on the lldb-server tests, as there the server is
started in a different way. Since this was a bit of a hack to start with, I am removing it until
I can solve the problem more holistically.

llvm-svn: 258501
2016-01-22 14:50:29 +00:00
Pavel Labath
e3e6be2aab Enable test log collection from remote debug servers
Summary:
We already have the ability to collect the server logs when doing local debugging. This enables
the collection of remote logs as well. This relies on specifying a relative path "server.log" for
LLDB_DEBUGSERVER_LOG_FILE when starting remote platform. Since we always set the platform working
directory to a fresh folder to avoid conflicts, the actual file path will always be different and
we can pick the logs up from there.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D16322

llvm-svn: 258414
2016-01-21 17:54:14 +00:00
Pavel Labath
bace94eda8 Remove skipIfLinuxClang decorator
it isn't used in the code anymore, and we're trying to cut down on the decorators.

llvm-svn: 258049
2016-01-18 15:01:14 +00:00
Pavel Labath
4da73573ab Guard against application of an XFAIL decorator on a class
This does not work and causes the class to be silently skipped, which is a bad idea. This makes
sure it cannot happen accidentaly. I've played with the idea of actually making the decorator
work at class level, but it proved too magic to do at this moment.

llvm-svn: 258048
2016-01-18 14:45:35 +00:00
Adrian McCarthy
aa1f270151 Fix detection of teardown hooks that are already bound.
llvm-svn: 257829
2016-01-14 22:52:16 +00:00
Omair Javaid
baa07490bc Xfail some Arm-Linux specific failures
Updated expectedFailureLinux decorator to reflect architecture

Marked some triaged failures as xfails on arm with updated expectedFailureLinux decorator

Differential revision: http://reviews.llvm.org/D15893

llvm-svn: 257405
2016-01-11 22:52:18 +00:00
Pavel Labath
773e86f255 Remove old flaky test rerun logic
Summary:
This removes the old logic for rerunning flaky tests. The new test runners will take care of
rerunning failing tests.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15855

llvm-svn: 256824
2016-01-05 10:44:36 +00:00
Adrian McCarthy
46155dd7a9 Allow test decorators to use lists (and not_in(list)) for arches
Differential Revision: https://mail.google.com/mail/u/0/?zx=w4areffgjbgg#inbox/151cb6afe6169bb0

llvm-svn: 256283
2015-12-22 21:01:21 +00:00
Pavel Labath
25e241b006 [test] Add ability to expect timeouts
Summary:
This adds ability to mark test that do not complete due to hangs, crashes, etc., as "expected",
to avoid flagging the build red for a known problem. Functionally, this extends the scope of the
existing expectedFailureXXX decorators to cover these states as well. Once this is in, I will
start replacing the magic list of failing tests in dosep.py with our regular annotations which
should hopefully make code simpler.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15530

llvm-svn: 255763
2015-12-16 12:09:45 +00:00
Siva Chandra
a386358c36 Make few adjustments after r255542.
Reviewers: zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15511

llvm-svn: 255584
2015-12-15 00:26:52 +00:00
Zachary Turner
8a927c4c10 Remove the multiplier loop.
This is leading to some kind of subtle issue related to local
functions and closures, so let's just go back to the old way for
now.

llvm-svn: 255567
2015-12-14 22:58:16 +00:00
Zachary Turner
e1eb5e39f5 Make skipIf decorator support not_in() functor.
llvm-svn: 255542
2015-12-14 21:26:49 +00:00
Zachary Turner
f098e4fb19 Make debug info specification use categories system.
Reviewed By: Tamas Berghammer, Pavel Labath
Differential Revision: http://reviews.llvm.org/D15428

llvm-svn: 255525
2015-12-14 18:49:16 +00:00
Pavel Labath
ffbf9e86b2 Make test categories composable
Summary:
Previously the add_test_categories would simply overwrite the current set of categories for a
method. This change makes the decorator truly "add" categories, by extending the current set of
categories instead of replacing it.

To do this, I have:
- replaced the getCategories() property on a method (which was itself a method), with a simple
  list property "categories". This makes add_test_categories easier to implement, and test
  categories isn't something which should change between calls anyway.
- rewritten the getCategoriesForTest function to merge method categories with the categories of
  the test case. Previously, it would just use the method categories if they were present. I have
  also greatly simplified this method. Originally, it would use a lot of introspection to enable
  it being called on various types of objects. Based on my tests, it was only ever being called
  on a test case. The new function uses much less introspection then the preivous one, so we
  should easily catch any stray uses, if there are any, as they will generate exceptions now.

Reviewers: zturner, tfiala, tberghammer

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15451

llvm-svn: 255493
2015-12-14 13:17:18 +00:00
Zachary Turner
742afdb3d2 Remove -S option from dotest.py.
llvm-svn: 255361
2015-12-11 19:21:49 +00:00
Todd Fiala
9187f27e32 Add test event marking a test as explicitly eligible for rerun if it is marked flakey.
This will be used in a future change to support rerunning flakey tests
that hit a test result isue in a low-load, single worker test runner phase.

This is implemented as an additive-style event rather than being
evaluated and added to the start_test event because the decorator code
only runs after the start_test event is created and sent.  i.e.
LLDBTestResult.startTest() runs before the test method decorators run.

llvm-svn: 255351
2015-12-11 18:06:47 +00:00
Kamil Rytarowski
f5d34b7b9d Add NetBSD support in the buildDriver and buildLibrary routines
Summary: NetBSD is like FreeBSD and Linux in these routines.

Reviewers: clay.chang, tfiala, emaste, joerg

Subscribers: lldb-commits, emaste

Differential Revision: http://reviews.llvm.org/D15374

llvm-svn: 255308
2015-12-10 22:56:56 +00:00
Zachary Turner
b08ab72427 Remove -w option from dotest.py.
llvm-svn: 255275
2015-12-10 18:50:32 +00:00
Zachary Turner
76972031fa Remove -k command line option from dotest.py.
This is part of an effort to remove unused command line options.

llvm-svn: 255143
2015-12-09 19:45:51 +00:00
Pavel Labath
7ead0b937c XFAIL TestReturnValue for remote Windows->Android tests
this also adds the ability to match the host platform to the expectedFailureAll decorator.

llvm-svn: 255105
2015-12-09 10:54:18 +00:00
Zachary Turner
d865c6b707 Remove the -c option from dotest.py.
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today.  git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.

llvm-svn: 255060
2015-12-08 22:15:48 +00:00
Zachary Turner
2155d5d301 Remove the -b option from dotest.py
This removes the blacklist option as part of an effort to remove
unused / unmaintained command line options from the test suite.

llvm-svn: 255040
2015-12-08 18:43:16 +00:00
Zachary Turner
aad25fb9a5 Remove +b option from dotest.py
llvm-svn: 255037
2015-12-08 18:36:05 +00:00
Tamas Berghammer
ccd6cffba3 Modify "platform connect" to connect to processes as well
The standard remote debugging workflow with gdb is to start the
application on the remote host under gdbserver (e.g.: gdbserver :5039
a.out) and then connect to it with gdb.

The same workflow is supported by debugserver/lldb-gdbserver with a very
similar syntax but to access all features of lldb we need to be
connected also to an lldb-platform instance running on the target.

Before this change this had to be done manually with starting a separate
lldb-platform on the target machine and then connecting to it with lldb
before connecting to the process.

This change modifies the behavior of "platform connect" with
automatically connecting to the process instance if it was started by
the remote platform. With this command replacing gdbserver in a gdb
based worflow is usually as simple as replacing the command to execute
gdbserver with executing lldb-platform.

Differential revision: http://reviews.llvm.org/D14952

llvm-svn: 255016
2015-12-08 14:08:19 +00:00
Zachary Turner
606e3a5221 Get rid of global variables in dotest.py
This moves all the global variables into a separate module called
`configuration`.  This has a number of advantages:

1. Configuration data is centrally maintained so it's easy to get
   a high level overview of what configuration data the test suite
   makes use of.
2. The method of sharing configuration data among different parts
   of the test suite becomes standardized.  Previously we would
   put some things into the `lldb` module, some things into the
   `lldbtest_config` module, and some things would not get shared.
   Now everything is shared through one module and is available to
   the entire test suite.
3. It opens the door to moving some of the initialization code into
   the `configuration` module, simplifying the implementation of
   `dotest.py`.

There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily.  This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable.  We can address these in the future.

llvm-svn: 254982
2015-12-08 01:15:30 +00:00
Kamil Rytarowski
49f9fb8d26 Add initial NetBSD support in lldbsuite/test/lldbtest.py
Summary:
Add new functions:

  - expectedFailureNetBSD()
  - expectedFlakeyNetBSD()
  - skipIfNetBSD()

Add new NetBSD entry in:

  - getPlatform()
  - getHostPlatform()

Assume that libc++ is installed and use the GNU toolchain

Reviewers: joerg, emaste, tfiala, clayborg

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15262

llvm-svn: 254948
2015-12-07 21:25:57 +00:00
Kamil Rytarowski
0b655da7e4 Define new builder_netbsd
Summary: This is used in tests.

Reviewers: emaste, tfiala, clayborg

Subscribers: zturner, lldb-commits, joerg

Differential Revision: http://reviews.llvm.org/D15065

llvm-svn: 254853
2015-12-05 18:46:56 +00:00
Siva Chandra
7dcad3178b Revert "Make skipIf support the not_in function (second attempt)."
Summary: This reverts commit 70dca28976ee8137acce2cc203dd394f4d761276.

Reviewers: amccarth, zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14881

llvm-svn: 253704
2015-11-20 20:30:36 +00:00
Adrian McCarthy
4d5d1dd6d5 Make skipIf support the not_in function (second attempt).
llvm-svn: 253683
2015-11-20 18:14:24 +00:00
Siva Chandra
d2e90129d9 Revert "Make skipIf support the not_in function."
Summary:
This reverts commit 251965377bdfb6227eea42c12a792c059e4e8a4b
as a test marked "skipIf(compiler='gcc')" runs when testing with GCC.

Reviewers: amccarth

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14852

llvm-svn: 253631
2015-11-20 01:54:24 +00:00
Adrian McCarthy
257da8ea50 Make skipIf support the not_in function.
llvm-svn: 253623
2015-11-20 00:11:38 +00:00
Enrico Granata
55d99f0e7c Cleanup work required to get the ASAN tests to run in the new test suite infrastructure; the tests are now xfailed on Darwin pending investigation
llvm-svn: 253604
2015-11-19 21:45:07 +00:00
Ying Chen
ca922bb9b9 Support unix-abstract-connect scheme as platform url in lldb testsuite
Reviewers: ovyalov

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14765

llvm-svn: 253488
2015-11-18 19:03:20 +00:00
Zachary Turner
48ef8d4c37 Fix some issues with swig & string conversion.
This patch fixes two issues:

1) Popen needs to be used with universal_newlines=True by default.
   This elicits automatic decoding from bytes -> string in Py3,
   and has no negative effects in other Py versions.
2) The swig typemaps for converting between string and (char*, int)
   did not work correctly when the length of the string was 0,
   indicating an error.  In this case we would try to construct a
   string from uninitialized data.
3) Ironically, the bug mentioned in #2 led to a test passing on
   Windows that was actually broken, because the test was written
   such that the assertion was never even getting checked, so it
   passed by default.  So we additionally fix this test to also
   fail if the method errors.  By fixing this test it's now broken
   on Windows, so we also xfail it.

llvm-svn: 253487
2015-11-18 18:40:16 +00:00
Zachary Turner
ba1057022c Python 3 - Skip a certain test for a particular (swig,python) combo.
Current versions of SWIG have a bug with Python 3 that causes
Python to assert when iterating over a generator.  This patch
skips the test for the right combination of Python version and
SWIG version.  I'm attempting to upstream a patch to SWIG to
fix this in a subsequent as-of-yet unreleased version, but
I don't know how long that will take.

llvm-svn: 253273
2015-11-16 23:58:20 +00:00
Ying Chen
0c35282c65 Re-Apply "Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll ..." with fix
Summary:
- Re-Commit r253106
- Initialize self.debug_info in Base::setUp()
- Fix argument order when calling check_list_or_lambda for compiler

Reviewers: jingham

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14673

llvm-svn: 253272
2015-11-16 23:41:02 +00:00
Zachary Turner
abdb839228 Add the ability to xfail or skip based on swig / python version.
llvm-svn: 253263
2015-11-16 22:40:30 +00:00
Pavel Labath
311cc7d133 Revert "Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll to reverse"
This reverts commit r253106.

llvm-svn: 253197
2015-11-16 11:11:10 +00:00
Jim Ingham
6d048942c5 Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll to reverse
the sense of the test.

llvm-svn: 253106
2015-11-14 00:20:33 +00:00
Zachary Turner
8d13fab183 Python 3 - Don't use unbuffered I/O in text mode.
This is unsupported in Python 3.  This could also have been fixed
by using "wb" instead of "w", but it doesn't seem like writing the
session log absolutely *needs* to be unbuffered.

llvm-svn: 252381
2015-11-07 01:08:15 +00:00
Zachary Turner
5cb8e67b17 Don't use module internal implementation details in our decorators.
We tried implementing something akin to a conditionalExpectedFailure
decorator for unittest2.  We did this by making use of some
implementation details of the unittest2 module.  In an effort to make
this work with unittest, this patch removes the reliance on the
implementation details.  I have a hard time wrapping my head around
how this all works with the deeply nested decorators, but the spirit
of the patch here is to do do the following: If the condition function
is true, use the original unittest2.expectedFailure decorator.  Otherwise
don't use any decorator, just call the test function.

Differential Revision: http://reviews.llvm.org/D14406
Reviewed By: tberghammer, labath

llvm-svn: 252326
2015-11-06 18:14:42 +00:00
Zachary Turner
c1b7cd72db Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.

When absolute imports are enabled, the import system changes in
a couple of ways:

1) The `import foo` syntax will *only* search sys.path.  If `foo`
   isn't in sys.path, it won't be found.  Period.  Without absolute
   imports, the import system will also search the same directory
   that the importing file resides in, so that you can easily
   import from the same folder.

2) From inside a package, you can use a dot syntax to refer to higher
   levels of the current package.  For example, if you are in the
   package lldbsuite.test.utility, then ..foo refers to
   lldbsuite.test.foo.  You can use this notation with the
   `from X import Y` syntax to write intra-package references.  For
   example, using the previous locationa s a starting point, writing
   `from ..support import seven` would import lldbsuite.support.seven

Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.

See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.

Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala

llvm-svn: 252191
2015-11-05 19:22:28 +00:00