The logic was incorrect because on windows, we need to look for
yaml2obj.EXE. I implement the search in terms of
distutils.spawn.find_executable, which should handle the platform
differences for us.
llvm-svn: 323744
Summary:
Adds new utilities that make it easier to write test cases for lldb acting as a client over a gdb-remote connection.
- A GDBRemoteTestBase class that starts a mock GDB server and provides an easy way to check client packets
- A MockGDBServer that, via MockGDBServerResponder, can be made to issue server responses that test client behavior.
- Utility functions for handling common data encoding/decoding
- Utility functions for creating dummy targets from YAML files
----
Split from the review at https://reviews.llvm.org/D42145, which was a new feature that necessitated the new testing capabilities.
Reviewers: clayborg, labath
Reviewed By: clayborg, labath
Subscribers: hintonda, davide, jingham, krytarowski, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D42195
Patch by Owen Shaw <llvm@owenpshaw.net>
llvm-svn: 323636
Summary:
This mechanism was mostly redundant with the file-based .categories
mechanism, and it was interfering with it, as any test which implemented
a getCategories method would not inherit the filesystem categories.
This patch removes it. The existing categories are preserved either by
adding a .categories file, or using the @add_test_categories decorator.
Reviewers: jingham, clayborg, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D39515
llvm-svn: 317277
dwarf&dwo versions were doing it, but gmodules and dsym weren't. All
this function does right now is pass OS=Android to make when targeting
android. This enables us to run dotest without manually passing --env
OS=Android.
llvm-svn: 317130
Normal customer devices won't be able to run these tests, we're hoping to get
a public facing bot set up at some point. Both devices pass the testsuite without
any errors or failures.
I have seen some instability with the armv7 test runs, I may submit additional patches
to address this. arm64 looks good.
I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same. I know it's a rather large patch.
One change I had to make specifically for iOS devices was that debugserver can't
create files. There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file. They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.
llvm-svn: 314132
Normal customer devices won't be able to run these devices, we're hoping to get
a public facing bot set up at some point. Both devices pass the testsuite without
any errors or failures.
I have seen some instability with the armv7 test runs, I may submit additional patches
to address this. arm64 looks good.
I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same. I know it's a rather large patch.
One change I had to make specifically for iOS devices was that debugserver can't
create files. There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file. They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.
llvm-svn: 314038
Normal customer devices won't be able to run these devices, we're hoping to get
a public facing bot set up at some point.
There will be some smaller follow-on patches. The changes to tools/lldb-server are
verbose and I'm not thrilled with having to skip all of these tests manually.
There are a few places where I'm making the assumption that "armv7", "armv7k", "arm64"
means it's an ios device, and I need to review & clean these up with an OS check
as well. (Android will show up as "arm" and "aarch64" so by pure luck they shouldn't
cause problems, but it's not an assumption I want to rely on).
I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same. I know it's a rather large patch.
One change I had to make specifically for iOS devices was that debugserver can't
create files. There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file. They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.
llvm-svn: 313932
For remote targets we need to call registerSharedLibrariesWithTarget to
make sure they are installed alongside main executable. This also
required a small fixup in the the mentioned function as in this case
"One" was both a directory name and a library name template. I fixed it
to make sure it checks that the string refers to a file before it
assumed it was a full library path.
llvm-svn: 303248
r298203 make SBPlatform::MakeDirectory less recursive, which breaks the
test suite creation of test directory hierarchy creation on the remote
target. Since the function was never fully recursive, and the name does
not imply recursiveness, I fix the problem by modifying the test runner
to do the recursion manually.
I also make the runner complain more loudly when it fails to create the
directory -- previously it just printed the error to stdout and caused
most of the tests to hang, which is not very helpful in diagnosing the
problem.
llvm-svn: 298261
Summary:
This has been broken at least since the new test result framework was
added, which was over a year ago. It looks like nobody has missed it
since.
Removing this makes the gmodules handling code saner, as it already did
not know how to handle the multiple-compilers case.
My motivation for this is libc++ data formatters support on android -- I
am trying make a central way of determining whether libc++ tests can be
run, and without this, I would have to resort to similar hacks as the
gmodules code.
Reviewers: jingham, zturner
Subscribers: danalbert, tfiala, lldb-commits
Differential Revision: https://reviews.llvm.org/D30779
llvm-svn: 297811
Make sure we don't generate extremely long file names for test trace log
file, as this can cause path-too-long errors. As the compilers in the
android ndk are deeply nested, it's very easy to trigger these.
I chose to output at most 4 path components -- this should keep the full
path for common cases like /usr/bin/gcc with room to spare, and should
be enough to uniquely identify the compiler for more deeply nested
cases.
llvm-svn: 296870
This is a large API change that removes the two functions from
StreamString that return a std::string& and a const std::string&,
and instead provide one function which returns a StringRef.
Direct access to the underlying buffer violates the concept of
a "stream" which is intended to provide forward only access,
and makes porting to llvm::raw_ostream more difficult in the
future.
Differential Revision: https://reviews.llvm.org/D26698
llvm-svn: 287152
Summary:
Convert tests using LLDB headers to use generateSource to put the right include paths in place regardless of whether or not you're building a framework.
This also abstracted generateSource out of TestPublicAPIHeaders.py into lldbtest.py.
Reviewers: tfiala, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D25887
llvm-svn: 285542
Summary:
dotest.py has a framework option that is not respected. This patch makes the framework path properly configurable via the --framework option.
This patch also adds a function to the lldbtest.Base class named "hasDarwinFramework" which allows us to not rely on the host platform to determine if a framework is present. If running on Darwin, and not building a framework, this will follow the *nix code paths which are appropriate for Darwin.
Reviewers: tfiala
Subscribers: lldb-commits, mgorny
Differential Revision: https://reviews.llvm.org/D25886
llvm-svn: 285541
This reverts commit r285357.
I committed this patch accidentally out of order. Will recommit when the change this depends on is landed.
llvm-svn: 285361
Summary:
Convert tests using LLDB headers to use generateSource to put the right include paths in place regardless of whether or not you're building a framework.
This also abstracted generateSource out of TestPublicAPIHeaders.py into lldbtest.py.
Reviewers: tfiala, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D25887
llvm-svn: 285357
Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden.
Reviewers: zturner, tfiala
Subscribers: labath, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D25714
llvm-svn: 284893
This change introduces optional marking of the column within a source
line where a thread is stopped. This marking will show up when the
source code for a thread stop is displayed, when the debug info
knows the column information, and if the optional column marking is
enabled.
There are two separate methods for handling the marking of the stop
column:
* via ANSI terminal codes, which are added inline to the source line
display. The default ANSI mark-up is to underline the column.
* via a pure text-based caret that is added in the appropriate column
in a newly-inserted blank line underneath the source line in
question.
There are some new options that control how this all works.
* settings set stop-show-column
This takes one of 4 values:
* ansi-or-caret: use the ANSI terminal code mechanism if LLDB
is running with color enabled; if not, use the caret-based,
pure text method (see the "caret" mode below).
* ansi: only use the ANSI terminal code mechanism to highlight
the stop line. If LLDB is running with color disabled, no
stop column marking will occur.
* caret: only use the pure text caret method, which introduces
a newly-inserted line underneath the current line, where
the only character in the new line is a caret that highlights
the stop column in question.
* none: no stop column marking will be attempted.
* settings set stop-show-column-ansi-prefix
This is a text format that indicates the ANSI formatting
code to insert into the stream immediately preceding the
column where the stop column character will be marked up.
It defaults to ${ansi.underline}; however, it can contain
any valid LLDB format codes, e.g.
${ansi.fg.red}${ansi.bold}${ansi.underline}
* settings set stop-show-column-ansi-suffix
This is the text format that specifies the ANSI terminal
codes to end the markup that was started with the prefix
described above. It defaults to: ${ansi.normal}. This
should be sufficient for the common cases.
Significant leg-work was done by Adrian Prantl. (Thanks, Adrian!)
differential review: https://reviews.llvm.org/D20835
reviewers: clayborg, jingham
llvm-svn: 282105
*** to conform to clang-format’s LLVM style. This kind of mass change has
*** two obvious implications:
Firstly, merging this particular commit into a downstream fork may be a huge
effort. Alternatively, it may be worth merging all changes up to this commit,
performing the same reformatting operation locally, and then discarding the
merge for this particular commit. The commands used to accomplish this
reformatting were as follows (with current working directory as the root of
the repository):
find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} +
find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ;
The version of clang-format used was 3.9.0, and autopep8 was 1.2.4.
Secondly, “blame” style tools will generally point to this commit instead of
a meaningful prior commit. There are alternatives available that will attempt
to look through this change and find the appropriate prior commit. YMMV.
llvm-svn: 280751
Summary:
This is a slightly reworked version of D16322, which I had reverted because it did not do what it
advertised. Differences from the previous version are:
- moved the code for cleaning up the remote working dir to a later point as it was removing the
log file before we could get to it.
- specialised log downloading code for gdb-remote tests is not needed, as this will cover that
use case as well.
Reviewers: tfiala
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D21898
llvm-svn: 274491
This change adds the capability of building test inferiors
with the -gmodules flag to enable module debug info support.
Windows is excluded per @zturner.
Reviewers: granata.enrico, aprantl, zturner, labath
Subscribers: zturner, labath, lldb-commits
Differential Revision: http://reviews.llvm.org/D19998
llvm-svn: 270848
Summary:
print build errors nicely in test output
This test infrastructure change adds a new Python exception
for test subject builds that fail. The output of the build
command is captured and propagated to both the textual test
output display code and to the test event system.
The ResultsFormatter objects have been modified to do something
more useful with this information. The xUnit formatter
now replaces the non-informative Python build error stacktrace
with the build error content. The curses ResultsFormatter
prints a 'B' for build errors rather than 'E'.
The xUnit output, in particular, makes it much easier for
developers to track down test subject build errors that cause
test failures when reports come in from CI.
Reviewers: granata.enrico
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D20252
llvm-svn: 269525
This change moves all the test event handling and its related
ResultsFormatter classes out of the packages/Python/lldbsuite/test dir
into a packages/Python/lldbsuite/test_event package. Formatters are
moved into a sub-package under that.
I am limiting the scope of this change to just the motion and a few
minor issues caught by a static Python checker (e.g. removing unused
import statements).
This is a pre-step for adding package-level tests to the test event
system. I also intend to simplify test event results formatter selection
after I make sure this doesn't break anybody.
See:
http://reviews.llvm.org/D19288
Reviewed by:
Pavel Labath
llvm-svn: 266885
Previously we had 3 different method to run shell commands on the
target and 4 copy of code waiting until a given file appears on the
target device (used for syncronization). This CL merges these methods
to 1 run_platform_command and 1 wait_for_file_on_target functions
located in some utility classes.
Differential revision: http://reviews.llvm.org/D18789
llvm-svn: 265398
Summary:
Debug info is used only by the client and lldb-server tests do not even have the client component
running, as they communicate with the server directly. Therefore, running the tests for each
debug info type is unnecessarry.
This adds general ability to mark a test class as not dependent on debug info, and marks all
lldb-server tests as such.
Reviewers: tberghammer, tfiala
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D18598
llvm-svn: 265017
self.expect() had two problems:
- If there was a substrs argument, then it overwrote the variable containing
the command to run with the last substr. That meant nonsense command text in
testsuite errors.
- The actual output is not printed, which makes fixing testsuite failures a bit
annoying (you end up having to use the -tv arguments to dotest).
This fixes both of these issues. We could do even better, pretty-printing the
criteria for "correct" output, but this at least makes dealing with errors a bit
better.
llvm-svn: 262950
Summary:
This does not yet give us a clean testsuite run but it does help with:
1. Actually building on linux
2. Run the testsuite with over 70% tests passing on linux.
Reviewers: tfiala, labath, zturner
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D17182
llvm-svn: 260721
This doesn't attempt to move every decorator. The reason for
this is that it requires touching every single test file to import
decorators.py. I would like to do this in a followup patch, but
in the interest of keeping the patches as bite-sized as possible,
I've only attempted to move the underlying common decorators first.
A few tests call these directly, so those tests are updated as part
of this patch.
llvm-svn: 259807
My eventual goal is to move all of the test decorators to their
own module such as `decorators.py`. But some of the decorators
use existing functions in `lldbtest.py` and conceptually the
functions are probably more appropriately placed in lldbplatformutil.
Moreover, lldbtest.py is a huge file with a ton of random utility
functions scattered around, so this patch also workds toward the
goal of reducing the footprint of this one module to a more
reasonable size.
So this patch moves some of them over to lldbplatformutil with the
eventual goal of moving decorators over to their own module.
Reviewed By: Tamas Berghammer, Pavel Labath
Differential Revision: http://reviews.llvm.org/D16830
llvm-svn: 259680
This decorator was used in only one test, and it's behaviour was quite complicated. It skipped
if:
- test was remote
- platform was *not* android
I am not aware of anyone running tests with this configuration (and even then, I am not aware of
a reason why the test should not pass), but if TestLoadUnload starts breaking for you after this
commit, please disable the test with
@expectedFailureAll(remote=True, oslist=[YOUR_PLATFORM])
llvm-svn: 259642
Previously we were returning a tuple of (bool, skip_reason) from
the tuple function. This makes for some awkward code, especially
since a value of True for the first argument implies that the
second argument is None, and a value of False implies that the
second argument is not None. So it was basically redundant, and
with this patch we simply return the skip reason or None directly.
llvm-svn: 259590
This should be no functional change, just a refactoring of the
skip decorators to all centralize on a single function,
`skipTestIfFn` that does all the logic. This allows easier
maintenance of the decorators and also centralizes all the
hard-to-understand logic in one place.
Reviewed by: Pavel Labath
Differential Revision: http://reviews.llvm.org/D16741
llvm-svn: 259543
r259433 introduced a regression, where if a compiler is specified without a path (e.g., CC=clang,
relying on the fact that clang is in $PATH), then the test suite would fail (at the compiler
version detection step) because realpath would interpret this as a path relative to cwd). The fix
is to perform the $PATH expansion (via `which`) before the realpath step.
llvm-svn: 259484
This patch attempts to solve the Python 2 / Python 3 incompatibilities by
introducing a new `encoded_file` abstraction that we use instead of
`io.open()`. The problem with the builtin implementation of `io.open` is
that `read` and `write` accept and return `unicode` objects, which are not
always convenient to work with in Python 2. We solve this by making
`encoded_file.open()` return the same object returned by `io.open()` but
with hooked `read()` and `write()` methods. These hooked methods will
accept binary or text data, and conditionally convert what it gets to a
`unicode` object using the correct encoding. When calling `read()` it
also does any conversion necessary to convert the output back into the
native `string` type of the running python version.
Differential Revision: http://reviews.llvm.org/D16736
llvm-svn: 259379
Instead of opening the file in unicode mode, we need only encode
data which potentially has non-ASCII characters as UTF8 before
writing. This should work across both Python versions, and is
also far simpler than anything else discussed.
llvm-svn: 258969
* basestring is not a thing anymore. Must use `six.string_types`.
* Must use from __future__ import print_function in every new test
file.
llvm-svn: 258967
