Summary:
While working on r370054, i've found it frustrating that the test output
was compeletely unhelpful in case of failures. Therefore I've decided to
improve that. In this I reuse the PExpectTest class, which was one of
our mechanisms for running pexpect tests, but which has gotten orhpaned
in the mean time.
I've replaced the existing send methods with a "expect" method, which
I've tried to design so that it has a similar interface to the expect
method in regular non-pexpect dotest tests (as it essentially does
something very similar). I've kept the ability to dump the transcript of
the pexpect communication to stdout in the "trace" mode, as that is a
very handy way to figure out what the test is doing. I've also removed
the "expect_string" method used in the existing tests -- I've found this
to be unhelpful because it hides the message that would be normally
displayed by the EOF exception. Although vebose, this message includes
some important information, like what strings we were searching for,
what were the last bits of lldb output, etc. I've also beefed up the
class to automatically disable the debug info test duplication, and
auto-skip tests when the host platform does not support pexpect.
This patch ports TestMultilineCompletion and TestIOHandlerCompletion to
the new class. It also deletes TestFormats as it is not testing anything
(definitely not formats) -- it was committed with the test code
commented out (r228207), and then the testing code was deleted in
r356000.
Reviewers: teemperor, JDevlieghere, davide
Subscribers: aprantl, lldb-commits
Differential Revision: https://reviews.llvm.org/D66954
llvm-svn: 370449
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
It used to be possible to enable logging through environment variables
read by dotest. This approach is deprecated, as stated in the dotest
help output. Instead --channel should be used.
Differential revision: https://reviews.llvm.org/D66920
llvm-svn: 370387
This removes the curses result formatter which appears to be broken.
Passing --curses to dotest.py screws up my terminal and doesn't run any
tests. It even crashes Python on occasion.
Differential revision: https://reviews.llvm.org/D66917
llvm-svn: 370386
This removes support for reading the LLDB_TEST_ARGUMENTS environment
variable and instead requires all arguments to be specified as part of
the invocation. This ensures that dotest.py invocations are easily
repeatable.
Differential revision: https://reviews.llvm.org/D66912
llvm-svn: 370278
This argument was used by dosep.py to pass information around from the
workers. With dosep.py gone, I'm fairly sure we don't need this any
longer.
llvm-svn: 370266
I was looking at the session directory logic for unrelated reasons and
noticed that the logic spread out across dotest. This simplifies things
a bit by moving the logic together.
llvm-svn: 370259
Disable the two failing tests until Raphael has a chance to investigate:
Failing Tests (2):
lldb-Suite :: functionalities/completion/TestCompletion.py
lldb-Suite :: functionalities/target_command/TestTargetCommand.py
llvm-svn: 370237
Now that all supported build systems create a valid dotest.py
invocation, we no longer need to guess the location of the lldb binary
and Python directory.
Differential revision: https://reviews.llvm.org/D66896
llvm-svn: 370234
Now that all supported build systems create a valid dotest.py
invocation, we no longer need to guess the directory where any of the
llvm tools live. Additionally, the current logic is incomplete: it
doesn't try to find any other tools than FileCheck, such as dsymutil for
example.
If no FileCheck is provided, we should print a warning and skip the
tests that need it, but that's not part of this patch.
Differential revision: https://reviews.llvm.org/D66893
llvm-svn: 370232
This patch removes the -q (quiet) flag and changing the default
behavior. Currently the flag serves two purposes that are somewhat
contradictory, as illustrated by the difference between the argument
name (quiet) and the configuration flag (parsable). On the one hand it
reduces output, but on the other hand it prints more output, like the
result of individual tests. My proposal is to guard the extra output
behind the verbose flag and always print the individual test results.
Differential revision: https://reviews.llvm.org/D66837
llvm-svn: 370226
Apparently inline tests stop running anything after an empty line
behind an self.expect, which is a very good approach that could
never cause people to write tests that never run.
This patch removes all the empty lines so that all this test
is actually run. Also fixes the broken expects that only passed
because they weren't run before.
llvm-svn: 370195
The refactoring patch for the option completion broke the completion
for ambiguous long options. As this feature was also untested (as
testing ambiguous options with the current test methods is impossible),
I just noticed now. This patch restores the old behavior and adds a
test for this feature.
llvm-svn: 370185
The results port was used by dosep.py to deal with test results coming
form different processes. With dosep.py gone, I don't think we need this
any longer.
Differential revision: https://reviews.llvm.org/D66811
llvm-svn: 370090
pexpect gives as raw data going to a terminal. This means that if the
completed line does not fit the emulated line, the returned data will
contain line breaks. On my machine these line breaks happened to be
inside the "iohandler/completion" string that the test was searching
for.
Work around this by telling pexpect to emulate a very wide terminal.
llvm-svn: 370054
Otherwise dotest doesn't run the test and just lets it always pass.
Also update the comment to explain that we do directory and not
file completion.
llvm-svn: 370047
On the command line we usually insert a space after a completion to indicate that
the completion was successful. After the completion API refactoring, this also
happens with directories which essentially breaks file path completion (as
adding a space terminates the path and starts a new arg). This patch restores the old
behavior by again allowing partial completions. Also extends the iohandler
and SB API tests as the implementation for this is different in Editline
and SB API.
llvm-svn: 370043
Today I discovered the skipLongRunningTest decorator and to my surprise
all the tests were passing without the decorator. They don't seem to be
that expensive either, they take a few seconds but we have tests that
take much longer than that. As such I propose to remove the decorator
and enable them by default.
Differential revision: https://reviews.llvm.org/D66774
llvm-svn: 369995
We have started to see the no_binary version of this test
fail. The reason is that the binary was being removed
before the spawn actually launched the inferior. Add a
simple filesystem based synchronization to avoid this race.
llvm-svn: 369930
With this patch dotest.py will print the full invocation whenever it
fails to parse its arguments. The dotest invocation is usually build up
with different inputs, potentially coming from CMake, lldb-dotest, lit
or passed directly. This can make debugging hard, especially on CI,
where there might be another layer of indirection. This aims to make
that a bit easier.
llvm-svn: 369922
It looks like running without this argument was supported
for legacy reasons, but a Xcode 11 beta made the argument
mandatory for our usecase.
llvm-svn: 369709
the platform in the setUp/tearDown methods. I want to migrate the
re-instatement of the correct plaform to the setUp base method but
haven't had time to look at that yet, so I want to land this handful
of fixes until I get to it.
Differential revision: https://reviews.llvm.org/D66331
llvm-svn: 369484
We cannot override the DWARF version in the CFLAGS_EXTRA because they
are used by tests that explicitly build without debug info. Instead, we
pass them through the regular CFLAGS.
llvm-svn: 369477
