The reproducer intentionally leak every object allocated during replay,
which means that modules never get orphaned. If this were to happen for
another reason, we might not be testing what we think we are. Assert
that there are no targets left at the end of a test and that the global
module cache is empty in the non-reproducer scenario.
Differential revision: https://reviews.llvm.org/D81612
We are leaking forked processes on macOS because the cleanup function
was checking the existence of /proc/pid which does not exist on macOS.
I've changed the code to be platform agnostic.
Summary:
It turns out that the order in which we provide completions for expressions is
nondeterministic. This leads to confusing user experience and also breaks the
reproducer tests (as two LLDB tests can go out of sync due to the
non-determinism in the completion lists)
The reason for the non-determinism is that the CompletionConsumer informs us
about decls in the order in which it finds declarations in the lookup store of
the DeclContexts it visits (mainly this snippet in SemaLookup.cpp):
``` lang=c++
// Enumerate all of the results in this context.
for (DeclContextLookupResult R :
Load ? Ctx->lookups()
: Ctx->noload_lookups(/*PreserveInternalState=*/false)) {
[...]
```
This storage of the lookup is sorted by pointer values (see the hash of
`DeclarationName`) and can therefore be non-deterministic. The LLDB code
completion consumer that receives these calls originally expected that the order
of declarations is defined by Clang, but it seems the API expects the client to
provide an order to the completions.
This patch fixes the issue as follows:
* We sort the completions we get from Clang alphabetically and also by the
priority value we get from Clang (with priority value sorting having precedence
over the alphabetical sorting)
* We make all the functions/variables that touch a completion before the sorting
const-qualified. The idea is that this should prevent that we never have
observable side-effect from touching these declarations in a non-deterministic
order (e.g., we don't try to complete the type by accident).
This way we behave like the other parts of Clang which also sort the results by
some deterministic value (usually the name or something computed from a name,
e.g., edit distance to a given string).
We most likely also need to fix the Clang code to make the loop I listed above
deterministic to prevent these issues in the future (tracked in rdar://63442513
). This wouldn't replace the functionality provided in this patch though as we
would still need the priority and overall alphabetical sorting.
Note: I had to increase the lldb-vscode completion limit to 100 as the tests
look for strings that aren't in the first 50 results anymore due to variable
names starting with letters like 'v' (which are now always shown much further
down in the list due to the alphabetical sorting).
Fixes rdar://63200995
Reviewers: JDevlieghere, clayborg
Reviewed By: JDevlieghere
Subscribers: mgrang, abidh
Differential Revision: https://reviews.llvm.org/D80292
Many tests use (commented out) print statement for debugging the test
itself. This patch adds a new trace method to lldbtest to reuse the
existing tracing infrastructure and replace these print statements.
Differential revision: https://reviews.llvm.org/D80448
The reproducers' working directory is set to the current working
directory when they are initialized. While this is not optimal, as the
cwd can change during a debug session, it has been sufficient so far.
The current approach doesn't work for the API test suite however because
dotest temporarily changes the directory to where the test's Python file
lives.
This patch adds an API to tell the reproducers what to set the CWD to.
This is a NO-OP in every mode but capture.
Differential revision: https://reviews.llvm.org/D79825
Summary:
If we don't have a current frame then we can still run many expressions
as long as we have an active target. With this patch `expect_expr` directly
calls the target's EvaluateExpression function when there is no current frame.
Reviewers: labath
Reviewed By: labath
Subscribers: JDevlieghere
Differential Revision: https://reviews.llvm.org/D77197
Global properties are shared between debugger instances and
if a test doesn't clear changes in settings it made,
this leads to side effects in other tests.
Differential Revision: https://reviews.llvm.org/D75537
This patch forces architecture "arm" if underlying os reports core
armv7l or armv8l. On linux systems 32 bit sysroot running on 64bit
AArch64 hardware reports armv7l or armv8l which is essently arm
32bit mode. This fixes 5 testcases on 32bit arm.
Some tests set settings and don't clean them up, this leads to side effects in other tests.
The patch removes a global debugger instance with a per-test debugger to avoid such effects.
From what I see, lldb.DBG was needed to determine the platform before a test is run,
lldb.selected_platform is used for this purpose now. Though, this required adding a new function
to the SBPlatform interface.
Differential Revision: https://reviews.llvm.org/D74903
Summary:
Currently the test suite runs with enabled automatically applied Clang fix-its for expressions.
This is causing that sometimes incorrect expressions in tests are still evaluated even though they
are actually incorrect. Let's disable this feature in the test suite so that we know when expressions
are wrong and leave the fix-it testing to the dedicated tests for that feature.
Also updates the `lang/cpp/operators/` test as it seems Clang needs the `struct` keywords
before C and would otherwise fail without fixits.
Reviewers: jingham, JDevlieghere, shafik
Reviewed By: JDevlieghere, shafik
Subscribers: shafik, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74957
Summary:
Currently when printing data types we include implicit scopes such as inline namespaces or anonymous namespaces.
This leads to command output like this (for `std::set<X>` with X being in an anonymous namespace):
```
(lldb) print my_set
(std::__1::set<(anonymous namespace)::X, std::__1::less<(anonymous namespace)::X>, std::__1::allocator<(anonymous namespace)::X> >) $0 = size=0 {}
```
This patch removes all the implicit scopes when printing type names in TypeSystemClang::GetDisplayTypeName
so that our output now looks like this:
```
(lldb) print my_set
(std::set<X, std::less<X>, std::allocator<X> >) $0 = size=0 {}
```
As previously GetDisplayTypeName and GetTypeName had the same output we actually often used the
two as if they are the same method (they were in fact using the same implementation), so this patch also
fixes the places where we actually want the display type name and not the actual type name.
Note that this doesn't touch the `GetTypeName` class that for example the data formatters use, so this patch
is only changes the way we display types to the user. The full type name can also still be found when passing
'-R' to see the raw output of a variable in case someone is somehow interested in that.
Partly fixes rdar://problem/59292534
Reviewers: shafik, jingham
Reviewed By: shafik
Subscribers: christof, JDevlieghere, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74478
Summary:
This creates a separate LLDB_TEST_SRC var to match the existing LLDB_TEST var. LLDB_TEST points to the test framework, LLDB_TEST_SRC points to the tests themselves.
The var points to the same place, but a future patch will move the tree + update var.
Reviewers: labath, JDevlieghere
Reviewed By: labath
Subscribers: merge_guards_bot, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71150
Pass the correct library directory from CMake to dotest.py when linking
liblldb, instead of trying to reconstruct the path from executable path.
This fixes link failures on platforms having non-null
LLVM_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D73767
Re-landing this now that (hopefully) all the failures this caused on the
bots have been addressed.
This patch changes the behavior of the substrs argument to self.expect.
Currently, the elements of substrs are unordered and as long as the
string appears in the output, the assertion passes.
We can be more precise by requiring that the substrings be ordered in
the way they appear. My hope is that this will make it harder to
accidentally pass a check because a string appears out of order.
Differential revision: https://reviews.llvm.org/D73766
This patch changes the behavior of the substrs argument to self.expect.
Currently, the elements of substrs are unordered and as long as the
string appears in the output, the assertion passes.
We can be more precise by requiring that the substrings be ordered in
the way they appear. My hope is that this will make it harder to
accidentally pass a check because a string appears out of order.
Differential revision: https://reviews.llvm.org/D73766
Currently if 'expect' fails and a custom msg is supplied, then lldbtest
will not print the actual command output. This makes it impossible to know
why the test actually failed. This just prints the command output even
if the msg parameter was supplied.
Summary:
This patch adds a new function to lldbtest: `expect_expr`. This function is supposed to replace the current approach
of calling `expect`/`runCmd` with `expr`, `p` etc.
`expect_expr` allows evaluating expressions and matching their value/summary/type/error message without
having to do any string matching that might allow unintended passes (e.g., `self.expect("expr 3+4", substrs=["7"])`
can unexpectedly pass for results like `(Class7) $0 = 7`, `(int) $7 = 22`, `(int) $0 = 77` and so on).
This only uses the function in a few places to test and demonstrate it. I'll migrate the tests in follow up commits.
Reviewers: JDevlieghere, shafik, labath
Reviewed By: labath
Subscribers: christof, abidh, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D70314
To do so, we need to register the sanitizer libraries with the target
so that they get uploaded before running. This patch adds a helper to
the test class to this effect.
The invocation shown by dotest.py to re-run a single test is misleading:
it ranges from missing arguments (best case scenario) to being totally
wrong (worst case scenario).
In the past I've tried to get it right, but given the dotest
architecture this is harder than it looks. Furthermore, we have pretty
good documentation on the website [1] for most use cases.
This patch removes the rerun invocation.
[1] https://lldb.llvm.org/resources/test.html
Before the test reorganization, everything was part of a single test
suite with a single module cache. Now that things are properly separated
this is no longer the case. Only the shell tests inherited the logic to
properly configure and wipe the module caches. This patch adds that
logic back for the API tests. While doing so, I noticed that we were
configuring a Clang module cache in CMake, but weren't actually using it
from dotest.py. I included a fix for that in this patch as well.
Differential revision: https://reviews.llvm.org/D68755
llvm-svn: 374386
Summary:
In r367234 we introduced a central place to hold the set up commands for
the various ways we have of launching lldb. However, a number of
commands still remained outside of that.
This patch moves the remaining set up commands into this function, which
allows us to remove manual clang module path setting code in
TestBatchMode.
One unfortunate victim of this approach is TestSTTYBeforeAndAfter which,
due to how it launches lldb (pexpect->expect->lldb), fails get the
quoting right. It would be possible to fix the quoting there, it would be a bit
icky, and none of the commands in this list are really relevant for what this
test is doing, so I just remove the commands outright.
Reviewers: JDevlieghere, jankratochvil
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D67173
llvm-svn: 371019
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
The refactoring patch for the option completion broke the completion
for ambiguous long options. As this feature was also untested (as
testing ambiguous options with the current test methods is impossible),
I just noticed now. This patch restores the old behavior and adds a
test for this feature.
llvm-svn: 370185
Today I discovered the skipLongRunningTest decorator and to my surprise
all the tests were passing without the decorator. They don't seem to be
that expensive either, they take a few seconds but we have tests that
take much longer than that. As such I propose to remove the decorator
and enable them by default.
Differential revision: https://reviews.llvm.org/D66774
llvm-svn: 369995
Summary:
This commit fixes some typo I found while exploring LLDB's codebase.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D66452
llvm-svn: 369313
On the matrix bot on GreenDragon [1] we want to run the test suite
against different DWARF versions. The idea here is not to replace
targeted tests for certain DWARF features, but rather to provide an easy
way to support this configuration.
[1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/
Differential revision: https://reviews.llvm.org/D66370
llvm-svn: 369272
Summary:
It seems this was an unintended side-effect of D26698. AFAICT, these
functions did return an empty string before that patch, and the patch
contained code which attempted to ensure that, but those efforts were
negated by ConstString::AsCString, which by default returns a nullptr
even for empty strings.
This patch:
- fixes the GetOutput/Error methods to really return empty strings
- adds and explicit test for that
- removes a workaround in lldbtest.py, which was masking this problem
from our other tests
Reviewers: jingham, clayborg
Subscribers: zturner, lldb-commits
Differential Revision: https://reviews.llvm.org/D65739
llvm-svn: 368806
rL357954 did increase `packet-timeout` 1sec -> 5secs. Which is IMO about the
maximum timeout reasonable for regular use. But for testsuite I think the
timeout should be higher as the testsuite runs in parallel and it can be run
even on slow hosts and with other load (moreover if it runs on some slow arch).
I have chosen 60 secs, that should be enough hopefully. Larger value could
make debugging with hanging `lldb-server` annoying.
This patch was based on this testsuite timeout:
http://lab.llvm.org:8014/builders/lldb-x86_64-fedora/builds/546/steps/test/logs/stdio
FAIL: test_connect (TestGDBRemoteClient.TestGDBRemoteClient)
Test connecting to a remote gdb server
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jkratoch/slave-lldb-x86_64-fedora/lldb-x86_64-fedora/llvm/tools/lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestGDBRemoteClient.py", line 13, in test_connect
process = self.connect(target)
File "/home/jkratoch/slave-lldb-x86_64-fedora/lldb-x86_64-fedora/llvm/tools/lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py", line 480, in connect
self.assertTrue(error.Success(), error.description)
AssertionError: False is not True : failed to get reply to handshake packet
Differential Revision: https://reviews.llvm.org/D65271
llvm-svn: 367234
Dotest contains code to clear DYLD_LIBRARY_PATH for the inferior to not
propagate sanitized builds. However, it's possible that we want to
inject a different library path with `--inferior-env`. To make that work
correctly, we need to do that *after* clearing DYLD_LIBRARY_PATH.
llvm-svn: 367054
Summary: lldb-mi has been removed, but there are still a bunch of references in the code base. This patch removes all of them.
Reviewers: JDevlieghere, jfb
Reviewed By: JDevlieghere
Subscribers: dexonsmith, ki.stfu, mgorny, abidh, jfb, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D64992
llvm-svn: 366590
Disable colors so we don't risk having unexpected ANSI codes in the test
output. Currently, the behavior of a test can change depending on
whether it's run under a color-supporting terminal, or under a dummy
terminal, for example when using lit or multiprocessing.
llvm-svn: 366356
This patch adds a dotest flag for setting environment variables for the
inferior. This is different from the current --env flag, which sets
variables in the debugger's environment. This allows us to set things
like LD_LIBRARY_PATH for testing.
Differential revision: https://reviews.llvm.org/D63790
llvm-svn: 364443
D55859 <https://reviews.llvm.org/D55859> has no effect for some of the
testcases so this patch extends it even for (all?) other testcases known to me.
LLDB was failing when LLDB prints errors reading system debug infos
(`*-debuginfo.rpm`, DWZ-optimized) which should never happen as LLDB testcases
should not be affected by system debug infos.
`lldb/packages/Python/lldbsuite/test/api/multithreaded/driver.cpp.template` is
using only SB API which does not expose `ModuleList` so I had to call
`HandleCommand()` there.
`lldb-test.cpp` could also use `HandleCommand` and then there would be no need
for `ModuleListProperties::SetEnableExternalLookup()` but I think it is cleaner
with API and not on based on text commands.
Differential Revision: https://reviews.llvm.org/D63339
llvm-svn: 363567
