`getCompilerVersion` assumes that `clang --version` prints out a string like `version [0-9\.]+`.
If clang is built from trunk, the version line might look like `clang version trunk (123abc)`.
Since there isn't any way of knowing by the commit id alone whether one commit is newer or older than another git commit (or clang version), assume that clang with a version id like this is very close to trunk. For example, any tests with `@skipIf(compiler="clang", compiler_version=['<', '8'])` should be run.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D85248
LLDB tests assume that tests are in the test tree (the `LLDB_TEST_SRC` env variable, configured by `dotest.py`).
If this assertion doesn't hold, tests fail in strange ways. An early place this goes wrong is in `compute_mydir` which does a simple length-based substring to get the relative path. Later, we use that path to chdir to. If the test file and test tree don't agree in realpath-ness (and therefore length), this will be a cryptic error of chdir-ing to a directory that does not exist.
The actual discrepency is that the places we look for `use_lldb_suite.py` don't use a realpath, but `dotest.py` does (see initialization of `configuration.testdirs`).
It doesn't particularly matter whether we use realpath or abspath to canonicalize things, but many places end up with implicit dependencies on the canonicalized pwd being a realpath, so make them realpath consistently. Also, in the `compute_mydir` method mentioned, raise an error if the path types don't agree.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85258
Add an option that allows the user to decide to not make the inferior is
responsible for its own TCC permissions. If you don't make the inferior
responsible, it inherits the permissions of its parent. The motivation
is the scenario of running the LLDB test suite from an external hard
drive. If the inferior is responsible, every test needs to be granted
access to the external volume. When the permissions are inherited,
approval needs to be granted only once.
Differential revision: https://reviews.llvm.org/D85237
Summary: Add printing of the output of stdout during compile errors, in
addition to stderr output.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D83425
Summary:
registerSharedLibrariesWithTarget was setting the library path
environment variable to the process build directory, but the function is
also accepting libraries in other directories (in which case they won't
be found automatically).
This patch makes the function set the path variable correctly for these
libraries too. This enables us to remove the code for setting the path
variable in TestWeakSymbols.py, which was working only accidentally --
it was relying on the fact that
launch_info.SetEnvironmentEntries(..., append=True)
would not overwrite the path variable it has set, but that is going to
change with D83306.
Reviewers: davide, jingham
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D83552
Remove custom tearDownHooks from GDB Remote tests as we now cleanup
subprocesses unconditionally. This also changes the termination order to
be the reverse of the creation order. I don't think anything is relying
on that right now, but it better fits the setup/teardown paradigm.
Remove the forkSubprocess method and its bookkeeping.
TestCreateAfterAttach is the only test using the fork method and I'm not
convinced it adds enough to warrant the maintenance. Pavel suggested the
same thing in D83815.
Summary:
Currently expect_expr will not run the expression if no target is selected. This
patch changes this behavior so that expect_expr will instead fall back to the
dummy target similar to what the `expression` command is doing. This way we
don't have to compile an empty executable to be able to use `expect_expr` (which
is a waste of resources for tests that just test generic type system features).
As a test I modernized the TestTypeOfDeclTypeExpr into a Python test +
expect_expr (as it relied on the dummy target fallback of the expression
command).
Reviewers: labath, JDevlieghere
Reviewed By: labath
Subscribers: abidh
Differential Revision: https://reviews.llvm.org/D83388
Always clean up subprocesses on tear down instead of relying on the
caller to do so. This is not only less error prone but also means the
tests can be more concise.
Differential revision: https://reviews.llvm.org/D83787
- Make the open more Pythonic.
- Remove the unused `cleanup` Make target.
- Remove commented-out/obvious/low-value comments.
- Cleanup the forked process PID list.
The glob expression for a test called "test" could match a log file for
a the test "test_foo". Instead of globbing, maintain an explicit list of
log files relevant to the current test.
Summary:
A lot of our tests do 'self.assertTrue(error.Success()'. The problem
with that is that when this fails, it produces a completely useless
error message (False is not True) and the most important piece of
information -- the actual error message -- is completely hidden.
Sometimes we mitigate that by including the error message in the "msg"
argument, but this has two additional problems:
- as the msg argument is evaluated unconditionally, one needs to be
careful to not trigger an exception when the operation was actually
successful.
- it requires more typing, which means we often don't do it
assertSuccess solves these problems by taking the entire SBError object
as an argument. If the operation was unsuccessful, it can format a
reasonable error message itself. The function still accepts a "msg"
argument, which can include any additional context, but this context now
does not need to include the error message.
To demonstrate usage, I replace a number of existing assertTrue
assertions with the new function. As this process is not easily
automatable, I have just manually updated a representative sample. In
some cases, I did not update the code to use assertSuccess, but I went
for even higher-level assertion apis (runCmd, expect_expr), as these are
even shorter, and can produce even better failure messages.
Reviewers: teemperor, JDevlieghere
Subscribers: arphaman, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D82759
The reproducer intentionally leak every object allocated during replay,
which means that modules never get orphaned. If this were to happen for
another reason, we might not be testing what we think we are. Assert
that there are no targets left at the end of a test and that the global
module cache is empty in the non-reproducer scenario.
Differential revision: https://reviews.llvm.org/D81612
We are leaking forked processes on macOS because the cleanup function
was checking the existence of /proc/pid which does not exist on macOS.
I've changed the code to be platform agnostic.
Summary:
It turns out that the order in which we provide completions for expressions is
nondeterministic. This leads to confusing user experience and also breaks the
reproducer tests (as two LLDB tests can go out of sync due to the
non-determinism in the completion lists)
The reason for the non-determinism is that the CompletionConsumer informs us
about decls in the order in which it finds declarations in the lookup store of
the DeclContexts it visits (mainly this snippet in SemaLookup.cpp):
``` lang=c++
// Enumerate all of the results in this context.
for (DeclContextLookupResult R :
Load ? Ctx->lookups()
: Ctx->noload_lookups(/*PreserveInternalState=*/false)) {
[...]
```
This storage of the lookup is sorted by pointer values (see the hash of
`DeclarationName`) and can therefore be non-deterministic. The LLDB code
completion consumer that receives these calls originally expected that the order
of declarations is defined by Clang, but it seems the API expects the client to
provide an order to the completions.
This patch fixes the issue as follows:
* We sort the completions we get from Clang alphabetically and also by the
priority value we get from Clang (with priority value sorting having precedence
over the alphabetical sorting)
* We make all the functions/variables that touch a completion before the sorting
const-qualified. The idea is that this should prevent that we never have
observable side-effect from touching these declarations in a non-deterministic
order (e.g., we don't try to complete the type by accident).
This way we behave like the other parts of Clang which also sort the results by
some deterministic value (usually the name or something computed from a name,
e.g., edit distance to a given string).
We most likely also need to fix the Clang code to make the loop I listed above
deterministic to prevent these issues in the future (tracked in rdar://63442513
). This wouldn't replace the functionality provided in this patch though as we
would still need the priority and overall alphabetical sorting.
Note: I had to increase the lldb-vscode completion limit to 100 as the tests
look for strings that aren't in the first 50 results anymore due to variable
names starting with letters like 'v' (which are now always shown much further
down in the list due to the alphabetical sorting).
Fixes rdar://63200995
Reviewers: JDevlieghere, clayborg
Reviewed By: JDevlieghere
Subscribers: mgrang, abidh
Differential Revision: https://reviews.llvm.org/D80292
Many tests use (commented out) print statement for debugging the test
itself. This patch adds a new trace method to lldbtest to reuse the
existing tracing infrastructure and replace these print statements.
Differential revision: https://reviews.llvm.org/D80448
The reproducers' working directory is set to the current working
directory when they are initialized. While this is not optimal, as the
cwd can change during a debug session, it has been sufficient so far.
The current approach doesn't work for the API test suite however because
dotest temporarily changes the directory to where the test's Python file
lives.
This patch adds an API to tell the reproducers what to set the CWD to.
This is a NO-OP in every mode but capture.
Differential revision: https://reviews.llvm.org/D79825
Summary:
If we don't have a current frame then we can still run many expressions
as long as we have an active target. With this patch `expect_expr` directly
calls the target's EvaluateExpression function when there is no current frame.
Reviewers: labath
Reviewed By: labath
Subscribers: JDevlieghere
Differential Revision: https://reviews.llvm.org/D77197
Global properties are shared between debugger instances and
if a test doesn't clear changes in settings it made,
this leads to side effects in other tests.
Differential Revision: https://reviews.llvm.org/D75537
This patch forces architecture "arm" if underlying os reports core
armv7l or armv8l. On linux systems 32 bit sysroot running on 64bit
AArch64 hardware reports armv7l or armv8l which is essently arm
32bit mode. This fixes 5 testcases on 32bit arm.
Some tests set settings and don't clean them up, this leads to side effects in other tests.
The patch removes a global debugger instance with a per-test debugger to avoid such effects.
From what I see, lldb.DBG was needed to determine the platform before a test is run,
lldb.selected_platform is used for this purpose now. Though, this required adding a new function
to the SBPlatform interface.
Differential Revision: https://reviews.llvm.org/D74903
Summary:
Currently the test suite runs with enabled automatically applied Clang fix-its for expressions.
This is causing that sometimes incorrect expressions in tests are still evaluated even though they
are actually incorrect. Let's disable this feature in the test suite so that we know when expressions
are wrong and leave the fix-it testing to the dedicated tests for that feature.
Also updates the `lang/cpp/operators/` test as it seems Clang needs the `struct` keywords
before C and would otherwise fail without fixits.
Reviewers: jingham, JDevlieghere, shafik
Reviewed By: JDevlieghere, shafik
Subscribers: shafik, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74957
Summary:
Currently when printing data types we include implicit scopes such as inline namespaces or anonymous namespaces.
This leads to command output like this (for `std::set<X>` with X being in an anonymous namespace):
```
(lldb) print my_set
(std::__1::set<(anonymous namespace)::X, std::__1::less<(anonymous namespace)::X>, std::__1::allocator<(anonymous namespace)::X> >) $0 = size=0 {}
```
This patch removes all the implicit scopes when printing type names in TypeSystemClang::GetDisplayTypeName
so that our output now looks like this:
```
(lldb) print my_set
(std::set<X, std::less<X>, std::allocator<X> >) $0 = size=0 {}
```
As previously GetDisplayTypeName and GetTypeName had the same output we actually often used the
two as if they are the same method (they were in fact using the same implementation), so this patch also
fixes the places where we actually want the display type name and not the actual type name.
Note that this doesn't touch the `GetTypeName` class that for example the data formatters use, so this patch
is only changes the way we display types to the user. The full type name can also still be found when passing
'-R' to see the raw output of a variable in case someone is somehow interested in that.
Partly fixes rdar://problem/59292534
Reviewers: shafik, jingham
Reviewed By: shafik
Subscribers: christof, JDevlieghere, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74478
Summary:
This creates a separate LLDB_TEST_SRC var to match the existing LLDB_TEST var. LLDB_TEST points to the test framework, LLDB_TEST_SRC points to the tests themselves.
The var points to the same place, but a future patch will move the tree + update var.
Reviewers: labath, JDevlieghere
Reviewed By: labath
Subscribers: merge_guards_bot, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71150
Pass the correct library directory from CMake to dotest.py when linking
liblldb, instead of trying to reconstruct the path from executable path.
This fixes link failures on platforms having non-null
LLVM_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D73767
Re-landing this now that (hopefully) all the failures this caused on the
bots have been addressed.
This patch changes the behavior of the substrs argument to self.expect.
Currently, the elements of substrs are unordered and as long as the
string appears in the output, the assertion passes.
We can be more precise by requiring that the substrings be ordered in
the way they appear. My hope is that this will make it harder to
accidentally pass a check because a string appears out of order.
Differential revision: https://reviews.llvm.org/D73766
This patch changes the behavior of the substrs argument to self.expect.
Currently, the elements of substrs are unordered and as long as the
string appears in the output, the assertion passes.
We can be more precise by requiring that the substrings be ordered in
the way they appear. My hope is that this will make it harder to
accidentally pass a check because a string appears out of order.
Differential revision: https://reviews.llvm.org/D73766
Currently if 'expect' fails and a custom msg is supplied, then lldbtest
will not print the actual command output. This makes it impossible to know
why the test actually failed. This just prints the command output even
if the msg parameter was supplied.
Summary:
This patch adds a new function to lldbtest: `expect_expr`. This function is supposed to replace the current approach
of calling `expect`/`runCmd` with `expr`, `p` etc.
`expect_expr` allows evaluating expressions and matching their value/summary/type/error message without
having to do any string matching that might allow unintended passes (e.g., `self.expect("expr 3+4", substrs=["7"])`
can unexpectedly pass for results like `(Class7) $0 = 7`, `(int) $7 = 22`, `(int) $0 = 77` and so on).
This only uses the function in a few places to test and demonstrate it. I'll migrate the tests in follow up commits.
Reviewers: JDevlieghere, shafik, labath
Reviewed By: labath
Subscribers: christof, abidh, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D70314
To do so, we need to register the sanitizer libraries with the target
so that they get uploaded before running. This patch adds a helper to
the test class to this effect.
The invocation shown by dotest.py to re-run a single test is misleading:
it ranges from missing arguments (best case scenario) to being totally
wrong (worst case scenario).
In the past I've tried to get it right, but given the dotest
architecture this is harder than it looks. Furthermore, we have pretty
good documentation on the website [1] for most use cases.
This patch removes the rerun invocation.
[1] https://lldb.llvm.org/resources/test.html
Before the test reorganization, everything was part of a single test
suite with a single module cache. Now that things are properly separated
this is no longer the case. Only the shell tests inherited the logic to
properly configure and wipe the module caches. This patch adds that
logic back for the API tests. While doing so, I noticed that we were
configuring a Clang module cache in CMake, but weren't actually using it
from dotest.py. I included a fix for that in this patch as well.
Differential revision: https://reviews.llvm.org/D68755
llvm-svn: 374386
Summary:
In r367234 we introduced a central place to hold the set up commands for
the various ways we have of launching lldb. However, a number of
commands still remained outside of that.
This patch moves the remaining set up commands into this function, which
allows us to remove manual clang module path setting code in
TestBatchMode.
One unfortunate victim of this approach is TestSTTYBeforeAndAfter which,
due to how it launches lldb (pexpect->expect->lldb), fails get the
quoting right. It would be possible to fix the quoting there, it would be a bit
icky, and none of the commands in this list are really relevant for what this
test is doing, so I just remove the commands outright.
Reviewers: JDevlieghere, jankratochvil
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D67173
llvm-svn: 371019
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
