In D89056 the default value for architecture was moved to `build` so that
all called functions see the same architecture value. It seems there are a
few functions that call buildDefault directly (and not via build), so
on some test configurations that set a custom arch value the architecture
value is no longer available.
This just adds the architecture code from build to buildDefault to get
the bots green again while I'm looking for a better solution.
This originally broke the TestQuoting which explicitly called buildDefault
instead of calling build() and marking the test as no_debug_info_test.
TestQuoting has been rewritten by now and is using `build`, so this should now
pass on all platforms.
Original summary:
The Darwin builder currently assumes in `getArchCFlags` that the passed `arch`
value is an actual string it can string.join with vendor/os/version/env strings:
```
triple = '-'.join([arch, vendor, os, version, env])
```
However this is not true for most tests as we just pass down the `arch=None`
default value from `TestBase.build`. This causes that if we actually end up in
this function we just error out when concatenating `None` with the other actual
strings of vendor/os/version/env. What we should do instead is check that if
there is no test-specific architecture that we fall back to the configuration's
architecture value.
It seems we already worked around this in `builder.getArchSpec` by explicitly
falling back to the architecture specified in the configuration.
This patch just moves this fallback logic to the top `build` function so that it
affects all functions called from `TestBase.build`.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D89056
The intention is not to allow stop-hook commands to query the
user, so this is correct. It also works around a deadlock in
switching to the Python Session to execute python based commands
in the stop hook when the Debugger stdin is backed by a FILE *.
Differential Revision: https://reviews.llvm.org/D90332
The Darwin builder currently assumes in `getArchCFlags` that the passed `arch`
value is an actual string it can string.join with vendor/os/version/env strings:
```
triple = '-'.join([arch, vendor, os, version, env])
```
However this is not true for most tests as we just pass down the `arch=None`
default value from `TestBase.build`. This causes that if we actually end up in
this function we just error out when concatenating `None` with the other actual
strings of vendor/os/version/env. What we should do instead is check that if
there is no test-specific architecture that we fall back to the configuration's
architecture value.
It seems we already worked around this in `builder.getArchSpec` by explicitly
falling back to the architecture specified in the configuration.
This patch just moves this fallback logic to the top `build` function so that it
affects all functions called from `TestBase.build`.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D89056
Add preconditions to `TestBase.expect()` that catch semantically invalid calls
that happen to succeed anyway. This also fixes the broken callsites caught by
these checks.
This prevents the following incorrect calls:
1. `self.expect("lldb command", "some substr")`
2. `self.expect("lldb command", "assert message", "some substr")`
Differential Revision: https://reviews.llvm.org/D88792
A few fixes while trying to figure out why tests are being skipped for arsenm:
- We check `$compiler -v`, but `-v` is `--verbose`, not `--version`. Use the long flag name.
- We check all lines matching `version ...`, but we should exit early for the first version string we see (which should be the main one). I'm not sure if this is the issue, but perhaps this is causing some users to skip some tests if another "version ..." is showing up later.
- Having `\.` in a python string is triggering pylint warnings, because it should be escaped as a regex string, e.g. `r'\.' However, `.` in a character class does not need to be escaped, as it matches only a literal `.` in that context.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D88051
targetHasSVE helper function was added to test for availability of SVE support
by connected platform. We now intend to use this function in other testcases
and I am moving it to a generic location in lldbtest.py to allow usage by
other upcoming testcases.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86872
This updates the errors reported by expect()
to something like:
```
Ran command:
"help"
Got output:
Debugger commands:
<...>
Expecting start string: "Debugger commands:" (was found)
Expecting end string: "foo" (was not found)
```
(see added tests for more examples)
This shows the user exactly what was run,
what checks passed and which failed. Along with
whether that check was supposed to pass.
(including what regex patterns matched)
These lines are also output to the test
trace file, whether the test passes or not.
Note that expect() will still fail at the first failed
check, in line with previous behaviour.
Also I have flipped the wording of the assert
message functions (.*_MSG) to describe failures
not successes. This makes more sense as they are
only shown on assert failures.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86792
TestCompletion is randomly failing on some bots. The error message however states
that the computed completions actually do contain the expected pid we're
looking for, so there shouldn't be any test failure.
The reason for that turns out to be that complete_from_to is actually used
for testing two different features. It can be used for testing what the
common prefix for the list of completions is and *also* for checking all the
possible completions that are returned for a command. Which one of the two
things should be checked can't be defined by a parameter to the function, but
is instead guessed by the test method instead based on the results that were
returned. If there is a common prefix in all completions, then that prefix
is searched and otherwise all completions are searched.
For TestCompletion's pid test this behaviour leads to the strange test failures.
If all the pid's that our test LLDB can see have a common prefix (e.g., it
can only see pids [123, 122, 10004, 10000] -> common prefix '1'), then
complete_from_to check that the common prefix contains our pid, which is
always fails ('1' doesn't contain '123' or any other valid pid). If there
isn't a common prefix (e.g., pids are [123, 122, 10004, 777]) then
complete_from_to will check the list of completions instead which works correctly.
This patch is fixing this by adding a simple check method that doesn't
have this behaviour and is simply searching the returned list of completions.
This should get the bots green while I'm working on a proper fix that fixes
complete_from_to.
This patch removes the rather confusing LLDB_LIB_DIR and LLDB_IMPLIB_DIR
environment variables. They are confusing because LLDB_LIB_DIR would
point to the bin subdirectory in the build root while LLDB_IMPLIB_DIR
would point to the lib subdirectory. The reason far this was
LLDB.framework, which gets build under bin.
This patch replaces their uses with configuration.lldb_framework_path
and configuration.lldb_libs_dir respectively.
Differential revision: https://reviews.llvm.org/D86817
EXP_MSG generates a message to show on assert
failure. Currently it looks like:
AssertionError: False is not True : '<cmd>'
returns expected result, got '<actual output>'
Which seems to say that the test failed but
also got the expected result.
It should say:
AssertionError: False is not True : '<cmd>'
returned unexpected result, got '<actual output>'
Reviewed By: teemperor, #lldb
Differential Revision: https://reviews.llvm.org/D86603
Rather than have different modules for different platforms, use
inheritance so we can have a Builer base class and optional child
classes that override platform specific methods.
Differential revision: https://reviews.llvm.org/D86174
TypeSystemClang::CreateTypedef was creating a typedef in the right
DeclContext, but it was not actually adding it as a child of the
context. The resulting inconsistent state meant that we would be unable
to reference the typedef from an expression directly, but we could use
them if they end up being pulled in by some previous subexpression
(because the ASTImporter will set up the correct links in the expression
ast).
This patch adds the typedef to the decl context it is created in.
Differential Revision: https://reviews.llvm.org/D86140
Right now the only places in the SB API where lldb:: ModuleSP instances are
destroyed are in SBDebugger::MemoryPressureDetected (where it's just attempted
but not guaranteed) and in SBDebugger::DeleteTarget (which will be removed in
D83933). Tests that directly create an lldb::ModuleSP and never create a target
therefore currently leak lldb::Module instances. This triggers the sanity checks
in lldbtest that make sure that the global module list is empty after a test.
This patch adds SBModule::GarbageCollectAllocatedModules as an explicit way to
clean orphaned lldb::ModuleSP instances. Also we now start calling this method
at the end of each test run and move the sanity check behind that call to make
this work. This way even tests that don't create targets can pass the sanity
check.
This fixes TestUnicodeSymbols.py when D83865 is applied (which makes that the
sanity checks actually fail the test).
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D83876
expect_expr currently can't verify the children of the result SBValue.
This patch adds the ability to check them. The idea is to have a CheckValue
class where one can specify what attributes of a SBValue should be checked.
Beside the properties we already check for (summary, type, etc.) this also
has a list of children which is again just a list of CheckValue object (which
can also have children of their own).
The main motivation is to make checking the children no longer based
on error-prone substring checks that allow tests to pass just because
for example the error message contains the expected substrings by accident.
I also expect that we can just have a variant of `expect_expr` for LLDB's
expression paths (aka 'frame var') feature.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D83792
`getCompilerVersion` assumes that `clang --version` prints out a string like `version [0-9\.]+`.
If clang is built from trunk, the version line might look like `clang version trunk (123abc)`.
Since there isn't any way of knowing by the commit id alone whether one commit is newer or older than another git commit (or clang version), assume that clang with a version id like this is very close to trunk. For example, any tests with `@skipIf(compiler="clang", compiler_version=['<', '8'])` should be run.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D85248
LLDB tests assume that tests are in the test tree (the `LLDB_TEST_SRC` env variable, configured by `dotest.py`).
If this assertion doesn't hold, tests fail in strange ways. An early place this goes wrong is in `compute_mydir` which does a simple length-based substring to get the relative path. Later, we use that path to chdir to. If the test file and test tree don't agree in realpath-ness (and therefore length), this will be a cryptic error of chdir-ing to a directory that does not exist.
The actual discrepency is that the places we look for `use_lldb_suite.py` don't use a realpath, but `dotest.py` does (see initialization of `configuration.testdirs`).
It doesn't particularly matter whether we use realpath or abspath to canonicalize things, but many places end up with implicit dependencies on the canonicalized pwd being a realpath, so make them realpath consistently. Also, in the `compute_mydir` method mentioned, raise an error if the path types don't agree.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85258
Add an option that allows the user to decide to not make the inferior is
responsible for its own TCC permissions. If you don't make the inferior
responsible, it inherits the permissions of its parent. The motivation
is the scenario of running the LLDB test suite from an external hard
drive. If the inferior is responsible, every test needs to be granted
access to the external volume. When the permissions are inherited,
approval needs to be granted only once.
Differential revision: https://reviews.llvm.org/D85237
Summary: Add printing of the output of stdout during compile errors, in
addition to stderr output.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D83425
Summary:
registerSharedLibrariesWithTarget was setting the library path
environment variable to the process build directory, but the function is
also accepting libraries in other directories (in which case they won't
be found automatically).
This patch makes the function set the path variable correctly for these
libraries too. This enables us to remove the code for setting the path
variable in TestWeakSymbols.py, which was working only accidentally --
it was relying on the fact that
launch_info.SetEnvironmentEntries(..., append=True)
would not overwrite the path variable it has set, but that is going to
change with D83306.
Reviewers: davide, jingham
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D83552
Remove custom tearDownHooks from GDB Remote tests as we now cleanup
subprocesses unconditionally. This also changes the termination order to
be the reverse of the creation order. I don't think anything is relying
on that right now, but it better fits the setup/teardown paradigm.
Remove the forkSubprocess method and its bookkeeping.
TestCreateAfterAttach is the only test using the fork method and I'm not
convinced it adds enough to warrant the maintenance. Pavel suggested the
same thing in D83815.
Summary:
Currently expect_expr will not run the expression if no target is selected. This
patch changes this behavior so that expect_expr will instead fall back to the
dummy target similar to what the `expression` command is doing. This way we
don't have to compile an empty executable to be able to use `expect_expr` (which
is a waste of resources for tests that just test generic type system features).
As a test I modernized the TestTypeOfDeclTypeExpr into a Python test +
expect_expr (as it relied on the dummy target fallback of the expression
command).
Reviewers: labath, JDevlieghere
Reviewed By: labath
Subscribers: abidh
Differential Revision: https://reviews.llvm.org/D83388
Always clean up subprocesses on tear down instead of relying on the
caller to do so. This is not only less error prone but also means the
tests can be more concise.
Differential revision: https://reviews.llvm.org/D83787
- Make the open more Pythonic.
- Remove the unused `cleanup` Make target.
- Remove commented-out/obvious/low-value comments.
- Cleanup the forked process PID list.
The glob expression for a test called "test" could match a log file for
a the test "test_foo". Instead of globbing, maintain an explicit list of
log files relevant to the current test.
Summary:
A lot of our tests do 'self.assertTrue(error.Success()'. The problem
with that is that when this fails, it produces a completely useless
error message (False is not True) and the most important piece of
information -- the actual error message -- is completely hidden.
Sometimes we mitigate that by including the error message in the "msg"
argument, but this has two additional problems:
- as the msg argument is evaluated unconditionally, one needs to be
careful to not trigger an exception when the operation was actually
successful.
- it requires more typing, which means we often don't do it
assertSuccess solves these problems by taking the entire SBError object
as an argument. If the operation was unsuccessful, it can format a
reasonable error message itself. The function still accepts a "msg"
argument, which can include any additional context, but this context now
does not need to include the error message.
To demonstrate usage, I replace a number of existing assertTrue
assertions with the new function. As this process is not easily
automatable, I have just manually updated a representative sample. In
some cases, I did not update the code to use assertSuccess, but I went
for even higher-level assertion apis (runCmd, expect_expr), as these are
even shorter, and can produce even better failure messages.
Reviewers: teemperor, JDevlieghere
Subscribers: arphaman, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D82759
The reproducer intentionally leak every object allocated during replay,
which means that modules never get orphaned. If this were to happen for
another reason, we might not be testing what we think we are. Assert
that there are no targets left at the end of a test and that the global
module cache is empty in the non-reproducer scenario.
Differential revision: https://reviews.llvm.org/D81612
We are leaking forked processes on macOS because the cleanup function
was checking the existence of /proc/pid which does not exist on macOS.
I've changed the code to be platform agnostic.
Summary:
It turns out that the order in which we provide completions for expressions is
nondeterministic. This leads to confusing user experience and also breaks the
reproducer tests (as two LLDB tests can go out of sync due to the
non-determinism in the completion lists)
The reason for the non-determinism is that the CompletionConsumer informs us
about decls in the order in which it finds declarations in the lookup store of
the DeclContexts it visits (mainly this snippet in SemaLookup.cpp):
``` lang=c++
// Enumerate all of the results in this context.
for (DeclContextLookupResult R :
Load ? Ctx->lookups()
: Ctx->noload_lookups(/*PreserveInternalState=*/false)) {
[...]
```
This storage of the lookup is sorted by pointer values (see the hash of
`DeclarationName`) and can therefore be non-deterministic. The LLDB code
completion consumer that receives these calls originally expected that the order
of declarations is defined by Clang, but it seems the API expects the client to
provide an order to the completions.
This patch fixes the issue as follows:
* We sort the completions we get from Clang alphabetically and also by the
priority value we get from Clang (with priority value sorting having precedence
over the alphabetical sorting)
* We make all the functions/variables that touch a completion before the sorting
const-qualified. The idea is that this should prevent that we never have
observable side-effect from touching these declarations in a non-deterministic
order (e.g., we don't try to complete the type by accident).
This way we behave like the other parts of Clang which also sort the results by
some deterministic value (usually the name or something computed from a name,
e.g., edit distance to a given string).
We most likely also need to fix the Clang code to make the loop I listed above
deterministic to prevent these issues in the future (tracked in rdar://63442513
). This wouldn't replace the functionality provided in this patch though as we
would still need the priority and overall alphabetical sorting.
Note: I had to increase the lldb-vscode completion limit to 100 as the tests
look for strings that aren't in the first 50 results anymore due to variable
names starting with letters like 'v' (which are now always shown much further
down in the list due to the alphabetical sorting).
Fixes rdar://63200995
Reviewers: JDevlieghere, clayborg
Reviewed By: JDevlieghere
Subscribers: mgrang, abidh
Differential Revision: https://reviews.llvm.org/D80292
Many tests use (commented out) print statement for debugging the test
itself. This patch adds a new trace method to lldbtest to reuse the
existing tracing infrastructure and replace these print statements.
Differential revision: https://reviews.llvm.org/D80448
The reproducers' working directory is set to the current working
directory when they are initialized. While this is not optimal, as the
cwd can change during a debug session, it has been sufficient so far.
The current approach doesn't work for the API test suite however because
dotest temporarily changes the directory to where the test's Python file
lives.
This patch adds an API to tell the reproducers what to set the CWD to.
This is a NO-OP in every mode but capture.
Differential revision: https://reviews.llvm.org/D79825
Summary:
If we don't have a current frame then we can still run many expressions
as long as we have an active target. With this patch `expect_expr` directly
calls the target's EvaluateExpression function when there is no current frame.
Reviewers: labath
Reviewed By: labath
Subscribers: JDevlieghere
Differential Revision: https://reviews.llvm.org/D77197
Global properties are shared between debugger instances and
if a test doesn't clear changes in settings it made,
this leads to side effects in other tests.
Differential Revision: https://reviews.llvm.org/D75537
This patch forces architecture "arm" if underlying os reports core
armv7l or armv8l. On linux systems 32 bit sysroot running on 64bit
AArch64 hardware reports armv7l or armv8l which is essently arm
32bit mode. This fixes 5 testcases on 32bit arm.
