When targeting iOS, the default dwarf version is 2 and not 4. Currently,
the test suite does not pick up on that because it invokes the test
compiler without a target triple. This patch fixes that and now
correctly skips tests that have a dwarf version specified in a skipIf
decorator.
rdar://84530477
Differential revision: https://reviews.llvm.org/D112325
This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804.
This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like:
(lldb) statistics dump
{
"expressionEvaluation": {
"failures": 0,
"successes": 0
},
"firstStopTime": 0.34164492800000001,
"frameVariable": {
"failures": 0,
"successes": 0
},
"launchOrAttachTime": 0.31969605400000001,
"targetCreateTime": 0.0040863039999999998
}
The top level keys are:
"expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts.
"frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts.
"targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled.
"launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs.
"firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test.
This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class.
Differential Revision: https://reviews.llvm.org/D111686
None of the commands we run really rely on shell features. Running them
with shell=False, simplifies the code as there is no need for elaborate
quoting.
Differential Revision: https://reviews.llvm.org/D111990
We had two sets of build<flavour> methods, whose bodies were largely
identical. This makes any kind of modification in their vicinity
repetitive and error-prone.
Replace each set with a single method taking an optional debug_info
parameter.
Differential Revision: https://reviews.llvm.org/D111989
We only had that ability for regular debugger launches. This meant that
it was not possible to use the normal dlopen patterns in attach tests.
This fixes that.
This patch adds a helper function to test target architecture is
AArch64 or not. This also tightens isAArch64* helpers by adding an
extra architecture check.
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D105483
At the moment nearly every test calls something similar to
`self.dbg.CreateTarget(self.getBuildArtifact("a.out"))` and them sometimes
checks if the created target is actually valid with something like
`self.assertTrue(target.IsValid(), "some useless text")`.
Beside being really verbose the error messages generated by this pattern are
always just indicating that the target failed to be created but now why.
This patch introduces a helper function `createTestTarget` to our Test class
that creates the target with the much more verbose `CreateTarget` overload that
gives us back an SBError (with a fancy error). If the target couldn't be created
the function prints out the SBError that LLDB returned and asserts for us. It
also defaults to the "a.out" build artifact path that nearly all tests are using
to avoid to hardcode "a.out" in every test.
I converted a bunch of tests to the new function but I'll do the rest of the
test suite as follow ups.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D102771
AArch64 kernel builds default to having /smaps and
the "VmFlags" line was added in 3.8. Long before MTE
was supported.
So we can assume that if you're AArch64 with MTE,
you can run this test.
The previous method of checking had a race condition
where the process we read smaps for, could finish before
we get to read the file.
I explored some alternatives but in the end I think
it's fine to just assume we have what we need.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D100493
Use a with block for reading the cpuinfo file.
When loading the file fails (or we're not on Linux)
return an empty string. Since all the callers are
going to do "x in self.getCPUInfo()".
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D99729
This patch adds a test case to test AArch64 dynamic register sets.
This tests for the availability of certain register sets and query
their registers accordingly.
Reviewed By: labath, DavidSpickett
Differential Revision: https://reviews.llvm.org/D96463
Linux systems can be configured (and most of them are configured that
way) to disable attaching to unrelated processes, /unless/ those
processes explicitly allow that.
Our test inferiors do that by explicitly calling prctl(PR_SET_PTRACER,
PR_SET_PTRACER_ANY) (a.k.a., lldb_enable_attach). This requires
additional synchronization to ensure that the test does not attempt
attach before that statement is executed.
This is working fine (albeit cumbersome) for most tests but
TestGdbRemoteAttachWait is special in that it wants to start the
inferior _after_ issuing the attach request. This means that the usual
synchronization method does not work.
This patch introduces a different solution -- enable attaching in the
test harness, before the process is launched. Besides fixing this
problem, this is also better because it avoids the need to add special
code to each attach test (which is a common error).
One gotcha here is that it won't work for remote test suites, as we
don't control launching there. However, we could add a similar option to
lldb-platform, or require that lldb-platform itself is started with
attaching enabled. At that point we could delete all lldb_enable_attach
logic.
The test was marked as remote-only, which means it was run ~never, and
accumulated various problems. This commit modifies the test to run
locally and includes a couple of other fixes necessary to make it run:
- moves the "invoke" method into the "Base" test class
- adds []'s around the IP address in a couple more places to make things
work with IPv6
The test is now marked as skipped when running the remote test suite. It
would be possible to make it run both locally and remotely, but this
would require writing a lot special logic for the remote case, and that
is not worth it.
lldb-server tests are a very special subclass of "api" tests. As they
communicate with lldb-server directly, they don't actually need most of
facilities provided by our TestBase class. In particular, they don't
need the ability to fork debug info flavours of tests (but they could
use debug server flavours).
This makes them inherit from "Base" instead. This avoids the need to
explicitly mark these tests as NO_DEBUG_INFO_TEST_CASE. Two additional
necessary tweaks were:
- move run_platform_command to the base (Base) class. This is used in
one test, and can be generally useful when running tests remotely.
- add a "build" method, forwarding to buildDefault. This is to avoid
updating each test case to use buildDefault (also, "build" sounds
better). It might be interesting to refactor the (Test)Base classes so
that all debug info flavour handling happens in TestBase, and the Base
class provides a simple build method automatically.
Kill (rather than detach) form the inferior if debugserver loses its
connection to lldb to prevent zombie processes.
Differential revision: https://reviews.llvm.org/D92908
Test runs log some of their output to files inside the LLDB session dir. This
session dir is shared between all tests, so all the tests have to make sure they
choose a unique file name inside that directory. We currently choose by default
`<test-class-name>-<test-method-name>` as the log file name. However, that means
that if not every test class in the test suite has a unique class name, then we
end up with a race condition as two tests will try to write to the same log
file.
I already tried in D83767 changing the format to use the test file basename
instead (which we already require to be unique for some other functionality),
but it seems the code for getting the basename didn't work on Windows.
This patch instead just changes that dotest stores the log files in the build
directory for the current test. We know that directory is unique for this test,
so no need to generate some unique file name now. Also removes all the
environment vars and parameters related to the now unused session dir.
The new log paths now look like this for a failure in 'TestCppOperators`:
```
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dwarf/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dsym/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_gmodules/Failure.log
```
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D92498
This extends the "memory region" command to
show tagged regions on AArch64 Linux when the MTE
extension is enabled.
(lldb) memory region the_page
[0x0000fffff7ff8000-0x0000fffff7ff9000) rw-
memory tagging: enabled
This is done by adding an optional "flags" field to
the qMemoryRegion packet. The only supported flag is
"mt" but this can be extended.
This "mt" flag is read from /proc/{pid}/smaps on Linux,
other platforms will leave out the "flags" field.
Where this "mt" flag is received "memory region" will
show that it is enabled. If it is not or the target
doesn't support memory tagging, the line is not shown.
(since majority of the time tagging will not be enabled)
Testing is added for the existing /proc/{pid}/maps
parsing and the new smaps parsing.
Minidump parsing has been updated where needed,
though it only uses maps not smaps.
Target specific tests can be run with QEMU and I have
added MTE flags to the existing helper scripts.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D87442
This adds `expect_var_path` to test variable paths so we no longer have to
use `frame var` and find substrs in the command output. The behaviour
is identical with `expect_expr` (and it also uses the same checking backend),
but it instead calls `GetValueForVariablePath` to evaluate the string as a variable
path.
Also rewrites a few of the tests that previously used `frame variable` to use
`expect_var_path`.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D90450
Depends on D89408.
This diff finally implements trace decoding!
The current interface is
$ trace load /path/to/trace/session/file.json
$ thread trace dump instructions
thread #1: tid = 3842849, total instructions = 22
[ 0] 0x40052d
[ 1] 0x40052d
...
[19] 0x400521
$ # simply enter, which is a repeat command
[20] 0x40052d
[21] 0x400529
...
This doesn't do any disassembly, which will be done in the next diff.
Changes:
- Added an IntelPTDecoder class, that is a wrapper for libipt, which is the actual library that performs the decoding.
- Added TraceThreadDecoder class that decodes traces and memoizes the result to avoid repeating the decoding step.
- Added a DecodedThread class, which represents the output from decoding and that for the time being only stores the list of reconstructed instructions. Later it'll contain the function call hierarchy, which will enable reconstructing backtraces.
- Added basic APIs for accessing the trace in Trace.h:
- GetInstructionCount, which counts the number of instructions traced for a given thread
- IsTraceFailed, which returns an Error if decoding a thread failed
- ForEachInstruction, which iterates on the instructions traced for a given thread, concealing the internal storage of threads, as plug-ins can decide to generate the instructions on the fly or to store them all in a vector, like I do.
- DumpTraceInstructions was updated to print the instructions or show an error message if decoding was impossible.
- Tests included
Differential Revision: https://reviews.llvm.org/D89283
In D89056 the default value for architecture was moved to `build` so that
all called functions see the same architecture value. It seems there are a
few functions that call buildDefault directly (and not via build), so
on some test configurations that set a custom arch value the architecture
value is no longer available.
This just adds the architecture code from build to buildDefault to get
the bots green again while I'm looking for a better solution.
This originally broke the TestQuoting which explicitly called buildDefault
instead of calling build() and marking the test as no_debug_info_test.
TestQuoting has been rewritten by now and is using `build`, so this should now
pass on all platforms.
Original summary:
The Darwin builder currently assumes in `getArchCFlags` that the passed `arch`
value is an actual string it can string.join with vendor/os/version/env strings:
```
triple = '-'.join([arch, vendor, os, version, env])
```
However this is not true for most tests as we just pass down the `arch=None`
default value from `TestBase.build`. This causes that if we actually end up in
this function we just error out when concatenating `None` with the other actual
strings of vendor/os/version/env. What we should do instead is check that if
there is no test-specific architecture that we fall back to the configuration's
architecture value.
It seems we already worked around this in `builder.getArchSpec` by explicitly
falling back to the architecture specified in the configuration.
This patch just moves this fallback logic to the top `build` function so that it
affects all functions called from `TestBase.build`.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D89056
The intention is not to allow stop-hook commands to query the
user, so this is correct. It also works around a deadlock in
switching to the Python Session to execute python based commands
in the stop hook when the Debugger stdin is backed by a FILE *.
Differential Revision: https://reviews.llvm.org/D90332
The Darwin builder currently assumes in `getArchCFlags` that the passed `arch`
value is an actual string it can string.join with vendor/os/version/env strings:
```
triple = '-'.join([arch, vendor, os, version, env])
```
However this is not true for most tests as we just pass down the `arch=None`
default value from `TestBase.build`. This causes that if we actually end up in
this function we just error out when concatenating `None` with the other actual
strings of vendor/os/version/env. What we should do instead is check that if
there is no test-specific architecture that we fall back to the configuration's
architecture value.
It seems we already worked around this in `builder.getArchSpec` by explicitly
falling back to the architecture specified in the configuration.
This patch just moves this fallback logic to the top `build` function so that it
affects all functions called from `TestBase.build`.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D89056
Add preconditions to `TestBase.expect()` that catch semantically invalid calls
that happen to succeed anyway. This also fixes the broken callsites caught by
these checks.
This prevents the following incorrect calls:
1. `self.expect("lldb command", "some substr")`
2. `self.expect("lldb command", "assert message", "some substr")`
Differential Revision: https://reviews.llvm.org/D88792
A few fixes while trying to figure out why tests are being skipped for arsenm:
- We check `$compiler -v`, but `-v` is `--verbose`, not `--version`. Use the long flag name.
- We check all lines matching `version ...`, but we should exit early for the first version string we see (which should be the main one). I'm not sure if this is the issue, but perhaps this is causing some users to skip some tests if another "version ..." is showing up later.
- Having `\.` in a python string is triggering pylint warnings, because it should be escaped as a regex string, e.g. `r'\.' However, `.` in a character class does not need to be escaped, as it matches only a literal `.` in that context.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D88051
targetHasSVE helper function was added to test for availability of SVE support
by connected platform. We now intend to use this function in other testcases
and I am moving it to a generic location in lldbtest.py to allow usage by
other upcoming testcases.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86872
This updates the errors reported by expect()
to something like:
```
Ran command:
"help"
Got output:
Debugger commands:
<...>
Expecting start string: "Debugger commands:" (was found)
Expecting end string: "foo" (was not found)
```
(see added tests for more examples)
This shows the user exactly what was run,
what checks passed and which failed. Along with
whether that check was supposed to pass.
(including what regex patterns matched)
These lines are also output to the test
trace file, whether the test passes or not.
Note that expect() will still fail at the first failed
check, in line with previous behaviour.
Also I have flipped the wording of the assert
message functions (.*_MSG) to describe failures
not successes. This makes more sense as they are
only shown on assert failures.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86792
TestCompletion is randomly failing on some bots. The error message however states
that the computed completions actually do contain the expected pid we're
looking for, so there shouldn't be any test failure.
The reason for that turns out to be that complete_from_to is actually used
for testing two different features. It can be used for testing what the
common prefix for the list of completions is and *also* for checking all the
possible completions that are returned for a command. Which one of the two
things should be checked can't be defined by a parameter to the function, but
is instead guessed by the test method instead based on the results that were
returned. If there is a common prefix in all completions, then that prefix
is searched and otherwise all completions are searched.
For TestCompletion's pid test this behaviour leads to the strange test failures.
If all the pid's that our test LLDB can see have a common prefix (e.g., it
can only see pids [123, 122, 10004, 10000] -> common prefix '1'), then
complete_from_to check that the common prefix contains our pid, which is
always fails ('1' doesn't contain '123' or any other valid pid). If there
isn't a common prefix (e.g., pids are [123, 122, 10004, 777]) then
complete_from_to will check the list of completions instead which works correctly.
This patch is fixing this by adding a simple check method that doesn't
have this behaviour and is simply searching the returned list of completions.
This should get the bots green while I'm working on a proper fix that fixes
complete_from_to.
This patch removes the rather confusing LLDB_LIB_DIR and LLDB_IMPLIB_DIR
environment variables. They are confusing because LLDB_LIB_DIR would
point to the bin subdirectory in the build root while LLDB_IMPLIB_DIR
would point to the lib subdirectory. The reason far this was
LLDB.framework, which gets build under bin.
This patch replaces their uses with configuration.lldb_framework_path
and configuration.lldb_libs_dir respectively.
Differential revision: https://reviews.llvm.org/D86817
EXP_MSG generates a message to show on assert
failure. Currently it looks like:
AssertionError: False is not True : '<cmd>'
returns expected result, got '<actual output>'
Which seems to say that the test failed but
also got the expected result.
It should say:
AssertionError: False is not True : '<cmd>'
returned unexpected result, got '<actual output>'
Reviewed By: teemperor, #lldb
Differential Revision: https://reviews.llvm.org/D86603
Rather than have different modules for different platforms, use
inheritance so we can have a Builer base class and optional child
classes that override platform specific methods.
Differential revision: https://reviews.llvm.org/D86174
TypeSystemClang::CreateTypedef was creating a typedef in the right
DeclContext, but it was not actually adding it as a child of the
context. The resulting inconsistent state meant that we would be unable
to reference the typedef from an expression directly, but we could use
them if they end up being pulled in by some previous subexpression
(because the ASTImporter will set up the correct links in the expression
ast).
This patch adds the typedef to the decl context it is created in.
Differential Revision: https://reviews.llvm.org/D86140
Right now the only places in the SB API where lldb:: ModuleSP instances are
destroyed are in SBDebugger::MemoryPressureDetected (where it's just attempted
but not guaranteed) and in SBDebugger::DeleteTarget (which will be removed in
D83933). Tests that directly create an lldb::ModuleSP and never create a target
therefore currently leak lldb::Module instances. This triggers the sanity checks
in lldbtest that make sure that the global module list is empty after a test.
This patch adds SBModule::GarbageCollectAllocatedModules as an explicit way to
clean orphaned lldb::ModuleSP instances. Also we now start calling this method
at the end of each test run and move the sanity check behind that call to make
this work. This way even tests that don't create targets can pass the sanity
check.
This fixes TestUnicodeSymbols.py when D83865 is applied (which makes that the
sanity checks actually fail the test).
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D83876
expect_expr currently can't verify the children of the result SBValue.
This patch adds the ability to check them. The idea is to have a CheckValue
class where one can specify what attributes of a SBValue should be checked.
Beside the properties we already check for (summary, type, etc.) this also
has a list of children which is again just a list of CheckValue object (which
can also have children of their own).
The main motivation is to make checking the children no longer based
on error-prone substring checks that allow tests to pass just because
for example the error message contains the expected substrings by accident.
I also expect that we can just have a variant of `expect_expr` for LLDB's
expression paths (aka 'frame var') feature.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D83792
`getCompilerVersion` assumes that `clang --version` prints out a string like `version [0-9\.]+`.
If clang is built from trunk, the version line might look like `clang version trunk (123abc)`.
Since there isn't any way of knowing by the commit id alone whether one commit is newer or older than another git commit (or clang version), assume that clang with a version id like this is very close to trunk. For example, any tests with `@skipIf(compiler="clang", compiler_version=['<', '8'])` should be run.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D85248
LLDB tests assume that tests are in the test tree (the `LLDB_TEST_SRC` env variable, configured by `dotest.py`).
If this assertion doesn't hold, tests fail in strange ways. An early place this goes wrong is in `compute_mydir` which does a simple length-based substring to get the relative path. Later, we use that path to chdir to. If the test file and test tree don't agree in realpath-ness (and therefore length), this will be a cryptic error of chdir-ing to a directory that does not exist.
The actual discrepency is that the places we look for `use_lldb_suite.py` don't use a realpath, but `dotest.py` does (see initialization of `configuration.testdirs`).
It doesn't particularly matter whether we use realpath or abspath to canonicalize things, but many places end up with implicit dependencies on the canonicalized pwd being a realpath, so make them realpath consistently. Also, in the `compute_mydir` method mentioned, raise an error if the path types don't agree.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85258
Add an option that allows the user to decide to not make the inferior is
responsible for its own TCC permissions. If you don't make the inferior
responsible, it inherits the permissions of its parent. The motivation
is the scenario of running the LLDB test suite from an external hard
drive. If the inferior is responsible, every test needs to be granted
access to the external volume. When the permissions are inherited,
approval needs to be granted only once.
Differential revision: https://reviews.llvm.org/D85237
