The logic behind --rerun-all-issues was removed when we switched to LIT
as the test driver. This patch just removes the dotest option and
corresponding entry in configuration.py.
This patch stores the --apple-sdk argument in the dotest configuration.
When it's set, use it instead of the triple to determine the current
platform.
Differential revision: https://reviews.llvm.org/D85537
In these two cases, use of `os.path.realpath` is problematic:
- The name of the compiler is significant [1] . For testing purposes, we might
provide a compiler called "clang" which is actually a symlink to some build
script (which does some flag processing before invoking the real clang). The
destination the symlink may not be called "clang", but we still want it to be
treated as such.
- When using a build system that puts build artifacts in an arbitrary build
location, and later creates a symlink for it (e.g. creates a
"<lldb root>/lldbsuite/test/dotest.py" symlinks that points to
"/build/artifact/<hash>/dotest.py"), looking at the realpath will not match
the "test" convention required here.
[1] See `Makefile.rules` in the lldb tree, e.g. we use different flags if the compiler is named "clang"
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85175
This reverts commit adb5c23f8c0d60eeec41dcbe21d1b26184e1c97d. It surprisingly fails on a windows build bot: http://lab.llvm.org:8011/builders/lldb-x64-windows-ninja/builds/18009
Will reland after some investigation and/or after adding some extra logging to help debug the issue.
If the test suite is misconfigured when it's run (a bad regexp, wrong test directory, etc.), the test suite may not discover any tests. When this happens, the test runner exits happily because no tests failed:
```
Ran 0 tests in 0.000s
RESULT: PASSED (0 passes, 0 failures, 0 errors, 0 skipped, 0 expected failures, 0 unexpected successes)
```
Change this to return an error so the misconfiguration can be more easily detected. Verified that `lldb-dotest -p TestDoesNotExist.py` successfully fails.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85169
After more investigation, I realised this part of the code is totally
unused. It was used for communicating the test results from the
"inferior" dotest process to the main "dosep" process running
everything. Now that everything is being orchestrated through lit, this
is not used for anything.
Summary:
My understanding is that this was added to make dotest interact well
with the GreenDragon bots, back when dotest was the main test driver.
Now that everything goes through lit (which has its own xunit
formatter), it seems largely irrelevant.
There are more cleanups that can be done after removing this be done
here, but this should be enough to test the waters.
Reviewers: JDevlieghere
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D83545
This gets rid of some surprising interplay between the flags.
Mainly needed because of Rosetta debugserver & Apple Silicon.
Differential Revision: https://reviews.llvm.org/D82804
and delete a bunch (but not all) redundant code. If you compare the remaining implementations of Platform*Simulator.cpp, there is still an obvious leftover cleanup task.
Specifically, this patch
- removes SDK initialization from dotest (there is equivalent but more
complete code in Makefile.rules)
- make Platform*Simulator inherit the generic implementation of
PlatformAppleSimulator (more can be done here)
- simplify the platform logic in Makefile.rules
- replace the custom SDK finding logic in Platform*Simulator with XcodeSDK
- adds a test for each supported simulator
Differential Revision: https://reviews.llvm.org/D81980
The different tools constructing dotest invocations (lit and
lldb-dotest) already print the command invocation so there's no need to
print it again in the dotest output.
My motivation for removing it is that it doesn't include the Python
interpreter and every time I accidentally copy it, the command fails
with an `ImportError`.
Differential revision: https://reviews.llvm.org/D81032
Don't use the environment to pass values to the builder that are present
in the dotest configuration module. A subsequent patch will pass the
remaining values through the configuration instead of the environment.
[intel-pt] Improve the way the test determines whether to run
- Now I'm creating a default value for the new test parameter
- I fixed a small mistake in the skipping logic of the test
... I forgot to clear the cmake cache when testing my diff
Summary:
@labath raised a concern on the way I was skipping this test. I think that was
fair and I found a better way.
Now I'm skipping if the CMAKE flag LLDB_BUILD_INTEL_PT is false.
I added an enabled_plugins entry in the dotest configuration, which gets
set by lit or lldb-dotest. The only available plugin right now is
'intel-pt', but I imagine it will be useful in the future for other
kinds of plugins that get determined at configuration time. I didn't
want to add a new argument option --enable-intel-pt or something or the
sort, as it wouldn't be useful for other cases.
Reviewers: labath, clayborg
Subscribers: lldb-commits, labath
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D77452
Make it possible to capture reproducers from the API test suite. Given
the symmetry between capture and replay, this patch also adds the
necessary code for replay. For now this is a NO-OP until the
corresponding reproducer instrumentation changes land.
For more info please refer to the RFC on lldb-dev:
http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html
Differential revision: https://reviews.llvm.org/D77588
Using the approach suggested by Pavel in D77588, this patch introduces a
new lldbconfig module that lives next to the lldb module. It makes it
possible to make the lldb module configurable before importing it. More
specifically it makes it possible to delay initializing the debugger,
which is needed for testing the reproducer.
Differential revision: https://reviews.llvm.org/D77661
Some tests set settings and don't clean them up, this leads to side effects in other tests.
The patch removes a global debugger instance with a per-test debugger to avoid such effects.
From what I see, lldb.DBG was needed to determine the platform before a test is run,
lldb.selected_platform is used for this purpose now. Though, this required adding a new function
to the SBPlatform interface.
Differential Revision: https://reviews.llvm.org/D74903
Summary:
This was added in 2018 (r339929), when we were still using the
hand-rolled test runner.
It does not seem to be relevant anymore. In fact as far as I can tell,
it's a big no-op now as the exclusive_test_subdir variable is never set.
Reviewers: vsk, JDevlieghere
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74551
Summary:
This creates a separate LLDB_TEST_SRC var to match the existing LLDB_TEST var. LLDB_TEST points to the test framework, LLDB_TEST_SRC points to the tests themselves.
The var points to the same place, but a future patch will move the tree + update var.
Reviewers: labath, JDevlieghere
Reviewed By: labath
Subscribers: merge_guards_bot, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71150
Pass the correct library directory from CMake to dotest.py when linking
liblldb, instead of trying to reconstruct the path from executable path.
This fixes link failures on platforms having non-null
LLVM_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D73767
The primary motivation for this is to add another dimension to the
Swift LLDB test matrix, but this seems generally useful.
Differential Revision: https://reviews.llvm.org/D72662
There already are decorators and "--excluded" option to mark test-cases/files
as expected to fail. However, when a new test file is added and it which relates
to a feature that a target doesn't support, this requires either adding decorators
to that file or modifying the file provided as "--excluded" option value.
The purpose of this patch is to avoid any modifications in such cases.
E.g. if a target doesn't support "watchpoints" and passes "--xfail-category watchpoint"
to dotest, a testing job will not fail after a new watchpoint-related test file is added.
Differential Revision: https://reviews.llvm.org/D71906
Summary: The test logic for running libc++ tests only looks to see if `/usr/include/c++/v1` exists. This adds a fallback for including libc++ tests as long as `$(CC) -stdlib=libc++` works.
Reviewers: labath, EricWF
Subscribers: ldionne, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71319
Before the test reorganization, everything was part of a single test
suite with a single module cache. Now that things are properly separated
this is no longer the case. Only the shell tests inherited the logic to
properly configure and wipe the module caches. This patch adds that
logic back for the API tests. While doing so, I noticed that we were
configuring a Clang module cache in CMake, but weren't actually using it
from dotest.py. I included a fix for that in this patch as well.
Differential revision: https://reviews.llvm.org/D68755
llvm-svn: 374386
Summary:
dotest.py currently requires a filterspec to be of the
form `TestCase.test_method`. This patch makes it more
flexible, so you can pass `TestModule.TestCase.test_method`
or `TestModule.TestCase` or `TestCase.test_method` or just
`test_method`.
This makes it more convenient to just copy a test name
out of the terminal after running a bunch of tests and use
it as a filterspec.
Reviewers: JDevlieghere, jasonmolenda, labath
Reviewed By: JDevlieghere
Subscribers: jingham, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D68545
llvm-svn: 373997
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
It used to be possible to enable logging through environment variables
read by dotest. This approach is deprecated, as stated in the dotest
help output. Instead --channel should be used.
Differential revision: https://reviews.llvm.org/D66920
llvm-svn: 370387
This removes the curses result formatter which appears to be broken.
Passing --curses to dotest.py screws up my terminal and doesn't run any
tests. It even crashes Python on occasion.
Differential revision: https://reviews.llvm.org/D66917
llvm-svn: 370386
This removes support for reading the LLDB_TEST_ARGUMENTS environment
variable and instead requires all arguments to be specified as part of
the invocation. This ensures that dotest.py invocations are easily
repeatable.
Differential revision: https://reviews.llvm.org/D66912
llvm-svn: 370278
This argument was used by dosep.py to pass information around from the
workers. With dosep.py gone, I'm fairly sure we don't need this any
longer.
llvm-svn: 370266
I was looking at the session directory logic for unrelated reasons and
noticed that the logic spread out across dotest. This simplifies things
a bit by moving the logic together.
llvm-svn: 370259
Now that all supported build systems create a valid dotest.py
invocation, we no longer need to guess the location of the lldb binary
and Python directory.
Differential revision: https://reviews.llvm.org/D66896
llvm-svn: 370234
Now that all supported build systems create a valid dotest.py
invocation, we no longer need to guess the directory where any of the
llvm tools live. Additionally, the current logic is incomplete: it
doesn't try to find any other tools than FileCheck, such as dsymutil for
example.
If no FileCheck is provided, we should print a warning and skip the
tests that need it, but that's not part of this patch.
Differential revision: https://reviews.llvm.org/D66893
llvm-svn: 370232
This patch removes the -q (quiet) flag and changing the default
behavior. Currently the flag serves two purposes that are somewhat
contradictory, as illustrated by the difference between the argument
name (quiet) and the configuration flag (parsable). On the one hand it
reduces output, but on the other hand it prints more output, like the
result of individual tests. My proposal is to guard the extra output
behind the verbose flag and always print the individual test results.
Differential revision: https://reviews.llvm.org/D66837
llvm-svn: 370226
