Some API tests (compiler calls) create a lot of garbage and cause
unexpected behavior in case of Windows host and Linux target, e.g.
```
lldb/test/API/commands/process/attach/%SystemDrive%/
lldb/test/API/functionalities/deleted-executable/%SystemDrive%/
lldb/test/API/functionalities/exec/%SystemDrive%/
lldb/test/API/functionalities/load_unload/%SystemDrive%/
lldb/test/API/functionalities/target-new-solib-notifications/%SystemDrive%/
lldb/test/API/functionalities/thread/create_after_attach/%SystemDrive%/
```
It can be fixed by transfer some standard Windows environment variables
into API tests.
Remove lldb-repro which was used to run the test suite against a
reproducer. The corresponding functionality has been removed from LLDB
so there's no need for the tool anymore.
Many LLDB's dotest.py based tests require the `make` tool. If it's not found in Path, they fail with an obscure error and show up as `UNRESOLVED`. On Windows, llvm-lit takes care of MSYS based testing tools like cat, printf, etc., but `make` is not part of that. Let's catch the situation early and check for it at configuration time.
This error isn't fatal: It should fail the build, but not immediately stop the configuration process. There might be other issues further down the line that can be caught in the same buildbot run.
1. This commit adds LLDB_TEST_PLATFORM_URL, LLDB_TEST_SYSROOT,
LLDB_TEST_PLATFORM_WORKING_DIR, LLDB_SHELL_TESTS_DISABLE_REMOTE cmake
flags to pass arguments for cross-compilation and remote running of both Shell&API tests.
2. To run Shell tests remotely, it adds 'platform select' and 'platform connect' commands to %lldb
substitution.
3. 'remote-linux' feature added to lit to disable tests failing with
remote execution.
4. A separate working directory is assigned to each test to avoid
conflicts during parallel test execution.
5. Remote Shell testing is run only when LLDB_TEST_SYSROOT is set for
building test sources. The recommended compiler for that is Clang.
---------
Co-authored-by: Vladimir Vereschaka <vvereschaka@accesssoftek.com>
The Python interpreter in Xcode cannot be copied because of a relative
RPATH. Our workaround would just use that Python interpreter directly
when it detects this. For the reasons explained in my previous commit,
that doesn't work in a virtual environment. Address this case by
creating a symlink to the "real" interpreter in the virtual environment.
On macOS, to make DYLD_INSERT_LIBRARIES and the Python shim work
together, we have a workaroud that copies the "real" Python interpreter
into the build directory. This doesn't work when running in a virtual
environment, as the copied interpreter cannot find the packages
installed in the virtual environment relative to itself.
Address this issue by copying the Python interpreter into the virtual
environment's `bin` folder, rather than the build folder, when the test
suite detects that it's being run inside a virtual environment.
I'm not thrilled about this solution because it puts a file outside the
build directory. However, given virtual environments are considered
disposable, this seems reasonable.
Avoid copying the Python interpreter when running in a virtual
environment as it will already have its own copy of the Python
interpreter. Also leave a breadcrumb that we're running with a different
Python interpreter.
that separates out language and version. To avoid reinventing the wheel
and introducing subtle incompatibilities, this API uses the table of
languages and versiond defined by the upcoming DWARF 6 standard
(https://dwarfstd.org/languages-v6.html). While the DWARF 6 spec is not
finialized, the list of languages is broadly considered stable.
The primary motivation for this is to allow the Swift language plugin to
switch between language dialects between, e.g., Swift 5.9 and 6.0 with
out introducing a ton of new language codes. On the main branch this
change is considered NFC.
Depends on https://github.com/llvm/llvm-project/pull/89980
The `LLDB_TEST_USE_VENDOR_PACKAGES` has defaulted to `Off` for a while.
Either installing `pexpect` or skipping those tests with
`-DLLDB_TEST_USER_ARGS=--skip-category=pexpect` seems to be enough that
we can fully remove this option.
This patch removes the `LLDB_TEST_USE_VENDOR_PACKAGES` cmake
configuration as well as the associated code to add
`third_party/Python/module` to the python path. I'll do the actual
deletion of `third_party/Python/module` in a followup PR in the
(unlikely, I hope) event this commit needs to be reverted.
…(#88436)"
This reverts commit 1f5d130df85c2d0550dc8687ad0fa1d96856c318. The
original commit checks that the host system is "Darwin" before setting
the `MallocNanoZone` envvar, but on the Shell lit config this attribute
does not exist at the point where it is being checked which leads to a
build failure.
This commit checks the host OS correctly.
When builds on Darwin are configured with ASan, running tests will emit
a warning about malloc's nano zone (`malloc: nano zone abandoned due to inability to reserve vm space`) that can interfere with the test
output and cause failures. Setting the environment variable
`MallocNanoZone` to 0 will remove this warning and allow the tests to
run as normal.
The goal here is to remove the third_party/Python/module tree, which
LLDB tests only use to `import pexpect`. This package is available on
`pip`, and I believe should not be hard to obtain.
However, in case it isn't easily available, deleting the tree right now
could cause disruption. This introduces a
`LLDB_TEST_USE_VENDOR_PACKAGES` cmake param that can be enabled, and the
tests will continue loading that tree. By default, it is enabled,
meaning there's really no change here. A followup change will disable it
by default once all known build bots are updated to include this
package. When disabled, an eager cmake check runs that makes sure
`pexpect` is available before waiting for the test to fail in an obscure
way.
Later, this option will go away, and when it does, we can delete the
tree too. Ideally this is not disruptive, and we can remove it in a week
or two.
Fix the bug where merge-fdata unconditionally outputs boltedcollection
line, regardless of whether input files have it set.
Test Plan:
Added bolt/test/X86/merge-fdata-nobat-mode.test which fails without this
fix.
There are some uses of "vscode" in
`lldb/packages/Python/lldbsuite/test/tools/lldb-dap/dap_server.py` where
I'm not sure if it's referring to the adapter or VS Code itself, so
those remain.
Set the halt_on_error runtime flag to make TSan errors fatal when
running the test suite. For the API tests the environment variables are
set conditionally on whether the TSan is enabled. The Shell and Unit
tests don't have that logic but setting the environment variable is
harmless. For consistency, I've also mirrored the ASAN option
(detect_stack_use_after_return=1) for the Shell tests.
Differential revision: https://reviews.llvm.org/D157152
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
In certain configurations, libc++ headers all exist in the same directory, and libc++ binaries exist in the same directory as lldb libs. When `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` is enabled (*and* the host is not Apple, which is why I assume this wasn't caught by others?), this is not the case: most headers will exist in the usual `include/c++/v1` directory, but `__config_site` exists in `include/$TRIPLE/c++/v1`. Likewise, the libc++.so binary exists in `lib/$TRIPLE/`, not `lib/` (where LLDB libraries reside).
This also adds the just-built-libcxx functionality to the lldb-dotest tool.
The `LIBCXX_` cmake config is borrowed from `libcxx/CMakeLists.txt`. I could not figure out a way to share the cmake config; ideally we would reuse the same config instead of copy/paste.
Reviewed By: JDevlieghere, fdeazeve
Differential Revision: https://reviews.llvm.org/D133973
The CMake variable LLDB_HAS_LIBCXX is passed to
`llvm_canonicalize_cmake_booleans`, which transforms TRUE/FALSE into
'1'/'0'. It also transforms undefined variables to '0'.
In particular, this means that the configuration script for LLDB API's
test always has _some_ value for the `has_libcxx` configuration:
```
config.has_libcxx = '@LLDB_HAS_LIBCXX@'
```
When deciding whether a libcxx exist, the testing scripts would only
check for the existence of `has_libcxx`, but not for its value. In other
words, because `if ('0')` is true in python we always think there is a
libcxx.
This was caught once D132940 was merged and most tests started to use
libcxx by default if `has_libcxx` is true. Prior to that, no failures
were seen because only tests are marked with
`@add_test_categories(["libc++"])` would require a libcxx, and these
would be filtered out on builds without the libcxx target. Furthermore,
the MacOS bots always build libcxx.
We fix this by making `has_libcxx` a boolean (instead of a string) and
by checking its value in the test configuration.
Differential Revision: https://reviews.llvm.org/D133639
When running LLDB API tests, a user can override test arguments with
LLDB_TEST_USER_ARGS. However, these flags used to be concatenated with a
CMake-derived variable LLDB_TEST_COMMON_ARGS, as below:
```
set(LLDB_DOTEST_ARGS ${LLDB_TEST_COMMON_ARGS};${LLDB_TEST_USER_ARGS}
CACHE INTERNAL STRING)
```
This is problematic, because LLDB_TEST_COMMON_ARGS must be processed
first, while LLDB_TEST_USER_ARGS args must be processed last, so that
user overrides are respected. Currently, if a user attempts to override
one of the "inferred" flags, the user's request is ignored. This is the
case, for example, with `--libcxx-include-dir` and
`--libcxx-library-dir`. Both flags are needed by the greendragon bots.
This commit removes the concatenation above, keeping the two original
variables throughout the entire flow, processing the user's flag last.
The variable LLDB_TEST_COMMON_ARGS needs to be a CACHE property, but it
is modified throughout the CMake file with `set` or `list` or `string`
commands, which don't work with properties. As such, a temporary
variable `LLDB_TEST_COMMON_ARGS_VAR` is created.
This was tested locally by invoking CMake with:
-DLLDB_TEST_USER_ARGS="--libcxx-include-dir=blah --libcxx-library-dir=blah2"
and checking that tests failed appropriately.
Differential Revision: https://reviews.llvm.org/D132642
This patch combines D129166 (to always pick the just-built libc++) and
D132257 (to allow customizing the libc++ for testing). The common goal
is to avoid picking up an unexpected libc++ for testing.
Differential revision: https://reviews.llvm.org/D132263
Make sure we use the libc++ from the build dir. Currently, by passing
-stdlib=libc++, we might pick up the system libc++. This change ensures
that if LLVM_LIBS_DIR is set, we try to use the libc++ from there.
Differential revision: https://reviews.llvm.org/D129166
Summary:
This is an essential piece of infrastructure for us to be
continuously testing debug info with BOLT. We can't only make changes
to a test repo because we need to change debuginfo tests to call BOLT,
hence, this diff needs to sit in our opensource repo. But when upstreaming
to LLVM, this should be kept BOLT-only outside of LLVM. When upstreaming,
we need to git diff and check all folders that are being modified by our
commits and discard this one (and leave as an internal diff).
To test BOLT in debuginfo tests, configure it with -DLLVM_TEST_BOLT=ON.
Then run check-lldb and check-debuginfo.
Manual rebase conflict history:
https://phabricator.intern.facebook.com/D29205224https://phabricator.intern.facebook.com/D29564078https://phabricator.intern.facebook.com/D33289118https://phabricator.intern.facebook.com/D34957174
Test Plan:
tested locally
Configured with:
-DLLVM_ENABLE_PROJECTS="clang;lld;lldb;compiler-rt;bolt;debuginfo-tests"
-DLLVM_TEST_BOLT=ON
Ran test suite with:
ninja check-debuginfo
ninja check-lldb
Reviewers: #llvm-bolt
Subscribers: ayermolo, phabricatorlinter
Differential Revision: https://phabricator.intern.facebook.com/D35317341
Tasks: T92898286
Avoid "TERM environment variable not set" by either propagating the TERM
environment variable or defaulting to vt100. All of our CI is already
doing this explicitly through the --env dotest arg, but it's easy to
forget when setting up a new job. I don't see any downside in making it
the default.
Remove the remaining references to LLDB_CAPTURE_REPRODUCER. I removed
the functionality in an earlier commit but forgot that there was a
corresponding test and logic to unset it in our test suite.
lit has grown a feature where it stores the runtimes of all tests.
Normally, these times should be stored in the build directory, but
because our API tests have set test_exec_root to point to the source
tree, it has ended up polluting our checkout and led to the
.lit_test_times.txt being committed to the repository.
Delete this file, and adjust the exec root of API tests. I've also
needed to adjust the root of Shell tests, in order to avoid the two
overlapping.
Make the API, Shell and Unit tests independent lit test suites. This
allows us to specify different dependencies and skip rebuilding all the
unit test (which is particularly expensive) when running check-lldb-api
or check-lldb-shell.
This does not change the autogenerated targets such as
check-lldb-shell-driver or the top level check-lldb target, which all
continue to work as before.
Differential revision: https://reviews.llvm.org/D98842
This variable is used to reducing the likelihood of hitting module cache
issues in CI where different branches can potentially run on the same
machine.
Replace the dotest command line options and various cmake variables,
which are used for passing the locations of llvm tools to the API tests
with a single variable, which points to the directory these tools are
placed in. Besides reducing repetition, this also makes things more
similar to how "normal" llvm tests are configured.
Differential Revision: https://reviews.llvm.org/D95261
Test runs log some of their output to files inside the LLDB session dir. This
session dir is shared between all tests, so all the tests have to make sure they
choose a unique file name inside that directory. We currently choose by default
`<test-class-name>-<test-method-name>` as the log file name. However, that means
that if not every test class in the test suite has a unique class name, then we
end up with a race condition as two tests will try to write to the same log
file.
I already tried in D83767 changing the format to use the test file basename
instead (which we already require to be unique for some other functionality),
but it seems the code for getting the basename didn't work on Windows.
This patch instead just changes that dotest stores the log files in the build
directory for the current test. We know that directory is unique for this test,
so no need to generate some unique file name now. Also removes all the
environment vars and parameters related to the now unused session dir.
The new log paths now look like this for a failure in 'TestCppOperators`:
```
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dwarf/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dsym/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_gmodules/Failure.log
```
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D92498
The new FreeBSDRemote plugin has reached feature parity on i386
and amd64 targets. Use it by default on these architectures, while
allowing the use of the legacy plugin via FREEBSD_LEGACY_PLUGIN envvar.
Revisit the method of switching plugins. Apparently, the return value
of PlatformFreeBSD::CanDebugProcess() is what really decides whether
the legacy or the new plugin is used.
Update the test status. Reenable the tests that were previously
disabled on FreeBSD and do not cause hangs or are irrelevant to FreeBSD.
Mark all tests that fail reliably as expectedFailure. For now, tests
that are flaky (i.e. produce unstable results) are left enabled
and cause unpredictable test failures.
Differential Revision: https://reviews.llvm.org/D90757
Replace the inline x86 watchpoint handling code with the reusable
NativeRegisterContextWatchpoint_x86. Implement watchpoint support
in NativeThreadFreeBSD and SIGTRAP handling for watchpoints.
Un-skip all concurrent_events tests as they pass with the new plugin.
Differential Revision: https://reviews.llvm.org/D90102
This patch adds several build system targets that run the normal test suite but
against the Watch/TV/iPhone simulators.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D89224
LIT uses a model where the test suite is configurable trough a
lit.site.cfg file. Most of the time we use the lit.site.cfg with values
that match the current build configuration, generated by CMake.
Nothing prevents you from running the test suite with a different
configuration, either by overriding some of these values from the
command line, or by passing a different lit.site.cfg.
The latter is currently tedious. Many configuration values are optional
but they still need to be set because lit.cfg.py is accessing them
directly. This patch changes the code to use getattr to return the
attribute if it exists. This makes it possible to specify a minimal
lit.site.cfg with only the mandatory/desired configuration values.
Differential revision: https://reviews.llvm.org/D86821
