The find-dynamic-unwind-info callback registration APIs in libunwind
limit the number of callbacks that can be registered. If we use multiple
UnwindInfoManager instances, each with their own own callback function
(as was the case prior to this patch) we can quickly exceed this limit
(see https://github.com/llvm/llvm-project/issues/126611).
This patch updates the UnwindInfoManager class to use a singleton
pattern, with the single instance shared between all LLVM JITs in the
process.
This change does _not_ apply to compact unwind info registered through
the ORC runtime (which currently installs its own callbacks).
As a bonus this change eliminates the need to load an IR "bouncer"
module to supply the unique callback for each instance, so support for
compact-unwind can be extended to the llvm-jitlink tools (which does not
support adding IR).
The system libunwind on older Darwins does not support JIT registration of
compact-unwind. Since the CompactUnwindManager utility discards redundant
eh-frame FDEs by default we need to remove the compact-unwind section first
when targeting older libunwinds in order to preserve eh-frames.
While LLJIT was already doing this as of eae6d6d18bd, MachOPlatform was not.
This was causing buildbot failures in the ORC runtime (e.g. in
https://green.lab.llvm.org/job/llvm.org/job/clang-stage1-RA/3479/).
This patch updates both LLJIT and MachOPlatform to check a bootstrap value,
"darwin-use-ehframes-only", to determine whether to forcibly preserve
eh-frame sections. If this value is present and set to true then compact-unwind
sections will be discarded, causing eh-frames to be preserved. If the value is
absent or set to false then compact-unwind will be used and redundant FDEs in
eh-frames discarded (FDEs that are needed by the compact-unwind section are
always preserved).
rdar://143895614
Re-enables compact-unwind support in JITLink, which was reverted in b04847b427d
due to buildbot failures.
The underlying cause for the failures on the buildbots was the lack of
compact-unwind registration support on older Darwin OSes. Since the
CompactUnwindManager pass now removes eh-frames by default we were left with
unwind-info that could not be registered. On x86-64, where eh-frame info is
produced by default the solution is to fall back to using eh-frames. On arm64
we simply can't support exceptions on older OSes.
This patch updates the EHFrameRegistrationPlugin to remove the compact-unwind
section (__LD,__compact_unwind) when installed, forcing use of eh-frames when
the EHFrameRegistrationPlugin is used. In LLJIT, the EHFrameRegistrationPlugin
continues to be used for all non-Darwin platform, and will be added on Darwin
platforms when the a CompactUnwindRegistrationPlugin instance can't be created
(e.g. due to missing support for compact-unwind info registration).
The lit.cfg.py script is updated to check whether the host OSes default unwind
info supports JIT registration, allowing tests to be disabled for older Darwin
OSes on arm64.
This reverts commit d6524c8dfa37634257050ca71d16e117b802181c. This
reverts commit b1bd73700a1fb6f450e0f6f9c405a9c8bde2cae7.
This was causing bot failures on Darwin
https://green.lab.llvm.org/job/llvm.org/job/clang-stage1-cmake-RA-incremental/7315/
Clang.Interpreter.simple-exception.cpp
Clang-Unit.Interpreter/ExceptionTests/_/ClangReplInterpreterExceptionTests/0.1
LLVM.ExecutionEngine/OrcLazy.minimal-throw-catch.ll
https://green.lab.llvm.org/job/llvm.org/job/clang-stage1-RA/3415/
ORC-x86_64-darwin.TestCases/Darwin/Generic.exceptions.cpp
ORC-x86_64-darwin.TestCases/Darwin/x86-64.lljit-ehframe.cpp
This reapplies 4f0325873fa (and follow up patches 26fc07d5d88, a001cc0e6cdc,
c9bc242e387, and fd174f0ff3e), which were reverted in 212cdc9a377 to
investigate bot failures (e.g.
https://lab.llvm.org/buildbot/#/builders/108/builds/8502)
The fix to address the bot failures was landed in d0052ebbe2e. This patch also
restricts construction of the UnwindInfoManager object to Apple platforms (as
it won't be used on other platforms).
Attempt to install the EHFrameRegistrationPlugin during GenericLLVMIRPlatform
setup, rather than object linking layer creation time.
Platform classes are responsible for exception handling: installing the plugin
unconditionally during linking-layer construction could result in frames being
registered more than once when native platform classes are used.
This is a precursor to re-landing compact unwind support (4f0325873fa).
This reverts 4f0325873faccfbe171bae4babceb65975ca892e and follow-up patches
(see below) while I investigate some ongoing failures on the buildbots.
---
Revert "[clang-repl] Try to XFAIL testcase on arm32 without affecting arm64
darwin."
This reverts commit fd174f0ff3e793fe96a6663b1488ed159cfe042f.
Revert "[clang-repl] The simple-exception test now passes on arm64-darwin."
This reverts commit c9bc242e387f4a4a3dfcd86561f3ec0ca8a72d62.
Revert "[ORC] Destroy defunct MaterializationUnits outside the session lock."
This reverts commit a001cc0e6cdcfa672b8aff9ce6d14782bb96356a.
Revert "[ORC] Add explicit narrowing casts to fix build errors."
This reverts commit 26fc07d5d88760ad659599184fd10181287d2d9e.
Revert "[ORC] Enable JIT support for the compact-unwind frame info format on
Darwin."
This reverts commit 4f0325873faccfbe171bae4babceb65975ca892e.
For Darwin/arm64 (including Apple Silicon Macs) this will enable exception
handling and stack unwinding in JIT'd code.
Darwin supports two unwind-info formats: DWARF eh-frames and compact-unwind. On
Darwin/x86-64 compilers usually produce both by default, and ORC supported
exceptions and unwinding via eh-frames (same as on Linux), discarding the
redundant compact-unwind info. On Darwin/arm64 compilers typically default to
producing compact-unwind only, with DWARF eh-frames as a fallback for functions
that can't be described in compact-unwind. Since ORC did not previously support
the compact-unwind format and eh-frames were not present ORC was unable to
handle exceptions or unwinding by default in Darwin/arm64 JIT'd code.
This patch enables support for the compact-unwind-info format, and contains
three major moving parts:
(1) The JITLink CompactUnwindManager class is responsible for transforming the
__compact_unwind records produced by the linker into the __unwind_info
tables that libunwind parses during unwinding. To enable this the
CompactUnwindManager class provides three JITLink passes: The
prepareForPrune pass that splits the __compact_unwind section into
single-record blocks, allowing unused records to be dead-stripped; the
processAndReserveUnwindInfo pass that reserves space for the final
__unwind_info section, and the writeUnwindInfo pass that writes the
__unwind_info section.
(2) The OrcTargetProcess UnwindInfoManager class maintains a table of
registered JIT'd __unwind_info and __eh_frame sections, and handles
requests from libunwind for unwind info sections (by registering a callback
with libunwind's __unw_add_find_dynamic_unwind_sections function).
(3) The Orc UnwindInfoRegistrationPlugin, which scans LinkGraphs for
__unwind_info and __eh_frame sections to register with the
UnwindInfoManager.
This commit adds the CompactUnwindManager passes to the default JITLink
pipelines for Darwin/arm64 and Darwin/x86-64, and UnwindInfoManager intances to
the SelfExecutorProcessControl class (when built for apple platforms) and the
llvm-jitlink-executor tool.
The LLJIT class will now create an UnwindInfoRegistrationPlugin when targeting
a process running on Darwin if it detects that an UnwindInfoManager is
available to handle the registrations.
The ORC runtime macho_platform class already supported libunwind callbacks, so
out-of-process execution and unwinding support will work when loading the ORC
runtime.
The llvm-jitlink tool will only support compact-unwind when the orc-runtime is
loaded, as the UnwindInfoRegistrationPlugin requires access to an IR compiler
to load a helper module and llvm-jitlink does not provide an IR compiler.
ErrorAsOutParameter's Error* constructor supports cases where an Error might not
be passed in (because in the calling context it's known that this call won't
fail). Most clients always have an Error present however, and for them an Error&
overload is more convenient.
With the help of @lhames, This pull request introduces the dlupdate
function in the ORC runtime. dlupdate enables incremental execution of
new initializers introduced in the REPL environment. Unlike traditional
dlopen, which manages initializers, code mapping, and library reference
counts, dlupdate focuses exclusively on running new initializers.
We can get a reference to the `ExecutionSession` from the
`ObjectLinkingLayer` argument, so there's no need to pass it in
separately.
This mirrors recent changes to `ElfNixPlatform` and `MachOPlatform` by
@lhames in
3dba4ca155
and
cc20dd285a.
With the help of @lhames, This pull request introduces the `dlupdate`
function in the ORC runtime. `dlupdate` enables incremental execution of
new initializers introduced in the REPL environment. Unlike traditional
`dlopen`, which manages initializers, code mapping, and library
reference counts, `dlupdate` focuses exclusively on running new
initializers.
This reverts commit edd6f0c544785d6f6276a24b94222e0064413cd1.
The newly added test uncovered a pre-existing issue on Arm 32 bit,
so as we did https://github.com/llvm/llvm-project/issues/94994, disable
it while we find the problem.
Constructors with the same priority should keep their relative order
that was specified. This is important for `clang-repl` with many `const`
variables after commit 05137ecfca ("[clang-repl] Emit const variables
only once").
This re-applies 6094b3b7db7, which was reverted in e7efd37c229 (and before that
in 1effa19de24) due to bot failures.
The test failures were fixed by having SelfExecutorProcessControl use an
InPlaceTaskDispatcher by default, rather than a DynamicThreadPoolTaskDispatcher.
This shouldn't be necessary (and indicates a concurrency issue elsewhere), but
InPlaceTaskDispatcher is a less surprising default, and better matches the
existing behavior (compilation on current thread by default), so the change
seems reasonable. I've filed https://github.com/llvm/llvm-project/issues/89870
to investigate the concurrency issue as a follow-up.
Coding my way home: 6.25133S 127.94177W
This re-applies 6094b3b7db7, which was reverted in a28557aadd8 due to broken
bots. As far as I can tell all failures were due to a missing #include <deque>,
which has been adedd in this commit.
Updates ExecutionSession to use the ExecutorProcessControl object's
TaskDispatcher rather than having a separate dispatch function. This gives the
TaskDispatcher a global view of all tasks to be executed, and provides a
single point to wait on for tasks to complete when shutting down the JIT.
The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.
This is a breaking change: clients who use to create a ThreadPool must
now create a DefaultThreadPool instead.
The JITLink AArch32 backend reached feature-parity with RuntimeDyld
on ELF-based systems. This patch changes the default JIT-linker in Orc's
LLJIT for these platforms.
This allows us to run clang-repl with JITLink on ARM and use all the
features we had with RuntimeDyld before. All existing tests for
clang-repl are passing.
For out-of-process support the DynamicLibrarySearchGenerator must go
through EPC, otherwise we lookup symbols from the host and not the
target process.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
LLJIT users may want to customize their native platform setup beyond what the
ExecutorNativePlatform helper permits. In this case LLJIT users can construct
the platform instance manually, and then add an ORCPlatformSupport instance
to forward initialize/deinitialize operations to the ORC runtime.
This re-applies e1a5bb59b91, which was reverted in e5f169f91a8 due to LSan
failures on some bots (see https://github.com/llvm/llvm-project/issues/67586).
The LSan failures were not caused by this patch (just exposed by it), so LSan
was disabled for the failing test in 47625fea5e3. This should be safe to
re-land now.
This change means that debugger support only needs to be linked in if it's
used. The code size of debugger support is expected to increase as we improve
it (e.g. pulling in DWARF parsing), so making it an optional extra is useful
for controlling final binary sizes.
This function will be run prior to platform setup to provide LLJIT clients with
a chance to customize the LLJIT instance (e.g. install plugins) before the JIT
runtime is loaded.
The motivating use-case is debugger support: We want to install the debugger
plugin before the runtime is loaded (during platform setup) so that the runtime
itself can be debugged. A patch to do this will be committed shortly.
JITLink for ppc64 has implemented all relocations implemented in rtdyld for ppc64 and has passed all existed lit tests and unittests in llvm/clang/compiler-rt. I propose making JITLink for ppc64 default in LLJIT.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D159175
For many interesting process-symbols setups we need access to the LLJIT instance
(e.g. to mangle symbols, or inspect the process triple). This patch updates the
ProcessSymbolsJITDylibSetupFunction to take an LLJIT reference and return the
process symbols JITDylib, which the callback must now create.
