Constructors with the same priority should keep their relative order
that was specified. This is important for `clang-repl` with many `const`
variables after commit 05137ecfca ("[clang-repl] Emit const variables
only once").
This re-applies 6094b3b7db7, which was reverted in e7efd37c229 (and before that
in 1effa19de24) due to bot failures.
The test failures were fixed by having SelfExecutorProcessControl use an
InPlaceTaskDispatcher by default, rather than a DynamicThreadPoolTaskDispatcher.
This shouldn't be necessary (and indicates a concurrency issue elsewhere), but
InPlaceTaskDispatcher is a less surprising default, and better matches the
existing behavior (compilation on current thread by default), so the change
seems reasonable. I've filed https://github.com/llvm/llvm-project/issues/89870
to investigate the concurrency issue as a follow-up.
Coding my way home: 6.25133S 127.94177W
This re-applies 6094b3b7db7, which was reverted in a28557aadd8 due to broken
bots. As far as I can tell all failures were due to a missing #include <deque>,
which has been adedd in this commit.
Updates ExecutionSession to use the ExecutorProcessControl object's
TaskDispatcher rather than having a separate dispatch function. This gives the
TaskDispatcher a global view of all tasks to be executed, and provides a
single point to wait on for tasks to complete when shutting down the JIT.
The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.
This is a breaking change: clients who use to create a ThreadPool must
now create a DefaultThreadPool instead.
The JITLink AArch32 backend reached feature-parity with RuntimeDyld
on ELF-based systems. This patch changes the default JIT-linker in Orc's
LLJIT for these platforms.
This allows us to run clang-repl with JITLink on ARM and use all the
features we had with RuntimeDyld before. All existing tests for
clang-repl are passing.
For out-of-process support the DynamicLibrarySearchGenerator must go
through EPC, otherwise we lookup symbols from the host and not the
target process.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
LLJIT users may want to customize their native platform setup beyond what the
ExecutorNativePlatform helper permits. In this case LLJIT users can construct
the platform instance manually, and then add an ORCPlatformSupport instance
to forward initialize/deinitialize operations to the ORC runtime.
This re-applies e1a5bb59b91, which was reverted in e5f169f91a8 due to LSan
failures on some bots (see https://github.com/llvm/llvm-project/issues/67586).
The LSan failures were not caused by this patch (just exposed by it), so LSan
was disabled for the failing test in 47625fea5e3. This should be safe to
re-land now.
This change means that debugger support only needs to be linked in if it's
used. The code size of debugger support is expected to increase as we improve
it (e.g. pulling in DWARF parsing), so making it an optional extra is useful
for controlling final binary sizes.
This function will be run prior to platform setup to provide LLJIT clients with
a chance to customize the LLJIT instance (e.g. install plugins) before the JIT
runtime is loaded.
The motivating use-case is debugger support: We want to install the debugger
plugin before the runtime is loaded (during platform setup) so that the runtime
itself can be debugged. A patch to do this will be committed shortly.
JITLink for ppc64 has implemented all relocations implemented in rtdyld for ppc64 and has passed all existed lit tests and unittests in llvm/clang/compiler-rt. I propose making JITLink for ppc64 default in LLJIT.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D159175
For many interesting process-symbols setups we need access to the LLJIT instance
(e.g. to mangle symbols, or inspect the process triple). This patch updates the
ProcessSymbolsJITDylibSetupFunction to take an LLJIT reference and return the
process symbols JITDylib, which the callback must now create.
This patch migrates uses of StringRef::{starts,ends}with_insensitive
to StringRef::{starts,ends}_with_insensitive so that we can use names
similar to those used in std::string_view. I'm planning to deprecate
StringRef::{starts,ends}with_insensitive once the migration is
complete across the code base.
Differential Revision: https://reviews.llvm.org/D150426
This reapplies 85c649bc02a, which was reverted in 35767e43d62 due to failures
with some example programs. The fix was to add export_executable_symbols to the
example programs.
This patch switches LLJIT's default JIT linker for ELF/x86-64 from RuntimeDyld
to JITLink.
Most clients should not be affected, but if you were explicitly accessing the
old RTDyldObjectLinkingLayer (e.g. to install JITEventListeners) you will need
to either force use of RuntimeDyld (following the example in
llvm/examples/OrcV2Examples/LLJITWithCustomObjectLinkingLayer), or switch to
using JITLink plugins instead.
Clients can now call LLJITBuilder::setEnableDebuggerSupport to enable
registration of debug info via the GDB-JIT registration protocol for JIT'd
code.
Calling LLJITBuilder::setEnableDebuggerSupport(true) will cause LLJITBuilder to
add either a DebugObjectManagerPlugin (if the object format on the triple is
ELF), or a GDBJITDebugInfoRegistrationPlugin (if the object format on the
triple is MachO). At present there is no support for debugging when using COFF.
This debugger registration support will only work when JITLink is used as the
underlying JIT linker.
This patch only addresses registration of JIT'd code by the JIT. To debug JIT'd
code you may also need to enable JIT'd code debugging in your debugger. E.g.
when debugging MachO JIT'd code under LLDB you will currently need to run
(lldb) set set plugin.jit-loader.gdb.enable on
to tell LLDB to listen for JIT'd code registration.
This reapplies 371cb1af61d, which was reverted in 0b2240eda01 due to bot
failures.
The clang-repl test failure is fixed by dropping the process symbols definition
generator that was manually attached to the main JITDylib, since LLJIT now
exposes process symbols by default. (The bug was triggered when JIT'd code used
the process atexit provided by the generator, rather than the JIT atexit which
has been moved into the platform JITDylib).
Any LLJIT clients that see crashes in static destructors should likewise remove
any process symbol generators attached to their main JITDylib.
This commit includes several related ergonomic improvements to LLJIT.
(1) Adds a default JITDylibSearchOrder to be appended to the initial link order
of JITDylibs created via LLJIT::createJITDylib (dropping any duplicate entries).
This was introduced to support automatic reflection of process symbols (see
(2) below), but has been made visible to clients as it's generically useful,
e.g. if clients have some extra set of libraries that they want to be visible
to JIT'd code by default.
The default JITDylibSearchOrder is only appended to the link order of JITDylibs
created via LLJIT::createJITDylib, and will not be apply to JITDylibs created by
directly calling the underlying ExecutionSession -- in that case clients can set
up the link order manually.
(2) Makes process symbols visible to JIT'd code by default via the new "Process"
JITDylib, which is added to the default link order.
LLJIT clients usually want symbols in the executor process to be accessible to
JIT'd code. Until now clients have been left to set this up themselves by adding
a DynamicLibrarySearchGenerator to the Main JITDylib. This patch adds a new
process symbols JITDylib that will be created by default (with an
EPCDynamicLibrarySearchGenerator attached) and added to the default link order,
making process symbols available to JIT'd code.
Clients who do not want process symbols to be visible to JIT'd code by default
can call setLinkProcessSymbolsByDefault(false) on their LLJITBuilder to disable
this:
LLJITBuilder()
...
.setLinkProcessSymbolsByDefault(false)
...
.create();
Clients can also call setProcessSymbolsJITDylibSetup to take over responsibility
for configuring the process symbols JITDylib (the callback that the client
supplies will be called on the bare process symbols JITDylib immediately after
it is created).
If setLinkProcessSymbolsByDefault(false) is called and no JITDylib setup
callback has been set then the process symbols JITDylib will not be created and
LLJIT::getProcessSymbolsJITDylib will return null.
(3) Adds an ExecutorNativePlatform utility that makes it easier to enable
native platform features.
Some object format features (e.g. native static initializers and thread locals)
require runtime support in the executing process. Support for these features in
ORC is implemented cooperatively between the ORC runtime and the LLVM Platform
subclasses (COFFPlatform, ELFNixPlatform, and MachOPlatform).
ExecutorNativePlatfrom simplifies the process of loading the ORC runtime and
creating the appropriate platform class for the executor process.
ExecutorNativePlatform takes a path to the ORC runtime (or a MemoryBuffer
containing the runtime) and other required runtimes for the executor platform
(e.g. MSVC on Windows) and then configures LLJIT with an appropriate platform
class based on the executor's target triple:
LLJITBuilder()
.setPlatformSetUp(ExecutorNativePlatform("/path/to/orc-runtime.a"));
(The ORC runtime is built as part of compiler-rt, and the exact name of the
archive is platform dependent).
The ORC runtime and platform symbols will be added to a new "Platform" JITDylib,
which will be added to the *front* of the default link order (so JIT'd code will
prefer symbol definitions in the platform/runtime to definitions in the executor
process).
ExecutorNativePlatform assumes that the Process JITDylib is available, as
the ORC runtime may depend on symbols provided by the executor process.
Differential Revision: https://reviews.llvm.org/D144276
ExecutorAddr was introduced in b8e5f918166 as an eventual replacement for
JITTargetAddress. ExecutorSymbolDef is introduced in this patch as a
replacement for JITEvaluatedSymbol: ExecutorSymbolDef is an (ExecutorAddr,
JITSymbolFlags) pair, where JITEvaluatedSymbol was a (JITTargetAddress,
JITSymbolFlags) pair.
A number of APIs had already migrated from JITTargetAddress to ExecutorAddr,
but many of ORC's internals were still using the older type. This patch aims
to address that.
Some public APIs are affected as well. If you need to migrate your APIs you can
use the following operations:
* ExecutorAddr::toPtr replaces jitTargetAddressToPointer and
jitTargetAddressToFunction.
* ExecutorAddr::fromPtr replace pointerToJITTargetAddress.
* ExecutorAddr(JITTargetAddress) creates an ExecutorAddr value from a
JITTargetAddress.
* ExecutorAddr::getValue() creates a JITTargetAddress value from an
ExecutorAddr.
JITTargetAddress and JITEvaluatedSymbol will remain in JITSymbol.h for now, but
the aim will be to eventually deprecate and remove these types (probably when
MCJIT and RuntimeDyld are deprecated).
Where the same dylib is loaded more than once we should just return the
JITDylib created by the first call rather than error out. This matches the
behavior of dlopen / LoadLibrary.
LLJIT::loadPlatformDynamicLibrary loads a dynamic library at a given path
(interpreted in the executor process -- the process containing the JIT'd code),
and returns a JITDylib (whose name is the given path) that reflects the symbols
in that library. LLJIT clients wishing to make the given symbols visible to
their JIT'd code can add this JITDylib to the link order of their JITDylib(s)
using JITDylib::addToLinkOrder.
The LLJIT::linkStaticLibraryInto overloads load a static library (or universal
binary) at a given path (interpreted in the controller process -- the process
containing the LLJIT instance) and adds its symbols to the given JITDylib.
The lli tool is updated to use LLJIT::linkStaticLibraryInto to implement the
extra-archive option.
LLJIT::loadPlatformDynamicLibrary is not tested in this patch as we don't have
a good way to produce dylibs in LLVM's regression test suite.
This reverts commit bdf5f9c3228d6ed1d7c6f87b3828a7d573b34c03, which was a work
in progress for https://reviews.llvm.org/D144276 and accidentally committed
early.
Simplifies the process of building an LLJIT instance that supports the native
platform features (initializers, TLV, etc.).
SetUpExecutorNativePlatform can be passed to LLJITBuilder::setPlatformSetUp
method. It takes a reference to the ORC runtime (as a path or an in-memory
archive) and automatically sets the platform for LLJIT's ExecutionSession based
on the executor process's triple.
Differential Revision: https://reviews.llvm.org/D144276
This implementation supports basic relocation types and adds EHFrame,
Got/Plt handling passes.
This patch also enables JIT support for LoongArch64.
With this patch, I successfully run hello.ll and simple_throw.ll
(which is generated from test-suite/SingleSource/Regression/C++/EH/simple_throw.cpp)
using the `lli` command with options `--jit-kind=orc --jit-linker=jitlink`.
Note: `hasJIT` property of LoongArch32 remains false as there is no
validation environment.
New changes: Since LoongArch does not support RuntimeDyld, JITLink is set
by default. Add a null-terminator to eh-frame sections. This should fix
the test failure on LoongArch bot.
(https://lab.llvm.org/staging/#/builders/236/builds/896)
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D141036
Since aedeb8d5570, which switched to EPC-based eh-frame registrationin LLJIT,
the eh-frame registration functions need to be forcibly linked into the target
process.
Failure to link the eh-frame registration functions triggered a test failure in
https://green.lab.llvm.org/green/job/clang-stage1-RA/31497, which was fixed by
forcibly linking the registration functions into that test case in saf2b2214b4
(rdar://101083784), however it has also caused some tests (e.g. the C API unit
tests) that depend on successful construction of an LLJIT instance to be
skipped.
Moving the forcible registration into LLJIT.cpp fixes the general issue.
Now that ExecutionSession objects alway have ExecutorProcessControl (EPC)
objects attached we can use EPCEHFrameRegistrar by default, rather than
InProcessEHFrameRegistrar. This allows LLJIT to work out-of-the-box with remote
EPCs on platforms that use JITLink, without requiring a custom
ObjectLinkingLayerCreator to override the eh-frame registrar.
Removes public bootstrap method that is not really necessary and not consistent with other platform API.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D132780
Fix "JIT session error: Symbols not found: [ DW.ref.__gxx_personality_v0 ] error" which happens when trying to use exceptions on ppc linux. To do this, it expands AutoClaimSymbols option in RTDyldObjectLinkingLayer to also claim weak symbols before they are tried to be resovled. In ppc linux, DW.ref symbols is emitted as weak hidden symbols in the later stage of MC pipeline. This means when using IRLayer (i.e. LLJIT), IRLayer will not claim responsibility for such symbols and RuntimeDyld will skip defining this symbol even though it couldn't resolve corresponding external symbol.
Reviewed By: sgraenitz
Differential Revision: https://reviews.llvm.org/D129175
RuntimeDyld does not support RISC-V, so it makes sense to enable
JITLink by default. This also makes relocations work without support
for a large code model.
Differential Revision: https://reviews.llvm.org/D129092
Define atexit symbol in GenericLLVMIRPlatformSupport so that it doesn't need to be defined by user.
On windows, llvm codegen emits atexit runtime calls to support global deinitializers as there is no lower function like cxa_atexit as in Itanium C++ ABI. ORC JIT user had to define custom atexit symbol manually. This was a hassle as it has to deal with dso_handle and cxa_atexit internals of LLJIT. If client didn't provide atexit definition, the default behaviour is just linking with host atexit function which is destined to fail as it calls dtors when the host program exits. This is after jit instances and buffers are freed, so users would see weird access violation exception from the uknown location. (in console application, the debugger thinks exception happened in scrt_common_main_seh)
This is a hack that has some caveats. (e.g. memory address is not identical) But, it's better than the situation described in the above. Ultimately, we will move on to ORC runtime that is able to solve the memory address issue properly.
Reviewed By: sgraenitz
Differential Revision: https://reviews.llvm.org/D128037
