Since LLVM supports `-ffat-lto-objects` we should enable this as an
option in the LLVM build. FatLTO should improve the time it takes to
build tests for LTO enabled builds of the compiler by not linking w/ the
bitcode portion of the object files, which should speed up build times
for LTO builds without disabling optimizations.
ld: warning: ignoring duplicate libraries:
This triggers quite frequently in llvm's build because CMake's library
depends mechanism doesn't de-duplicate libraries on the link line.
Duplication is necessary for ELF platforms, but means something subtly
different on Darwin platforms, hence the warning. Since we don't have
much control over that from CMake, just disable the warning wholesale
whenever the linker is detected to support it.
Add the ability to set the number of tablegen jobs that can run in
parallel
similar to the LLVM_PARALLEL_[COMPILE|LINK]_JOBS options that already
exist.
Summary:
Turns out that we need this for the header install directory. I didn't
notice because I had old values cached. Revert for now until I think of
a better way to suppress the error.
Summary:
I've begun treating GPU runtimes builds as cross-compiling with the LLVM
infrastructure. However, we include a lot of random stuff that the GPU
build isn't prepared to handle. This currently emits a warning, and
while it's not striclty necessary, is annoying. This patch suppresses it
by not including the standard GNU install directory resources when used
from the GPU.
1. Replace the obsolete `llvm_add_unittests()` CMake function with an
explicit check for `TARGET llvm_gtest`. This is more consistent with the
rest of the code, and it makes it possible to avoid checking out
`third-party` tree.
2. Add `LLDUnitTests` test dependency to standalone builds. It is
defined unconditionally, and actually necessary to ensure that unit
tests will be built.
Currently `add_lit_target` sets the `USES_TERMINAL` CMake option. When
using Ninja, this forces all lit testsuite targets into the
single-threaded `console` pool.
This PR adds a new option `LLVM_PARALLEL_LIT` which drops the
`USES_TERMINAL` flag, allowing Ninja to run them in parallel.
The default setting (`LLVM_PARALLEL_LIT=OFF`) retains the existing
behavior of serial testsuite execution.
This symlink was added in 91a384621e5b762d9c173ffd247cfeadd5f436a2 to
maintain backwards compatibility, but it needs to point to
libLLVM.so.$MAJOR.$MINOR rather than libLLVM.so. This works better for
distros that ship libLLVM.so and libLLVM.so.$MAJOR.$MINOR in separate
packages and also prevents mistakes like
libLLVM-19.so -> libLLVM.so -> libLLVM.so.18.1
Fixes#82647
Summary:
This block of code is used to prevent a GPU-based cross compiling build
from taking incompatible arguments. However this incorrectly used the
LLVM default triple instead of the runtimes target. Fix that so the bots
can continue to default the triple to NVPTX.
LLVM will intercept errors using assert() when
LLVM_ENABLE_EXPENSIVE_CHECKS is ON. So an explicit check is added.
---------
Co-authored-by: Hua Tian <akiratian@tencent.com>
Summary:
This is a massive patch because it reworks the entire build and
everything that depends on it. This is not split up because various bots
would fail otherwise. I will attempt to describe the necessary changes
here.
This patch completely reworks how the GPU build is built and targeted.
Previously, we used a standard runtimes build and handled both NVPTX and
AMDGPU in a single build via multi-targeting. This added a lot of
divergence in the build system and prevented us from doing various
things like building for the CPU / GPU at the same time, or exporting
the startup libraries or running tests without a full rebuild.
The new appraoch is to handle the GPU builds as strict cross-compiling
runtimes. The first step required
https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC`
target to build for the GPU without touching the other targets. This
means that the GPU uses all the same handling as the other builds in
`libc`.
The new expected way to build the GPU libc is with
`LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`.
The second step was reworking how we generated the embedded GPU library
by moving it into the library install step. Where we previously had one
`libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This
patch includes the necessary clang / OpenMP changes to make that not
break the bots when this lands.
We unfortunately still require that the NVPTX target has an `internal`
target for tests. This is because the NVPTX target needs to do LTO for
the provided version (The offloading toolchain can handle it) but cannot
use it for the native toolchain which is used for making tests.
This approach is vastly superior in every way, allowing us to treat the
GPU as a standard cross-compiling target. We can now install the GPU
utilities to do things like use the offload tests and other fun things.
Some certain utilities need to be built with
`--target=${LLVM_HOST_TRIPLE}` as well. I think this is a fine
workaround as we
will always assume that the GPU `libc` is a cross-build with a
functioning host.
Depends on https://github.com/llvm/llvm-project/pull/81557
We need to do this now that we are bumping the minor release number when
we create the release branch.
This also results in a slight change to the library names for LLVM. The
main library now has a more convential library name:
'libLLVM.so.$major.$minor'. The old library name: libLLVM-$major.so is
now a symlink that points to the new library. However, the symlink is
not present in the build directory. It is only present in the install
directory.
The library name was changed because it helped to keep the CMake changes
more simple.
Fixes#76273
A colleague observes that switching the default value of
LLVM_EXPERIMENTAL_DEBUGINFO_ITERATORS to "On" hasn't flipped the value
in their CMakeCache.txt. This probably means that everyone with an
existing build tree is going to not have support built in, meaning
everyone in LLVM would need to clean+rebuild their worktree when we flip
the switch on... which doesn't sound good.
So instead, just delete the flag and everything it does, making everyone
build and run ~400 lit tests in RemoveDIs mode. None of the buildbots
have had trouble with this, so it Should Be Fine (TM).
(Sending for review as this is changing various comments, and touches
several different areas -- I don't want to get too punchy).
Include LLVM_ENABLE_HTTPLIB along with httplib package finding in
LLVMConfig.cmake, as this dependency is needed by LLVMDebuginfod that is
now used by LLDB. Without it, building LLDB standalone fails with:
```
CMake Error at /usr/lib/llvm/19/lib64/cmake/llvm/LLVMExports.cmake:90 (set_target_properties):
The link interface of target "LLVMDebuginfod" contains:
httplib::httplib
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
/usr/lib/llvm/19/lib64/cmake/llvm/LLVMConfig.cmake:357 (include)
cmake/modules/LLDBStandalone.cmake:9 (find_package)
CMakeLists.txt:34 (include)
```
Summary:
This patch is an attempt to make the `find_package(FFI)` support in LLVM
prefer to provide the static library version if present. This is
currently
an optional library for building `libffi`, and its presence implies that
it should likely be used. This patch is an attempt to fix some problems
observed with testing programs linked against `libffi` on many different
systems that could have conflicting paths. Linking it statically
prevents this.
This patch adds the `ffi_static` target for this library.
The support introduced in 675702f356b0c3a540fa2e8af4192f7d658b2988 is
not working correctly in all scenarios. Instead of setup_host_tool
function, we can use the existing targets introduced by add_tablegen
macro.
When crosscompiling tools for a different architecture, we need to build
native libc-hdrgen which can be achieved using the existing CMake
support for crosscompiling tablegen tools.
Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld
-flavor`, would not longer work on Windows, when these tools were linked
as part of `llvm-driver`. This was caused by `InitLLVM` which was part
of the `*_main()` function of these tools, which in turn calls
`windows::GetCommandLineArguments`. That function completly replaces
argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by
`llvm-driver` prior to calling these tools was reset.
`InitLLVM` is now called by the `llvm-driver`. Any tool that
participates in (or is part of) the `llvm-driver` doesn't call
`InitLLVM` anymore.
illumos has an older version of the Solaris linker that does not
support the GNU version script compat nor version scripts and does
not support -Bsymbolic-functions. Treat illumos linker separately.
The libclang/CMakeLists part lifted from NetBSD's pkgsrc.
Build tested on Solaris 11.4 and OpenIndiana 2023.10.
/usr/bin/ld --version
ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.3260
ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.1790 (illumos)
In newer SDKs, ld64 is called ld-classic. Look for it first, as ld in
those SDKs is still missing some functionality some tests depend on.
This enables running the tests from check-llvm-tools-lto with newer SDKs
on ARM64 macOS.
There is a bit of an issue with how `mlir-linalg-ods-yaml-gen` is
classified in the MLIR build. Due to it being a tool, it is excluded
from the install when using `-DLLVM_BUILD_TOOLS=OFF`. However, it is a
necessary component of the build, so it can cause build issues with
users of the installed LLVM, and so I think it should not be excluded.
It is a tablegen-like tool, so my solution is to reclassify it that way
in the build.
When the top-level CMake invocation has `CMAKE_VERBOSE_MAKEFILE=ON`,
indicating the user wants to have verbose builds (i.e. all executed
commands explicitly echoed), some of the subprojects and runtimes (such
as compiler-rt, libcxx, etc) do not build in verbose mode. For example,
with Ninja:
```
[ 99% 6252/6308] cd /build/runtimes/builtins-bins && /usr/local/bin/cmake --build .
[ 0% 6/308] Building C object CMakeFiles/clang_rt.builtins-i386.dir/absvti2.c.o
[ 0% 7/308] Building C object CMakeFiles/clang_rt.builtins-i386.dir/absvdi2.c.o
[ 0% 8/308] Building C object CMakeFiles/clang_rt.builtins-i386.dir/absvsi2.c.o
...
```
This is because `llvm_ExternalProject_Add()` and `add_custom_libcxx()`
use CMake's `ExternalProject_Add()` function to configure such
subproject builds, and do not pass through the `CMAKE_VERBOSE_MAKEFILE`
setting.
Similar to what is done in `clang/CMakeLists.txt`, add
`-DCMAKE_VERBOSE_MAKEFILE=ON` to the `ExternalProject_Add()` invocations
in `llvm_ExternalProject_Add()` and `add_custom_libcxx()`, whenever the
top-level CMake invocation had `CMAKE_VERBOSE_MAKEFILE` turned on.
When building debug version of LLVM with
`LLVM_USE_RELATIVE_PATHS_IN_FILES=On` would cause source paths to be
incorrect, and be prefixed by the build directory. This lead to source
locations like the following: `../build/llvm/...`. Such paths do not
exist, and existing debuggers can't adjust their search location because
of the incorrect prefix. Ultimately, this happened because the relative
path creation goes in the wrong direction, from source-dir to build-dir
instead of from build-dir to source-dir.
This patch swaps the directionality of the relative paths so that they
get a proper prefix from the build directory. Given a build dir at
`/build` and a project directory at `/llvm-project`, we get source
locations like: `../llvm-project/llvm/lib/Transforms/...`, which a
debugger can resolve once pointed to the correct project directory.
This adds dlltool to the list of tools which don't get excluded from
installation when LLVM_INSTALL_TOOLCHAIN_ONLY is set.
The most important effect here is that this tool will now be included in
the official Windows release.
While llvm-lib reuses the dlltool machinary internally and has many of
the same capabilities, it does not expose the functionality controller
by the '-k' flag, which is currently the only way to create import
libraries for i386 with stdcall symbols from a module definition alone.
We avoid changing llvm-lib tool, since it is designed to emulate LIB.EXE
from MSVC toolchain, and as this functionality is not supported there,
we would have had to introduce an LLVM extension flag in order to
support it.
See https://reviews.llvm.org/D36548 for reference on rationale for
dlltool '-k' flag.
This file is a list of files and their required include dirs, used by
the TableGen LSP server
(https://mlir.llvm.org/docs/Tools/MLIRLSP/#tablegen-lsp-language-server--tblgen-lsp-server).
Initialy this only included MLIR TableGen files, so I've expanded that
by moving it into llvm so all projects that use the llvm `tablegen`
function will be added to the file.
You could already do some things in llvm TableGen files without this,
but were limited with files that don't include their dependencies using
`include`, only with command line arguments.
Once those are in the yml file, the language server sees all that and go
to definition etc. all works.
With LTO, gcc's IPA passes might drop the foo() function and then the test
will pass even on platforms where __builtin_thread_pointer is unavailable.
On PPC64, we get this as a result:
```
llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp:361:61: error: ‘__builtin_thread_pointer’ is not supported on this targ
```
Just mark the function in the CMake configure test with the 'used' attribute to
avoid it being optimised out. The test then behaves correctly with -flto.
Tested with e.g. 'powerpc64le-linux-gnu-gcc -O2 -flto a.c'.
Reported-by: matoro
Reviewed-by: maskray
Closes: https://github.com/llvm/llvm-project/pull/70968
Signed-off-by: Sam James <sam@gentoo.org>
When the user specifies `LIBCXX_HARDENING_MODE` on the command line in a
bootstrapping build, it should override the setting implied by
`LLVM_ENABLE_ASSERTIONS`.
This PR adds options to let CMake calculate the ninja job pools
depending on free memory and available cores.
You can provide memory requirements for each compile and link job which
is checked against CMake AVAILABLE_PHYSICAL_MEMORY and
NUMBER_OF_LOGICAL_CORES. [This information are available since CMake
3.0](https://cmake.org/cmake/help/v3.0/command/cmake_host_system_information.html).
This is very helpful in CI environments with multiple jobs per
environment or a VM with multiple users.
Its different to LLVM_PARALLEL_LINK_JOBS / LLVM_PARALLEL_COMPILE_JOBS
(or ninja -j 1) because it tries to use the resources more efficient
without being terminated. Only downside currently is that compile and
link jobs can run at the same time so there is an offset for link job
memory suggested which is added to the documentation.
The definitions aren't added as cache because if I understand it
correctly this would break it because values could be outdated.
If a CMake project doesn't enable the C language, then the CMake FFI and
Terminfo find modules will fail their checks for compilation and
linking.
This commit allows projects to enable only C++ by first checking if a C
compiler is set before testing C source compilation; if not, it checks
whether C++ compilation succeeds.
Fixes#53950
`CMAKE_{C/CXX}_FLAGS` affects all targets in LLVM. This can
be undesirable in situations, like the case of enabling thinLTO,
where `-flto` is added to every source file. In reality, we only
care about optimizing a select few of binaries, such as clang or lld,
that dominate the compilation pipeline. Auxiliary binaries in a
distribution and not on the critical path can be kept non-optimized.
This PR adds support of per-target linker flags, which can solve the
thinLTO problem by negating the effects of LTO via targeted linker
flags on the targets. The example of negating thinLTO
above can be done by doing the following:
```
set(LLVM_llvm-dwarfdump_LINKER_FLAGS "-Wl,--lto-O0" CACHE STRING "Custom linker flags to llvm-dwarfdump")
set(LLVM_lldb_LINKER_FLAGS "-Wl,--lto-O0" CACHE STRING "Custom linker flags to lldb")
```
There's other applications where this could be used (e.g. avoid
optimizing host tools for build speed improvement etc.).
I've generalized this so that users can apply their desired flags to
targets that are generated by `llvm_add_library` or
`add_llvm_executable`.
Internally, our toolchain builds were on average 1.4x faster when
selectively choosing the binaries that we want optimized.
...behind an experimental CMAKE option that's off by default.
This patch adds a new ilist-iterator-like class that can carry two extra bits
as well as the usual node pointer. This is part of the project to remove
debug-intrinsics from LLVM: see the rationale here [0], they're needed to
signal whether a "position" in a BasicBlock includes any debug-info before or
after the iterator.
This entirely duplicates ilist_iterator, attempting re-use showed it to be a
false economy. It's enable-able through the existing ilist_node options
interface, hence a few sites where the instruction-list type needs to be
updated. The actual main feature, the extra bits in the class, aren't part of
the class unless the cmake flag is given: this is because there's a
compile-time cost associated with it, and I'd like to get everything in-tree
but off-by-default so that we can do proper comparisons.
Nothing actually makes use of this yet, but will do soon, see the Phab patch
stack.
[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939
Differential Revision: https://reviews.llvm.org/D153777
Currently, If the C or C++ compiler supports the `-gsplit-dwarf` option it is added to _all_ compilers.
If a project decides to use another language, such as Swift, this option will be sent to that compiler as well, regardless whether that compiler supports it or not (Swift doesnot).
This patch uses [generator expressions](https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html) to limit the `-gsplit-dwarf` option to only those compilers that support it (C and C++).
It's no longer possible to submit bitcode apps to the Apple App Store.
The tools
used to create xar archived bitcode sections inside MachO files have
been
discontinued. Additionally, the xar APIs have been deprecated since
macOS 12,
so this change removes unnecessary code from objdump and all
dependencies on
libxar.
This fixes rdar://116600767
This is one of the most common issues new users face, especially as so
many reccomended CMake configurations include `-DLLVM_ENABLE_LLD=ON`.
I don't want the error message to get too long but let's at least say
that there are 2 main reasons for the failure. If it's not those then
maybe folks will find the actual problem while trying to discount them.
The new message looks like:
```
CMake Error at cmake/modules/HandleLLVMOptions.cmake:330 (message):
Host compiler does not support '-fuse-ld=not_a_linker'. Please make sure
that 'not_a_linker' is installed and that your host compiler can compile a
simple program when given the option '-fuse-ld=not_a_linker'.
Call Stack (most recent call first):
CMakeLists.txt:910 (include)
```
