This reverts the changes from 91a384621e5b762d9c173ffd247cfeadd5f436a2
for Windows targets. The changes in that commit don't work as expected
for Windows targets (those parts of llvm_add_library don't quite behave
the same for Windows), while the previous status quo (producing a
library named "libLLVM-<major>.dll") is the defacto standard way of
doing versioned library names there, contrary to on Unix.
After that commit, the library always ended up named "libLLVM.dll",
executables linking against it would reference "libLLVM.dll", and
"libLLVM-<major>.dll" was provided as a symlink.
Thus revert this bit back to as it were, so that executables actually
link against a versioned libLLVM, and no separate symlink is needed.
The only thing that might be improved compared to the status quo as it
was before these changes, is that the import library is named
"lib/libLLVM-<major>.dll.a", while the common style would be to name it
plainly "lib/libLLVM.dll.a" (even while it produces references to
"libLLVM-<major>.dll", but none of these had that effect for Windows
targets.
(As a side note, the llvm-shlib library can be built for MinGW, but not
currently in MSVC configurations.)
This symlink was added in 91a384621e5b762d9c173ffd247cfeadd5f436a2 to
maintain backwards compatibility, but it needs to point to
libLLVM.so.$MAJOR.$MINOR rather than libLLVM.so. This works better for
distros that ship libLLVM.so and libLLVM.so.$MAJOR.$MINOR in separate
packages and also prevents mistakes like
libLLVM-19.so -> libLLVM.so -> libLLVM.so.18.1
Fixes#82647
We need to do this now that we are bumping the minor release number when
we create the release branch.
This also results in a slight change to the library names for LLVM. The
main library now has a more convential library name:
'libLLVM.so.$major.$minor'. The old library name: libLLVM-$major.so is
now a symlink that points to the new library. However, the symlink is
not present in the build directory. It is only present in the install
directory.
The library name was changed because it helped to keep the CMake changes
more simple.
Fixes#76273
illumos has an older version of the Solaris linker that does not
support the GNU version script compat nor version scripts and does
not support -Bsymbolic-functions. Treat illumos linker separately.
The libclang/CMakeLists part lifted from NetBSD's pkgsrc.
Build tested on Solaris 11.4 and OpenIndiana 2023.10.
/usr/bin/ld --version
ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.3260
ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.1790 (illumos)
As for now, 'extract_symbols.py' can use several tools to extract
symbols from object files and libraries and to guess if the target is
32-bit Windows. The tools are being found via PATH, so in most cases,
they are just system tools. This approach has a number of limitations,
in particular:
* System tools may not be able to handle the target format in case of
cross-platform builds,
* They cannot read symbols from LLVM bitcode files, so the staged LTO
build with plugins is not supported,
* The auto-selected tools may be suboptimal (see D113557),
* Support for multiple tools for a single task increases the complexity
of the script code.
The patch proposes using LLVM's own tools to solve these issues.
Specifically, 'llvm-readobj' detects the target platform, and 'llvm-nm'
reads symbols from all supported formats, including bitcode files. The
tools can be built in Release mode for the host platform or overridden
using CMake settings 'LLVM_READOBJ' and 'LLVM_NM' respectively. The
implementation also supports using precompiled tools via
'LLVM_NATIVE_TOOL_DIR'.
Differential Revision: https://reviews.llvm.org/D149119
libLLVM.so is empty if it is built with Android NDK (`-DLLVM_BUILD_LLVM_DYLIB=ON`). The patch fixes it.
Reviewed By: xbolva00
Differential Revision: https://reviews.llvm.org/D140268
64-bit atomics are used in llvm/ADT/Statistic.h, which means that users
of libLLVM.so might also have to link with libatomic. To avoid having
to special-case the library here, we simply add all `LLVM_SYSTEM_LIBS`
as public link dependencies to libLLVM.
This fixes a build failure on PowerPC 32-bit.
Reviewed By: beanz
Differential Revision: https://reviews.llvm.org/D132799
A simple sed doing these substitutions:
- `${LLVM_BINARY_DIR}/\$\{CMAKE_CFG_INTDIR}/lib(${LLVM_LIBDIR_SUFFIX})?\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/\$\{CMAKE_CFG_INTDIR}/bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`
where `\>` means "word boundary".
The only manual modifications were reverting changes in
- `compiler-rt/cmake/Modules/CompilerRTUtils.cmake`
because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.
There are many more occurrences without `CMAKE_CFG_INTDIR`, but those are left for D132316 as they have proved somewhat tricky to fix.
This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.
Reviewed By: sebastian-ne
Differential Revision: https://reviews.llvm.org/D133828
Adds too many dependencies: many libraries in LLVM_SYSTEM_LIBS are
arguably not required for users of libLLVM.
This reverts commit 44ffc13f2eb6188a86ae88ea1e942e9ac354db9b.
64-bit atomics are used in llvm/ADT/Statistic.h, which means that users
of libLLVM.so might also have to link with libatomic. To avoid having
to special-case the library here, we simply add all `LLVM_SYSTEM_LIBS`
as public link dependencies to libLLVM.
This fixes a build failure on PowerPC 32-bit.
Reviewed By: beanz
Differential Revision: https://reviews.llvm.org/D132799
A simple sed doing these substitutions:
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?lib(${LLVM_LIBDIR_SUFFIX})?\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`
where `\>` means "word boundary".
The only manual modifications were reverting changes in
- `compiler-rt/cmake/Modules/CompilerRTUtils.cmake
- `runtimes/CMakeLists.txt`
because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.
This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.
Reviewed By: sebastian-ne
Differential Revision: https://reviews.llvm.org/D132316
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.
`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.
I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.
Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi
Differential Revision: https://reviews.llvm.org/D130586
While build llvm-project as a sub-project on windows, met a build error:
libllvm-c.exports /llvm/bin\llvm-nm.exe: error: ...builds/rel64ninja/./lib/LLVMDemangle.lib: no such file or directory
The libllvm-c.exports, libllvm-c.args, and lib/*.lib should under LLVM_BINARY_DIR, using CMAKE_BINARY_DIR will cause 'no such file' error while llvm-project built as a sub-project.
https://reviews.llvm.org/D47381 / eb46c95c3e7aeba4d183ca614fe238067eddf97f
changed the triples set up by GetHostTriple.cmake for i686 MSVC
from i686-pc-win32 to i686-pc-windows-msvc without changing
the corresponding condition in llvm-shlib.
Since then, the 32 bit x86 build of LLVM-C.dll has contained no
exported symbols at all.
Differential Revision: https://reviews.llvm.org/D109493
This is an ELF specific option which isn't supported for Windows/MinGW
targets, even if the MinGW linker otherwise uses an ld.bfd like linker
interface.
Differential Revision: https://reviews.llvm.org/D105148
llvm-dev message: https://lists.llvm.org/pipermail/llvm-dev/2021-May/150465.html
In an ELF shared object, a default visibility defined symbol is preemptible by
default. This creates some missed optimization opportunities.
-Bsymbolic-functions is more aggressive than our current -fvisibility-inlines-hidden
(present since 2012) as it applies to all function definitions. It can
* avoid PLT for cross-TU function calls && reduce dynamic symbol lookup
* reduce dynamic symbol lookup for taking function addresses and optimize out GOT/TOC on x86-64/ppc64
In a -DLLVM_TARGETS_TO_BUILD=X86 build, the number of JUMP_SLOT decreases from 12716 to 1628, and the number of GLOB_DAT decreases from 1918 to 1313
The built clang with `-DLLVM_LINK_LLVM_DYLIB=on -DCLANG_LINK_CLANG_DYLIB=on` is significantly faster.
See the Linux kernel build result https://bugs.archlinux.org/task/70697
Note: the performance of -fno-semantic-interposition -Bsymbolic-functions
libLLVM.so and libclang-cpp.so is close to a PIE binary linking against
`libLLVM*.a` and `libclang*.a`. When the host compiler is Clang,
-Bsymbolic-functions is the major contributor. On x86-64 (with GOTPCRELX) and
ppc64 ELFv2, the GOT/TOC relocations can be optimized.
Some implication:
Interposing a subset of functions is no longer supported.
(This is fragile on ELF and unsupported on Mach-O at all. For Mach-O we don't
use `ld -interpose` or `-flat_namespace`)
Compiling a program which takes the address of any LLVM function with
`{gcc,clang} -fno-pic` and expects the address to equal to the address taken
from libLLVM.so or libclang-cpp.so is unsupported. I am fairly confident that
llvm-project shouldn't have different behaviors depending on such pointer
equality (as we've been using -fvisibility-inlines-hidden which applies to
inline functions for a long time), but if we accidentally do, users should be
aware that they should not make assumption on pointer equality in `-fno-pic`
mode.
See more on https://maskray.me/blog/2021-05-09-fno-semantic-interposition
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D102090
llvm-dev message: https://lists.llvm.org/pipermail/llvm-dev/2021-May/150465.html
In an ELF shared object, a default visibility defined symbol is preemptible by default.
This creates some missed optimization opportunities. -fno-semantic-interposition can optimize -fPIC:
* in Clang: avoid GOT/PLT cost for variable access/function calls to external linkage definition in the same TU
* in GCC: enable interprocedural optimizations (including inlining) and avoid PLT
See https://gist.github.com/MaskRay/2d4dfcfc897341163f734afb59f689c6 for more information.
-Bsymbolic-functions is more aggressive than -fvisibility-inlines-hidden (present since 2012) as it applies
to all function definitions. It can
* avoid PLT for cross-TU function calls && reduce dynamic symbol lookup
* reduce dynamic symbol lookup for taking function addresses and optimize out GOT/TOC on x86-64/ppc64
With both options, the libLLVM.so and libclang-cpp.so performance should
be closer to PIE binary linking against `libLLVM*.a` and `libclang*.a`
(In a -DLLVM_TARGETS_TO_BUILD=X86 build, the number of JUMP_SLOT decreases from 12716 to 1628, and the number of GLOB_DAT decreases from 1918 to 1313
The built clang with `-DLLVM_LINK_LLVM_DYLIB=on -DCLANG_LINK_CLANG_DYLIB=on` is significantly faster.
See the Linux kernel build result https://bugs.archlinux.org/task/70697
)
Some implication:
Interposing a subset of functions is no longer supported.
(This is fragile anyway and cannot really be supported. For Mach-O we don't use
`ld -interpose`, so interposition is not supported on Mach-O at all.)
Compiling a program which takes the address of any LLVM function with
`{gcc,clang} -fno-pic` and expects the address to equal to the address taken
from libLLVM.so or libclang-cpp.so is unsupported. I am fairly confident that
llvm-project shouldn't have different behaviors depending on such pointer
equality (as we've been using -fvisibility-inlines-hidden which applies to
inline functions for a long time), but if we accidentally do, users should be
aware that they should not make assumption on pointer equality in `-fno-pic`
mode.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D102090
This makes it possible to build libLLVM.so without first creating a
static library for each component. In the case where only libLLVM.so is
built (i.e. ninja LLVM) this eliminates 150 linker jobs.
Reviewed By: stellaraccident
Differential Revision: https://reviews.llvm.org/D95727
This patch optionally replaces the CRT allocator (i.e., malloc and free) with rpmalloc (mixed public domain licence/MIT licence) or snmalloc (MIT licence) or mimalloc (MIT licence). Please note that the source code for these allocators must be available outside of LLVM's tree.
To enable, use `cmake ... -DLLVM_INTEGRATED_CRT_ALLOC=D:/git/rpmalloc -DLLVM_USE_CRT_RELEASE=MT` where `D:/git/rpmalloc` has already been git clone'd from `https://github.com/mjansson/rpmalloc`. The same applies to snmalloc and mimalloc.
When enabled, the allocator will be embeded (statically linked) into the LLVM tools & libraries. This currently only works with the static CRT (/MT), although using the dynamic CRT (/MD) could potentially work as well in the future.
When enabled, this changes the memory stack from:
new/delete -> MS VC++ CRT malloc/free -> HeapAlloc -> VirtualAlloc
to:
new/delete -> {rpmalloc|snmalloc|mimalloc} -> VirtualAlloc
The goal of this patch is to bypass the application's global heap - which is thread-safe thus inducing locking - and instead take advantage of a modern lock-free, thread cache, allocator. On a 6-core Xeon Skylake we observe a 2.5x decrease in execution time when linking a large scale application with LLD and ThinLTO (12 min 20 sec -> 5 min 34 sec), when all hardware threads are being used (using LLD's flag /opt:lldltojobs=all). On a dual 36-core Xeon Skylake with all hardware threads used, we observe a 24x decrease in execution time (1 h 2 min -> 2 min 38 sec) when linking a large application with LLD and ThinLTO. Clang build times also see a decrease in the range 5-10% depending on the configuration.
Differential Revision: https://reviews.llvm.org/D71786
The new option works like the existing LLVM_TABLEGEN, and
LLVM_CONFIG_PATH options. Instead of building llvm-nm, the build uses
the executable defined by LLVM_NM.
This is useful for cross-compilation scenarios where the host cannot run
the cross-compiled tool, and recursing into another cmake build is not
an option (due to required DEFINE's, for example).
Reviewed By: smeenai
Differential Revision: https://reviews.llvm.org/D83022
This reverts commit 35edd704e0fda09e8e634515c0b451d4a8b6b914.
Revert the revert and extend the patch further to account for the use of
the `PYTHONINTERP_FOUND`.
This is primarily motivated by the desire to move from Python2 to
Python3. `PYTHON_EXECUTABLE` is ambiguous. This explicitly identifies
the python interpreter in use. Since the LLVM build seems to be able to
completed successfully with python3, use that across the build. The old
path aliases `PYTHON_EXECUTABLE` to be treated as Python3.
This reverts commit cd84bfb8142bc7ff3a07a188ffb809f1d86d1fd7. Although
this passed the CI in phabricator, some of the bots are missing python3
packages, revert it temporarily.
This is primarily motivated by the desire to move from Python2 to
Python3. `PYTHON_EXECUTABLE` is ambiguous. This explicitly identifies
the python interpreter in use. Since the LLVM build seems to be able to
completed successfully with python3, use that across the build. The old
path aliases `PYTHON_EXECUTABLE` to be treated as Python3.
Summary:
Most libraries are defined in the lib/ directory but there are also a
few libraries defined in tools/ e.g. libLLVM, libLTO. I'm defining
"Component Libraries" as libraries defined in lib/ that may be included in
libLLVM.so. Explicitly marking the libraries in lib/ as component
libraries allows us to remove some fragile checks that attempt to
differentiate between lib/ libraries and tools/ libraires:
1. In tools/llvm-shlib, because
llvm_map_components_to_libnames(LIB_NAMES "all") returned a list of
all libraries defined in the whole project, there was custom code
needed to filter out libraries defined in tools/, none of which should
be included in libLLVM.so. This code assumed that any library
defined as static was from lib/ and everything else should be
excluded.
With this change, llvm_map_components_to_libnames(LIB_NAMES, "all")
only returns libraries that have been added to the LLVM_COMPONENT_LIBS
global cmake property, so this custom filtering logic can be removed.
Doing this also fixes the build with BUILD_SHARED_LIBS=ON
and LLVM_BUILD_LLVM_DYLIB=ON.
2. There was some code in llvm_add_library that assumed that
libraries defined in lib/ would not have LLVM_LINK_COMPONENTS or
ARG_LINK_COMPONENTS set. This is only true because libraries
defined lib lib/ use LLVMBuild.txt and don't set these values.
This code has been fixed now to check if the library has been
explicitly marked as a component library, which should now make it
easier to remove LLVMBuild at some point in the future.
I have tested this patch on Windows, MacOS and Linux with release builds
and the following combinations of CMake options:
- "" (No options)
- -DLLVM_BUILD_LLVM_DYLIB=ON
- -DLLVM_LINK_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_BUILD_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_LINK_LLVM_DYLIB=ON
Reviewers: beanz, smeenai, compnerd, phosek
Reviewed By: beanz
Subscribers: wuzish, jholewinski, arsenm, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, nhaehnle, mgorny, mehdi_amini, sbc100, jgravelle-google, hiraditya, aheejin, fedor.sergeev, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, steven_wu, rogfer01, MartinMosbeck, brucehoult, the_o, dexonsmith, PkmX, jocewei, jsji, dang, Jim, lenary, s.egerton, pzheng, sameer.abuasal, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D70179
Due to a misstake with r365902 that tried to simplify the install with
toolchain logic LLVM-C.dll was no longer being installed.
Patch By: Jakob Bornecrantz
llvm-svn: 370271
Summary:
This will simplify the macros by allowing us to remove the hard-coded
list of libraries that should be installed when
LLVM_INSTALL_TOOLCHAIN_ONLY is enabled.
Reviewers: beanz, smeenai
Reviewed By: beanz
Subscribers: aheejin, mehdi_amini, mgorny, steven_wu, dexonsmith, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D64580
llvm-svn: 365902
Summary:
The MinGW driver for lld does not support the --version-script option.
For GNU ld, it's a no-op since LLVM.dll exports all symbols.
Reviewers: srhines, mstorsjo
Subscribers: mgorny, fedor.sergeev, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D63743
llvm-svn: 364343
When we're cross-compiling, build and use a native llvm-nm instead of
attempting to use the one from the target's build tree.
A nice follow-up would be to add a cache variable to allow specifying a
path to an external native llvm-nm instead of building one ourselves,
similar to LLVM_TABLEGEN and LLVM_CONFIG_PATH.
Differential Revision: https://reviews.llvm.org/D60025
llvm-svn: 357487
As discovered in D56774 the command line gets to long, so use a response file
to give the script the libs. This change has been tested and is confirmed
working for me.
Commited on behalf of Jakob Bornecrantz.
Differential Revision: https://reviews.llvm.org/D56781
llvm-svn: 356443
They were breaking the Windows build when using MSBuild, see the
discussion on D56781.
r351833: "Use response file when generating LLVM-C.dll"
> Use response file when generating LLVM-C.dll
>
> As discovered in D56774 the command line gets to long, so use a response file to give the script the libs. This change has been tested and is confirmed working for me.
>
> Commited on behalf of Jakob Bornecrantz
>
> Differential Revision: https://reviews.llvm.org/D56781
r352250: "Build LLVM-C.dll by default on windows and enable in release package"
> Build LLVM-C.dll by default on windows and enable in release package
>
> With the fixes to the building of LLVM-C.dll in D56781 this should now
> be safe to land. This will greatly simplify dealing with LLVM for people
> that just want to use the C API on windows. This is a follow up from
> D35077.
>
> Patch by Jakob Bornecrantz!
>
> Differential revision: https://reviews.llvm.org/D56774
llvm-svn: 352492
As discovered in D56774 the command line gets to long, so use a response file to give the script the libs. This change has been tested and is confirmed working for me.
Commited on behalf of Jakob Bornecrantz
Differential Revision: https://reviews.llvm.org/D56781
llvm-svn: 351833
Summary:
Hello!
This commit adds a LLVM-C target that is always built on MSVC. A big fat warning, this is my first cmake code ever so there is a fair bit of I-have-no-idea-what-I'm-doing going on here. Which is also why I placed it outside of llvm-shlib as I was afraid of breaking things of other people. Secondly llvm-shlib builds a LLVM.so which exports all symbols and then does a thin library that points to it, but on Windows we do not build a LLVM.dll so that would have complicated the code more.
The patch includes a python script that calls dumpbin.exe to get all of the symbols from the built libraries. It then grabs all the symbols starting with LLVM and generates the export file from those. The export file is then used to create the library just like the LLVM-C that is built on darwin.
Improvements that I need help with, to follow up this review.
- Get cmake to make sure that dumpbin.exe is on the path and wire the full path to the script.
- Use LLVM-C.dll when building llvm-c-test so we can verify that the symbols are exported.
- Bundle the LLVM-C.dll with the windows installer.
Why do this? I'm building a language frontend which is self-hosting, and on windows because of various tooling issues we have a problem of consuming the LLVM*.lib directly on windows. Me and the users of my projects using LLVM would be greatly helped by having LLVM-C.dll built and shipped by the Windows installer. Not only does LLVM takes forever to build, you have to run a extra python script in order to get the final DLL.
Any comments, thoughts or help is greatly appreciated.
Cheers, Jakob.
Patch by: Wallbraker (Jakob Bornecrantz)
Reviewers: compnerd, beanz, hans, smeenai
Reviewed By: beanz
Subscribers: xbolva00, bhelyer, Memnarch, rnk, fedor.sergeev, chapuni, smeenai, john.brawn, deadalnix, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D35077
llvm-svn: 339151
Summary:
This option is no longer needed since r300496 added symbol
versioning by default
Reviewers: sylvestre.ledru, beanz, mgorny
Reviewed By: mgorny
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D49835
llvm-svn: 338751
Fuchsia uses ELF as a file format and LLD as the linker so we can
use the same implementation as other ELF based platforms.
Differential Revision: https://reviews.llvm.org/D46991
llvm-svn: 332570
Shared-library build on Solaris requires --whole-archive to be specified (option accepted by all available linkers).
At the same time, --version-script can not be handled by Solaris-ld, so it should be skipped.
-M is of no use here, since there is no syntax in Solaris-ld mapfiles that allows to version all global symbols,
not just the named ones (at least this is my impression from digging deep into the docs).
Patch by Fedor Sergeev <fedor.sergeev@oracle.com>
llvm-svn: 308490
