Summary:
This patch moves the RPC server handling to be a header only utility
stored in the `shared/` directory. This is intended to be shared within
LLVM for the loaders and `offload/` handling.
Generally, this makes it easier to share code without weird
cross-project binaries being plucked out of the build system. It also
allows us to soon move the loader interface out of the `libc` project so
that we don't need to bootstrap those and can build them in LLVM.
Extract Flang's runtime library to use the LLVM_ENABLE_RUNTIME
mechanism. It will only become active when
`LLVM_ENABLE_RUNTIMES=flang-rt` is used, which also changes the
`FLANG_INCLUDE_RUNTIME` to `OFF` so the old runtime build rules do not
conflict. This also means that unless `LLVM_ENABLE_RUNTIMES=flang-rt` is
passed, nothing changes with the current build process.
Motivation:
* Consistency with LLVM's other runtime libraries (compiler-rt, libc,
libcxx, openmp offload, ...)
* Allows compiling the runtime for multiple targets at once using the
LLVM_RUNTIME_TARGETS configuration options
* Installs the runtime into the compiler's per-target resource directory
so it can be automatically found even when cross-compiling
Also see RFC discussion at
https://discourse.llvm.org/t/rfc-use-llvm-enable-runtimes-for-flangs-runtime/80826
Summary:
When building a project in a runtime mode, the compilation database is a
separate CMake invocation. So its `compile_commands.json` file will be
placed elsewhere in the `runtimes/runtime-bins` directory. This is
somewhat annoying for ongoing development when a runtimes build is
necessary. This patch adds some CMake magic to merge the two files.
Fixed issue w/ standalone runtimes build by checking if the LLVM src and
CMake src are the same.
Summary:
When building a project in a runtime mode, the compilation database is a
separate CMake invocation. So its `compile_commands.json` file will be
placed elsewhere in the `runtimes/runtime-bins` directory. This is
somewhat annoying for ongoing development when a runtimes build is
necessary. This patch adds some CMake magic to merge the two files.
Summary:
We should now use the official™ way to include the files from
`libc/shared`. This required some code to make sure that it's not
included twice if multiple people use it as well as a sanity check on
the directory.
This relands #86209 which was reverted because ./bin/llvm no longer
accepted test paths in the source tree instead of the build tree. This was
happening because `add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit`
was called before all tsst suites were registered, and therefore it was
missing the source->build dir mappings.
Original commit message:
I am currently trying to test the LLVM runtimes (including compiler-rt)
against an installed LLVM tree rather than a build tree (since that is
no longer available). Currently, the runtimes build of compiler-rt assumes
that LLVM_BINARY_DIR is writable since it uses configure_file() to write
there during the CMake configure stage. Instead, generate this file inside
CMAKE_CURRENT_BINARY_DIR, which will match LLVM_BINARY_DIR when invoked
from llvm/runtimes/CMakeLists.txt.
I also needed to make a minor change to the hwasan tests: hwasan_symbolize
was previously found in the LLVM_BINARY_DIR, but since it is generated as
part of the compiler-rt build it is now inside the CMake build directory
instead. I fixed this by passing the output directory to lit as
config.compiler_rt_bindir and using llvm_config.add_tool_substitutions().
For testing that we no longer write to the LLVM install directory as
part of testing or configuration, I created a read-only bind mount and
configured the runtimes builds as follows:
```
$ sudo mount --bind --read-only ~/llvm-install /tmp/upstream-llvm-readonly
$ cmake -DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_COMPILER=/tmp/upstream-llvm-readonly/bin/clang \
-DCMAKE_CXX_COMPILER=/tmp/upstream-llvm-readonly/bin/clang++ \
-DLLVM_INCLUDE_TESTS=TRUE -DLLVM_ENABLE_ASSERTIONS=TRUE \
-DCOMPILER_RT_INCLUDE_TESTS=TRUE -DCOMPILER_RT_DEBUG=OFF \
-DLLVM_ENABLE_RUNTIMES=compiler-rt \
-DCMAKE_DISABLE_FIND_PACKAGE_LLVM=TRUE \
-DCMAKE_DISABLE_FIND_PACKAGE_Clang=TRUE \
-G Ninja -S ~/upstream-llvm-project/runtimes \
-B ~/upstream-llvm-project/runtimes/cmake-build-debug-llvm-git
```
Pull Request: https://github.com/llvm/llvm-project/pull/114307
The variables are all `constexpr`, which implies `inline`. Since they
aren't `constexpr` in C++03 they're also not `inline` there. Because of
that we define them out-of-line currently. Instead we can use the C++17
extension of `inline` variables, which results in the same weak
definitions of the variables but without having all the boilerplate.
The runtimes used to support a build mode called the "Standalone build",
which isn't supported anymore (and hasn't been for a few years).
However, various places in the code still contained stuff whose only
purpose was to support that build mode, and some outdated documentation.
This patch cleans that up (although I probably missed some).
- Remove HandleOutOfTreeLLVM.cmake which isn't referenced anymore
- Remove the LLVM_PATH CMake variable which isn't used anymore
- Update some outdated documentation referencing standalone builds
I am currently trying to test the LLVM runtimes (including compiler-rt)
against an installed LLVM tree rather than a build tree (since that is
no longer available). Currently, the runtimes build of compiler-rt assumes
that LLVM_BINARY_DIR is writable since it uses configure_file() to write
there during the CMake configure stage. Instead, generate this file inside
CMAKE_CURRENT_BINARY_DIR, which will match LLVM_BINARY_DIR when invoked
from llvm/runtimes/CMakeLists.txt.
I also needed to make a minor change to the hwasan tests: hwasan_symbolize
was previously found in the LLVM_BINARY_DIR, but since it is generated as
part of the compiler-rt build it is now inside the CMake build directory
instead. I fixed this by passing the output directory to lit as
config.compiler_rt_bindir and using llvm_config.add_tool_substitutions().
For testing that we no longer write to the LLVM install directory as
part of testing or configuration, I created a read-only bind mount and
configured the runtimes builds as follows:
```
$ sudo mount --bind --read-only ~/llvm-install /tmp/upstream-llvm-readonly
$ cmake -DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_COMPILER=/tmp/upstream-llvm-readonly/bin/clang \
-DCMAKE_CXX_COMPILER=/tmp/upstream-llvm-readonly/bin/clang++ \
-DLLVM_INCLUDE_TESTS=TRUE -DLLVM_ENABLE_ASSERTIONS=TRUE \
-DCOMPILER_RT_INCLUDE_TESTS=TRUE -DCOMPILER_RT_DEBUG=OFF \
-DLLVM_ENABLE_RUNTIMES=compiler-rt \
-DLLVM_BINARY_DIR=/tmp/upstream-llvm-readonly \
-G Ninja -S ~/upstream-llvm-project/runtimes \
-B ~/upstream-llvm-project/runtimes/cmake-build-debug-llvm-git
```
Reviewed By: ldionne
Pull Request: https://github.com/llvm/llvm-project/pull/86209
While these flags semantically are relevant only for C++, we do add them
to CMAKE_REQUIRED_FLAGS if they are detected. All flags in that variable
are used both when testing compilation of C and C++ (and for detecting
libraries, which uses the C compiler driver).
Therefore, to be sure we safely can add the flags to
CMAKE_REQUIRED_FLAGS, test for the option with the C language.
This should fix compilation with GCC; newer versions of GCC do support
the -nostdlib++ option, but it's only supported by the C++ compiler
driver, not the C driver. (However, many builds of GCC also do accept
the option with the C driver, if GCC was compiled with Ada support
enabled, see [1]. That's why this issue isn't noticed in all
configurations with GCC.)
Clang does support these options in both C and C++ driver modes.
This should fix#90332.
[1]
https://github.com/llvm/llvm-project/issues/90332#issuecomment-2325099254
Implements std::from_chars for float and double.
The implementation uses LLVM-libc to do the real parsing. Since this is
the first time libc++
uses LLVM-libc there is a bit of additional infrastructure code. The
patch is based on the
[RFC] Project Hand In Hand (LLVM-libc/libc++ code sharing)
https://discourse.llvm.org/t/rfc-project-hand-in-hand-llvm-libc-libc-code-sharing/77701
When using Clang as a compiler, use Clang to normalize the triple that's
used to construct path for runtime library build and install paths. This
ensures that paths are consistent and avoids the issue where the build
uses a different triple spelling.
Differential Revision: https://reviews.llvm.org/D140925
The GCC build has gotten to the point where it's often hard to find the
actual error in the build log. We should look into enabling these
warnings again in the future, but it looks like a lot of them are
bogous.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
In a nutshell, this moves our libomptarget code to populate the offload
subproject.
With this commit, users need to enable the new LLVM/Offload subproject
as a runtime in their cmake configuration.
No further changes are expected for downstream code.
Tests and other components still depend on OpenMP and have also not been
renamed. The results below are for a build in which OpenMP and Offload
are enabled runtimes. In addition to the pure `git mv`, we needed to
adjust some CMake files. Nothing is intended to change semantics.
```
ninja check-offload
```
Works with the X86 and AMDGPU offload tests
```
ninja check-openmp
```
Still works but doesn't build offload tests anymore.
```
ls install/lib
```
Shows all expected libraries, incl.
- `libomptarget.devicertl.a`
- `libomptarget-nvptx-sm_90.bc`
- `libomptarget.rtl.amdgpu.so` -> `libomptarget.rtl.amdgpu.so.18git`
- `libomptarget.so` -> `libomptarget.so.18git`
Fixes: https://github.com/llvm/llvm-project/issues/75124
---------
Co-authored-by: Saiyedul Islam <Saiyedul.Islam@amd.com>
Summary:
I've begun treating GPU runtimes builds as cross-compiling with the LLVM
infrastructure. However, we include a lot of random stuff that the GPU
build isn't prepared to handle. This currently emits a warning, and
while it's not striclty necessary, is annoying. This patch suppresses it
by not including the standard GNU install directory resources when used
from the GPU.
This appears to have caused a variety of breakages, see comments on the PR.
> Summary:
> There are a few default options that LLVM adds that can be problematic
> for runtimes builds. These options are generally intended to handle
> building LLVM itself, but are also added when building in a runtimes
> mode. One such issue I've run into is that in `libc` we deliberately use
> `--target` to use a different device toolchain, which doesn't support
> some linker arguments passed via `-Wl`. This is observed in
> https://github.com/llvm/llvm-project/pull/73030 when attempting to use
> these options.
>
> This patch completely removes these default arguments.
>
> The consensus is that any issues created by this patch should ultimately
> be solved on a per-runtime basis.
This reverts commit ee922e6ebfb6aab722f6b0f7cfc0c20af636a250.
Summary:
There are a few default options that LLVM adds that can be problematic
for runtimes builds. These options are generally intended to handle
building LLVM itself, but are also added when building in a runtimes
mode. One such issue I've run into is that in `libc` we deliberately use
`--target` to use a different device toolchain, which doesn't support
some linker arguments passed via `-Wl`. This is observed in
https://github.com/llvm/llvm-project/pull/73030 when attempting to use
these options.
This patch completely removes these default arguments.
The consensus is that any issues created by this patch should ultimately
be solved on a per-runtime basis.
This reverts commit 79b03306af5c11d354fa90db8bfd7818cd811ef5.
I am seeing very confusing errors with the `libomptarget` test suite
when using dlopen / other flags. Reverting to investigate why this is.
Summary:
There are a few default options that LLVM adds that can be problematic
for runtimes builds. These options are generally intended to handle
building LLVM itself, but are also added when building in a runtimes
mode. One such issue I've run into is that in `libc` we deliberately use
`--target` to use a different device toolchain, which doesn't support
some linker arguments passed via `-Wl`. This is observed in
https://github.com/llvm/llvm-project/pull/73030 when attempting to use
these options.
This patch completely removes these default arguments.
The consensus is that any issues created by this patch should ultimately
be solved on a per-runtime basis.
I could probably break this commit into more pieces.
---
This patch adds libc++ support for Android L (Android 5.0+) and up,
tested using the Android team's current compiler, a recent version of
the AOSP sysroot, and the x86[-64] Android Emulator.
CMake and Lit Configuration:
Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake
to cross-compile to Android without using CMake's built-in NDK support
(which only works with an actual packaged NDK).
Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++
(and libc++abi) for Android. This file configures libc++ to match what
the NDK distributes, e.g.:
- libc++_shared.so (includes libc++abi objects, there is no
libc++abi.so). libunwind is linked statically but not exported.
- libc++_static.a (does not include libc++abi) and libc++abi.a
- `std::__ndk1` namespace
- All the libraries are built with `__ANDROID_API__=21`, even when they
are linked to something targeting a higher API level.
(However, when the Android LLVM team builds these components, they do
not use these CMake cache files. Instead they use Python scripts to
configure the builds. See
https://android.googlesource.com/toolchain/llvm_android/.)
Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android
NDK's libc++_shared.so. These files can target old or new Android
devices. The Android LLVM team uses these test files to test libc++ for
both arm/arm64 and x86/x86_64 architectures.
The Android testing mode works by setting %{executor} to adb_run.py,
which uses `adb push` and `adb shell` to run tests remotely. adb_run.py
always runs tests as the "shell" user even on an old emulator where "adb
unroot" doesn't work. The script has workarounds for old Android
devices. The script uses a Unix domain socket on the host
(--job-limit-socket) to restrict concurrent adb invocations. Compiling
the tests is a major part of libc++ testing run-time, so it's desirable
to exploit all the host cores without overburdening the test devices,
which can have far fewer cores.
BuildKite CI:
Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang
and an Android sysroot to build libc++, then starts an Android emulator
container to run tests.
Run the emulator and an adb server in a separate Docker container
(libcxx-ci-android-emulator), and create a separate Docker image for
each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the
container's adb server. Running the only adb server inside the container
makes cleanup more reliable between test runs, e.g. the adb client
doesn't create a `~/.android` directory and the adb server can be
restarted along with the emulator using docker stop/run. (N.B. The
emulator insists on connecting to an adb server and will start one
itself if it can't connect to one.)
The suffix to the android-ndk-* job is a label that concisely specifies
an Android SDK emulator image. e.g.:
- "system-images;android-21;default;x86" ==> 21-def-x86
- "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64
Fixes: https://github.com/llvm/llvm-project/issues/69270
Differential Revision: https://reviews.llvm.org/D139147
Since 78d649a417b48cb8a2ba2e755f0e7c8fb8b1bb83 the recommended way to
pass an executor is to use the _TEST_PARAMS variable, which means we now
pass more complicated value (including ones that may contain multiple
`=`) as part of this variable. However, the `REGEX REPLACE` being used
has greedy matches so everything up to the last = becomes part of the
variable name which results in invalid syntax in the generated lit
config file.
This was noticed due to builder failures for those using the
CrossWinToARMLinux.cmake cache file.
---------
Co-authored-by: Vladimir Vereschaka <vvereschaka@accesssoftek.com>
This reverts commit 6403287eff71a3d6f6c862346d6ed3f0f000eb70.
This is failing on all but 1 of Linaro's flang builders.
CMake Error at /home/tcwg-buildbot/worker/clang-aarch64-full-2stage/llvm/flang-rt/unittests/CMakeLists.txt:37 (message):
Target llvm_gtest not found.
See discourse thread https://discourse.llvm.org/t/rfc-support-cmake-option-to-control-link-type-built-for-flang-runtime-libraries/71602/18 for full details.
Flang-rt is the new library target for the flang runtime libraries. It builds the Flang-rt library (which contains the sources of FortranRuntime and FortranDecimal) and the Fortran_main library. See documentation in this patch for detailed description (flang-rt/docs/GettingStarted.md).
This patch aims to:
- integrate Flang's runtime into existing llvm infrasturcture so that Flang's runtime can be built similarly to other runtimes via the runtimes target or via the llvm target as an enabled runtime
- decouple the FortranDecimal library sources that were used by both compiler and runtime so that different build configurations can be applied for compiler vs runtime
- add support for running flang-rt testsuites, which were created by migrating relevant tests from `flang/test` and `flang/unittest` to `flang-rt/test` and `flang-rt/unittest`, using a new `check-flang-rt` target.
- provide documentation on how to build and use the new FlangRT runtime
Reviewed By: DanielCChen
Differential Revision: https://reviews.llvm.org/D154869
There are some issues in `llvm-libgcc` before this patch:
Commit c5a20b518203613497fa864867fc232648006068 ([llvm-libgcc] initial
commit)
uses `$<TARGET_OBJECTS:unwind_static>` to get libunwind objects, which
is empty.
The built library is actually a shared version of libclang_rt.builtins.
When configuring with `llvm/CMakeLists.txt`, target `llvm-libgcc`
requires a
corresponding target in `llvm-libgcc/CMakeLists.txt`.
Per target installation is not handled by `llvm-libgcc`, which is not
consistent
with `libunwind`.
This patch fixes those issues by:
Reusing target `unwind_shared` in `libunwind`, linking
`compiler-rt.builtins`
objects into it, and applying version script.
Adding target `llvm-libgcc`, creating symlinks, and utilizing cmake's
dependency
and component mechanism to ensure link targets will be built and
installed along
with symlinks.
Mimicking `libunwind` to handle per target installation.
It is quite hard to set necessary options without further modifying the
order of
runtime projects in `runtimes/CMakeLists.txt`. So though this patch
reveals the
possibility of co-existence of `llvm-libgcc` and
`compiler-rt`/`libunwind` in
`LLVM_ENABLE_RUNTIMES`, we still inhibit it to minimize influence on
other
projects, considering that `llvm-libgcc` is only intended for toolchain
vendors,
and not for casual use.
This reverts commit d763c6e5e2d0a6b34097aa7dabca31e9aff9b0b6.
Adds the patch by @hans from
https://github.com/llvm/llvm-project/issues/62719
This patch fixes the Windows build.
d763c6e5e2d0a6b34097aa7dabca31e9aff9b0b6 reverted the reviews
D144509 [CMake] Bumps minimum version to 3.20.0.
This partly undoes D137724.
This change has been discussed on discourse
https://discourse.llvm.org/t/rfc-upgrading-llvms-minimum-required-cmake-version/66193
Note this does not remove work-arounds for older CMake versions, that
will be done in followup patches.
D150532 [OpenMP] Compile assembly files as ASM, not C
Since CMake 3.20, CMake explicitly passes "-x c" (or equivalent)
when compiling a file which has been set as having the language
C. This behaviour change only takes place if "cmake_minimum_required"
is set to 3.20 or newer, or if the policy CMP0119 is set to new.
Attempting to compile assembly files with "-x c" fails, however
this is workarounded in many cases, as OpenMP overrides this with
"-x assembler-with-cpp", however this is only added for non-Windows
targets.
Thus, after increasing cmake_minimum_required to 3.20, this breaks
compiling the GNU assembly for Windows targets; the GNU assembly is
used for ARM and AArch64 Windows targets when building with Clang.
This patch unbreaks that.
D150688 [cmake] Set CMP0091 to fix Windows builds after the cmake_minimum_required bump
The build uses other mechanism to select the runtime.
Fixes#62719
Reviewed By: #libc, Mordante
Differential Revision: https://reviews.llvm.org/D151344
This reverts commit 65429b9af6a2c99d340ab2dcddd41dab201f399c.
Broke several projects, see https://reviews.llvm.org/D144509#4347562 onwards.
Also reverts follow-up commit "[OpenMP] Compile assembly files as ASM, not C"
This reverts commit 4072c8aee4c89c4457f4f30d01dc9bb4dfa52559.
Also reverts fix attempt "[cmake] Set CMP0091 to fix Windows builds after the cmake_minimum_required bump"
This reverts commit 7d47dac5f828efd1d378ba44a97559114f00fb64.
These will be replaced by CMake's check_linker_flag once we update
the minimum CMake version 3.20.
Differential Revision: https://reviews.llvm.org/D145716
`-Wall` on clang-cl is equivalent to `-Weverything` on clang. We already add the correct warning flag depending whether we are in an MSVC-like environment, so just remove it from the list of flags that get passed unconditionally.
Spies: libcxx-commits
Differential Revision: https://reviews.llvm.org/D146378
