Summary:
This patch removed the `-ffreestanding` and `-fno-builtin` flags from
the publically installed version of the GPU library. The presense of
`-fno-builtin` caused issues that prevented all inlining in the GPU C
library, see
https://discourse.llvm.org/t/rfc-libc-ffreestanding-fno-builtin/75883.
Previously, I attempted to fix this by loosening the restriction that
`"no-builtins"` functions cannot be inlined into functions without that
attribute. However, that opened up a lot of extra issues that stalled
that approach.
This patch instead removes that and instead passes `-fno-builtin-<xyz>`
for the few calls that are known to be problematic. I believe this works
in general as the GPU backends do not emit any libcalls and the
implementations of most of these simply reduce to builtins right now.
This is a very useful patch as we can now actually inline calls.
PR for adding microbenchmarking infrastructure for NVPTX. `nvlink`
cannot perform LTO, so we cannot inline `libc` functions and this
function call overhead is not adjusted for during microbenchmarking.
Summary:
This patch adds `CMAKE_REQUIRED_FLAGS` for the GPU build so checks like
`check_cxx_compiler_flags` work as expected. This is required because we
need to hack around the potential lack of `nvlink` and `ptxas` for NVPTX
targets and the fact that the AMDGPU target needs `-nogpulib` to avoid
errors on lack of ROCm. This makes a few of the checks pass and also
allows us to just check `-mcpu=native` for architecture detection
instead of finding the tools manually.
Reverts llvm/llvm-project#95419 and Reland #95358.
This PR is full of temporal fixes. After a discussion with @lntue, it is
better to avoid further changes to the cmake infrastructure for now as a
rework to the cmake utilities will be landed in the future.
Summary:
These tools need to be built before we can do the library creation
stage. If they are generated in the same build then this is not
guaranteed so we should add explicit dependencies.
Summary:
If the user does not have the selected backend enabled, we should still
be able to build the LLVM-IR an ddistribute it. This patch makes logic
to suppress tests if the backend can't build it, as well as removing a
flag for the building that's only present int he NVPTX backend.
In full build mode, the fuzzing tests fail to build. This PR disabled MPFR related tests in full build
```
[2/4] Building CXX object projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o
FAILED: projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o
/usr/bin/clang++ -DLIBC_NAMESPACE=__llvm_libc_19_0_0_git -I/home/schrodingerzy/Documents/llvm/llvm-project/build/projects/libc/fuzzing/stdio -I/home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio -I/home/schrodingerzy/Documents/llvm/llvm-project/libc -isystem /home/schrodingerzy/Documents/llvm/llvm-project/build/projects/libc/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fsanitize=fuzzer -O2 -g -DNDEBUG -std=c++17 -MD -MT projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o -MF projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o.d -o projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o -c /home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp:19:
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/utils/MPFRWrapper/mpfr_inc.h:21:
In file included from /usr/include/mpfr.h:53:
In file included from /usr/include/gmp.h:35:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/iosfwd:38:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/bits/requires_hosted.h:31:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/c++config.h:679:
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:44:5: error: function-like macro '__GLIBC_PREREQ' is not defined
44 | #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
| ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:55:5: error: function-like macro '__GLIBC_PREREQ' is not defined
55 | #if __GLIBC_PREREQ(2, 26) \
| ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:66:6: error: function-like macro '__GLIBC_PREREQ' is not defined
66 | # if __GLIBC_PREREQ(2, 27)
| ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:78:6: error: function-like macro '__GLIBC_PREREQ' is not defined
78 | # if __GLIBC_PREREQ(2, 34)
| ^
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp:19:
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/utils/MPFRWrapper/mpfr_inc.h:21:
In file included from /usr/include/mpfr.h:53:
In file included from /usr/include/gmp.h:35:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/iosfwd:42:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/bits/postypes.h:40:
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:64:11: error: no member named 'mbstate_t' in the global namespace
64 | using ::mbstate_t;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:143:11: error: no member named 'btowc' in the global namespace
143 | using ::btowc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:144:11: error: no member named 'fgetwc' in the global namespace
144 | using ::fgetwc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:145:11: error: no member named 'fgetws' in the global namespace
145 | using ::fgetws;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:146:11: error: no member named 'fputwc' in the global namespace
146 | using ::fputwc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:147:11: error: no member named 'fputws' in the global namespace
147 | using ::fputws;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:148:11: error: no member named 'fwide' in the global namespace
148 | using ::fwide;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:149:11: error: no member named 'fwprintf' in the global namespace
149 | using ::fwprintf;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:150:11: error: no member named 'fwscanf' in the global namespace
150 | using ::fwscanf;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:151:11: error: no member named 'getwc' in the global namespace
151 | using ::getwc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:152:11: error: no member named 'getwchar' in the global namespace
152 | using ::getwchar;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:153:11: error: no member named 'mbrlen' in the global namespace
153 | using ::mbrlen;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:154:11: error: no member named 'mbrtowc' in the global namespace
154 | using ::mbrtowc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:155:11: error: no member named 'mbsinit' in the global namespace
155 | using ::mbsinit;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:156:11: error: no member named 'mbsrtowcs' in the global namespace
156 | using ::mbsrtowcs;
| ~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
```
Summary:
Debug builds don't optimize out certain parts of the code that end up
making the GPU backend crash. This results in regular builds not being
successful just to build the testing objects. Disable them for now in
debug mode.
Summary:
The libc build has a few utilties that need to be built before we can do
everything in the full build. The one requirement currently is the
`libc-hdrgen` binary. If we are doing a full build runtimes mode we
first add `libc` to the projects list and then only use the `projects`
portion to buld the `libc` portion. We also use utilities for the GPU
build, namely the loader utilities. Previously we would build these
tools on-demand inside of the cross-build, which tool some hacky
workarounds for the dependency finding and target triple. This patch
instead just builds them similarly to libc-hdrgen and then passses them
in. We now either pass it manually it it was built, or just look it up
like we do with the other `clang` tools.
Depends on https://github.com/llvm/llvm-project/pull/84664
Summary:
This patch fixes some issues with building the GPU target manually
without the runtimes bootstrapping build. This fixes the install
directory and sets the default namespace to something more sensible if
not set. Also I got rid of the check on `-mcpu=native`. it was a neat
trick, but CMake in its INFINITE wisdom does not allow you to set link
flags on the compiler flag check. So I just went with the old tool
usage.
Summary:
The other test locations only give these messages when we are in verbose
logging mode. The average user does not care about which tests are not
being built, and most platforms will have missing tests.
Summary:
We use this extra directory for offloading languages like CUDA or
OpenMP. We made the '/gpu' directory for the regular headers but not the
others. Fix that for now.
Currently, `libc` fails when building on redhat because the triple
format uses `redhat` instead of `linux` (The same problem as openSUSE).
This PR changes `libc` to accept `redhat` as a valid Linux triple.
---------
Co-authored-by: Joseph Huber <huberjn@outlook.com>
Summary:
Recent changes added an include path in the float128 type that used the
internal `libc` path to find the macro. This doesn't work once it's
installed because we need to search from the root of the install dir.
This patch adds "include/" to the include path so that our inclusion
of installed headers always match the internal use.
Summary:
This warning is intended to indicate if `-march=native` returns
different values in a single compilation sense. As it stands we don't
care and it absolutely spams the test output if you run it on a machine
with more than one GPU like any cluster machine. Disable these warnings
everywhere we compile.
Summary:
The math directory likes to do architecture specific implementations of
these math functions. For the GPU case it was complicated by the fact
that both NVPTX and AMDGPU had to go through the same code paths. Since
reworking the GPU target this is no longer the case and we can simply
use the same scheme. This patch moves all the old code into two separate
directories. This likely results in a net increase in code, but it's
easier to reason with.
Summary:
Recent patches have allowed us to treat these libraries as direct
builds. This makes it easier to simply build them to a single LLVM-IR
file. This matches the way these files are presented by the ROCm and
CUDA toolchains and makes it easier to work with.
Summary:
The GPU target has recently been changed to support standard `libc`
build rules. This means we should be able to build for it both in
`LLVM_ENABLE_PROJECTS` mode, or targeting the runtimes directory
directly as in the LLVM `libc` documentation. Previously this failed
because the version check on the compiler was too strict and the
`--target=` options were not being set on the link jobs unless in CMake
cross compiliation mode. This patch fixes those so the following config
should work now to build the GPU target directly if using NVPTX.
```
cmake ../runtimes -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang \
-DLLVM_ENABLE_RUNTIMES=libc -DLLVM_RUNTIMES_TARGET=nvptx64-nvidia-cuda \
-DLLVM_DEFAULT_TARGET_TRIPLE=nvptx64-nvidia-cuda \
-DLIBC_HDRGEN_EXE=/path/to/hdrgen/libc-hdrgen \
-DLLVM_LIBC_FULL_BUILD=ON -GNinja
```
Summary:
We need some extra tools for the GPU build. Normally we search for these
from the build itself, but in the case of a `LLVM_PROJECTS_BUILD` or
some other kind of external build, this directory will not be populated.
However, the GPU build already requires that the compiler is an
up-to-date clang, which should always have these present next to the
binary. Simply add this as a fallback search path. Generally we want it
to be the second, because it would pick up someone install and then
become stale.
Summary:
This patch silences two warnings that may occur during the building of
GPU tests. These are not informative or helpful and just make the test
output longer.
Summary:
This is a massive patch because it reworks the entire build and
everything that depends on it. This is not split up because various bots
would fail otherwise. I will attempt to describe the necessary changes
here.
This patch completely reworks how the GPU build is built and targeted.
Previously, we used a standard runtimes build and handled both NVPTX and
AMDGPU in a single build via multi-targeting. This added a lot of
divergence in the build system and prevented us from doing various
things like building for the CPU / GPU at the same time, or exporting
the startup libraries or running tests without a full rebuild.
The new appraoch is to handle the GPU builds as strict cross-compiling
runtimes. The first step required
https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC`
target to build for the GPU without touching the other targets. This
means that the GPU uses all the same handling as the other builds in
`libc`.
The new expected way to build the GPU libc is with
`LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`.
The second step was reworking how we generated the embedded GPU library
by moving it into the library install step. Where we previously had one
`libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This
patch includes the necessary clang / OpenMP changes to make that not
break the bots when this lands.
We unfortunately still require that the NVPTX target has an `internal`
target for tests. This is because the NVPTX target needs to do LTO for
the provided version (The offloading toolchain can handle it) but cannot
use it for the native toolchain which is used for making tests.
This approach is vastly superior in every way, allowing us to treat the
GPU as a standard cross-compiling target. We can now install the GPU
utilities to do things like use the offload tests and other fun things.
Some certain utilities need to be built with
`--target=${LLVM_HOST_TRIPLE}` as well. I think this is a fine
workaround as we
will always assume that the GPU `libc` is a cross-build with a
functioning host.
Depends on https://github.com/llvm/llvm-project/pull/81557
Summary:
I neglected the fact that `activemask` is a 6.2 or 6.3 feature, so
building this on older machines is incorrect. Bump this up to 6.3 for
now so it works. In the future we will try to get rid of the N
architecture business.
