Summary:
These tools need to be built before we can do the library creation
stage. If they are generated in the same build then this is not
guaranteed so we should add explicit dependencies.
Summary:
If the user does not have the selected backend enabled, we should still
be able to build the LLVM-IR an ddistribute it. This patch makes logic
to suppress tests if the backend can't build it, as well as removing a
flag for the building that's only present int he NVPTX backend.
In full build mode, the fuzzing tests fail to build. This PR disabled MPFR related tests in full build
```
[2/4] Building CXX object projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o
FAILED: projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o
/usr/bin/clang++ -DLIBC_NAMESPACE=__llvm_libc_19_0_0_git -I/home/schrodingerzy/Documents/llvm/llvm-project/build/projects/libc/fuzzing/stdio -I/home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio -I/home/schrodingerzy/Documents/llvm/llvm-project/libc -isystem /home/schrodingerzy/Documents/llvm/llvm-project/build/projects/libc/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fsanitize=fuzzer -O2 -g -DNDEBUG -std=c++17 -MD -MT projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o -MF projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o.d -o projects/libc/fuzzing/stdio/CMakeFiles/libc.fuzzing.stdio.printf_float_conv_fuzz.dir/printf_float_conv_fuzz.cpp.o -c /home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp:19:
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/utils/MPFRWrapper/mpfr_inc.h:21:
In file included from /usr/include/mpfr.h:53:
In file included from /usr/include/gmp.h:35:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/iosfwd:38:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/bits/requires_hosted.h:31:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/c++config.h:679:
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:44:5: error: function-like macro '__GLIBC_PREREQ' is not defined
44 | #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
| ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:55:5: error: function-like macro '__GLIBC_PREREQ' is not defined
55 | #if __GLIBC_PREREQ(2, 26) \
| ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:66:6: error: function-like macro '__GLIBC_PREREQ' is not defined
66 | # if __GLIBC_PREREQ(2, 27)
| ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu/bits/os_defines.h:78:6: error: function-like macro '__GLIBC_PREREQ' is not defined
78 | # if __GLIBC_PREREQ(2, 34)
| ^
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp:19:
In file included from /home/schrodingerzy/Documents/llvm/llvm-project/libc/utils/MPFRWrapper/mpfr_inc.h:21:
In file included from /usr/include/mpfr.h:53:
In file included from /usr/include/gmp.h:35:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/iosfwd:42:
In file included from /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/bits/postypes.h:40:
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:64:11: error: no member named 'mbstate_t' in the global namespace
64 | using ::mbstate_t;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:143:11: error: no member named 'btowc' in the global namespace
143 | using ::btowc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:144:11: error: no member named 'fgetwc' in the global namespace
144 | using ::fgetwc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:145:11: error: no member named 'fgetws' in the global namespace
145 | using ::fgetws;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:146:11: error: no member named 'fputwc' in the global namespace
146 | using ::fputwc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:147:11: error: no member named 'fputws' in the global namespace
147 | using ::fputws;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:148:11: error: no member named 'fwide' in the global namespace
148 | using ::fwide;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:149:11: error: no member named 'fwprintf' in the global namespace
149 | using ::fwprintf;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:150:11: error: no member named 'fwscanf' in the global namespace
150 | using ::fwscanf;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:151:11: error: no member named 'getwc' in the global namespace
151 | using ::getwc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:152:11: error: no member named 'getwchar' in the global namespace
152 | using ::getwchar;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:153:11: error: no member named 'mbrlen' in the global namespace
153 | using ::mbrlen;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:154:11: error: no member named 'mbrtowc' in the global namespace
154 | using ::mbrtowc;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:155:11: error: no member named 'mbsinit' in the global namespace
155 | using ::mbsinit;
| ~~^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cwchar:156:11: error: no member named 'mbsrtowcs' in the global namespace
156 | using ::mbsrtowcs;
| ~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
```
Summary:
Debug builds don't optimize out certain parts of the code that end up
making the GPU backend crash. This results in regular builds not being
successful just to build the testing objects. Disable them for now in
debug mode.
Summary:
The libc build has a few utilties that need to be built before we can do
everything in the full build. The one requirement currently is the
`libc-hdrgen` binary. If we are doing a full build runtimes mode we
first add `libc` to the projects list and then only use the `projects`
portion to buld the `libc` portion. We also use utilities for the GPU
build, namely the loader utilities. Previously we would build these
tools on-demand inside of the cross-build, which tool some hacky
workarounds for the dependency finding and target triple. This patch
instead just builds them similarly to libc-hdrgen and then passses them
in. We now either pass it manually it it was built, or just look it up
like we do with the other `clang` tools.
Depends on https://github.com/llvm/llvm-project/pull/84664
Summary:
This patch fixes some issues with building the GPU target manually
without the runtimes bootstrapping build. This fixes the install
directory and sets the default namespace to something more sensible if
not set. Also I got rid of the check on `-mcpu=native`. it was a neat
trick, but CMake in its INFINITE wisdom does not allow you to set link
flags on the compiler flag check. So I just went with the old tool
usage.
Summary:
The other test locations only give these messages when we are in verbose
logging mode. The average user does not care about which tests are not
being built, and most platforms will have missing tests.
Summary:
We use this extra directory for offloading languages like CUDA or
OpenMP. We made the '/gpu' directory for the regular headers but not the
others. Fix that for now.
Currently, `libc` fails when building on redhat because the triple
format uses `redhat` instead of `linux` (The same problem as openSUSE).
This PR changes `libc` to accept `redhat` as a valid Linux triple.
---------
Co-authored-by: Joseph Huber <huberjn@outlook.com>
Summary:
Recent changes added an include path in the float128 type that used the
internal `libc` path to find the macro. This doesn't work once it's
installed because we need to search from the root of the install dir.
This patch adds "include/" to the include path so that our inclusion
of installed headers always match the internal use.
Summary:
This warning is intended to indicate if `-march=native` returns
different values in a single compilation sense. As it stands we don't
care and it absolutely spams the test output if you run it on a machine
with more than one GPU like any cluster machine. Disable these warnings
everywhere we compile.
Summary:
The math directory likes to do architecture specific implementations of
these math functions. For the GPU case it was complicated by the fact
that both NVPTX and AMDGPU had to go through the same code paths. Since
reworking the GPU target this is no longer the case and we can simply
use the same scheme. This patch moves all the old code into two separate
directories. This likely results in a net increase in code, but it's
easier to reason with.
Summary:
Recent patches have allowed us to treat these libraries as direct
builds. This makes it easier to simply build them to a single LLVM-IR
file. This matches the way these files are presented by the ROCm and
CUDA toolchains and makes it easier to work with.
Summary:
The GPU target has recently been changed to support standard `libc`
build rules. This means we should be able to build for it both in
`LLVM_ENABLE_PROJECTS` mode, or targeting the runtimes directory
directly as in the LLVM `libc` documentation. Previously this failed
because the version check on the compiler was too strict and the
`--target=` options were not being set on the link jobs unless in CMake
cross compiliation mode. This patch fixes those so the following config
should work now to build the GPU target directly if using NVPTX.
```
cmake ../runtimes -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang \
-DLLVM_ENABLE_RUNTIMES=libc -DLLVM_RUNTIMES_TARGET=nvptx64-nvidia-cuda \
-DLLVM_DEFAULT_TARGET_TRIPLE=nvptx64-nvidia-cuda \
-DLIBC_HDRGEN_EXE=/path/to/hdrgen/libc-hdrgen \
-DLLVM_LIBC_FULL_BUILD=ON -GNinja
```
Summary:
We need some extra tools for the GPU build. Normally we search for these
from the build itself, but in the case of a `LLVM_PROJECTS_BUILD` or
some other kind of external build, this directory will not be populated.
However, the GPU build already requires that the compiler is an
up-to-date clang, which should always have these present next to the
binary. Simply add this as a fallback search path. Generally we want it
to be the second, because it would pick up someone install and then
become stale.
Summary:
This patch silences two warnings that may occur during the building of
GPU tests. These are not informative or helpful and just make the test
output longer.
Summary:
This is a massive patch because it reworks the entire build and
everything that depends on it. This is not split up because various bots
would fail otherwise. I will attempt to describe the necessary changes
here.
This patch completely reworks how the GPU build is built and targeted.
Previously, we used a standard runtimes build and handled both NVPTX and
AMDGPU in a single build via multi-targeting. This added a lot of
divergence in the build system and prevented us from doing various
things like building for the CPU / GPU at the same time, or exporting
the startup libraries or running tests without a full rebuild.
The new appraoch is to handle the GPU builds as strict cross-compiling
runtimes. The first step required
https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC`
target to build for the GPU without touching the other targets. This
means that the GPU uses all the same handling as the other builds in
`libc`.
The new expected way to build the GPU libc is with
`LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`.
The second step was reworking how we generated the embedded GPU library
by moving it into the library install step. Where we previously had one
`libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This
patch includes the necessary clang / OpenMP changes to make that not
break the bots when this lands.
We unfortunately still require that the NVPTX target has an `internal`
target for tests. This is because the NVPTX target needs to do LTO for
the provided version (The offloading toolchain can handle it) but cannot
use it for the native toolchain which is used for making tests.
This approach is vastly superior in every way, allowing us to treat the
GPU as a standard cross-compiling target. We can now install the GPU
utilities to do things like use the offload tests and other fun things.
Some certain utilities need to be built with
`--target=${LLVM_HOST_TRIPLE}` as well. I think this is a fine
workaround as we
will always assume that the GPU `libc` is a cross-build with a
functioning host.
Depends on https://github.com/llvm/llvm-project/pull/81557
Summary:
I neglected the fact that `activemask` is a 6.2 or 6.3 feature, so
building this on older machines is incorrect. Bump this up to 6.3 for
now so it works. In the future we will try to get rid of the N
architecture business.
Usage of uninitialized memory is a top memory safety issue in C++ codebases.
Help mitigate this somewhat by default initialize stack allocations to a
pattern (0xAA repeating).
Clang has received optimizations to sink these into control flow paths that
access such values to minimize the overhead of these added initializations.
If there's a measurable slowdown, we can add
-ftrivial-auto-var-init-max-size=<N> for some value N bytes if we have any
large stack allocations, or add attribute uninitialized to any variable
declarations.
Unsupported until GCC 12.1 / Clang 8.
Increases file size of libc.a from a full build by +8.79Ki (+0.2%).
fixes https://github.com/llvm/llvm-project/issues/78743
- For normal objects, the patch removes `RTTI` and exceptions in `fullbuild`
- For FP tests, the patch adds links to `stdc++` and `gcc_s` if `MPFR` is used.
This reverts commit 6886a52d6dbefff77f33de12ff85d654e2557f81.
Most of the errors observed in postsubmit have been addressed. We can
fix-forward the remaining ones.
Link: https://lab.llvm.org/buildbot/#/changes/117129
Summary:
A previous patch added a dependency on the stack protectors, this was
not built on the GPU targets so every test was disabled. It turns out
that disabled tests still get targets so we need to specifically check
if the it is in the target's set of entrypoints before we can use it.
Another patch, because the build-bot was down, snuck in that prevented
the new math tests from being run. The problem is that the `signal.h`
header requires target specific definitions but was being used
unconditionally. I have made changes that disable building this header
if the file is not defined in the config. This required disbaling the
signal_to_string utility, so that will simply be missing from targets
that don't define it.
This allows individual object files to override the common compile
commands in
their local CMakeLists' add_object_library call.
For example, the common compile commands contain -Wall and -Wextra.
Before
this patch, the per object COMPILE_OPTIONS were prepended to these, so
that
builds of individual object files could not individually disable
specific
diagnostics from those groups explicitly.
After this patch, the per-object file compile objects are appended to
the list
of compiler flags, enabling this use case.
ARGN is a bit of cmake magic; let's be explicit in the APPEND that we're
appending the compile options.
Link: #77007
* separate initialization routines into _start and do_start for all
architectures.
* lift do_start as a separate object library to avoid code duplication.
* (addtionally) address the problem of building hermetic libc with
-fstack-pointer-*
The `crt1.o` is now a merged result of three components:
```
___
|___ x86_64
| |_______ start.cpp.o <- _start (loads process initial stack and aligns stack pointer)
| |_______ tls.cpp.o <- init_tls, cleanup_tls, set_thread_pointer (TLS related routines)
|___ do_start.cpp.o <- do_start (sets up global variables and invokes the main function)
```
Profiling cmake shows that a significant time configuring `libc` folder
is spent on running `get_object_files_for_test` in the `test` folder (13
sec in `libc/test` folder / 16 sec in `libc` folder). By caching all
needed objects for each target instead of resolving every time, the time
cmake spends on configuring `libc/test` folder is reduced to ~1s.
Summary:
The GPU tests tend to fail when run massively in parallel. This is why
we use a CMake job pool to limit it to 1 in most cases. We should
default to the configuration that is most likely to work, that being a
single thread. There aren't enough GPU tests for this to be a massive
increase in test time on the bots, so we should default to what works
guaranteed.
