Summary:
Currently we dispatch the writing mode off of a runtime enum passed in
by the constructor. This causes very unfortunate codegen for the GPU
targets where we get worst-case codegen because of the unused function
pointer for `sprintf`. Instead, this patch moves all of this to a
template so it can be masked out. This results in no dynamic stack and
uses 60 VGPRs instead of 117. It also compiles about 5x as fast.
* strcpy doesn't need to depend on memcpy
* qsort_test_helper has been generalized and doesn't need to depend on
qsort.
This is a small cleanup to unblock the work outlined in #130327.
This reverts commit 1e6e845d49a336e9da7ca6c576ec45c0b419b5f6 because it
changed the 1st parameter of adjust() to be unsigned, but libc itself
calls adjust() with a negative argument in align_backward() in
op_generic.h.
Full build precommit bots were failing due to mis-alignment of atomics
in hermetic tests. This PR enforces the alignment for the bump allocator
of hermetic test framework.
Fixes https://github.com/llvm/llvm-project/issues/128185.
A temporary fix based on discussions in #128079
Currently, baremetal targets are failing to build because the scanf
internals require FILE* (either from the system's libc or our libc).
Normally we'd just turn off the broken entrypoint, but since the scanf
internals are built separately that doesn't work. This patch adds extra
conditions to building those internals, which we can hopefully remove
once we have a proper way to build scanf for embedded.
Another followup fix to #121215
The new cmake wouldn't define the readerat all if the target wasn't GPU
or didn't have a definition of FILE. This patch rewrites the cmake to be
more general.
As a followup, I'd like to make `use_system_file` consistent between
/test and /src. Currently in /src it includes the `COMPILE_OPTIONS` and
in /test it does not.
Summary:
The scan operation implemented here only works if there are contiguous
ones in the executation mask that can be used to propagate the result.
There are two solutions to this, one is to enter 'whole-wave-mode' and
forcibly turn them back on, or to do this serially. This implementation
does the latter because it's more portable, but checks to see if the
parallel fast-path is applicable.
Needs to be backported for correct behavior and because it fixes a
failing libc test.
Summary:
These helpers are very useful but currently absent. They allow the user
to get a bitmask representing the matches within the warp. I have made
an executive decision to drop the `predicate` return from `match_all`
because it's easily testable with `match_all() == __activemask()`.
Implements the posix-specified strftime conversions for the default
locale, along with comprehensive unit tests. This reuses a lot of design
from printf, as well as the printf writer.
Roughly based on #111305, but with major rewrites.
Use the trick from gtest to allow `ASSERT_...` and `EXPECT_...`
macros to be used in braceless `if` without producing warnings
about the nested `if`-`else` that results.
Use a combination of polynomial approximation and Newton-Raphson
iterations in 64-bit and 128-bit integers to improve the performance of
sqrtf128. The correct rounding is provided by squaring the result and
comparing it with the argument.
Performance improvement using the newly added perf test:
- My function = the improved implementation from this PR
- Other function = current implementation using
`libc/src/__support/FPUtil/generic/sqrt.h`
```
Performance tests with inputs in denormal range:
-- My function --
Total time : 1260765265 ns
Average runtime : 125.951 ns/op
Ops per second : 7939623 op/s
-- Other function --
Total time : 7160726518 ns
Average runtime : 715.357 ns/op
Ops per second : 1397902 op/s
-- Average runtime ratio --
Mine / Other's : 0.176067
Performance tests with inputs in normal range:
-- My function --
Total time : 373003808 ns
Average runtime : 37.2631 ns/op
Ops per second : 26836189 op/s
-- Other function --
Total time : 7353398916 ns
Average runtime : 734.605 ns/op
Ops per second : 1361275 op/s
-- Average runtime ratio --
Mine / Other's : 0.0507254
```
---------
Co-authored-by: Alexei Sibidanov <sibid@uvic.ca>
Fixes:
llvm-project/libc/test/integration/src/pthread/pthread_rwlock_test.cpp:59:29:
warning: missing field '__preference' initializer
[-Wmissing-field-initializers]
59 | pthread_rwlock_t rwlock = PTHREAD_RWLOCK_INITIALIZER;
| ^
Also, add a test that demonstrates the same issue for
PTHREAD_MUTEX_INITIALIZER, and fix that, too.
PTHREAD_ONCE_INIT does not have this issue and does have test coverage.
When cross compiling the libc-stdbit-tests, the existing tests trigger numerous
instances of -Wimplicit-int-conversion. The truncation of these implicit
promotions is intentional.
This reverts commit bada9220b87e73c0f4a498b82f883e17eda928d1.
Revert "[libc][stdfix] Implement fixed point `countlsfx` functions in llvm-libc (#125356)"
This reverts commit f2a1103b323492160d7d27a1575fbda709b49036.
`man 3 signal`'s declaration has a face _only a mother could love_.
sighandler_t and __sighandler_t are not defined in the C standard, or POSIX.
They are helpful typedefs provided by glibc and the Linux kernel UAPI headers
respectively since working with function pointers' syntax can be painful. But
we should not rely on them; in C++ we have `auto*` and `using` statements.
Remove the proxy header, and only include a typedef for sighandler_t when
targeting Linux, for compatibility with glibc.
Fixes: #125598
This test was problematic, and also unnecessary. It's not really
a test of the libc functionality or ABI. That's already covered
by the LlvmLibcStackChkFail.Death test. The Smash test was in
fact just testing that the compiler produces the call in the
expected situation. That's a compiler test, not a libc test.
It's not really feasible to make a test like this both reliable
and safe. Since it's not something libc needs to test, it's not
worth trying.
Summary:
The CUDA impelementation has long supported the `width` argument on its
shuffle instrucitons, which makes it more difficult to replace those
uses with this helper. This patch just correctly implements that for
AMDGPU and NVPTX so it's equivalent to `__shfl_sync` in CUDA. This will
ease porting.
Fortunately these get optimized out correctly when passing in known
widths.
Previously this test was entirely disabled under asan, but not
hwasan. Instead of disabling the test, make the test compatible
with both asan and hwasan by disabling sanitizers only on the
subroutine that does the stack-smashing.
The existing options for bin→dec float conversion are all based on the
Ryū algorithm, which generates 9 output digits at a time using a table
lookup. For users who can't afford the space cost of the table, the
table-lookup subroutine is replaced with one that computes the needed
table entry on demand, but the algorithm is otherwise unmodified.
The performance problem with computing table entries on demand is that
now you need to calculate a power of 10 for each 9 digits you output.
But if you're calculating a custom power of 10 anyway, it's easier to
just compute one, and multiply the _whole_ mantissa by it.
This patch adds a header file alongside `float_dec_converter.h`, which
replaces the whole Ryū system instead of just the table-lookup routine,
implementing this alternative simpler algorithm. The result is accurate
enough to satisfy (minimally) the accuracy demands of IEEE 754-2019 even
in 128-bit long double. The new float128 test cases demonstrate this by
testing the cases closest to the 39-digit rounding boundary.
In my tests of generating 39 output digits (the maximum number supported
by this algorithm) this code is also both faster and smaller than the
USE_DYADIC_FLOAT version of the existing Ryū code.
When IntegerToString converts a BigInt into decimal, it determines each
digit by computing `n % 10` and then resets n to `n / 10`, until the
number becomes zero. The div and mod operations are done using
`BigInt::divide_unsigned`, which uses the simplest possible bit-by-bit
iteration, which is a slow algorithm in general, but especially so if
the divisor 10 must first be promoted to a BigInt the same size as the
dividend. The effect is to make each division take quadratic time, so
that the overall decimal conversion is cubic – and the division is
quadratic in the number of _bits_, so the constant of proportionality is
also large.
In this patch I've provided custom code to extract decimal digits much
faster, based on knowing that the divisor is always 10, and processing a
word at a time. So each digit extraction is linear-time with a much
smaller constant of proportionality.
Full comments are in the code. The general strategy is to do the
reduction mod 10 first to determine the output digit; then subtract it
off, so that what's left is guaranteed to be an exact multiple of 10;
and finally divide by 10 using modular-arithmetic techniques rather than
reciprocal-approximation-based ordinary integer division.
I didn't find any existing tests of IntegerToString on a BigInt, so I've
added one.
While GCC's -Wdeprecated is on by default and doesn't do much,
Clang's -Wdeprecated enables many more things. More apply in
C++20, so switch a test file that tickled one to using that. In
future, C++20 should probably be made the baseline for compiling
all the libc code.
This PR aims to add the groundwork to test the precision of libc complex
functions against MPC. I took `cargf` as a test to verify that the infra
works fine.
Summary:
This is a blocker on another patch in the OpenMP runtime. The problem is
that NVIDIA truly doesn't handle RPC-based allocations very well. It
cannot reliably update the MMU while a kernel is running and it will
usually deadlock if called from a separate thread due to internal use of
TLS.
This patch just removes the definition of `malloc` and `free` for NVPTX.
The result here is that they will be undefined, which is the cue for the
`nvlink` linker to define them for us. So, as far as `libc` is concerned
it still implements malloc.
Update string_utils' string_length to work with char* or wchar_t*, so that it
may be reusable when implementing wmemchr, wcspbrk, wcsrchr, wcsstr.
Link: #121183
Link: #124027
Co-authored-by: Nick Desaulniers <ndesaulniers@google.com>
---------
Co-authored-by: Tristan Ross <tristan.ross@midstall.com>
