This patch contains a number of small portability improvements for the
test suite, making it easier to run the test suite with other standard
library implementations.
- Guard checks for _LIBCPP_HARDENING_MODE to avoid -Wundef
- Avoid defining _LIBCPP_HARDENING_MODE even when no hardening mode is
specified -- we should use the default mode of the library in that case.
- Add missing includes and qualify a few function calls.
- Avoid opening namespace std to forward declare stdlib containers. The
test suite should represent user code, and user code isn't allowed to do
that.
In D144319, Clang tried to land a change that would cause some functions
that are not supposed to return nullptr to optimize better. As reported
in https://reviews.llvm.org/D144319#4203982, libc++ started seeing
failures in its CI shortly after this change was landed.
As explained in D146379, the reason for these failures is that libc++'s
throwing `operator new` can in fact return nullptr when compiled with
exceptions disabled. However, this contradicts the Standard, which
clearly says that the throwing version of `operator new(size_t)` should
never return nullptr. This is actually a long standing issue. I've
previously seen a case where LTO would optimize incorrectly based on the
assumption that `operator new` doesn't return nullptr, an assumption
that was violated in that case because libc++.dylib was compiled with
-fno-exceptions.
Unfortunately, fixing this is kind of tricky. The Standard has a few
requirements for the allocation functions, some of which are impossible
to satisfy under -fno-exceptions:
1. `operator new(size_t)` must never return nullptr
2. `operator new(size_t, nothrow_t)` must call the throwing version and
return nullptr on failure to allocate
3. We can't throw exceptions when compiled with -fno-exceptions
In the case where exceptions are enabled, things work nicely.
`new(size_t)` throws and `new(size_t, nothrow_t)` uses a try-catch to
return nullptr. However, when compiling the library with
-fno-exceptions, we can't throw an exception from `new(size_t)`, and we
can't catch anything from `new(size_t, nothrow_t)`. The only thing we
can do from `new(size_t)` is actually abort the program, which does not
make it possible for `new(size_t, nothrow_t)` to catch something and
return nullptr.
This patch makes the following changes:
1. When compiled with -fno-exceptions, the throwing version of `operator
new` will now abort on failure instead of returning nullptr on failure.
This resolves the issue that the compiler could mis-compile based on the
assumption that nullptr is never returned. This constitutes an API and
ABI breaking change for folks compiling the library with -fno-exceptions
(which is not the general public, who merely uses libc++ headers but use
a shared library that has already been compiled). This should mostly
impact vendors and other folks who compile libc++.dylib themselves.
2. When the library is compiled with -fexceptions, the nothrow version
of `operator new` has no change. When the library is compiled with
-fno-exceptions, the nothrow version of `operator new` will now check
whether the throwing version of `operator new` has been overridden. If
it has not been overridden, then it will use an implementation
equivalent to that of the throwing `operator new`, except it will return
nullptr on failure to allocate (instead of terminating). However, if the
throwing `operator new` has been overridden, it is now an error NOT to
also override the nothrow `operator new`. Indeed, there is no way for us
to implement a valid nothrow `operator new` without knowing the exact
implementation of the throwing version.
In summary, this change will impact people who fall into the following
intersection of conditions:
- They use the libc++ shared/static library built with `-fno-exceptions`
- They do not override `operator new(..., std::nothrow_t)`
- They override `operator new(...)` (the throwing version)
- They use `operator new(..., std::nothrow_t)`
We believe this represents a small number of people.
Fixes#60129
rdar://103958777
Differential Revision: https://reviews.llvm.org/D150610
...in attempt to fix the FreeBSD CI.
I noticed that suddenly some tests in the latest PRs fail to compile on
FreeBSD (`SIGILL` and `SIGTRAP` not defined). This tries to resolve
the issue.
Co-authored-by: Zingam <zingam@outlook.com>
In the hardening modes that can be used in production (`fast` and
`extensive`), make a failed assertion invoke a trap instruction rather
than calling verbose abort. In the debug mode, still keep calling
verbose abort to provide a better user experience and to allow us to
keep our existing testing infrastructure for verifying assertion
messages. Since the debug mode by definition enables all assertions, we
can be sure that we still check all the assertion messages in the
library when running the test suite in the debug mode.
The main motivation to use trapping in production is to achieve better
code generation and reduce the binary size penalty. This way, the
assertion handler can compile to a single instruction, whereas the
existing mechanism with verbose abort results in generating a function
call that in general cannot be optimized away (made worse by the fact
that it's a variadic function, imposing an additional penalty). See the
[RFC](https://discourse.llvm.org/t/rfc-hardening-in-libc/73925) for more
details. Note that this mechanism can now be completely [overridden at
CMake configuration
time](https://github.com/llvm/llvm-project/pull/77883).
This patch also significantly refactors `check_assertion.h` and expands
its test coverage. The main changes:
- when overriding `verbose_abort`, don't do matching inside the function
-- just print the error message to `stderr`. This removes the need to
set a global matcher and allows to do matching in the parent process
after the child finishes;
- remove unused logic for matching source locations and for using
wildcards;
- make matchers simple functors;
- introduce `DeathTestResult` that keeps data about the test run,
primarily to make it easier to test.
In addition to the refactoring, `check_assertion.h` can now recognize
when a process exits due to a trap.
Previously there were two ways to override the verbose abort function
which gets called when a hardening assertion is triggered:
- compile-time: define the `_LIBCPP_VERBOSE_ABORT` macro;
- link-time: provide a definition of `__libcpp_verbose_abort` function.
This patch adds a new configure-time approach: the vendor can provide
a path to a custom header file which will get copied into the build by
CMake and included by the library. The header must provide a definition
of the
`_LIBCPP_ASSERTION_HANDLER` macro which is what will get called should
a hardening assertion fail. As of this patch, overriding
`_LIBCPP_VERBOSE_ABORT` will still work, but the previous mechanisms
will be effectively removed in a follow-up patch, making the
configure-time mechanism the sole way of overriding the default handler.
Note that `_LIBCPP_ASSERTION_HANDLER` only gets invoked when a hardening
assertion fails. It does not affect other cases where
`_LIBCPP_VERBOSE_ABORT` is currently used (e.g. when an exception is
thrown in the `-fno-exceptions` mode).
The library provides a default version of the custom header file that
will get used if it's not overridden by the vendor. That allows us to
always test the override mechanism and reduces the difference in
configuration between the pristine version of the library and
a platform-specific version.
Rather than allow for a message to be considered a match for the actual
assertion if it is anywhere in the assertion text, make sure that the
expected and the actual assertion are identical.
Addresses #77701
This makes exception handling a lot simpler, since we don't have to convert any exceptions this way. Is also properly handles all the user-thrown exceptions.
Reviewed By: ldionne, #libc
Spies: arichardson, mstorsjo, libcxx-commits
Differential Revision: https://reviews.llvm.org/D154238
We introduced an optimization to std::sort in 4eddbf9f10a6. However,
that optimization led to issues where users that were passing invalid
comparators to std::sort could start seeing OOB reads. This led to
the revert of the std::sort optimization from the LLVM 16 release
(see https://llvm.org/D146421).
This patch introduces _LIBCPP_ASSERTs to the places in the algorithm
where we make an assumption that the comparator will be consistent and
hence avoid a bounds check based on that. If the comparator happens not
to be consistent with itself, these are the places where we would
incorrectly go out of bounds. This allows users that enable libc++
assertions to catch such misuse at the cost of basically a bounds
check. For users that do not enable libc++ assertions (which is 99.9%
of users since assertions are off by default), this is basically a
no-op, and in fact the assertion will turn into a __builtin_assume,
making it explicit to the compiler that it can rely on the fact that
we're not going out of bounds.
I think this patch strikes the right balance. Folks that want absolute
performance will get what they want, since it is a precondition for the
comparator to be consistent, so the bounds checks are technically not
mandatory. Folks who want more safety *already* need to be enabling
libc++ assertions to catch other types of bugs (like operator[] OOB),
so this solution should also work for them.
I do think we have a lot of work towards popularizing the use of libc++
assertions and integrating it better so that users don't have to know
about the obscure _LIBCPP_ENABLE_ASSERTIONS macro to enable them, but
that's a separate concern.
rdar://106897934
Differential Revision: https://reviews.llvm.org/D147089
With the goal of reusing that handler to do other things besides
handling assertions (such as terminating when an exception is thrown
under -fno-exceptions), the name `__libcpp_assertion_handler` doesn't
really make sense anymore.
Furthermore, I didn't want to use the name `__libcpp_abort_handler`,
since that would give the impression that the handler is called
whenever `std::abort()` is called, which is not the case at all.
Differential Revision: https://reviews.llvm.org/D130562
Instead of taking a fixed set of arguments, use variadics so that
we can pass arbitrary arguments to the handler. This is the first
step towards using the handler to handle other non-assertion-related
failures, like std::unreachable and an exception being thrown in
-fno-exceptions mode, which would improve user experience by including
additional information in crashes (right now, we call abort() without
additional information).
Differential Revision: https://reviews.llvm.org/D130507
This patch adds a lightweight assertion handler mechanism that can be
overriden at link-time in a fashion similar to `operator new`.
This is a third take on https://llvm.org/D121123 (which allowed customizing
the assertion handler at compile-time), and https://llvm.org/D119969
(which allowed customizing the assertion handler at runtime only).
This approach is, I think, the best of all three explored approaches.
Indeed, replacing the assertion handler in user code is ergonomic,
yet we retain the ability to provide a custom assertion handler when
deploying to older platforms that don't have a default handler in
the dylib.
As-is, this patch provides a pretty good amount of backwards compatibility
with the previous debug mode:
- Code that used to set _LIBCPP_DEBUG=0 in order to get basic assertions
in their code will still get basic assertions out of the box, but
those assertions will be using the new assertion handler support.
- Code that was previously compiled with references to __libcpp_debug_function
and friends will work out-of-the-box, no changes required. This is
because we provide the same symbols in the dylib as we used to.
- Code that used to set a custom __libcpp_debug_function will stop
compiling, because we don't provide that declaration anymore. Users
will have to migrate to the new way of setting a custom assertion
handler, which is extremely easy. I suspect that pool of users is
very limited, so breaking them at compile-time is probably acceptable.
The main downside of this approach is that code being compiled with
assertions enabled but deploying to an older platform where the assertion
handler didn't exist yet will fail to compile. However users can easily
fix the problem by providing a custom assertion handler and defining
the _LIBCPP_AVAILABILITY_CUSTOM_ASSERTION_HANDLER_PROVIDED macro to
let the library know about the custom handler. In a way, this is
actually a feature because it avoids a load-time error that one would
otherwise get when trying to run the code on the older target.
Differential Revision: https://reviews.llvm.org/D121478
Prior to this patch, there was no distinction between tests that check
basic assertions and tests that check full-fledged iterator debugging
assertions. Both were disabled when support for the debug mode is not
provided in the dylib, which is stronger than it needs to be.
Furthermore, all of the tests using "debug_macros.h" that contain more
than one assertion in them were broken -- any code after the first
assertion would never be executed.
This patch refactors all of our assertion-related tests to:
1. Be enabled whenever they can, i.e. basic assertions tests are run
even when the debug mode is disabled.
2. Use the superior `check_assertion.h` (previously `debug_mode_helper.h`)
instead of `debug_macros.h`, which allows multiple assertions in the
same program.
3. Coalesce some tests into the same file to make them more readable.
4. Use consistent naming for test files -- no more db{1,2,3,...,10} tests.
This is a large but mostly mechanical patch.
Differential Revision: https://reviews.llvm.org/D121462
