We weren't applying the libdir subdir to header directories but this is
necessary for correctness when building e.g. ASan variant. This change
also updates path construction logic accross all runtimes and ensures
they're consistent.
Instead of using FOO_TEST_DEPS global variables that don't get updated
properly from subdirectories, use targets to propagate the dependencies
across directories.
In essence, this header has always been related to configuration of
the library but we didn't want to put it inside <__config> due to
complexity reasons. Now that we have sub-headers in <__config>, we
can move <__availability> to it and stop including it everywhere since
we already obtain the required macros via <__config>.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
This patch removes the two-level backend dispatching mechanism we had in
the PSTL. Instead of selecting both a PSTL backend and a PSTL CPU
backend, we now only select a top-level PSTL backend. This greatly
simplifies the PSTL configuration layer.
While this patch technically removes some flexibility from the PSTL
configuration mechanism because CPU backends are not considered
separately, it opens the door to a much more powerful configuration
mechanism based on chained backends in a follow-up patch.
This is a step towards overhauling the PSTL dispatching mechanism.
This reverts commit 4109b18ee5de1346c2b89a5c89b86bae5c8631d3.
It looks like the automatic detection has false positives. This broke
the following build https://github.com/llvm/llvm-project/pull/85262
The libc++ CMake build aims to provide a general-purpose configuration
that matches the canonical LLVM setup (libc++ / libc++abi / libunwind),
not the configuration used for compatibility on any given system. For
these "compatibility" configurations, we use CMake caches like
Apple.cmake and FreeBSD.cmake.
Defaulting to libcxxrt on FreeBSD makes it look as though we're trying
to build the system compatible configuration on FreeBSD, which isn't
really correct and causes confusion more than anything else.
Fixes#84476
Installs the source files of the experimental libc++ modules. These
source files (.cppm) are used by the Clang to build the std and
std.compat modules.
The design of this patch is based on a discussing in SG-15 on
12.12.2023. (SG-15 is the ISO C++ Tooling study group):
- The modules are installed at a location, that is not known to build
systems and compilers.
- Next to the library there will be a module manifest json file.
This json file contains the information to build the module from the
libraries sources. This information includes the location where the
sources are installed. @ruoso supplied the specification of this json
file.
- If possible, the compiler has an option to give the location of the
module manifest file
(https://github.com/llvm/llvm-project/pull/76451).
Currently there is no build system support, but it expected to be added
in the future.
Fixes: https://github.com/llvm/llvm-project/issues/73089
Revert "Revert #76246 and #76083"
This reverts commit 5c150e7eeba9db13cc65b329b3c3537b613ae61d.
Adds a small fix that should properly disable the tests on Windows.
Unfortunately the original poster has not provided feedback and the
original patch did not fail in the LLVM CI infrastructure.
Modules are known to fail on Windows due to non compliance of the
C library. Currently not having this patch prevents testing on other
platforms.
These cause test build failures on Windows.
This reverts the following commits:
57ca74843586c9a93c425036c5538aae0a2cfa60
d06ae33ec32122bb526fb35025c1f0cf979f1090
Previously there were two ways to override the verbose abort function
which gets called when a hardening assertion is triggered:
- compile-time: define the `_LIBCPP_VERBOSE_ABORT` macro;
- link-time: provide a definition of `__libcpp_verbose_abort` function.
This patch adds a new configure-time approach: the vendor can provide
a path to a custom header file which will get copied into the build by
CMake and included by the library. The header must provide a definition
of the
`_LIBCPP_ASSERTION_HANDLER` macro which is what will get called should
a hardening assertion fail. As of this patch, overriding
`_LIBCPP_VERBOSE_ABORT` will still work, but the previous mechanisms
will be effectively removed in a follow-up patch, making the
configure-time mechanism the sole way of overriding the default handler.
Note that `_LIBCPP_ASSERTION_HANDLER` only gets invoked when a hardening
assertion fails. It does not affect other cases where
`_LIBCPP_VERBOSE_ABORT` is currently used (e.g. when an exception is
thrown in the `-fno-exceptions` mode).
The library provides a default version of the custom header file that
will get used if it's not overridden by the vendor. That allows us to
always test the override mechanism and reduces the difference in
configuration between the pristine version of the library and
a platform-specific version.
This removes the entire modules testing infrastructure.
The current infrastructure uses CMake to generate the std and std.compat
module. This requires quite a bit of plumbing and uses CMake. Since
CMake introduced module support in CMake 3.26, modules have a higher
CMake requirement than the rest of the LLVM project. (The LLVM project
requires 3.20.) The main motivation for this approach was how libc++
generated its modules. Every header had its own module partition. This
was changed to improve performance and now only two modules remain. The
code to build these can be manually crafted.
A followup patch will reenable testing modules, using a different
approach.
I recently came across LIBCXXABI_USE_LLVM_UNWINDER and was surprised to
notice it was disabled by default. Since we build libunwind by default
and ship it in the LLVM toolchain, it would seem to make sense that
libc++ and libc++abi rely on libunwind for unwinding instead of using
the system-provided unwinding library (if any).
Most importantly, using the system unwinder implies that libc++abi is
ABI compatible with that system unwinder, which is not necessarily the
case. Hence, it makes a lot more sense to instead default to using the
known-to-be-compatible LLVM unwinder, and let vendors manually select a
different unwinder if desired.
As a follow-up change, we should probably apply the same default to
compiler-rt.
Differential Revision: https://reviews.llvm.org/D150897Fixes#77662
rdar://120801778
When we use the -nostdlib++ flag, we don't need to explicitly link
against compiler-rt, since the compiler already links against it by
default. This simplifies the flags that we need to use when building
with Clang and GCC, and opens the door to further simplifications since
most platforms won't need to detect whether libgcc and libgcc_s are
supported anymore.
Furthermore, on platforms where -nostdlib++ is used, this patch prevents
manually linking compiler-rt *before* other system libraries. For
example, Apple platforms have several compiler-rt symbols defined in
libSystem.dylib. If we manually link against compiler-rt, we end up
overriding the default link order preferred by the compiler and
potentially using the symbols from the clang-provided libclang_rt.a
library instead of the system provided one.
Note that we don't touch how libunwind links against compiler-rt when it
builds the .so/.a because libunwind currently doesn't use -nodefaultlibs
and we want to avoid rocking the boat too much.
rdar://119506163
This commit introduces basic annotations for `std::basic_string`,
mirroring the approach used in `std::vector` and `std::deque`.
Initially, only long strings with the default allocator will be
annotated. Short strings (_SSO - short string optimization_) and strings
with non-default allocators will be annotated in the near future, with
separate commits dedicated to enabling them. The process will be similar
to the workflow employed for enabling annotations in `std::deque`.
**Please note**: these annotations function effectively only when libc++
and libc++abi dylibs are instrumented (with ASan). This aligns with the
prevailing behavior of Memory Sanitizer.
To avoid breaking everything, this commit also appends
`_LIBCPP_INSTRUMENTED_WITH_ASAN` to `__config_site` whenever libc++ is
compiled with ASan. If this macro is not defined, string annotations are
not enabled. However, linking a binary that does **not** annotate
strings with a dynamic library that annotates strings, is not permitted.
Originally proposed here: https://reviews.llvm.org/D132769
Related patches on Phabricator:
- Turning on annotations for short strings:
https://reviews.llvm.org/D147680
- Turning on annotations for all allocators:
https://reviews.llvm.org/D146214
This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.
The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.
This Pull Request introduces basic annotations for `std::basic_string`.
Long strings exhibit structural similarities to `std::vector` and will
be annotated accordingly. Short strings are already implemented, but
will be turned on separately in a forthcoming commit. Look at [a
comment](https://github.com/llvm/llvm-project/pull/72677#issuecomment-1850554465)
below to read about SSO issues at current moment.
Due to the functionality introduced in
[D132522](dd1b7b797a),
the `__sanitizer_annotate_contiguous_container` function now offers
compatibility with all allocators. However, enabling this support will
be done in a subsequent commit. For the time being, only strings with
the default allocator will be annotated.
If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com
There are a few drive-by fixes:
- Since the combination RTTI disabled and exceptions enabled do not
work, this combination is prohibited.
- A small NFC in any fixing clang-tidy.
The code in the Buildkite configuration is prepared for using the std
module. There are more fixes needed for that configuration which will be
done in a separate commit.
Patch 4b1fe097f9a3882f437bc3b829ef02331e28a8d6 introduced a bug when
building libc++ for Fuchsia, it disabled the libc++.so linker script by
default. This patch restores its original behavior.
We were detecting which sanitizer flags to use when building
libc++.dylib but we were never actually adding those flags to the
targets, which means that our sanitized builds would basically build the
dylib without any sanitizers enabled.
We really shouldn't be depending on far away configuration options like
LLVM_HAVE_LINK_VERSION_SCRIPT here. This patch simplifies the enablement
of the linker scripts and as a result gets rid of an undesirable
dependency on HandleLLVMOptions.cmake.
As a drive-by, the patch also stops taking into account whether Python3
is available. This should have no bearing on whether we generate a
linker script or not, which is required for correctness. If someone
tries to build libc++ and generate a linker script but Python3 is not
available, they should get an error instead of silently getting an
incorrect installation of the library.
1. Instead of using individual "boolean" macros, have an "enum" macro
`_LIBCPP_HARDENING_MODE`. This avoids issues with macros being
mutually exclusive and makes overriding the hardening mode within a TU
more straightforward.
2. Rename the safe mode to debug-lite.
This brings the code in line with the RFC:
https://discourse.llvm.org/t/rfc-hardening-in-libc/73925Fixes#65101
This is necessary in order to implement some papers like P2467R1, which
require using C++23 declarations in the dylib. It is a good habit to
keep building the dylib with a recent standard version regardless.
With this patch, we also stop strictly enforcing that the targets are
built with C++23. Concretely, C++23 will soon be required in order to
build the dylib, but not enforcing it strictly works around some issues
like the documentation bots using an old and unsupported compiler. Since
these bots do not actually build the library, not strictly enforcing the
C++ Standard makes our CMake build more resilient to these kinds of
situation. This is just a workaround though, the better way of going
about would be to update the compiler on the documentation bot but we
don't seem to have control over that.
We support GCC 13, which supports the flag. This allows simplifying the
CMake logic around the use of -nostdlib++. Note that there are other
places where we don't assume -nostdlib++ yet in the build, but this
patch is intentionally trying to be small because this part of our CMake
is pretty tricky.
To allow for a smoother transition, keep the safe mode working as is in
the LLVM 18 release (the first release that aims to make hardening
available), then deprecate it in LLVM 19.
On Apple platforms, we always support the -nostdlib++ flag. Hence, it is
not necessary to manually link against system libraries. In fact, doing
so causes us to link against libSystem explicitly, which messes up with
the order of libraries we should use. Indeed:
Before patch, using the system unwinder (LIBCXXABI_USE_LLVM_UNWINDER = OFF)
===========================================================================
$ otool -L lib/{libc++.1.dylib,libc++abi.1.dylib,libunwind.1.dylib}
lib/libc++.1.dylib:
@rpath/libc++.1.dylib
/usr/lib/libSystem.B.dylib
@rpath/libc++abi.1.dylib
lib/libc++abi.1.dylib:
@rpath/libc++abi.1.dylib
/usr/lib/libSystem.B.dylib
lib/libunwind.1.dylib:
@rpath/libunwind.1.dylib
/usr/lib/libSystem.B.dylib
After patch, using the system unwinder (LIBCXXABI_USE_LLVM_UNWINDER = OFF)
===========================================================================
$ otool -L lib/{libc++.1.dylib,libc++abi.1.dylib,libunwind.1.dylib}
lib/libc++.1.dylib:
@rpath/libc++.1.dylib
@rpath/libc++abi.1.dylib
/usr/lib/libSystem.B.dylib
lib/libc++abi.1.dylib:
@rpath/libc++abi.1.dylib
/usr/lib/libSystem.B.dylib
lib/libunwind.1.dylib:
@rpath/libunwind.1.dylib
/usr/lib/libSystem.B.dylib
Before patch, with the LLVM unwinder (LIBCXXABI_USE_LLVM_UNWINDER = ON)
=======================================================================
$ otool -L lib/{libc++.1.dylib,libc++abi.1.dylib,libunwind.1.dylib}
lib/libc++.1.dylib:
@rpath/libc++.1.dylib
/usr/lib/libSystem.B.dylib
@rpath/libc++abi.1.dylib
@rpath/libunwind.1.dylib
lib/libc++abi.1.dylib:
@rpath/libc++abi.1.dylib
/usr/lib/libSystem.B.dylib
@rpath/libunwind.1.dylib
lib/libunwind.1.dylib:
@rpath/libunwind.1.dylib
/usr/lib/libSystem.B.dylib
After patch, with the LLVM unwinder (LIBCXXABI_USE_LLVM_UNWINDER = ON)
======================================================================
$ otool -L lib/{libc++.1.dylib,libc++abi.1.dylib,libunwind.1.dylib}
lib/libc++.1.dylib:
@rpath/libc++.1.dylib
@rpath/libc++abi.1.dylib
@rpath/libunwind.1.dylib
/usr/lib/libSystem.B.dylib
lib/libc++abi.1.dylib:
@rpath/libc++abi.1.dylib
@rpath/libunwind.1.dylib
/usr/lib/libSystem.B.dylib
lib/libunwind.1.dylib:
@rpath/libunwind.1.dylib
/usr/lib/libSystem.B.dylib
As we can see, libSystem appears before the just-built libraries before
the patch, which causes the libunwind.dylib bundled in libSystem.dylib
to be used instead of the just-built libunwind.dylib.
We didn't notice the issue until recently when I tried to update the
macOS CI builders to macOS 13.5, where it is necessary to use the right
libunwind library (the exact reason still needs to be investigated).
The safe mode is in-between the hardened and the debug modes, extending
the checks contained in the hardened mode with certain checks that are
relatively cheap and prevent common sources of errors but aren't
security-critical. Thus, the safe mode trades off some performance for
a wider set of checks, but unlike the debug mode, it can still be used
in production.
Differential Revision: https://reviews.llvm.org/D158823
This is the first step to implement time zone support in libc++. This
adds the complete tzdb_list class and a minimal tzdb class. The tzdb
class only contains the version, which is used by reload_tzdb.
Next to these classes it contains documentation and build system support
needed for time zone support. The code depends on the IANA Time Zone
Database, which should be available on the platform used or provided by
the libc++ vendors.
The code is labeled as experimental since there will be ABI breaks
during development; the tzdb class needs to have the standard headers.
Implements parts of:
- P0355 Extending <chrono> to Calendars and Time Zones
Addresses:
- LWG3319 Properly reference specification of IANA time zone database
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D154282
The building of the std module has been moved from `params.py` and
`dsl.py` to a `lit.local.cfg` for the entire test suite. In theory this
change allows testing modules in most configurations, except:
- combined with clang modules
- C++ versions that don't support the std module
Currently only C++23 with all parts enabled works.
C++26 is expected to work properly with CMake 3.27. That versions of CMake
knows how to invoke clang using C++26.
The parts disabled modi of libc++ have not been modularized yet.
It still is the goal that in the future CMake will be able to do the work
done in `lit.local.cfg`. Doing this in CMake would require a more mature
libc++ implementation.
Thanks a lot to @ldionne for giving hints how to enable modules in a
`lit.local.cfg`.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D157625
This patch fixes a few CMake options that were set using incorrect
mechanisms.
CMake's man page for the -D <var>=<value> option states: If a command in
the project sets the type to PATH or FILEPATH, then the <value> will be
converted to an absolute path. That's not what we want for most of the
paths we have as configuration options. Otherwise, using -D to set the
configuration option results in an absolute path being used, which
breaks things.
option() denotes a boolean variable, but what was desired was a
string/list variable. Fix this to prevent cmake from changing any
non-empty user provided values to 'ON'.
Differential Revision: https://reviews.llvm.org/D157926
If CMAKE_MSVC_RUNTIME_LIBRARY isn't set, then CMake defaults to
the debug CRT, if CMAKE_BUILD_TYPE is set to Debug. If
CMAKE_MSVC_RUNTIME_LIBRARY is set though, that overrides any
implicit defaults.
Match this in libcxx's own manual linking logic. This allows
decoupling the debug CRT from the CMAKE_BUILD_TYPE and allows users
to configure their builds exactly how they want.
Differential Revision: https://reviews.llvm.org/D155561
This respects the CMAKE_MSVC_RUNTIME_LIBRARY option for selecting
the right CRT to use.
Add a CI configuration that tests building this way.
Based on a patch by Andrew Ng.
The test config files end up accumulating and duplicating a fair
bit of cmake-specific logic here; if preferred, we could also add
that in `libcxx/test/CMakeLists.txt` and export a few more variables
to `cmake-bridge.cfg.in` instead.
Differential Revision: https://reviews.llvm.org/D155560
The library msvcrt.lib pulls in ucrt.lib and vcruntime.lib anyway,
there's no need to manually link against the individual dependencies.
This matches how the tests link against libraries - they only link
against msvcrt and msvcprt, not directly against ucrt and vcruntime.
Differential Revision: https://reviews.llvm.org/D155555
Our threading support layer is currently a huge mess. There are too many
configurations with too many confusing names, and none of them are tested
in the usual CI. Here's a list of names related to these configurations:
LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY
_LIBCPP_BUILDING_THREAD_LIBRARY_EXTERNAL
LIBCXXABI_BUILD_EXTERNAL_THREAD_LIBRARY
_LIBCPP_HAS_THREAD_LIBRARY_EXTERNAL
LIBCXX_HAS_EXTERNAL_THREAD_API
_LIBCPP_HAS_THREAD_API_EXTERNAL
This patch cleans this up by removing the ability to build libc++ with
an "external" threading library for testing purposes, removing 4 out of
6 "names" above. That setting was meant to be used by libc++ developers,
but we don't use it in-tree and it's not part of our CI.
I know the ability to use an external threading API is used by some folks
out-of-tree, and this patch doesn't change that. This only changes the
way they will have to test their external threading support. After this
patch, the intent would be for them to set `-DLIBCXX_HAS_EXTERNAL_THREAD_API=ON`
when building the library, and to provide their usual `<__external_threading>`
header when they are testing the library. This can be done easily now
that we support custom lit configuration files in test suites.
The motivation for this patch is that our threading support layer is
basically unmaintainable -- anything beyond adding a new "backend" in
the slot designed for it requires incredible attention. The complexity
added by this setting just doesn't pull its weigh considering the
available alternatives.
Concretely, this will also allow future patches to clean up
`<__threading_support>` significantly.
Differential Revision: https://reviews.llvm.org/D154466
`_LIBCPP_ENABLE_ASSERTIONS` was used to enable the "safe" mode in
libc++. Libc++ now provides the hardened mode and the debug mode that
replace the safe mode.
For backward compatibility, enabling `_LIBCPP_ENABLE_ASSERTIONS` now
enables the hardened mode. Note that the hardened mode provides
a narrower set of checks than the previous "safe" mode (only
security-critical checks that are performant enough to be used in
production).
Differential Revision: https://reviews.llvm.org/D154997
It's no longer necessary to define `_DEBUG` because we use our own
internal assertion mechanism in libc++.
Differential Revision: https://reviews.llvm.org/D155038
This patch only adds new configuration knobs -- the actual assertions
will be added in follow-up patches.
Differential Revision: https://reviews.llvm.org/D153902
MSVC is set for actual msvc and clang-cl. LIBXX_TARGETTING_CLANG_CL is
still used over in runtimes to configure warning flags.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D154698
Previously we added both `-GR-` and `-fno-rtti` if RTTI was disabled.
When building with clang 16.x, that caused this error in part of the build:
```
clang-16: error: argument unused during compilation: '-G R-' [-Werror,-Wunused-command-line-argument]
```
I think the strange message is because clang is seeing `R-` as the argument
to `-G`, which is a valid clang option.
`-GR-` is an alternate syntax for the `/GR-` option for MSVC
(the dash means disable RTTI):
https://learn.microsoft.com/en-us/cpp/build/reference/gr-enable-run-time-type-information?view=msvc-170
This error is sort of fixed by cd18efb61d759405956dbd30e4b5f2720d8e1783
but not intentionally. Also, we'd have to wait for 17.x to benefit from that.
The proper fix here is to only add `-GR-` if we are building with MSVC
or the MSVC-like clang-cl, and add `-fno-rtti` if not.
Reviewed By: #libc, simon_tatham, michaelplatings, ldionne
Differential Revision: https://reviews.llvm.org/D154582
Whether we include operator new and delete into libc++ has always
been a build time setting, and piggy-backing on a macro like
_LIBCPP_DISABLE_NEW_DELETE_DEFINITIONS is inconsistent with how
we handle similar cases for e.g. LIBCXX_ENABLE_RANDOM_DEVICE. Instead,
simply avoid including new.cpp in the sources of the library when we
do not wish to include these operators in the build.
This also makes us much closer to being able to share the definitions
between libc++ and libc++abi, since we could technically build those
definitions into a standalone static library and decide whether we link
it into libc++abi.dylib or libc++.dylib.
Differential Revision: https://reviews.llvm.org/D153272
The patch is based on D144994.
D151030 added the module definitions for the module std.
This patch wires in the module and enables the basic testing.
Some notable features are missing:
- There is no test that libc++ can be fully imported as a module.
- This lacks the parts for the std.compat module.
- The module is not shipped with libc++.
Implements parts of
- P2465R3 Standard Library Modules std and std.compat
Reviewed By: ldionne, aaronmondal, #libc
Differential Revision: https://reviews.llvm.org/D151814
LIBCXX_ENABLE_FILESYSTEM should represent whether the platform has
support for a filesystem, not just whether we support <filesystem>.
This patch slightly generalizes the setting to also encompass whether
we provide <fstream>, since that only makes sense when a filesystem is
supported.
Differential Revision: https://reviews.llvm.org/D152168
Back in 2020 [1], we went very close to enabling Filesystem on MSVC
by disabling int128_t, but decided to wait because MSVC support
for int128_t was supposed to come shortly after. Since it's not
there yet, I propose turning off int128_t support by default on MSVC.
This will make <filesystem> available by default on MSVC, and most
importantly will open the possibility for changing
LIBCXX_ENABLE_FILESYSTEM to mean "the system doesn't have support
for a filesystem" instead of simply "don't build the std::filesystem
library", which is what I'm really after with this change.
In a way, this is a resurection of D91139.
[1]: https://reviews.llvm.org/D91139#2429595
Differential Revision: https://reviews.llvm.org/D134912
