This removes the need for macOS nodes in Buildkite. It also moves to the
proper way of testing backdeployment, which is to actually run on the
target OS itself, instead of using packaged dylibs from previous OS
versions and trying to emulate backdeployment with DYLD_LIBRARY_PATH.
As a drive-by change, also fix a few back-deployment annotations that
were incorrect and add support for minor versions in the Lit feature
determining availability from the target triple.
This is a re-application of bc6bd3bc1e9 which was reverted in
f11abac6524 because it broke the Clang pre-commit CI.
Original commit message:
This patch rewrites the modulemap to have fewer top-level modules.
Previously, our modulemap had one top level module for each header in
the library, including private headers. This had the well-known problem
of making compilation times terrible, in addition to being somewhat
against the design principles of Clang modules.
This patch provides almost an order of magnitude compilation time
improvement when building modularized code (certainly subject to
variations). For example, including <ccomplex> without a module cache
went from 22.4 seconds to 1.6 seconds, a 14x improvement.
To achieve this, one might be tempted to simply put all the headers in a
single top-level module. Unfortunately, this doesn't work because libc++
provides C compatibility headers (e.g. stdlib.h) which create cycles
when the C Standard Library headers are modularized too. This is
especially tricky since base systems are usually not modularized: as far
as I know, only Xcode 16 beta contains a modularized SDK that makes this
issue visible. To understand it, imagine we have the following setup:
// in libc++'s include/c++/v1/module.modulemap
module std {
header stddef.h
header stdlib.h
}
// in the C library's include/module.modulemap
module clib {
header stddef.h
header stdlib.h
}
Now, imagine that the C library's <stdlib.h> includes <stddef.h>,
perhaps as an implementation detail. When building the `std` module,
libc++'s <stdlib.h> header does `#include_next <stdlib.h>` to get the C
library's <stdlib.h>, so libc++ depends on the `clib` module.
However, remember that the C library's <stdlib.h> header includes
<stddef.h> as an implementation detail. Since the header search paths
for libc++ are (and must be) before the search paths for the C library,
the C library ends up including libc++'s <stddef.h>, which means it
depends on the `std` module. That's a cycle.
To solve this issue, this patch creates one top-level module for each C
compatibility header. The rest of the libc++ headers are located in a
single top-level `std` module, with two main exceptions. First, the
module containing configuration headers (e.g. <__config>) has its own
top-level module too, because those headers are included by the C
compatibility headers.
Second, we create a top-level std_core module that contains several
dependency-free utilities used (directly or indirectly) from the __math
subdirectory. This is needed because __math pulls in a bunch of stuff,
and __math is used from the C compatibility header <math.h>.
As a direct benefit of this change, we don't need to generate an
artificial __std_clang_module header anymore to provide a monolithic
`std` module, since our modulemap does it naturally by construction.
A next step after this change would be to look into whether math.h
really needs to include the contents of __math, and if so, whether
libc++'s math.h truly needs to include the C library's math.h header.
Removing either dependency would break this annoying cycle.
Thanks to Eric Fiselier for pointing out this approach during a recent
meeting. This wasn't viable before some recent refactoring, but wrapping
everything (except the C headers) in a large module is by far the
simplest and the most effective way of doing this.
Fixes#86193
This allows catching OOB accesses inside `unique_ptr<T[]>` when the size
of the allocation is known. The size of the allocation can be known when
the unique_ptr has been created with make_unique & friends or when the
type necessitates an array cookie before the allocation.
This is a re-aplpication of 45a09d181 which had been reverted in
f11abac6 due to unrelated CI failures.
This reverts 3 commits:
45a09d1811d5d6597385ef02ecf2d4b7320c37c5
24bc3244d4e221f4e6740f45e2bf15a1441a3076
bc6bd3bc1e99c7ec9e22dff23b4f4373fa02cae3
The GitHub pre-merge CI has been broken since this PR went in. This
change reverts it to see if I can get the pre-merge CI working again.
This allows catching OOB accesses inside `unique_ptr<T[]>` when the size
of the allocation is known. The size of the allocation can be known when
the unique_ptr has been created with make_unique & friends or when the
type necessitates an array cookie before the allocation.
This patch rewrites the modulemap to have fewer top-level modules.
Previously, our modulemap had one top level module for each header in
the library, including private headers. This had the well-known problem
of making compilation times terrible, in addition to being somewhat
against the design principles of Clang modules.
This patch provides almost an order of magnitude compilation time
improvement when building modularized code (certainly subject to
variations). For example, including <ccomplex> without a module cache
went from 22.4 seconds to 1.6 seconds, a 14x improvement.
To achieve this, one might be tempted to simply put all the headers in a
single top-level module. Unfortunately, this doesn't work because libc++
provides C compatibility headers (e.g. stdlib.h) which create cycles
when the C Standard Library headers are modularized too. This is
especially tricky since base systems are usually not modularized: as far
as I know, only Xcode 16 beta contains a modularized SDK that makes this
issue visible. To understand it, imagine we have the following setup:
// in libc++'s include/c++/v1/module.modulemap
module std {
header stddef.h
header stdlib.h
}
// in the C library's include/module.modulemap
module clib {
header stddef.h
header stdlib.h
}
Now, imagine that the C library's <stdlib.h> includes <stddef.h>,
perhaps as an implementation detail. When building the `std` module,
libc++'s <stdlib.h> header does `#include_next <stdlib.h>` to get the C
library's <stdlib.h>, so libc++ depends on the `clib` module.
However, remember that the C library's <stdlib.h> header includes
<stddef.h> as an implementation detail. Since the header search paths
for libc++ are (and must be) before the search paths for the C library,
the C library ends up including libc++'s <stddef.h>, which means it
depends on the `std` module. That's a cycle.
To solve this issue, this patch creates one top-level module for each C
compatibility header. The rest of the libc++ headers are located in a
single top-level `std` module, with two main exceptions. First, the
module containing configuration headers (e.g. <__config>) has its own
top-level module too, because those headers are included by the C
compatibility headers.
Second, we create a top-level std_core module that contains several
dependency-free utilities used (directly or indirectly) from the __math
subdirectory. This is needed because __math pulls in a bunch of stuff,
and __math is used from the C compatibility header <math.h>.
As a direct benefit of this change, we don't need to generate an
artificial __std_clang_module header anymore to provide a monolithic
`std` module, since our modulemap does it naturally by construction.
A next step after this change would be to look into whether math.h
really needs to include the contents of __math, and if so, whether
libc++'s math.h truly needs to include the C library's math.h header.
Removing either dependency would break this annoying cycle.
Thanks to Eric Fiselier for pointing out this approach during a recent
meeting. This wasn't viable before some recent refactoring, but wrapping
everything (except the C headers) in a large module is by far the
simplest and the most effective way of doing this.
Fixes#86193
We were incorrectly computing whether a FTM has been implemented.
Instead of checking whether any version of the FTM is implemented for
the current Standard, we need to make sure that the correct version of
the FTM has been implemented.
As a drive-by fix, also correctly close the file that we load JSON from,
which was forgotten.
Works towards P0619R4 / #99985.
The use of `std::get_temporary_buffer` and `std::return_temporary_buffer`
are replaced with `unique_ptr`-based RAII buffer holder.
Escape hatches:
- `_LIBCPP_ENABLE_CXX20_REMOVED_TEMPORARY_BUFFER` restores
`std::get_temporary_buffer` and `std::return_temporary_buffer`.
Drive-by changes:
- In `<syncstream>`, states that `get_temporary_buffer` is now removed,
because `<syncstream>` is added in C++20.
This significantly simplifies the code, improves compile times and
improves the object layout of types using `__compressed_pair` in the
unstable ABI. The only downside is that this is extremely ABI sensitive
and pedantically breaks the ABI for empty final types, since the address
of the subobject may change. The ABI of the whole object should not be
affected.
Fixes#91266Fixes#93069
Before changing the compiler version in
https://github.com/llvm/llvm-project/pull/108761, we first of all need
to upgrade the HEAD version to `Clang-20` in the Docker files and push
new builder images to the CI.
We waited before supporting std::jthread fully because we wanted to
investigate other implementation strategies (in particular one involving
std::mutex). Since then, we did some benchmarking and decided that we
wouldn't be moving forward with std::mutex. Hence, there is no real
reason to punt on making std::jthread & friends non-experimental.
The _LIBCPP_HAS_NO_THREADS carve-out does not result in hard errors
anymore, but the patch that changed that forgot to update the header
restrictions we use to auto-generate several files.
We can also remove the restrictions for the no-localization build and
no-wide-characters, but doing it is less straightforward so I'm leaving
it out of this patch.
When we introduced the machinery for transitive includes validation, at
some point we stopped including the full set of transitive includes in
the CSV files and instead only tracked the set of public headers
included *directly* by a top-level header.
The reason for doing that was so that the CSV files containing
"transitive" includes could be used to draw the dependency graph of
libc++ headers. However, the downside was that it made the contents of
the CSV files much harder to interpret.
In particular, many changes that modify the CSV files do not in fact
modify the effective set of transitive includes, which is confusing.
This patch goes back to storing the full set of transitive includes in
the CSV files and removes the ability to graph the libc++ includes
directly from those CSV files, which we never actually used.
We need a forward-declaration so that we can know about std::byte from
some type traits without having to include std::byte's definition, which
(circularly) depends back on type traits.
When bringing up a new cross compiler from scratch, we build
libunwind/libcxx in a setup where the toolchain is incomplete and unable
to perform the normal linker checks; this requires a few special cases
in the CMake files.
We simulate that scenario by removing the libc++ headers, libunwind and
libc++ libraries from the installed toolchain.
We need to set CMAKE_CXX_COMPILER_WORKS since CMake fails to probe the
compiler. We need to set CMAKE_CXX_COMPILER_TARGET, since LLVM's
heuristics fail when CMake hasn't been able to probe the environment
properly. (This is normal; one has to set those options when setting up
such a toolchain from scratch.)
This adds CI coverage for these build scenarios, which otherwise seldom
are tested by some build flow (but are essential when setting up a cross
compiler from scratch).
Many headers include `<cstddef>` just for size_t, and pulling in
additional content (e.g. the traits used for std::byte) is unnecessary.
To solve this problem, this patch splits up `<cstddef>` into
subcomponents so that headers can include only the parts that they
actually require.
This has the added benefit of making the modules build a lot stricter
with respect to IWYU, and also providing a canonical location where we
define `std::size_t` and friends (which were previously defined in
multiple headers like `<cstddef>` and `<ctime>`).
After this patch, there's still many places in the codebase where we
include `<cstddef>` when `<__cstddef/size_t.h>` would be sufficient.
This patch focuses on removing `<cstddef>` includes from __type_traits
to make these headers non-circular with `<cstddef>`. Additional
refactorings can be tackled separately.
This re-formats a few headers that had become out-of-sync with respect
to formatting since we ran clang-format on the whole codebase. There's
surprisingly few instances of it.
This patch replaces 'tags' in the CSV status pages by inline notes
that optionally describe more details about the paper/LWG issue.
Tags were not really useful anymore because we have a vastly superior
tagging system via Github issues, and keeping the tags up-to-date
between CSV files and Github is going to be really challenging.
This patch also adds support for encoding custom notes in the CSV
files via Github issues. To encode a note in the CSV file, the
body (initial description) of a Github issue can be edited to contain
the following markers:
BEGIN-RST-NOTES
text that will be added as a note in the RST
END-RST-NOTES
Amongst other things, this solves the problem of conveying that a
paper has been implemented as a DR, and it gives a unified way to
add notes to the status pages from Github.
This patch implements https://wg21.link/P2747R2.
The library changes affect direct `operator new` and `operator new[]`
calls even when the core language changes are absent.
The changes are not available for MS ABI because the `operator new` and
`operator new[]` are from VCRuntime's `<vcruntime_new.h>`. A feature
request was submitted for that [1].
As a drive-by change, the patch reformatted the whole `new.pass.cpp` and
`new_array.pass.cpp` tests.
Closes#105427
[1]: https://developercommunity.visualstudio.com/t/constexpr-for-placement-operator-newope/10730304.
We always strive to test libc++ as close as possible to the way we are
actually shipping it. This was approximated reasonably well by setting
up the minimal driver flags when running the test suite, however we were
running the test suite against the library located in the build
directory.
This patch improves the situation by installing the library (the
headers, the built library, modules, etc) into a fake location and then
running the test suite against that fake "installation root".
This should open the door to getting rid of the temporary copy of the
headers we make during the build process, however this is left for a
future improvement.
Note that this adds quite a bit of verbosity whenever running the test
suite because we install the headers beforehand every time. We should be
able to override this to silence it, however CMake doesn't currently
give us a way to do that, see https://gitlab.kitware.com/cmake/cmake/-/issues/26085.
This patch implements https://wg21.link/p2609r3.
The test code was originally authored by JMazurkiewicz.
Notes:
- P2609R3 is not officially a Defect Report, but MSVC STL
implements it in C++20 mode.
Moreover, P2609R3 and P2997R1 touch exactly the same set of
concepts, and MSVC STL and libc++ have already treated P2997R1
as a DR.
- This patch also adjusted feature-test macros.
+ In C++20 mode, the value of __cpp_lib_ranges should be `202110L` because
- `202202L` covers `range_adaptor_closure` (P2387R3), and
- `202207L` covers move-only types in range adaptors (P2494R2).
And all of these changes are only available since C++23 mode.
+ In C++23 mode, the value should be `202406L` because
- `202211L` covers removing poison overloads (P2602R2),
- `202302L` covers relaxing projected value types (P2609R3), and
- `202406L` covers removing requirements on `iter_common_reference_t` (P2997R1).
And all of these changes are already or being implemented.
Fixes#105253.
Co-authored-by: Jakub Mazurkiewicz <mazkuba3@gmail.com>
This patch decouples macOS CI testing from BuildKite, which makes the
maintenance of macOS CI easier and more accessible to all contributors.
Right now, the macOS CI is running entirely on machines owned by the
LLVM Foundation with only a small set of contributors having direct
access to them. In particular, updating these machines is currently
a very time-consuming manual process that requires taking the machines
offline, and using Github-provided instances makes that an order of
magnitude easier.
The story for performing back-deployment testing still needs to be
figured out, so for now we are retaining some jobs under BuildKite.
This trait is implemented in C++26 conditionally on the compiler
supporting the __builtin_is_virtual_base_of intrinsic. I believe only
tip-of-trunk Clang currently implements that builtin.
Closes#105432
This enhances the Github - CSV synchronization script to understand
some of the idioms we use in the CSV status files, like |Nothing To Do|
and others.
We already documented that libc++ was C++14 complete, but we still
documented the status of C++14. Since that is redundant (and I suspect
the C++14 status page was missing some stuff), simply remove them.
This script can be run manually to synchronize the CSV files that we use
to track Standards Conformance with the Github issues that track our
implementation of LWG issues and papers.
The generator makes a few changes to the output
- removes the synopsis, it did not really show what was implemented
correctly.
- the output now is clang-format clean.
This code uses the new FTM data structure. Since the contents of this
structure are not up-to-date the code is only used in its tests.
This fixes testing with MinGW, if built without
__USE_MINGW_ANSI_STDIO=1.
On x86 MinGW, such a configuration fails printf tests with long doubles
due to mismatches between 80 and 64 bit long doubles - but on ARM,
there's no such issue, so building without __USE_MINGW_ANSI_STDIO=1 is
perfectly valid there.
Add another similar XFAIL to a libcxxabi test; this test isn't executed
in MSVC environments, so no XFAIL has been needed so far.
The base of android-buildkite-builder is buildkite-builder, not
android-build-base. android-build-base is only used for its /opt/android
directory, so move the Docker installation step into
android-buildkite-builder.
Install bzip2 for extracting ndk_platform.tar.bz2.
Add "set -e" to RUN heredocs to catch failing commands.
~~NB: This PR depends on #78876. Ignore the first commit when reviewing,
and don't merge it until #78876 is resolved. When/if #78876 lands, I'll
clean this up.~~
This partially restores parity with the old, since removed debug build.
We now can re-enable a bunch of the disabled tests. Some things of note:
- `bounded_iter`'s converting constructor has never worked. It needs a
friend declaration to access the other `bound_iter` instantiation's
private fields.
- The old debug iterators also checked that callers did not try to
compare iterators from different objects. `bounded_iter` does not
currently do this, so I've left those disabled. However, I think we
probably should add those. See
https://github.com/llvm/llvm-project/issues/78771#issuecomment-1902999181
- The `std::vector` iterators are bounded up to capacity, not size. This
makes for a weaker safety check. This is because the STL promises not to
invalidate iterators when appending up to the capacity. Since we cannot
retroactively update all the iterators on `push_back()`, I've instead
sized it to the capacity. This is not as good, but at least will stop
the iterator from going off the end of the buffer.
There was also no test for this, so I've added one in the `std`
directory.
- `std::string` has two ambiguities to deal with. First, I opted not to
size it against the capacity. https://eel.is/c++draft/string.require#4
says iterators are invalidated on an non-const operation. Second,
whether the iterator can reach the NUL terminator. The previous debug
tests and the special-case in https://eel.is/c++draft/string.access#2
suggest no. If either of these causes widespread problems, I figure we
can revisit.
- `resize_and_overwrite.pass.cpp` assumed `std::string`'s iterator
supported `s.begin().base()`, but I see no promise of this in the
standard. GCC also doesn't support this. I fixed the test to use
`std::to_address`.
- `alignof.compile.pass.cpp`'s pointer isn't enough of a real pointer.
(It needs to satisfy `NullablePointer`, `LegacyRandomAccessIterator`,
and `LegacyContiguousIterator`.) `__bounded_iter` seems to instantiate
enough to notice. I've added a few more bits to satisfy it.
Fixes#78805
The Android Emulator has started printing this message, so pass the
`-no-metrics` option:
```
##############################################################################
## WARNING - ACTION REQUIRED ##
## Consider using the '-metrics-collection' flag to help improve the ##
## emulator by sending anonymized usage data. Or use the '-no-metrics' ##
## flag to bypass this warning and turn off the metrics collection. ##
## In a future release this warning will turn into a one-time blocking ##
## prompt to ask for explicit user input regarding metrics collection. ##
## ##
## Please see '-help-metrics-collection' for more details. You can use ##
## '-metrics-to-file' or '-metrics-to-console' flags to see what type of ##
## data is being collected by emulator as part of usage statistics. ##
##############################################################################
```
Implementing the Hermite polynomials which are part of C++17's
mathematical special functions. The goal is to get early feedback which
will make implementing the other functions easier. Integration of
functions in chunks (e.g. `std::hermite` at first, then `std::laguerre`,
etc.) might make sense as well (also see note on boost.math below).
I started out from this abandoned merge request:
https://reviews.llvm.org/D58876 .
The C++23 standard defines them in-terms of `/* floating-point type */`
arguments. I have not looked into that.
Note, there is still an ongoing discussion on discourse whether
importing boost.math is an option.
Implements [P1223R5][] completely.
Includes an implementation of `find_last`, `find_last_if`, and
`find_last_if_not`.
[P1223R5]: https://wg21.link/p1223r5
In order to test libc++ under the "Apple System Library" configuration,
we need to run the tests using DYLD_LIBRARY_PATH. This is required
because libc++ gets an install_name of /usr/lib when built as a system
library, which means that we must override the copy of libc++ used by
the whole process. This effectively reverts 2cf2f1b, which was the wrong
solution for the problem I was having.
Of course, this assumes that the just-built libc++ is sufficient to
replace the system library, which is not actually the case
out-of-the-box. Indeed, the system library contains a few symbols that
are not provided by the upstream library, leading to undefined symbols
when replacing the system library by the just-built one.
To solve this problem, we separately build shims that provide those
missing symbols and we manually link against them when we build
executables in the tests. While this is somewhat brittle, it provides a
localized and unintrusive way to allow testing the Apple system
configuration in an upstream environment, which has been a frequent
request.
