As a follow-up to #121013 (which optimized `ranges::copy`) and #121026
(which optimized `ranges::copy_backward`), this PR enhances the
performance of `std::ranges::{move, move_backward}` for
`vector<bool>::iterator`, addressing a subtask outlined in issue #64038.
The optimizations bring performance improvements analogous to those
achieved for the `{copy, copy_backward}` algorithms: up to 2000x for
aligned moves and 60x for unaligned moves. Moreover, comprehensive
tests covering up to 4 storage words (256 bytes) with odd and even bit
sizes are provided, which validate the proposed optimizations in this
patch.
This patch adds a large number of missing includes in the libc++ headers
and the test suite. Those were found as part of the effort to move
towards a mostly monolithic top-level std module.
We've introduced `__constexpr_memmove` a while ago, which simplified the
implementation of the copy and move lowering a bit. This allows us to
remove some of the boilerplate.
These headers have become very small by using compiler builtins, often
containing only two declarations. This merges these headers, since
there doesn't seem to be much of a benefit keeping them separate.
Specifically, `is_{,_nothrow,_trivially}{assignable,constructible}` are
kept and the `copy`, `move` and `default` versions of these type traits
are moved in to the respective headers.
This patch runs clang-format on all of libcxx/include and libcxx/src, in
accordance with the RFC discussed at [1]. Follow-up patches will format
the benchmarks, the test suite and remaining parts of the code. I'm
splitting this one into its own patch so the diff is a bit easier to
review.
This patch was generated with:
find libcxx/include libcxx/src -type f \
| grep -v 'module.modulemap.in' \
| grep -v 'CMakeLists.txt' \
| grep -v 'README.txt' \
| grep -v 'libcxx.imp' \
| grep -v '__config_site.in' \
| xargs clang-format -i
A Git merge driver is available in libcxx/utils/clang-format-merge-driver.sh
to help resolve merge and rebase issues across these formatting changes.
[1]: https://discourse.llvm.org/t/rfc-clang-formatting-all-of-libc-once-and-for-all
Resolves Issue #69083
The `_CopySegment` helper for `ranges::copy(join_view, out)` is not
`constexpr` causing rejection in `libc++` in a static assertion context
as in the issue snippet.
This simplifies the code inside copy/move and makes it easier to apply the optimization to other algorithms.
Reviewed By: ldionne, Mordante, #libc
Spies: arichardson, libcxx-commits
Differential Revision: https://reviews.llvm.org/D151265
We plan to add concepts for checking that iterators actually provide what they claim to. This is to avoid people thinking that these type traits actually check the iterator requirements in more detail.
Reviewed By: ldionne, #libc
Spies: Mordante, libcxx-commits, wenlei
Differential Revision: https://reviews.llvm.org/D150801
This has multiple benefits:
- The optimizations are also performed for the `ranges::` versions of the algorithms
- Code duplication is reduced
- it is simpler to add this optimization for other segmented iterators,
like `ranges::join_view::iterator`
- Algorithm code is removed from `<deque>`
Reviewed By: ldionne, huixie90, #libc
Spies: mstorsjo, sstefan1, EricWF, libcxx-commits, mgorny
Differential Revision: https://reviews.llvm.org/D132505
This reverts commit a6e1080b87db8fbe0e1afadd96af5a3c0bd5e279.
Fix the conditions when the `memmove` optimization can be applied and refactor them out into a reusable type trait, fix and significantly expand the tests.
Differential Revision: https://reviews.llvm.org/D139235
Instead of using `reverse_iterator`, share the optimization between the 4 algorithms. The key observation here that `memmove` applies to both `copy` and `move` identically, and to their `_backward` versions very similarly. All algorithms now follow the same pattern along the lines of:
```
if constexpr (can_memmove<InIter, OutIter>) {
memmove(first, last, out);
} else {
naive_implementation(first, last, out);
}
```
A follow-up will delete `unconstrained_reverse_iterator`.
This patch removes duplication and divergence between `std::copy`, `std::move` and `std::move_backward`. It also improves testing:
- the test for whether the optimization is used only applied to `std::copy` and, more importantly, was essentially a no-op because it would still pass if the optimization was not used;
- there were no tests to make sure the optimization is not used when the effect would be visible.
Differential Revision: https://reviews.llvm.org/D130695
This was discussed on Discord with the consensus that we should rename the macros.
Reviewed By: ldionne, Mordante, var-const, avogelsgesang, jloser, #libc
Spies: libcxx-commits
Differential Revision: https://reviews.llvm.org/D131498
Also fix `ranges::stable_sort` and `ranges::inplace_merge` to support
proxy iterators now that their internal implementations can correctly
dispatch `rotate`.
Differential Revision: https://reviews.llvm.org/D130758
This patch also adds a new optimization to `std::move`. It unwraps three `reverse_iterator`s if the wrapped iterator is a `contiguous_iterator` and the iterated type is trivially_movable. This allows us to simplify `ranges::move_backward` to a forward to `std::move` without any pessimization.
Reviewed By: var-const, #libc
Spies: libcxx-commits, mgorny
Differential Revision: https://reviews.llvm.org/D126616
This commit reverts 5aaefa51 (and also partly 7f285f48e77 and b6d75682f9,
which were related to the original commit). As landed, 5aaefa51 had
unintended consequences on some downstream bots and didn't have proper
coverage upstream due to a few subtle things. Implementing this is
something we should do in libc++, however we'll first need to address
a few issues listed in https://reviews.llvm.org/D106124#3349710.
Differential Revision: https://reviews.llvm.org/D120683
libc++ has started splicing standard library headers into much more
fine-grained content for maintainability. It's very likely that outdated
and naive tooling (some of which is outside of LLVM's scope) will
suggest users include things such as <__ranges/access.h> instead of
<ranges>, and Hyrum's law suggests that users will eventually begin to
rely on this without the help of tooling. As such, this commit
intends to protect users from themselves, by making it a hard error for
anyone outside of the standard library to include libc++ detail headers.
Differential Revision: https://reviews.llvm.org/D106124
Basically a rebase of D104980; most of that patch had already happened
via gradual drive-by changes, but this finishes it up.
Don't touch the inclusions from `<__functional_base>`, `<__hash_table>`,
or `<__locale>`; those could be removed if we propagated the
inclusions up to the includers of those files, but there are lots
of those includers.
`<algorithm>`, `<functional>`, and `<memory>` already include `<utility>`
at the top level. `<iterator>` did not, so I've added it there.
Differential Revision: https://reviews.llvm.org/D119020
The NFC part of D116809. We still want to enforce this in CI,
but the mechanism for that is still to-be-determined.
Differential Revision: https://reviews.llvm.org/D116809
Moves:
* `std::move`, `std::forward`, `std::declval`, and `std::swap` into
`__utility/${FUNCTION_NAME}`.
* `std::swap_ranges` and `std::iter_swap` into
`__algorithm/${FUNCTION_NAME}`
Differential Revision: https://reviews.llvm.org/D103734
This is a fairly mechanical change, it just moves each algorithm into
its own header. This is intended to be a NFC.
This commit re-applies 7ed7d4ccb899, which was reverted in 692d7166f771
because the Modules build got broken. The modules build has now been
fixed, so we're re-committing this.
Differential Revision: https://reviews.llvm.org/D103583
Attribution note
----------------
I'm only committing this. This commit is a mix of D103583, D103330 and
D104171 authored by:
Co-authored-by: Christopher Di Bella <cjdb@google.com>
Co-authored-by: zoecarver <z.zoelec2@gmail.com>
This is a fairly mechanical change, it just moves each algorithm into its own header. This is a NFC.
Note: during this change, I burned down all the includes, so this follows "include only and exactly what you use."
Differential Revision: https://reviews.llvm.org/D103583
