Commit 8306968b592d942cc49bde2e387061e673a9fbb7 deleted file
`compiler-rt/lib/memprof/memprof_meminfoblock.h`, but didn't remove it
from MEMPROF_HEADERS in `compiler-rt/lib/memprof/CMakeLists.txt`.
Remove unneeded leftover line in
`compiler-rt/lib/memprof/CMakeLists.txt`.
p.s.
GH #54777 reported a llvm14 build failure due to the existence of the
leftover line, but I'm unable to reproduce the build failure with llvm19
trunk.
By storing possible test vectors instead of combinations of conditions,
the restriction is dramatically relaxed.
This introduces two options to `cc1`:
* `-fmcdc-max-conditions=32767`
* `-fmcdc-max-test-vectors=2147483646`
This change makes coverage mapping, profraw, and profdata incompatible
with Clang-18.
- Bitmap semantics changed. It is incompatible with previous format.
- `BitmapIdx` in `Decision` points to the end of the bitmap.
- Bitmap is packed per function.
- `llvm-cov` can understand `profdata` generated by `llvm-profdata-18`.
RFC:
https://discourse.llvm.org/t/rfc-coverage-new-algorithm-and-file-format-for-mc-dc/76798
--
Change(s) since llvmorg-19-init-14288-g7ead2d8c7e91
- Update compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
MacOS 15.0 and iOS 18.0 added a new sysctl to fetch a bitvector of all
the hw.optional.arm.FEAT_*'s in one go. Using this has a perf advantage
over doing multiple round-trips to the kernel and back, but since it's
not present in older oses, we still need the slow fallback.
This was reverted in https://github.com/llvm/llvm-project/pull/95435
because it broke Android static hwasan binaries. This reland limits the
change to !SANITIZER_ANDROID.
Original commit message:
When set to non-zero, the HWASan runtime will map the shadow base at the
specified constant address.
This is particularly useful in conjunction with the existing compiler
option 'hwasan-mapping-offset', which bakes a hardcoded constant address
into the instrumentation.
---------
Co-authored-by: Vitaly Buka <vitalybuka@gmail.com>
When an uninstrumented libatomic is used with a TSan instrumented
memcpy, TSan may report a data race in circumstances where writes are
arguably safe.
This occurs because __atomic_compare_exchange won't be instrumented in
an uninstrumented libatomic, so TSan doesn't know that the subsequent
memcpy is race-free.
On the other hand, pthread_mutex_(un)lock will be intercepted by TSan,
meaning an uninstrumented libatomic will not report this false-positive.
pthread_mutexes also may try a number of different strategies to acquire
the lock, which may bound the amount of time a thread has to wait for a
lock during contention.
While pthread_mutex_lock has a larger overhead (due to the function call
and some dispatching), a dispatch to libatomic already predicates a lack
of performance guarantees.
It broke tests on Mac, see comment on the PR.
> Use more flexable regex ([0-9]+) for frame number checks. Since the
> frame numbers might change if some functions are not inlined.
>
> Similar to
> *
> 0360f3218a
> *
> 404bc5ca2a
This reverts commit 98174fb6ec9784d9eb7be313ee1317ca6e703d90
and the follow-up commit 86d8aec97fd9e009e9b360ffeee744d3acce730d.
To detect features we either use HWCAPs or directly extract system
register bitfields and compare with a value. In many cases equality
comparisons give wrong results for example FEAT_SVE is not set if SVE2
is available (see the issue #93651). I am also making the access to
__aarch64_cpu_features atomic.
The corresponding PR for the ACLE specification is
https://github.com/ARM-software/acle/pull/322.
When running some tests with --gtest_repeat=100 --gtest_shuffle, I
encountered some problems because the allocator wasn't torn down
completely, and the singleton pointer ended up pointing to a
use-after-scope'd object.
This patch has a couple of fixes and niceties:
1. Removing the once-init stuff from tests, now that it's implicitly
done in GuardedPoolAllocator::installAtFork() anyway.
2. Calling uninitTestOnly() in the late_init test.
3. Resetting the HasReportedBadPoolAccess when the signal handlers are
installed (allowing for --gtest_repeat w/ recoverable mode).
4. Adding a check and resetting the singleton pointer in
uninitTestOnly().
Use more flexable regex ([0-9]+) for frame number checks. Since the
frame numbers might change if some functions are not inlined.
Similar to
*
0360f3218a
*
404bc5ca2a
Add missing __DATA sections that the objc runtime expects to register.
This fixes running objc code that makes use of `@protocol` references
and `__attribute__((objc_nonlazy_class))` classes.
rdar://129368761
BytesInBG is always greater or equal to BG->BytesInBGAtLastCheckpoint.
Note that the bug led to unnecessary attempts of page releasing and
doesn't have critical impact on the correctness.
`std::aligned_storage` is deprecated with C++23, see
[here](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p1413r3.pdf).
This replaces the usages of `std::aligned_storage` within compiler-rt
with an aligned `std::byte` array.
I will provide patches for other subcomponents as well.
DFSan's sscanf is incorrect
(https://github.com/llvm/llvm-project/issues/94769), which results in
erroneous matches when scraping RSS from /proc/maps. This patch works
around the issue by using strstr as a secondary check.
It also adds a loose validity check for the initial RSS measurement, to
guard against regressions in get_rss_kb().
Fixes https://github.com/llvm/llvm-project/issues/91287
This test case shows a limitation of DFSan's sscanf implementation
(introduced in https://reviews.llvm.org/D153775): it simply ignores
ordinary characters in the format string, instead of actually comparing
them against the input. This may change the semantics of instrumented
programs.
Importantly, this also means that DFSan's release_shadow_space.c test,
which relies on sscanf to scrape the RSS from /proc/maps output, will
incorrectly match lines that don't contain RSS information. As a result,
it adding together numbers from irrelevant output (e.g., base
addresses), resulting in test flakiness
(https://github.com/llvm/llvm-project/issues/91287).
This function is called during very early startup and which can result
in a crash on FreeBSD. The sigaction() function in libc is indirected
via a table so that it can be interposed by the threading library
rather than calling the syscall directly. In the crash I was observing
this table had not yet been relocated, so we ended up jumping to an
invalid address. To avoid this problem we can call __sys_sigaction,
which calls the syscall directly and in FreeBSD 15 is part of libsys
rather than libc, so does not depend on libc being fully initialized.
…ty with high-entropy ASLR
With high-entropy ASLR (e.g., 32-bits == 16TB), the allocator base of
0x700000000000 (112TB) may collide with the placement of the libraries
(e.g., on Linux, the mmap base could be 128TB - 16TB == 112TB). This
results in a segfault in the test case.
This patch moves the allocator base below the PIE program segment,
inspired by fb77ca05ffb4f8e666878f2f6718a9fb4d686839. As per that patch:
1) we are leaving the old behavior for Apple 2) since ASLR cannot be set
above 32-bits for x86-64 Linux, we expect this new layout to be durable.
Note that this is only changing a test case, not the behavior of
sanitizers. Sanitizers have their own settings for initializing the
allocator base.
Reproducer:
1. ninja check-sanitizer # Just to build the test binary needed below;
no need to actually run the tests here
2. sudo sysctl vm.mmap_rnd_bits=32 # Increase ASLR entropy
3. for f in `seq 1 10000`; do echo $f;
GTEST_FILTER=*SizeClassAllocator64Dense
./projects/compiler-rt/lib/sanitizer_common/tests/Sanitizer-x86_64-Test
> /tmp/x; if [ $? -ne 0 ]; then cat /tmp/x; fi; done
This reverts commit b62b7a42bbee4a3bbf9094808f460fdc9c119bd7 and
a5729b71d844c1444f7d348dc2d4ea5b98de5ec5.
This commit broke compilation for systems that lack pthreads.
PTHREAD_MUTEX_INITIALIZER is zeroes for glibc and musl, but it improves
conformance and might work with more libc implementations.
Follow-up to #94374
Pull Request: https://github.com/llvm/llvm-project/pull/94387
When an uninstrumented libatomic is used with a TSan instrumented
memcpy, TSan may report a data race in circumstances where writes are
arguably safe.
This occurs because __atomic_compare_exchange won't be instrumented in
an uninstrumented libatomic, so TSan doesn't know that the subsequent
memcpy is race-free.
On the other hand, pthread_mutex_(un)lock will be intercepted by TSan,
meaning an uninstrumented libatomic will not report this false-positive.
pthread_mutexes also may try a number of different strategies to acquire
the lock, which may bound the amount of time a thread has to wait for a
lock during contention.
While pthread_mutex_lock has a larger overhead (due to the function
call and some dispatching), a dispatch to libatomic already predicates
a lack of performance guarantees.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
0e96eebc7f
accidentally turned the prior patch
(57a507930b)
into a no-op because this macro is always defined (as either 1 or 0).
This patch changes it to correctly use #if.
This reverts commit fe82a3da36196157c0caa1ef2505186782f750d1.
This broke LLDB on MacOS due to a missing symbol during linking.
The fix has been applied in c6c08eee37bada190bd1aa4593c88a5e2c8cdaac.
Original commit message:
The terminfo dependency introduces a significant nonhermeticity into the
build. It doesn't respect `--no-undefined-version` meaning that it's not
a dependency that can be built with Clang 17+. This forces maintainers
of source-based distributions to implement patches or ignore linker
errors.
Remove it to reduce the closure size and improve portability of
LLVM-based tools. Users can still use command line arguments to toggle
color support expliticly.
Fixes#75490Closes#53294#23355
So far, these tests have been disabled in mingw build configurations
(built as asan-dynamic), but these were enabled in
246234ac70faa1e3281a2bb83dfc4dd206a7d59c, exposing the issue.
(That commit is currently reverted, but will probably be relanded in
some form soon.)
Re-Apply: 246234ac70faa1e3281a2bb83dfc4dd206a7d59c
Originally #81677
The static asan runtime on windows had various buggy hacks to ensure loaded dlls got the executable's copy of asan, these never worked all that well, so we have eliminated the static runtime altogether and made the dynamic runtime work for applications linking any flavor of the CRT.
Among other things this allows non-asan-instrumented applications to load asan-instrumented dlls that link against the static CRT.
Co-authored-by: Amy Wishnousky <amyw@microsoft.com>
The data-layout independent constant folding currently has some rather
gnarly code for canonicalizing GEP indices to reduce "notional
overindexing", and then infers inbounds based on that canonicalization.
Now that we canonicalize to i8 GEPs, this canonicalization is
essentially useless, as we'll discard it as soon as the GEP hits the
data-layout aware constant folder anyway. As such, I'd like to remove
this code entirely.
This shouldn't have any impact on optimization capabilities.
In https://github.com/llvm/llvm-project/pull/88323, I changed the logic
within `add_compiler_rt_runtime` to only explicitly code sign the
resulting library if an older version of Apple's ld64 was in use. This
was based on the assumption that newer versions of ld64 and the new
Apple linker always ad-hoc sign their output binaries. This is true in
most cases, but not when using Apple's new linker with the
`-darwin-target-variant` flag to build Mac binaries that are compatible
with Catalyst.
Rather than adding increasingly complicated logic to detect the exact
scenarios that require explicit code signing, I've opted to always
explicitly code sign when using any Apple linker. We instead detect and
use the 'linker-signed' codesigning option when possible to match the
signatures that the linker would otherwise create. This avoids having
non-'linker-signed' ad-hoc signatures which was the underlying problem
that https://github.com/llvm/llvm-project/pull/88323 was intended to
address.
Co-authored-by: Mark Rowe <markrowe@chromium.org>
This is one of the major changes we (Microsoft) have made in the version
of asan we ship with Visual Studio.
@amyw-msft wrote a blog post outlining this work at
https://devblogs.microsoft.com/cppblog/msvc-address-sanitizer-one-dll-for-all-runtime-configurations/
> With Visual Studio 2022 version 17.7 Preview 3, we have refactored the
MSVC Address Sanitizer (ASan) to depend on one runtime DLL regardless of
the runtime configuration. This simplifies project onboarding and
supports more scenarios, particularly for projects statically linked
(/MT, /MTd) to the C Runtimes. However, static configurations have a new
dependency on the ASan runtime DLL.
> Summary of the changes:
> ASan now works with /MT or /MTd built DLLs when the host EXE was not
compiled with ASan. This includes Windows services, COM components, and
plugins.
Configuring your project with ASan is now simpler, since your project
doesn’t need to uniformly specify the same [runtime
configuration](https://learn.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=msvc-170)
(/MT, /MTd, /MD, /MDd).
ASan workflows and pipelines for /MT or /MTd built projects will need to
ensure the ASan DLL (clang_rt.asan_dynamic-<arch>.dll) is available on
PATH.
The names of the ASan .lib files needed by the linker have changed (the
linker normally takes care of this if not manually specifying lib names
via /INFERASANLIBS)
You cannot mix ASan-compiled binaries from previous versions of the MSVC
Address Sanitizer (this is always true, but especially true in this
case).
Here's the description of these changes from our internal PR
1. Build one DLL that includes everything debug mode needs (not included
here, already contributed upstream).
* Remove #if _DEBUG checks everywhere.
* In some places, this needed to be replaced with a runtime check. In
asan_win.cpp, IsDebugRuntimePresent was added where we are searching for
allocations prior to ASAN initialization.
* In asan_win_runtime_functions.cpp and interception_win.cpp, we need to
be aware of debug runtime DLLs even when not built with _DEBUG.
2. Redirect statically linked functions to the ASAN DLL for /MT
* New exports for each of the C allocation APIs so that the statically
linked portion of the runtime can call them (see asan_malloc_win.cpp,
search MALLOC_DLL_EXPORT). Since we want our stack trace information to
be accurate and without noise, this means we need to capture stack frame
info from the original call and tell it to our DLL export. For this, I
have reused the __asan_win_new_delete_data used for op new/delete
support from asan_win_new_delete_thunk_common.h and moved it into
asan_win_thunk_common.h renamed as __asan_win_stack_data.
* For the C allocation APIs, a new file is included in the
statically-linked /WHOLEARCHIVE lib - asan_malloc_win_thunk.cpp. These
functions simply provide definitions for malloc/free/etc to be used
instead of the UCRT's definitions for /MT and instead call the ASAN DLL
export. /INFERASANLIBS ensures libucrt.lib will not take precedence via
/WHOLEARCHIVE.
* For other APIs, the interception code was called, so a new export is
provided: __sanitizer_override_function.
__sanitizer_override_function_by_addr is also provided to support
__except_handler4 on x86 (due to the security cookie being per-module).
3. Support weak symbols for /MD
* We have customers (CoreCLR) that rely on this behavior and would force
/MT to get it.
* There was sanitizer_win_weak_interception.cpp before, which did some
stuff for setting up the .WEAK section, but this only worked on /MT. Now
stuff registered in the .WEAK section is passed to the ASAN DLL via new
export __sanitizer_register_weak_function (impl in
sanitizer_win_interception.cpp). Unlike linux, multiple weak symbol
registrations are possible here. Current behavior is to give priority on
module load order such that whoever loads last (so priority is given to
the EXE) will have their weak symbol registered.
* Unfortunately, the registration can only occur during the user module
startup, which is after ASAN DLL startup, so any weak symbols used by
ASAN during initialization will not be picked up. This is most notable
for __asan_default_options and friends (see asan_flags.cpp). A mechanism
was made to add a callback for when a certain weak symbol was
registered, so now we process __asan_default_options during module
startup instead of ASAN startup. This is a change in behavior, but
there's no real way around this due to how DLLs are.
4. Build reorganization
* I noticed that our current build configuration is very MSVC-specific
and so did a bit of reworking. Removed a lot of
create_multiple_windows_obj_lib use since it's no longer needed and it
changed how we needed to refer to each object_lib by adding runtime
configuration to the name, conflicting with how it works for non-MSVC.
* No more Win32 static build, use /MD everywhere.
* Building with /Zl to avoid defaultlib warnings.
In addition:
* I've reapplied "[sanitizer][asan][win] Intercept _strdup on Windows
instead of strdup" which broke the previous static asan runtime. That
runtime is gone now and this change is required for the strdup tests to
work.
* I've modified the MSVC clang driver to support linking the correct
asan libraries, including via defining _DLL (which triggers different
defaultlibs and should result in the asan dll thunk being linked, along
with the dll CRT (via defaultlib directives).
* I've made passing -static-libsan an error on windows, and made
-shared-libsan the default. I'm not sure I did this correctly, or in the
best way.
* Modified the test harnesses to add substitutions for the dynamic and
static thunks and to make the library substitutions point to the dynamic
asan runtime for all test configurations on windows. Both the static and
dynamic windows test configurations remain, because they correspond to
the static and dynamic CRT, not the static and dynamic asan runtime
library.
---------
Co-authored-by: Amy Wishnousky <amyw@microsoft.com>
The original pull request
(https://github.com/llvm/llvm-project/pull/92838) was reverted due to a
PowerPC buildbot breakage
(df626dd11c).
This reland limits the scope of the change to non-PowerPC platforms. I
am unaware of any PowerPC use cases that would benefit from a larger
kNumStackOriginDescrs constant.
Original CL description: This increases the constant size of
kNumStackOriginDescrs to 4M (64GB of BSS across two arrays), which ought
to be enough for anybody.
This is the easier alternative suggested by eugenis@ in
https://github.com/llvm/llvm-project/pull/92826.
Before this change, the FDR BufferQueue iterator could access oob memory
due to checks of the form `!Buffers[Offset].Used && Offset != Max`. This
allows access to `Buffers[Max]`, which is past the end of the `Buffers`
array. This can lead to crashes when that memory is not mapped. Fix this
by testing `Offset != Max` first.
Previously, some xray trampolines would modify condition codes (before
saving/after restoring flags) due to stack alignment instructions, which
use add/sub.
I am not aware of issues that this causes in practice (outside of the
situation described in https://github.com/llvm/llvm-project/pull/89364,
which is only problematic due to a different bug). Nevertheless, it
seems nicer and less error-prone for xray instrumentation to be as
unobstrusive/preserve as much state as possible.
The terminfo dependency introduces a significant nonhermeticity into the
build. It doesn't respect `--no-undefined-version` meaning that it's not
a dependency that can be built with Clang 17+. This forces maintainers
of source-based distributions to implement patches or ignore linker
errors.
Remove it to reduce the closure size and improve portability of
LLVM-based tools. Users can still use command line arguments to toggle
color support expliticly.
Fixes#75490Closes#53294#23355
