This reverts commit 36ca9a29025a2f678096e9545fa2ec44e8432592.
We were using the `time` as the seed while choosing a new TSD. To make
the access of TSDs evenly distributed, we require a higher precision in
`time`. Otherwise, many threads may result in having the same random
access pattern on TSDs because they share the same `time` in certain
period. On Linux, CLOCK_MONOTONIC_COARSE usually adopts 4 ms precision.
This is way higher than the average accessing time of TSD (which is
usually less than 1 us). As a result, when multiple threads try to
select a new TSD in a 4 ms interval, they share the same `time` seed and
end up choosing and congesting on the same TSD.
compiler-rt may depend on libc (memset etc). Likewise a libc built by
clang may depend on compiler-rt builtins.
This circular dependency doesn't matter much once they're both compiled.
The easy compilation order to build both from source is:
1. install libc headers somewhere
2. build compiler-rt builtins against those headers
3. build libc against compiler-rt builtins
This patch relaxes the cmake sanity check to pass without requiring a
libc library. That allows the above sequence to work. Otherwise one
needs to build a static libc, then use that to pass the compiler-rt
cmake check, then build a normal libc.
Do not abort if a vector cannot increase its own capacity. In that case,
push_back calls silently fail.
Modify the ScopedString implementation so that it no longer requires two
passes to do the format. Move the helper functions to be private member
functions so that they can use push_back directly. This allows the
capacity to be increased under the hood and/or silently discards data if
the capacity is exceeded and cannot be increased.
Add new tests for the Vector and ScopedString for capacity increase
failures.
Doing this so that if a map call fails, and we are attempting to write
an error string, we can still get some of the message dumped. This also
avoids crashing in Scudo code, and makes the caller handle any failures.
In glibc versions before 2.33. `libc_nonshared.a` defines
`__fxstat/__fxstat64` but there is no `fstat/fstat64`. glibc 2.33 added
`fstat/fstat64` and obsoleted `__fxstat/__fxstat64`. Ports added after
2.33 do not provide `__fxstat/__fxstat64`, so our `fstat/fstat64`
interceptors using `__fxstat/__fxstat64` interceptors would lead to
runtime failures on such ports (LoongArch and certain RISC-V ports).
Similar to https://reviews.llvm.org/D118423, refine the conditions that
we define fstat{,64} interceptors. `fstat` is supported by musl/*BSD
while `fstat64` is glibc only.
Fixes#83844.
This PR adds callbacks to mark futex syscalls as blocking. Unfortunately
we didn't have a mechanism before to mark syscalls as a blocking call,
so I had to implement it, but it mostly reuses the `BlockingCall`
implementation
[here](96819daa3d/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp (L362-L380)).
The issue includes some information but this issue was discovered
because Rust uses futexes directly. So most likely we need to update
Rust as well to use these callbacks.
Also see the latest comments in #85188 for some context.
I also sent another PR #84162 to mark `pthread_*_lock` calls as
blocking.
Fixes#83561.
When a thread is blocked on a mutex and we send an async signal to that
mutex, it never arrives because tsan thinks that `pthread_mutex_lock` is
not a blocking function. This patch marks `pthread_*_lock` functions as
blocking so we can successfully deliver async signals like `SIGPROF`
when the thread is blocked on them.
See the issue also for more details. I also added a test, which is a
simplified version of the compiler explorer example I posted in the
issue.
Please let me know if you have any other ideas or things to improve!
Happy to work on them.
Also I filed #83844 which is more tricky because we don't have a libc
wrapper for `SYS_futex`. I'm not sure how to intercept this yet. Please
let me know if you have ideas on that as well. Thanks!
Reverting #85188 with follow up patches.
This reverts commit 362d26366d0175f01ffb6085eb747a6e40f01147.
This reverts commit c9bdeabdf4b46fbf1f6a9fcbf9cd61d460b18c08.
This reverts commit 6bc6e1ace9fa8453e164fa04b5d9acd5a77e089a.
This reverts commit 01fa550ff654d6724e6da54c877032baeddff14b.
This reverts commit ddcbab37ac0e5743a8d39be3dd48d967f4c85504.
/Zl is the equivalent of -nodefaultlibs. The idea here is to make sure
that the asan runtime doesn't have any defaultlibs directives, which
makes it easier to mix an asan runtime built with the dynamic CRT with
an application built with the static CRT (or vise-versa).
This is part of the overall effort to remove the static asan runtime on
windows entirely: https://github.com/llvm/llvm-project/pull/81677
Co-authored-by: Amy Wishnousky <amyw@microsoft.com>
The HOST_LINK_VERSION is a hardcoded string in Darwin clang that detects
the linker version at configure time. The driver uses this information
to build the correct set of arguments for the linker. This patch detects
the linker version again during compiler-rt configuration and passes it
to the tests. This allows a clang built on a machine with a new linker
to run compiler-rt tests on a machine with an old linker.
rdar://125198603
The greendragon was recently moved and now it runs on somewhat newer
macOS
version - which breaks some sanitizers tests
rdar://125052915
Co-authored-by: Mariusz Borsa <m_borsa@apple.com>
Since this standalone build configuration uses the runtime libraries that
are being built just now, we need to ensure that e.g. the TSan unit tests
depend on the tsan runtime library. Also fix TSAN_DEPS being overridden
to not include the tsan runtime (commit .....).
This change fixes a build race seen in the CI checks for
TsanRtlTest-x86_64-Test in https://github.com/llvm/llvm-project/pull/83088.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83650
When building optimized versions of the runtime libraries the compiler
is generally able to elide these references, but when building them
for maximum debug info (with -O0), these references remain which causes
the test suite to fail for tests that do not pull in the C++ standard
library.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/84613
Some buildbots (e.g.,
https://lab.llvm.org/buildbot/#/builders/18/builds/16061/steps/10/logs/stdio)
have recently started complaining about
```
cast from 'SignalHandlerType' (aka 'void (*)(int, void *, void *)') to 'sa_sigaction_t' (aka 'void (*)(int, siginfo_t *, void *)') converts to incompatible function type [-Werror,-Wcast-function-type-strict]
219 | sigact.sa_sigaction = (sa_sigaction_t)handler;
```
This patch does an intermediate cast to `'(void (*) (void))'` to suppress the warning.
N.B. SignalHandlerType has `'void*'` instead of `'siginfo_t*'` because it is typedef'ed in sanitizer_common/sanitizer_common.h, which does not have access to the header (signal.h) that defines siginfo_t; we therefore cannot fix SignalHandlerType.
DFSan's shadow mappings are incompatible with 32 bits of ASLR entropy
('sudo sysctl vm.mmap_rnd_bits=32; ninja check-dfsan') and it is
difficult to fix this via increasing the size of the shadow mappings,
due to the overhead of shadow memory. This patch works around the issue
by detecting if the memory layout is incompatible, and if so,
re-exec'ing without ASLR.
DFSan and MSan share copy-pasted shadow memory code, hence this
workaround is ported from MSan:
- "[msan] Re-exec with no ASLR if memory layout is incompatible on
Linux"
(58f7251820)
- "[msan] Add 'MappingDesc::ALLOCATOR' type and check it is available"
(af2bf86a37)
(which in turn are inspired by TSan: "Re-exec TSan with no ASLR if
memory layout is incompatible on Linux"
(0784b1eefa
))
aeubanks had remarked in
https://github.com/llvm/llvm-project/pull/85142#issuecomment-2004442883
that this issue occurs in Chromium:
https://ci.chromium.org/ui/p/chromium/builders/try/linux_upload_clang/5066/overview
memprof often fails when ASLR entropy is too high ('sudo sysctl
vm.mmap_rnd_bits=32; ninja check-memprof'), which is the default setting
for newer versions of Ubuntu
(https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/commit/?h=hwe-6.5-next--2024.03.04-1--auto&id=6b522637c6a7dabd8530026ae933fb5ff17e877f).
This patch fixes the issue by moving the allocator base, analogously to
ASan (https://reviews.llvm.org/D148280).
Explanation from the ASan patch: when CONFIG_ARCH_MMAP_RND_BITS == 32,
it will frequently conflict with memprof's allocator, because the PIE
program segment base address of 0x555555555554 plus an ASLR shift of up
to ((2**32) * 4K == 0x100000000000) will sometimes exceed memprof's
hardcoded base address of 0x600000000000. We fix this by simply moving
the allocator base to 0x500000000000, which is below the PIE program
segment base address. This is cleaner than trying to move it to another
location that is sandwiched between the PIE program and library
segments, because if either of those grow too large, it will collide
with the allocator region.
Note that we will never need to change this base address again (unless
we want to increase the size of the allocator), because ASLR cannot be
set above 32-bits for x86-64 Linux (the PIE program segment and library
segments would collide with each other; see also ARCH_MMAP_RND_BITS_MAX
in https://github.com/torvalds/linux/blob/master/arch/x86/Kconfig).
With https://github.com/llvm/llvm-project/pull/83088, we now need the
runtimes to be built before running test if
COMPILER_RT_TEST_STANDALONE_BUILD_LIBS is true, since otherwise we
get failures running `ninja check-all` such as the following:
```
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.fuzzer-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-basic-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-fdr-x86_64.a: No such file or directory
```
This is a follow-up to 058e9b03 which started removing these checks
and it should make it easier to stop forcing COMPILER_RT_STANDALONE_BUILD
for runtimes builds in the future.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83651
this DCHECK was not valid for hwasan_load1, and was not necessary for
the. the function is written without any assumptions of alignment of the
pointer.
Updated the corelimit.cpp test to handle buildbots with RLIMIT_CORE
rlim_max already set to zero.
Original commit message:
I was trying to debug why `ninja check-compiler-rt` was taking so long
to run on my system and after some debugging it turned out that most of
the time was being spent generating core dumps.
On many current Linux systems, coredumps are no longer dumped in the CWD
but instead piped to a utility such as systemd-coredumpd that stores
them in a deterministic location. This can be done by setting the
kernel.core_pattern sysctl to start with a '|'. However, when using such
a setup the kernel ignores a coredump limit of 0 (since there is no file
being written) and we can end up piping many gigabytes of data to
systemd-coredumpd which causes the test suite to freeze for a long time.
While most piped coredump handlers do respect the crashing processes'
RLIMIT_CORE, this is notable not the case for Debian's systemd-coredump
due to a local patch that changes sysctl.d/50-coredump.conf to ignore
the specified limit and instead use RLIM_INFINITY
(https://salsa.debian.org/systemd-team/systemd/-/commit/64599ffe44f0d).
Fortunately there is a workaround: the kernel recognizes the magic value
of 1 for RLIMIT_CORE to disable coredumps when piping. One byte is also
too small to generate any coredump, so it effectively behaves as if we
had set the value to zero.
The alternative to using RLIMIT_CORE=1 would be to use prctl() with the
PR_SET_DUMPABLE flag, however that also prevents ptrace(), so makes it
impossible to attach a debugger.
Fixes: https://github.com/llvm/llvm-project/issues/45797
This reverts commit 0b9f19a9880eb786871194af116f223d2ad30c52.
Contains test changes from
https://github.com/llvm/llvm-project/pull/81677 that are simple test
changes. For the most part just makes the tests allow more flexibility
in the callstacks produced by asan.
Note: this PR has the exact changes from
https://github.com/llvm/llvm-project/pull/81677 in addition to a revert
commit to remove the ones that require other things from that PR. This
will be squashed, ofc. I just left things unsquashed for reviewing
pleasure :).
This is a non-functional-change
This ports the change from TSan
(0784b1eefa).
Testing notes: run 'sudo sysctl vm.mmap_rnd_bits=32; ninja check-msan'
before and after this patch.
N.B. aggressive ASLR may also cause the app to overlap with the
allocator region; for MSan, this was fixed in
af2bf86a37
Blocking that signal causes inter-blocking for profilers that monitor
threads through that signal.
Update tests accordingly to use an uncaught signal.
This is a recommit of 6f3f659ce9ab91002b4a490b0ce4b085981383cd with the
tests fixed.
Fix#83844 and #83561
MSan divides the virtual address space into APP, INVALID, SHADOW and
ORIGIN memory. The allocator usually just steals a bit of the APP
address space: typically the bottom portion of the PIE binaries section,
which works because the Linux kernel maps from the top of the PIE
binaries section. However, if ASLR is very aggressive, the binary may
end up mapped in the same location where the allocator wants to live;
this results in a segfault.
This patch adds in a MappingDesc::ALLOCATOR type and enforces that the
memory range for the allocator is not occupied by anything else.
Since the allocator range information is not readily available in
msan.h, we duplicate the information from msan_allocator.cpp.
Note: aggressive ASLR can also lead to a different type of failure,
where the PIE binaries/libraries are mapped entirely outside of the
APP/ALLOCATOR sections; that will be addressed in a separate patch
(https://github.com/llvm/llvm-project/pull/85142).
Reapply "[sanitizer][asan][win] Intercept _strdup on Windows instead of
strdup
This includes test changes and interface changes that are duplicated in
https://github.com/llvm/llvm-project/pull/81677
This reverts commit 03dc87e93937bf25a48bc77d0006c59f37b1855d.
Instead of explicitly disabling a feature by declaring the variable and
set it to false, this change supports the optional flags. I.e., you can
skip certain flags if you are not using it.
This optional feature supports both forms,
1. Value: A parameter for a feature. E.g., EnableRandomOffset
2. Type: A C++ type implementing a feature. E.g., ConditionVariableT
On the other hand, to access the flags will be through one of the
wrappers, BaseConfig/PrimaryConfig/SecondaryConfig/CacheConfig
(CacheConfig is embedded in SecondaryConfig). These wrappers have the
getters to access the value and the type. When adding a new feature, we
need to add it to `allocator_config.def` and mark the new variable with
either *_REQUIRED_* or *_OPTIONAL_* macro so that the accessor will be
generated properly.
In addition, also remove the need of `UseConditionVariable` to flip
on/off of condition variable. Now we only need to define the type of
condition variable.
I was trying to debug why `ninja check-compiler-rt` was taking so long
to run on my system and after some debugging it turned out that most of
the time was being spent generating core dumps.
On many current Linux systems, coredumps are no longer dumped in the CWD
but instead piped to a utility such as systemd-coredumpd that stores
them in a deterministic location. This can be done by setting the
kernel.core_pattern sysctl to start with a '|'. However, when using such
a setup the kernel ignores a coredump limit of 0 (since there is no file
being written) and we can end up piping many gigabytes of data to
systemd-coredumpd which causes the test suite to freeze for a long time.
While most piped coredump handlers do respect the crashing processes'
RLIMIT_CORE, this is notable not the case for Debian's systemd-coredump
due to a local patch that changes sysctl.d/50-coredump.conf to ignore
the specified limit and instead use RLIM_INFINITY
(https://salsa.debian.org/systemd-team/systemd/-/commit/64599ffe44f0d).
Fortunately there is a workaround: the kernel recognizes the magic value
of 1 for RLIMIT_CORE to disable coredumps when piping. One byte is also
too small to generate any coredump, so it effectively behaves as if we
had set the value to zero.
The alternative to using RLIMIT_CORE=1 would be to use prctl() with the
PR_SET_DUMPABLE flag, however that also prevents ptrace(), so makes it
impossible to attach a debugger.
Fixes: https://github.com/llvm/llvm-project/issues/45797
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83701
All these unit tests already include ${COMPILER_RT_GTEST_SOURCE} as an
input source file and the target llvm_gtest does not exist for
standalone builds. Currently the DEPS argument is ignored for standalone
builds so the missing target is not a problem, but as part of fixing a
build race for standalone builds I am planning to include those
dependencies in COMPILER_RT_TEST_STANDALONE_BUILD_LIBS configurations.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83649
On AArch64 with BTI, we have to start functions with the appropriate
BTI hint to indicate that the function is a valid call target.
To support interceptors with AArch64 BTI, add "BTI c".
Most sanitizers don't support static linking. One primary reason is the
incompatibility with interceptors. `GetTlsSize` is another reason.
asan/memprof use `__interception::DoesNotSupportStaticLinking`
(`_DYNAMIC` reference) to reject -static at link time. Port this
detector to other sanitizers. dfsan actually supports -static for
certain cases. Don't touch dfsan.
