guard_acquire is a helper function used to implement TSan's
__cxa_guard_acquire and pthread_once interceptors.
https://reviews.llvm.org/D54664 introduced optional hooks to support
cooperative multi-threading. It worked by marking the entire
guard_acquire call as a potentially blocking region.
In principle, only the contended case needs to be a potentially blocking
region. This didn't matter for __cxa_guard_acquire because the compiler
emits an inline fast path before calling __cxa_guard_acquire. That is,
once we call __cxa_guard_acquire at all, we know we're in the contended
case.
https://reviews.llvm.org/D107359 then unified the __cxa_guard_acquire
and pthread_once interceptors, adding the hooks to pthread_once.
However, unlike __cxa_guard_acquire, pthread_once callers are not
expected to have an inline fast path. The fast path is inside the
function.
As a result, TSan unnecessarily calls into the cooperative
multi-threading engine on every pthread_once call, despite applications
generally expecting pthread_once to be fast after initialization. Fix
this by deferring the hooks to the contended case inside guard_acquire.
In certain cases, the SDK headers declare
`OSSpinLock*` APIs as macros (instead of
functions), so users can be transparently
forwarded to non-deprecated APIs.
When enabled, building of TSan interceptors failed
because these macros interfere with the
interceptor machinery, i.e., they prevent proper
forward declaration of intercepted APIs.
In a previous change [1], we misattributed this to
the deprecation of `OSSpinLock*` APIs.
[1] ae484c21c05668f84b13304c28bc39f753e493de
rdar://143193907
Linking this runtime requires C++ ABI, which breaks -nostdlib++ builds.
However, UBSAN C++ runtime is only needed for CFI and VPTR checks.
Unblocks #120370.
For some targets uptr is mapped to unsigned int and size_t to unsigned
long and sizeof(int)==sizeof(long) holds. Still, these are distinct
types and type checking may fail. Therefore, replace uptr by
usize/SIZE_T wherever a size_t is expected.
Part of #116957
The INSTALL_BYPRODUCTS ExternalProject_Add() argument was only added in
CMake 3.26 and the current minimum is 3.20. Work around this by using an
explicit ExternalProject_Add_Step() call for the install step with a
BYPRODUCTS argument. We can't keep using the `install` name here since that
is reserved by the CMake implementation and results in errors when used.
This commit should be reverted once LLVM depends on CMake 3.26.
Pull Request: https://github.com/llvm/llvm-project/pull/115677
Using the build tree is somewhat fragile since the layout is not
guaranteed to be stable and means the tests are tightly coupled to the
libc++/libc++abi build tree layout. Instead update the ExternalProject
to install the library and headers and do not add the build tree to
the include/linker flags.
Pull Request: https://github.com/llvm/llvm-project/pull/115077
In C++ it's UB to use undeclared values as enum.
And there is support __ATOMIC_HLE_ACQUIRE and
__ATOMIC_HLE_RELEASE need such values.
So use `int` in TSAN interface, and mask out
irrelevant bits and cast to enum ASAP.
`ThreadSanitizer.cpp` already declare morder parameterd
in these functions as `i32`.
This may looks like a slight change, as we
previously didn't mask out additional bits for `fmo`,
and `NoTsanAtomic` call. But from implementation
it's clear that they are expecting exact enum.
Reverts llvm/llvm-project#115032
Reapply llvm/llvm-project#114724
In C++ it's UB to use undeclared values as enum.
And there is support `__ATOMIC_HLE_ACQUIRE` and
`__ATOMIC_HLE_RELEASE` need such values.
Internal implementation was switched to `class enum`,
where that behavior is defined. But interface is C, so
we just switch to `int`.
`DlSymAllocator` allows early allocations, when
tsan is not yet initialized, e.g. from `dlsym`.
All other sanitizers with interceptors already use
`DlSymAllocator`.
Existing `in_symbolizer()` tsan logic is very similar.
However, we need to keep both as `DlSymAllocator`
does not support large allocations, needed for Symolizer.
Fixes asan, msan crash on check added in #108684.
The #108684 includes reproducer of the issue.
Change interface of `GetThreadStackAndTls` to
set `tls_begin` and `tls_end` at the same time.
We can use an internal linkage variable to make it clear the variable is
not exported. The special section .preinit_array is a GC root.
Pull Request: https://github.com/llvm/llvm-project/pull/98584
In extremely rare cases (estimated 1 in 3 million), minor allocations
that happen after the memory layout was checked in
InitializePlatformEarly() [1] may result in the memory layout
unexpectedly being incompatible in InitializePlatform(). We fix this by
adding another memory layout check (and opportunity to re-exec without
ASLR) in InitializePlatform().
To improve future debuggability, this patch also dumps the process map
if the memory layout is unexpectedly incompatible.
[1]
```
__sanitizer::InitializePlatformEarly();
__tsan::InitializePlatformEarly();
#if !SANITIZER_GO
InitializeAllocator(); // <-- ~8MB mmap'ed
ReplaceSystemMalloc();
#endif
if (common_flags()->detect_deadlocks)
ctx->dd = DDetector::Create(flags()); // <-- ~4MB mmap'ed
Processor *proc = ProcCreate(); // <-- ~1MB mmap'ed
ProcWire(proc, thr);
InitializeInterceptors(); <-- ~3MB mmap'ed
InitializePlatform();
```
MemoryAccessRangeT overestimates the size of the shadow region by 8x,
occasionally leading to assertion failure:
```
RawShadow* shadow_mem = MemToShadow(addr);
...
// Check that end of shadow is valid
if (!IsShadowMem(shadow_mem + size * kShadowCnt - 1)) {
DCHECK(IsShadowMem(shadow_mem + size * kShadowCnt - 1));
```
It is erroneous for two separate reasons:
- it uses kShadowCnt (== 4) instead of kShadowMultiplier (== 2)
- since shadow_mem is a RawShadow*, pointer arithmetic is multiplied by
sizeof(RawShadow) == 4
This patch fixes the calculation, and also improves the debugging
information.
The assertion error was observed on a buildbot
(https://lab.llvm.org/staging/#/builders/89/builds/656/steps/13/logs/stdio):
```
Bad shadow addr 0x3000000190bc (7fffffffe85f)
ThreadSanitizer: CHECK failed: tsan_rtl_access.cpp:690 "((IsShadowMem(shadow_mem + size * kShadowCnt - 1))) != (0)" (0x0, 0x0) (tid=2202676)
```
Notice that 0x3000000190bc is not the correct shadow for the end address
0x7fffffffe85f.
This error is more commonly observed on high-entropy ASLR systems, since
ASLR may be disabled (if the randomized memory layout is incompatible),
leading to an allocation near the boundaries of the high app memory
region (and therefore a shadow end that may be erroneously calculated to
be past the end of the shadow region). Also note that the assertion is
guarded by SANITIZER_DEBUG.
---------
Co-authored-by: Vitaly Buka <vitalybuka@gmail.com>
Start functions with BTI in order to identify the function as a valid
branch target.
Also add the BTI marker to tsan_rtl_aarch64.S.
With this patch, libclang_rt.tsan.so can now be generated with
DT_AARCH64_BTI_PLT when built with -mbranch-protection=standard.
According to https://reviews.llvm.org/D114250
this was to handle Mac specific issue, however
the test is Linux only.
The test effectively prevents to lock main allocator
on fork, but we do that on Linux for other
sanitizers for years, and need to do the same
for TSAN to avoid deadlocks.
We use REAL() calls in interceptors, but
DEFINE_REAL_PTHREAD_FUNCTIONS has nothing to do
with them and only used for internal maintenance
threads.
This is done to avoid confusion like in #96456.
Sometime tsan runtimes calls, like
`__tsan_mutex_create ()`, need to store a stack
in the StackDepot, and the Depot may need to start
and maintenance thread.
Example:
```
__sanitizer::FutexWait ()
__sanitizer::Semaphore::Wait ()
__sanitizer::Mutex::Lock ()
__tsan::SlotLock ()
__tsan::SlotLocker::SlotLocker ()
__tsan::Acquire ()
__tsan::CallUserSignalHandler ()
__tsan::ProcessPendingSignalsImpl ()
__tsan::ProcessPendingSignals ()
__tsan::ScopedInterceptor::~ScopedInterceptor ()
___interceptor_mmap ()
pthread_create ()
__sanitizer::internal_start_thread ()
__sanitizer::(anonymous namespace)::CompressThread::NewWorkNotify ()
__sanitizer::StackDepotNode::store ()
__sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1, 20>::Put ()
__tsan::CurrentStackId ()
__tsan::MutexCreate ()
__tsan_mutex_create ()
```
pthread_create() implementation may hit other
interceptors recursively, which may invoke
ProcessPendingSignals, which deadlocks.
Alternative solution could be block interceptors
closer to TSAN runtime API function, like
`__tsan_mutex_create`, or just before
`StackDepotPut``, but it's not needed for most
calls, only when new thread is created using
`real_pthread_create`.
I don't see a reasonable way to create a
regression test.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
In glibc versions before 2.33. `libc_nonshared.a` defines
`__fxstat/__fxstat64` but there is no `fstat/fstat64`. glibc 2.33 added
`fstat/fstat64` and obsoleted `__fxstat/__fxstat64`. Ports added after
2.33 do not provide `__fxstat/__fxstat64`, so our `fstat/fstat64`
interceptors using `__fxstat/__fxstat64` interceptors would lead to
runtime failures on such ports (LoongArch and certain RISC-V ports).
Similar to https://reviews.llvm.org/D118423, refine the conditions that
we define fstat{,64} interceptors. `fstat` is supported by musl/*BSD
while `fstat64` is glibc only.
Fixes#83844.
This PR adds callbacks to mark futex syscalls as blocking. Unfortunately
we didn't have a mechanism before to mark syscalls as a blocking call,
so I had to implement it, but it mostly reuses the `BlockingCall`
implementation
[here](96819daa3d/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp (L362-L380)).
The issue includes some information but this issue was discovered
because Rust uses futexes directly. So most likely we need to update
Rust as well to use these callbacks.
Also see the latest comments in #85188 for some context.
I also sent another PR #84162 to mark `pthread_*_lock` calls as
blocking.
Fixes#83561.
When a thread is blocked on a mutex and we send an async signal to that
mutex, it never arrives because tsan thinks that `pthread_mutex_lock` is
not a blocking function. This patch marks `pthread_*_lock` functions as
blocking so we can successfully deliver async signals like `SIGPROF`
when the thread is blocked on them.
See the issue also for more details. I also added a test, which is a
simplified version of the compiler explorer example I posted in the
issue.
Please let me know if you have any other ideas or things to improve!
Happy to work on them.
Also I filed #83844 which is more tricky because we don't have a libc
wrapper for `SYS_futex`. I'm not sure how to intercept this yet. Please
let me know if you have ideas on that as well. Thanks!
Reverting #85188 with follow up patches.
This reverts commit 362d26366d0175f01ffb6085eb747a6e40f01147.
This reverts commit c9bdeabdf4b46fbf1f6a9fcbf9cd61d460b18c08.
This reverts commit 6bc6e1ace9fa8453e164fa04b5d9acd5a77e089a.
This reverts commit 01fa550ff654d6724e6da54c877032baeddff14b.
This reverts commit ddcbab37ac0e5743a8d39be3dd48d967f4c85504.
