Reverts llvm/llvm-project#98469
We can't add this dependency
```
OBJECT_LIBS RTSanitizerCommon
RTSanitizerCommonLibc
```
safestack is security hardening, and RTSanitizerCommon is too fat for
that.
MSan may segfault inside a signal handler, if MSan instrumentation is
trying to access thread-local storage that has already been destroyed.
This fixes the issue by blocking asychronous signals inside
MsanThread::TSDDtor. This is based on an idea suggested by Paul
Pluzhnikov (block async signals in MsanThread::Destroy()) and refined by
Vitaly Buka.
Note: ed8565cf0b64ea5e88cc94f321b1870bb105d09d changed *BlockSignals to
only block asynchronous signals, despite the name.
In extremely rare cases (estimated 1 in 3 million), minor allocations
that happen after the memory layout was checked in
InitializePlatformEarly() [1] may result in the memory layout
unexpectedly being incompatible in InitializePlatform(). We fix this by
adding another memory layout check (and opportunity to re-exec without
ASLR) in InitializePlatform().
To improve future debuggability, this patch also dumps the process map
if the memory layout is unexpectedly incompatible.
[1]
```
__sanitizer::InitializePlatformEarly();
__tsan::InitializePlatformEarly();
#if !SANITIZER_GO
InitializeAllocator(); // <-- ~8MB mmap'ed
ReplaceSystemMalloc();
#endif
if (common_flags()->detect_deadlocks)
ctx->dd = DDetector::Create(flags()); // <-- ~4MB mmap'ed
Processor *proc = ProcCreate(); // <-- ~1MB mmap'ed
ProcWire(proc, thr);
InitializeInterceptors(); <-- ~3MB mmap'ed
InitializePlatform();
```
Even with the `-u __safestack_init` link order fixed on Solaris, there
are still several safestack test issues left:
* While 540fd42c755f20f7b79c6c79493ec36d8cb9b3d3 enabled safestack on
Solaris in the driver unconditionally, it ignored that Solaris also
exists on SPARC and forgot to enable SPARC support for the runtime lib.
This patch fixes that.
- The tests fail to link with undefined references to
`__sanitizer_internal_memset` etc. These are from
`sanitizer_redefine_builtins.h`. Definitions live in
`sanitizer_libc.cpp.o`. This patch adds them to the safestack runtime
lib as is already the case e.g. for asan and ubsan. Why GNU ld allows
the link to complete with those references undefined is beyond me.
- The `pthread*.c` tests `FAIL` with
``` safestack CHECK failed:
/vol/llvm/src/llvm-project/local/compiler-rt/lib/safestack/safestack.cpp:227
size ```
The problem is that `pthread_attr_init` initializes the `stacksize`
attribute to 0, signifying the default. Unless explicitly overridded, it
stays that way. I think this is allowed by XPG7. Since safestack cannot
deal with this, I set `size` to the defaults documented in
`pthread_create(3C)`. Unfortunately, there's no macro for those values
outside of private `libc` headers.
- The Solaris `syscall` interface isn't stable. This is not just a
theoretical concern, but the syscalls have changed incompatibly several
times in the past. Therefore this patch switches the implementations of
`TgKill` (where `SYS_lwp_kill` doesn't exist on Solaris 11.4 anyway),
`Mmap`, `Munmap`, and `Mprotect` to the same `_REAL*` solution already
used in `sanitizer_solaris.cpp`. Instead of duplicating what's already
in `sanitizer_common`, it seems way better to me to just reuse those
implementations, though. A subsequent patch does just that.
With those changes, safestack compiles and all tests `PASS`, so the
tests are re-enabled for good.
Tested on `amd64-pc-solaris2.11`, `sparcv9-sun-solaris2.11`,
`x86_64-pc-linux-gnu`, and `sparc64-unknown-linux-gnu`.
When compiling for the ILP32E/LP64E ABIs, even on a RISC-V machine with
`i`, we should be using the ILP32E/LP64E save/restore routines, so use
the right preprocessor macro.
#94322 defines .preinit_array to initialize nsan early.
DT_PREINIT_ARRAY can only be used with the main executable. GNU ld would
complain when a DSO has .preinit_array .
MemoryAccessRangeT overestimates the size of the shadow region by 8x,
occasionally leading to assertion failure:
```
RawShadow* shadow_mem = MemToShadow(addr);
...
// Check that end of shadow is valid
if (!IsShadowMem(shadow_mem + size * kShadowCnt - 1)) {
DCHECK(IsShadowMem(shadow_mem + size * kShadowCnt - 1));
```
It is erroneous for two separate reasons:
- it uses kShadowCnt (== 4) instead of kShadowMultiplier (== 2)
- since shadow_mem is a RawShadow*, pointer arithmetic is multiplied by
sizeof(RawShadow) == 4
This patch fixes the calculation, and also improves the debugging
information.
The assertion error was observed on a buildbot
(https://lab.llvm.org/staging/#/builders/89/builds/656/steps/13/logs/stdio):
```
Bad shadow addr 0x3000000190bc (7fffffffe85f)
ThreadSanitizer: CHECK failed: tsan_rtl_access.cpp:690 "((IsShadowMem(shadow_mem + size * kShadowCnt - 1))) != (0)" (0x0, 0x0) (tid=2202676)
```
Notice that 0x3000000190bc is not the correct shadow for the end address
0x7fffffffe85f.
This error is more commonly observed on high-entropy ASLR systems, since
ASLR may be disabled (if the randomized memory layout is incompatible),
leading to an allocation near the boundaries of the high app memory
region (and therefore a shadow end that may be erroneously calculated to
be past the end of the shadow region). Also note that the assertion is
guarded by SANITIZER_DEBUG.
---------
Co-authored-by: Vitaly Buka <vitalybuka@gmail.com>
Some failures in Rtsan-aarch64-NoInstTest have been reported.
This reverts commit ed17431bf7489c87ea81a6a67af5c7c206da0080.
This reverts commit b81fcd01bde51eb8976b81a2c0c19fc0645cd2ff.
so that `clang -fsanitize=numerical -shared-libsan` will use
`libclang_rt.nsan.so` on Linux.
Shared runtime is preferred for some platforms (Android, Apple, Fuchsia;
though they are not supported yet) and helps plugin use cases (#98302).
* Update `ninja nsan` to build `libclang_rt.nsan.so`
* Fix `nsan.syms.extra`: `nsan_*` is unneeded. Add `__ubsan_*` so that
`-fsanitize=numerical,undefined -shared-libsan` works.
* Move allocation functions to `nsan_malloc_linux.cpp`. While Apple
platforms aren't supported yet, this separation makes it easier to add
Apple support.
* Delete interceptors for very obsoleted pvalloc/valloc but retain
memalign.
* Replace `HandleEarlyAlloc` with `DlsymAlloc`.
Pull Request: https://github.com/llvm/llvm-project/pull/98415
Modify the timer code to keep track of the maximum time that any call
takes.
Add a method to get a ScopedString representing the timer data. Allows
the testing of the timer on all platforms.
Add new unit tests for the maximum, and a lot of new tests.
On some systems (e.g., at least two AArch64 Linux instances), the
process map can have:
```
fffffffdf000-1000000000000 ... [stack]
```
instead of:
```
fffffffdf000- ffffffffffff
```
The stack top value is larger than `GetMaxUserVirtualAddress()`, which
violates the precondition that shadow memory calculations expect. This
patch fixes the issue by saturating off-by-one values (and also adds
checks for more flagrant violations).
This fixes an issue that was observed with DFSan on AArch64 Linux (with
high-entropy ASLR, resulting in ASLR being disabled on some runs):
```
==11057==ERROR: DataflowSanitizer failed to allocate 0x1600000800000 (387028101365760) bytes at address 4fffff800000 (errno: 12)
```
(https://lab.llvm.org/staging/#/builders/90/builds/552/steps/9/logs/stdio)
This was trying to allocate a shadow at `[0x4fffff800000, 0x4fffff800000
+ 0x1600000800000] = [0x4fffff800000, 0x1b00000000000]`. Notice that the
end of the shadow region - an invalid value - is equal to
`MEM_TO_SHADOW(0x1000000000000)`, where `MEM_TO_SHADOW` is defined as
`(mem ^ 0xB00000000000ULL)`.
Summary:
This patch adds initial support to build the `builtins` library for GPU
targets. Primarily this requires adding a few new architectures for
`amdgcn` and `nvptx64`. I built this using the following invocations.
```console
$ cmake ../compiler-rt -DCMAKE_C_COMPILER=clang
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Release -GNinja
-DCMAKE_C_COMPILER_TARGET=<nvptx64-nvidia-cuda|amdgcn-amd-amdhsa>
-DCMAKE_CXX_COMPILER_TARGET=<nvptx64-nvidia-cuda|amdgcn-amd-amdhsa>
-DCMAKE_C_COMPILER_WORKS=1 -DCMAKE_CXX_COMPILER_WORKS=1
-DLLVM_CMAKE_DIR=../cmake/Modules -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
-C ../compiler-rt/cmake/caches/GPU.cmake
```
Some pointers would be appreciated for how to test this using a standard
(non-default target only) build.
GPU builds are somewhat finnicky. We only expect this to be built with a
sufficiently new clang, as it's the only compiler that supports the
target and output we distribute. Distribution is done as LLVM-IR blobs
for now.
GPUs have little backward compatibility, so linking object files is
left to a future patch.
More work is necessary to build correctly for all targets and ship into
the correct clang resource directory. Additionally we need to use the
`libc` project's support for running unit tests.
They are checked in the parent CMakeLists in
`compiler_rt_build_runtime` and `compiler_rt_test_runtime`.
There are non-redundant checks when a sanitizer checks
for the presense of another one. They should not be removed.
This changes the behavior of `BlockSignals` and `ScopedBlockSignals` to
block only asynchronous signals.
This extension is intended to be used in a future fix for MSan (block
async signals during `MsanThread::Destroy`).
Follow on to #92460 (reporting old android builds failing) and #98264
(powerpc failing to discover gtests).
Getting back to stability by getting back down to a very basic set of
supported arches and OS's. In the future if there is demand we can build
it back up. Especially when I understand how to test these systems, or
have people who want to do the work on them.
---------
Co-authored-by: David Trevelyan <david.trevelyan@gmail.com>
Fixing test failures in #98256
There are two issues being fixed here:
```
ld.lld: error: undefined symbol: dlvsym
ld.lld: error: undefined symbol: dlvsym
```
Fixed by linking dl
```
FAILED: ... -lstdc++ -no-pie -pthread -latomic -m64
/opt/rh/devtoolset-11/root/usr/lib/gcc/ppc64-redhat-linux/11/../../../../bin/ld: cannot find /usr/lib64/libatomic.so.1
```
Fixed by removing atomic from the dependencies, I don't think we were
even using it.
I added in all of the dependencies used by the parent library in the
tests, hopefully getting ahead of any other issues, adding rt and m,
even if we haven't had issues with them yet
Follow up to #98219
This reverts commit
[14f7450](14f745074d)
Ensure that -pthread is explicitly linked when running the rtsan tests.
Issue this fixes:
```
FAILED: compiler-rt/lib/rtsan/tests/Rtsan-powerpc64le-NoInstTest /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests/Rtsan-powerpc64le-NoInstTest
cd /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests && /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./bin/clang++ RtsanNoInstTestObjects.rtsan_preinit.cpp.powerpc64le.o RtsanNoInstTestObjects.rtsan_test_context.cpp.powerpc64le.o RtsanNoInstTestObjects.rtsan_test_main.cpp.powerpc64le.o RtsanNoInstTestObjects.gtest-all.cc.powerpc64le.o RtsanNoInstTestObjects.gmock-all.cc.powerpc64le.o /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests/libRTRtsanTest.powerpc64le.a -o /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests/./Rtsan-powerpc64le-NoInstTest -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -resource-dir=/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./lib/../lib/clang/19 -Wl,-rpath,/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./lib/../lib/clang/19/lib/powerpc64le-unknown-linux-gnu -lstdc++ -no-pie -latomic -m64 -fno-function-sections
/usr/bin/ld: RtsanNoInstTestObjects.gtest-all.cc.powerpc64le.o: undefined reference to symbol 'pthread_getspecific@@GLIBC_2.17'
//usr/lib64/libpthread.so.0: error adding symbols: DSO missing from command line
```
Follow up to #92460
DEPS llvm_gtest is not used by compiler-rt,
compiler-rt compiles them with COMPILER_RT_GOOGLETEST_SOURCES.
This reverts commit e217f98ac3ee55755ee6ea3b97551cf8283528c9.
Introducing the main runtime of realtime sanitizer. For more
information, please see the [discourse
thread](https://discourse.llvm.org/t/rfc-nolock-and-noalloc-attributes/76837).
We have also put together a [reviewer support
document](https://github.com/realtime-sanitizer/radsan/blob/doc/review-support/doc/review.md)
to show what our intention is.
This review introduces the sanitizer backend. This includes:
* CMake build files (largely adapted from asan).
* Main RTSan architecture (the external API, thread local context,
stack).
* Interceptors.
* Many unit tests.
Please see the [reviewer support
document](https://github.com/realtime-sanitizer/radsan/blob/doc/review-support/doc/review.md)
for what our next steps are. We are moving in lockstep with this PR
#84983 for the codegen coming up next.
Note to reviewers: If you see support documentation mention "RADSan",
this was the "old acronym" for the realtime sanitizer, they refer to the
same thing. If you see it let us know and we can correct it (especially
in the llvm codebase)
---------
Co-authored-by: David Trevelyan <david.trevelyan@gmail.com>
Disables LSan in order to remove a 1% flake rate in this test.
There's some logic in LeakSanitizer and its integration into libFuzzer
that will disable LSan and re-run the input. This only happens when more
malloc()s are detected than free()s. Under high system load, this
appears to be possible as the "more mallocs than frees" is dependent on
walltime. In these instances, the number of runs ends up being `n + 1`,
which is undesirable.
This reverts commit f6616e99c71c15d530060346ec29c3246d7fc235.
Was causing buildbot failures on Windows. I also remember seeing a
AMDGPU buildbot failing somewhere on a warning as they have -Werror
enabled.
This reverts commit 2039e130649d8469bc85fa31ba7422d1d3739f90.
This relands commit 19cf8deabe1124831164987f1b9bf2f806c0a875.
Added some additional preprocessor directives to ensure that Host.cpp
only includes cpuid.h when being built on x86.
This patch unifies the implementation of getAMDProcessorTypeAndSubtype
between compiler-rt and LLVM.
This patch is intended to be a step towards pulling these functions out
into identical .inc files to better facilitate code sharing between LLVM
and compiler-rt.
This reverts commit 19cf8deabe1124831164987f1b9bf2f806c0a875.
This was causing quite a few buildbot failures (see the PR description).
Reverting for now while I have time to sort it out. Seems like it should
just be conditional preprocessor macros for X86 however.
This patch makes the host/feature detection in compiler-rt and LLVM use
the functions provided in cpuid.h(__get_cpuid, __get_cpuid_count)
instead of inline assembly. This simplifies the implementation and moves
any inline assembly away to a more common place.
A while ago, some similar cleanup was attempted, but this ended up
resulting in some compilation errors due to toolchain minimum version
issues (https://bugs.llvm.org/show_bug.cgi?id=30384). After the
reversion landed, there have been no attempts since then to clean up the
code, even though the minimum supported compilers now support the
relevant functions (https://godbolt.org/z/o1Mjz8ndv).
