15246 Commits

Author SHA1 Message Date
Philip Reames
9b955f77a1 Attempt to stablize compiler-rt/test/asan/TestCases/strncpy-overflow.cpp
This attempts to adjust the test to still exercise the expected codepath after D115904.  This test is fundementally rather fragile.

Unfortunately, I have not been able to confirm this workaround either does, or does not, work.  Attempting check-all with compiler-rt blows through an additional 30GB of disk space so my build config which exceeds my local disk space.
2021-12-20 17:51:11 -08:00
Vitaly Buka
700d16b6d6 [tsan] Fix Darwin crash after D115759
Remove global constructor which may or may not be needed for Android,
at it breaks Darwin now.
2021-12-20 17:05:41 -08:00
Dmitry Vyukov
4c5476b066 tsan: fix NULL deref in TraceSwitchPart
There is a small chance that the slot may be not queued in TraceSwitchPart.
This can happen if the slot has kEpochLast epoch and another thread
in FindSlotAndLock discovered that it's exhausted and removed it from
the slot queue. kEpochLast can happen in 2 cases: (1) if TraceSwitchPart
was called with the slot locked and epoch already at kEpochLast,
or (2) if we've acquired a new slot in SlotLock in the beginning
of the function and the slot was at kEpochLast - 1, so after increment
in SlotAttachAndLock it become kEpochLast.

If this happens we crash on ctx->slot_queue.Remove(thr->slot).
Skip the requeueing if the slot is not queued.
The slot is exhausted, so it must not be ctx->slot_queue.

The existing stress test triggers this with very small probability.
I am not sure how to make this condition more likely to be triggered,
it evaded lots of testing.

Depends on D116040.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D116041
2021-12-20 18:55:51 +01:00
Dmitry Vyukov
2eb3e20461 tsan: fix deadlock during race reporting
SlotPairLocker calls SlotLock under ctx->multi_slot_mtx.
SlotLock can invoke global reset DoReset if we are out of slots/epochs.
But DoReset locks ctx->multi_slot_mtx as well, which leads to deadlock.

Resolve the deadlock by removing SlotPairLocker/multi_slot_mtx
and only lock one slot for which we will do RestoreStack.
We need to lock that slot because RestoreStack accesses the slot journal.
But it's unclear why we need to lock the current slot.
Initially I did it just to be on the safer side (but at that time
we dit not lock the second slot, so it was easy just to lock the current slot).

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D116040
2021-12-20 18:52:48 +01:00
Ellis Hoag
ac719d7c9a [InstrProf] Don't profile merge by default in lightweight mode
Profile merging is not supported when using debug info profile
correlation because the data section won't be in the binary at runtime.
Change the default profile name in this mode to `default_%p.proflite` so
we don't use profile merging.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115979
2021-12-20 09:51:49 -08:00
Julian Lettner
64f4041725 [TSan][Darwin] Fix shadow mapping for iOS simulator on Apple Silicon
With the introduction of Apple Silicon `defined(__aarch64__)` is not a
reliable way to check for the platform anymore.

We want to use the "normal" `Mapping48AddressSpace` mapping everywhere
except devices, including the iOS simulators on AS.

Relevant revisions:
https://reviews.llvm.org/D35147
https://reviews.llvm.org/D86377
https://reviews.llvm.org/D107743
https://reviews.llvm.org/D107888

Differential Revision: https://reviews.llvm.org/D115843
2021-12-17 15:59:43 -08:00
Julian Lettner
4399f3b6b0 [TSan][Darwin] Make malloc_size interceptor more robust
Previously we would crash in the TSan runtime if the user program passes
a pointer to `malloc_size()` that doesn't point into app memory.

In these cases, `malloc_size()` should return 0.

For ASan, we fixed a similar issue here:
https://reviews.llvm.org/D15008

Radar-Id: rdar://problem/86213149

Differential Revision: https://reviews.llvm.org/D115947
2021-12-17 15:38:08 -08:00
Ellis Hoag
3429c7c328 [InstrProf][NFC] Require zlib for debug info tests
There was a build failure on the `instrprof-debug-info-correlate.c` test
because zlib was missing so we need to require it to run the test.

Differential Revision: https://reviews.llvm.org/D115970
2021-12-17 14:06:05 -08:00
Ellis Hoag
65d7fd0239 [Try2][InstrProf] Add Correlator class to read debug info
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).

This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

This was first landed in https://reviews.llvm.org/D114566 but had to be reverted due to build errors.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115915
2021-12-17 10:45:59 -08:00
Martin Storsjö
cc324af0d6 Revert "Port __sanitizer::StopTheWorld to Windows"
This reverts commit 3f5f687e2e8bf8b68c8a6ae0e000a8a78bfa70e5.

That commit broke building for mingw, where the sanitizers are
built with -nostdinc++, while the added source file includes
the C++ standard library's <algorithm>.

Additionally, the new code fails to build for i386, as it
unconditionally uses the CONTEXT member Rsp.
2021-12-17 20:41:14 +02:00
Nico Weber
5dd8ff7380 [asan/mac] Fix remaining -Wformat warnings
AARCH64_GET_REG() is used to initialize uptrs, and after D79132
the ptrauth branch of its implementation explicitly casts to uptr.

The non-ptrauth branch returns ucontext->uc_mcontext->__ss.__fp (etc),
which has either type void* or __uint64_t (ref usr/include/mach/arm/_structs.h)
where __uint64_t is a unsigned long long (ref usr/include/arm/_types.h).
uptr is an unsigned long (ref
compiler-rt/lib/sanitizer_common/sanitizer_internal_defs.h). So explicitly
cast to uptr in this branch as well, so that AARCH64_GET_REG() has a
well-defined type.

Then change DUMPREGA64() tu use %lx instead of %llx since that's the right type
for uptr. (Most other places in compiler-rt print uptrs as %p and cast the arg
to (void*), but there are explicit 0x%016 format strings in the surroundings,
so be locally consistent with that.)

No behavior change, in the end it's just 64-bit unsigneds by slightly different
names.
2021-12-17 09:44:09 -05:00
Nico Weber
44f0e14537 [asan/mac] Fix a few -Wformat warnings
All other places cast uptrs to (void*) when printing it via
%p after D113099, so do that here too.
2021-12-17 09:17:52 -05:00
Nico Weber
d412f35395 [asan/mac] Fix a -Wformat warning
internal_getpid() is manually cast to int in all other places that
pass its result to a printf string, so do that here too for now.
2021-12-17 09:17:52 -05:00
Clemens Wasser
3f5f687e2e Port __sanitizer::StopTheWorld to Windows
This also makes the sanitizer_stoptheworld_test cross-platform by using the STL, rather than pthread.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115204
2021-12-16 23:29:15 -08:00
Vitaly Buka
5ed03c1e10 [sanitizer] Simplify sanitizer_stoptheworld_test 2021-12-16 23:24:17 -08:00
Ellis Hoag
bdc68ee70f Revert "[InstrProf] Add Correlator class to read debug info"
Also reverts an attempt to fix the build errors https://reviews.llvm.org/D115911

The original diff https://reviews.llvm.org/D114566 causes some build
errors that I need to investigate.

https://lab.llvm.org/buildbot/#/builders/118/builds/7037

This reverts commit 95946d2f8589b5450d8f9a9e9b4f7adf44386f8b.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115913
2021-12-16 16:28:19 -08:00
Vitaly Buka
b4618f576e [NFC][sanitizer] Remove unused variable 2021-12-16 15:24:45 -08:00
Ellis Hoag
95946d2f85 [InstrProf] Add Correlator class to read debug info
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).

This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D114566
2021-12-16 15:18:12 -08:00
Ellis Hoag
58d9c1aec8 [Try2][InstrProf] Attach debug info to counters
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.

Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

The original diff https://reviews.llvm.org/D114565 was reverted because of the `Instrumentation/InstrProfiling/debug-info-correlate.ll` test, which is fixed in this commit.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115693
2021-12-16 14:20:30 -08:00
Vitaly Buka
bc9688f4e4 [sanitizer] Define SANITIZER_ARM SANITIZER_X64 SANITIZER_ARM64 2021-12-16 13:26:33 -08:00
Vitaly Buka
72847ec8bf [sanitizer] Clang-format sanitizer_platform.h 2021-12-16 13:26:33 -08:00
Emily Shi
fcfcc64ce9 [compiler-rt][NFC] add note to interface_symbols_linux.cpp about darwin
There were changes made to the linux version of this test that were not made for darwin
(see https://reviews.llvm.org/D115837) and this caused downstream failures.

Adding comment to this test to remind people to edit interface_symbols_darwin.cpp.
There is the reverse of this comment in the darwin file to remind us to edit the linux version already.

Differential Revision: https://reviews.llvm.org/D115899
2021-12-16 13:08:03 -08:00
Emily Shi
db38527966 [compiler-rt] removed memory access callback from asan interface for darwin
These symbols were introduced in https://reviews.llvm.org/D114558 but were not properly guarded for mac which caused downstream CI failures

Reviewed By: kubamracek, yln

Differential Revision: https://reviews.llvm.org/D115837
2021-12-16 13:03:47 -08:00
Clemens Wasser
9991ab5df9 [sanitizer] Portable sanitizer_stoptheworld_test
Part of D115204
2021-12-16 12:53:53 -08:00
Vitaly Buka
9cf0ea3558 [sanitizer] Clang format before D115204 2021-12-16 12:52:36 -08:00
Julian Lettner
dcf11af24b [Sanitizer] Disable LSan when testing on iOS simulators
LSan (`ASAN_OPTIONS=detect_leaks=1`) is supported on macOS, but disabled
by default on Darwin (`SANITIZER_MAC`):
```
COMMON_FLAG(bool, detect_leaks, !SANITIZER_MAC, "Enable memory leak detection.")
```

We enable it here for ASan tests to prevent regressions (per comment).
However, LSan is not supported for the iOS simulator and the tests fail
when it is enabled.

Make this "Is macOS?" check more precise since the current one (`Darwin
&& x86_64`) has two issues:
 * Includes the simulators
 * Excludes macOS on Apple Silicon

This will allow us to (re)enable simulator testing on Green dragon to
give open source better feedback about sanitizer changes:
https://green.lab.llvm.org

rdar://86529234

Differential Revision: https://reviews.llvm.org/D115816
2021-12-16 12:37:45 -08:00
Clemens Wasser
66d1fdb29b [lsan] Format CAN_SANITIZE_LEAKS defines
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115584
2021-12-16 11:55:16 -08:00
Matt Kulukundis
406b538dea Add a flag to force tsan's background thread
Reviewed By: dvyukov, vitalybuka

Differential Revision: https://reviews.llvm.org/D115759
2021-12-16 11:47:33 -08:00
Julian Lettner
3a1eb1cf2a [TSan] Make test fail more predictably
This test would hang when the system ran out of resources and we fail to
create all 300 threads.

Differential Revision: https://reviews.llvm.org/D115845
2021-12-16 08:33:32 -08:00
Vitaly Buka
c209b7e3d5 [sanitizer] Update global_symbols.txt 2021-12-16 00:15:20 -08:00
Julian Lettner
8f1ea2e85c [TSan][Darwin] Fix CheckAndProtect() for MappingAppleAarch64
In the new TSan runtime refactoring this line was changed:
```
ProtectRange(MetaShadowEnd(), TraceMemBeg());
-->
ProtectRange(MetaShadowEnd(), HeapMemBeg());
```

But for `MappingAppleAarch64` the app heap comes before the shadow,
resulting in:
```
CHECK failed: tsan_platform_posix.cpp:83 "((beg)) <= ((end))" (0xe00000000, 0x200000000)
```

rdar://86521924

Differential Revision: https://reviews.llvm.org/D115834
2021-12-15 18:03:58 -08:00
Julian Lettner
a7cbe198ce [Darwin] Remove workaround for symbolication in iOS simulator runtimes
A while ago we added some code to the sanitizer runtimes for iOS
simulators to allow `atos` (external process) to inspect the sanitized
process during report generation to enable symbolication.  This was done
by setting the `__check_mach_ports_lookup` env var early during process
startup which came with a couple of complications.

This workaround is not required anymore and removing it fixes TSan in
the iOS simulator after the new TSan runtime landed.
(https://reviews.llvm.org/D112603)

Relevant/reverted revisions:
https://reviews.llvm.org/D78178
https://reviews.llvm.org/D78179
https://reviews.llvm.org/D78525

rdar://86472733

Differential Revision: https://reviews.llvm.org/D115767
2021-12-15 17:22:13 -08:00
Stephen Hines
cce4a7258b [compiler-rt][AArch64] Add a workaround for Exynos 9810
Big.LITTLE Heterogeneous architectures, as described by ARM [1],
require that the instruction set architecture of the big and little
cores be compatible. However, the Samsung Exynos 9810 is known to
have different ISAs in its core.
According to [2], some cores are ARMv8.2 and others are ARMv8.0.

Since LSE is for ARMv8.1 and later, it should be disabled
for this broken CPU.

[1] https://developer.arm.com/documentation/den0024/a/big-LITTLE-Technology
[2] https://github.com/golang/go/issues/28431

Patch by: Byoungchan Lee (byoungchan.lee@gmx.com)
Reviewed By: srhines

Differential Revision: https://reviews.llvm.org/D114523
2021-12-14 19:51:46 -08:00
Lang Hames
5255ca9726 [llvm-jitlink] Update handling of library options.
Adds -L<search-path> and -l<library> options that are analogous to ld's
versions.

Each instance of -L<search-path> or -l<library> will apply to the most recent
-jd option on the command line (-jd <name> creates a JITDylib with the given
name). Library names will match against JITDylibs first, then llvm-jitlink will
look through the search paths for files named <search-path>/lib<library>.dylib
or <search-path>/lib<library>.a.

The default "main" JITDylib will link against all JITDylibs created by -jd
options, and all JITDylibs will link against the process symbols (unless
-no-process-symbols is specified).

The -dlopen option is renamed -preload, and will load dylibs into the JITDylib
for the ORC runtime only.

The effect of these changes is to make it easier to describe a non-trivial
program layout to llvm-jitlink for testing purposes. E.g. the following
invocation describes a program consisting of three JITDylibs: "main" (created
implicitly) containing main.o, "Foo" containing foo1.o and foo2.o, and linking
against library "bar" (not a JITDylib, so it must be a .dylib or .a on disk)
and "Baz" (which is a JITDylib), and "Baz" containing baz.o.

llvm-jitlink \
  main.o \
  -jd Foo foo1.o foo2.o -L${HOME}/lib -lbar -lBaz
  -jd Baz baz.o
2021-12-15 13:49:30 +11:00
Kirill Stoimenov
da7e9e3410 [ASan] Added NO_EXEC_STACK_DIRECTIVE to assembly callback file.
This is present in our assembly files. It should fix decorate_proc_maps.cpp failures because of shadow memory being allocated as executable.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115552
2021-12-14 18:34:17 -08:00
Kirill Stoimenov
c13524856b [ASan] Shared optimized callbacks implementation.
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114558
2021-12-14 15:55:14 +00:00
Ellis Hoag
c809da7d9c Revert "[InstrProf] Attach debug info to counters"
This reverts commit 800bf8ed29fbcaa9436540e83bc119ec92e7d40f.

The `Instrumentation/InstrProfiling/debug-info-correlate.ll` test was
failing because I forgot the `llc` commands are architecture specific.
I'll follow up with a fix.

Differential Revision: https://reviews.llvm.org/D115689
2021-12-13 18:15:17 -08:00
Ellis Hoag
800bf8ed29 [InstrProf] Attach debug info to counters
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.

Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D114565
2021-12-13 17:51:22 -08:00
Arthur Eubanks
fff876c20e Revert "[ASan] Shared optimized callbacks implementation."
This reverts commit ebc31d2e7297d4002c72e5c7b9761c9559428ffc.

Causes test failures on Windows, see comments on D114558.
2021-12-13 15:26:55 -08:00
Arthur Eubanks
a357fad868 Revert "[ASan] Added NO_EXEC_STACK_DIRECTIVE to assembly callback file."
This reverts commit e31a5e0ba540fcebf115d2fcde2be8a250c485c1.

Reverting dependent change D114558.
2021-12-13 15:26:55 -08:00
Noah Shutty
34491ca729 [Symbolizer][Debuginfo] Add debuginfod client to llvm-symbolizer.
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.

Reviewed By: jhenderson, vitalybuka

Differential Revision: https://reviews.llvm.org/D113717
2021-12-13 23:00:32 +00:00
Vitaly Buka
75504789bd [NFC][asan] XFAIL a test on THUMB
Followup to D114934
2021-12-13 12:07:13 -08:00
Petr Hosek
a65389849d Revert "[CMake] Update internal libc++ configuration"
This reverts commit e286fefa8caaddea521685a8de1879acca82c67c since
the issue affecting the sanitizer-ppc64le-linux builder is resolved.
2021-12-13 11:14:00 -08:00
Petr Hosek
e286fefa8c [CMake] Update internal libc++ configuration
This is an attempt to address 458ead66dc378ee5b9cbeb453a9cb7c3de768b2e
by better matching the previous configuration.
2021-12-13 11:07:53 -08:00
Dmitry Vyukov
9fb8058a80 tsan: enable the new runtime
This enables the new runtime (D112603) by default.

Depends on D112603.

Differential Revision: https://reviews.llvm.org/D115624
2021-12-13 12:50:13 +01:00
Dmitry Vyukov
b332134921 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-12-13 12:48:34 +01:00
Dmitry Vyukov
b088833375 tsan: deflake dlopen_static_tls.cpp
Currently the test calls dlclose in the thread
concurrently with the main thread calling a function
from the dynamic library. This is not good.
Wait for the main thread to call the function
before calling dlclose.

Depends on D115612.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D115613
2021-12-13 12:01:40 +01:00
Dmitry Vyukov
7de546e9e8 tsan: deflake flush_memory.cpp
The test contains a race and checks that it's detected.
But the race may not be detected since we are doing aggressive flushes
and if the state flush happens between racing accesses, tsan won't
detect the race). So return 1 to make the test deterministic
regardless of the race.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D115612
2021-12-13 12:01:30 +01:00
Florian Mayer
edd2b99a57 [sanitizers] include build ids in stacks on linux.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D114294
2021-12-10 14:24:03 -08:00
Kirill Stoimenov
e31a5e0ba5 [ASan] Added NO_EXEC_STACK_DIRECTIVE to assembly callback file.
This is present in our assembly files. It should fix decorate_proc_maps.cpp failures because of shadow memory being allocated as executable.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115552
2021-12-10 21:53:13 +00:00