Fixes error: ISO C++20 considers use of overloaded operator '==' (with
operand types 'MacosVersion' and 'MacosVersion') to be ambiguous despite
there being a unique best viable function
[-Werror,-Wambiguous-reversed-operator].
This converts the comparison operator from a non-symmetric operator
(const VersionBase<VersionType>& (as "this") and const VersionType &).
into a symmetric operator
Relands #135068
Co-authored-by: Ivan Tadeu Ferreira Antunes Filho <antunesi@google.com>
In powerpc64-unknown-linux-musl, signal.h does not include asm/ptrace.h,
which causes "member access into incomplete type 'struct pt_regs'"
errors. Include the header explicitly to fix this.
Also in sanitizer_linux_libcdep.cpp, there is a usage of TlsPreTcbSize
which is not defined in such a platform. Guard the branch with macro.
This adds an experimental flag that will keep track of where the manual memory poisoning (`__asan_poison_memory_region`) is called from, and print the stack trace if the poisoned region is accessed. (Absent this flag, ASan will tell you what code accessed a poisoned region, but not which code set the poison.)
This implementation performs best-effort record keeping using ring buffers, as suggested by Vitaly. The size of each ring buffer is set by the `poison_history_size` flag.
Summary:
Previously, we removed the special handling for the code object version
global. I erroneously thought that this meant we cold get rid of this
weird `-Xclang` option. However, this also emits an LLVM IR module flag,
which will then cause linking issues.
Fixes error: ISO C++20 considers use of overloaded operator '==' (with
operand types 'MacosVersion' and 'MacosVersion') to be ambiguous despite
there being a unique best viable function
[-Werror,-Wambiguous-reversed-operator].
This converts the comparison operator from a non-symmetric operator
(const VersionBase<VersionType>& (as "this") and const VersionType &).
into a symmetric operator
This is an optional mechanism that automatically detects roots. It's a best-effort mechanism, and its main goal is to *avoid* pointing at the message pump function as a root. This is the function that polls message queue(s) in an infinite loop, and is thus a bad root (it never exits).
High-level, when collection is requested - which should happen when a server has already been set up and handing requests - we spend a bit of time sampling all the server's threads. Each sample is a stack which we insert in a `PerThreadCallsiteTrie`. After a while, we run for each `PerThreadCallsiteTrie` the root detection logic. We then traverse all the `FunctionData`, find the ones matching the detected roots, and allocate a `ContextRoot` for them. From here, we special case `FunctionData` objects, in `__llvm_ctx_profile_get_context, that have a `CtxRoot` and route them to `__llvm_ctx_profile_start_context`.
For this to work, on the llvm side, we need to have all functions call `__llvm_ctx_profile_release_context` because they _might_ be roots. This comes at a slight (percentages) penalty during collection - which we can afford since the overall technique is ~5x faster than normal instrumentation. We can later explore conditionally enabling autoroot detection and avoiding this penalty, if desired.
Note that functions that `musttail call` can't have their return instrumented this way, and a subsequent patch will harden the mechanism against this case.
The mechanism could be used in combination with explicit root specification, too.
The try-compile mechanism requires that `CMAKE_REQUIRED_FLAGS` is a
space-separated string instead of a list of flags. The original code
expanded `BUILTIN_FLAGS` into `CMAKE_REQUIRED_FLAGS` as a
space-separated string and then would overwrite `CMAKE_REQUIRED_FLAGS`
with `TARGET_${arch}_CFLAGS` prepended to the unexpanded
`BUILTIN_CFLAGS_${arch}`. This resulted in the first two arguments being
passed into the try-compile invocation, but dropping the other arguments
listed in `BUILTIN_CFLAGS_${arch}`.
This patch appends `TARGET_${arch}_CFLAGS` and `BUILTIN_CFLAGS_${arch}` to
`CMAKE_REQUIRED_FLAGS` before expanding CMAKE_REQUIRED_FLAGS as a
space-separated string. This passes any pre-set required flags, in addition to
all of the builtin and target flags to the Float16 detection.
With the refactoring in PR #133744, `__llvm_ctx_profile_start_context` doesn't need to be marked `SANITIZER_NO_THREAD_SAFETY_ANALYSIS` because `tryStartContextGivenRoot` (where the bulk of the logic went) is.
An initial patch for supporting automated root detection. The auto-detector is introduced subsequently, but this patch introduces a datastructure for capturing sampled stacks, per thread, in a trie, and inferring from such samples which functions are reasonable roots.
Summary:
When we were first porting to COV5, this lead to some ABI issues due to
a change in how we looked up the work group size. Bitcode libraries
relied on the builtins to emit code, but this was changed between
versions. This prevented the bitcode libraries, like OpenMP or libc,
from being used for both COV4 and COV5. The solution was to have this
'none' functionality which effectively emitted code that branched off of
a global to resolve to either version.
This isn't a great solution because it forced every TU to have this
variable in it. The patch in
https://github.com/llvm/llvm-project/pull/131033 removed support for
COV4 from OpenMP, which was the only consumer of this functionality.
Other users like HIP and OpenCL did not use this because they linked the
ROCm Device Library directly which has its own handling (The name was
borrowed from it after all).
So, now that we don't need to worry about backward compatibility with
COV4, we can remove this special handling. Users can still emit COV4
code, this simply removes the special handling used to make the OpenMP
device runtime bitcode version agnostic.
This PR partially reverts 83e180c and instead opts to not define the GPU entry
point on Darwin platforms. Marking `__llvm_write_custom_profile` as used
was causing issues on embedded platforms.
In the caching part of the secondary path, when about to try to release
memory to the OS, we always wait while acquiring the lock. However, if
multiple threads are attempting this at the same time, all other threads
will likely do nothing when the release call is made. Change the
algorithm to skip the release if there is another release in process.
Also, pull the lock outside the releaseOlderThan function. This is so
that in the store path, we use the tryLock and skip if another thread is
releasing. But in the path where a forced release call is being made,
that call will wait for release to complete which guarantees that all
entries are released when requested.
Introduced a cmake option that is disabled by default that suppresses
searching via the PATH variable for a symbolizer. The option will be
enabled for downstream builds where the user will need to specify the
symbolizer path more explicitly, e.g., by using ASAN_SYMBOLIZER_PATH.
guard_acquire is a helper function used to implement TSan's
__cxa_guard_acquire and pthread_once interceptors.
https://reviews.llvm.org/D54664 introduced optional hooks to support
cooperative multi-threading. It worked by marking the entire
guard_acquire call as a potentially blocking region.
In principle, only the contended case needs to be a potentially blocking
region. This didn't matter for __cxa_guard_acquire because the compiler
emits an inline fast path before calling __cxa_guard_acquire. That is,
once we call __cxa_guard_acquire at all, we know we're in the contended
case.
https://reviews.llvm.org/D107359 then unified the __cxa_guard_acquire
and pthread_once interceptors, adding the hooks to pthread_once.
However, unlike __cxa_guard_acquire, pthread_once callers are not
expected to have an inline fast path. The fast path is inside the
function.
As a result, TSan unnecessarily calls into the cooperative
multi-threading engine on every pthread_once call, despite applications
generally expecting pthread_once to be fast after initialization. Fix
this by deferring the hooks to the contended case inside guard_acquire.
This generalizes https://github.com/llvm/llvm-project/pull/131975 to non-32-bit Linux (i.e., 64-bit Linux).
This works around an edge case in 64-bit Linux, whereby the memory layout is incompatible if the stack size is unlimited AND ASLR entropy is 31+ bits (see https://github.com/google/sanitizers/issues/856#issuecomment-2747076811).
More generally, this "re-exec without ASLR if layout is incompatible" is a hammer that can work around most shadow mapping issues, without incurring the overhead of using a dynamic shadow.
Function definition added in #131975 was missing 'Try' (do or do not ...). My guess is buildbots
mostly didn't trip because usage was gated by 'if (sizeof(uptr) ==
32)', which is rare among buildbots.
A bare-bones version of LLVM's unique_function: this behaves like a
std::unique_function, except that it supports move only callable types.
This will be used in upcoming improvements to the ORC runtime.
The "extensible_" prefix on these files was inherited from LLVM, where it
distinguished the dynamic RTTI APIs from the LLVM's custom static RTTI APIs.
In the ORC runtime these files will be used to hold all of our RTTI APIs
(the current dynamic ones, and any static ones added in the future), so we
shouldn't use this prefix.
Observed in Wine when trying to intercept `ExitThread`, which forwards
to `ntdll.RtlExitUserThread`.
`gdb` interprets it as `xchg %ax,%ax`.
`llvm-mc` outputs simply `nop`.
```
==Asan-i386-calls-Dynamic-Test.exe==964==interception_win: unhandled instruction at 0x7be27cf0: 66 90 55 89 e5 56 50 8b
```
```
Wine-gdb> bt
#0 0x789a1766 in __interception::GetInstructionSize (address=<optimized out>, rel_offset=<optimized out>) at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/interception/interception_win.cpp:983
#1 0x789ab480 in __sanitizer::SharedPrintfCode(bool, char const*, char*) () at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_printf.cpp:311
#2 0x789a18e7 in __interception::OverrideFunctionWithHotPatch (old_func=2078440688, new_func=2023702608, orig_old_func=warning: (Internal error: pc 0x792f1a2c in read in CU, but not in symtab.)warning: (Error: pc 0x792f1a2c in address map, but not in symtab.)0x792f1a2c) at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/interception/interception_win.cpp:1118
#3 0x789a1f34 in __interception::OverrideFunction (old_func=2078440688, new_func=2023702608, orig_old_func=warning: (Internal error: pc 0x792f1a2c in read in CU, but not in symtab.)warning: (Error: pc 0x792f1a2c in address map, but not in symtab.)0x792f1a2c) at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/interception/interception_win.cpp:1224
#4 0x789a24ce in __interception::OverrideFunction (func_name=0x78a0bc43 <vtable for __asan::AsanThreadContext+1163> "ExitThread", new_func=2023702608, orig_old_func=warning: (Internal error: pc 0x792f1a2c in read in CU, but not in symtab.)warning: (Error: pc 0x792f1a2c in address map, but not in symtab.)0x792f1a2c) at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/interception/interception_win.cpp:1369
#5 0x789f40ef in __asan::InitializePlatformInterceptors () at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/asan/asan_win.cpp:190
#6 0x789e0c3c in __asan::InitializeAsanInterceptors () at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:802
#7 0x789ee6b5 in __asan::AsanInitInternal () at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:442
#8 0x789eefb0 in __asan::AsanInitFromRtl () at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:522
#9 __asan::AsanInitializer::AsanInitializer (this=<optimized out>) at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:542
#10 __cxx_global_var_init () at C:/llvm-mingw/llvm-mingw/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:546
...
Wine-gdb> disassemble /r 2078440688,2078440688+20
Dump of assembler code from 0x7be27cf0 to 0x7be27d04:
0x7be27cf0 <_RtlExitUserThread@4+0>: 66 90 xchg %ax,%ax
...
```
Mechanism to keep the compiler-rt and llvm view of `FunctionData` in sync. Since CtxInstrContextNode.h is exactly the same on both sides (there's an existing test, `compiler-rt/test/ctx_profile/TestCases/check-same-ctx-node.test`, checking that), we capture the structure in a macro that is then generated as `struct` fields on the compiler-rt side, and as `Type` objects on the llvm side. The macro needs to be told how to render a few kinds of fields.
If we add more fields to FunctionData that can be described by the current known types of fields, then the llvm side would automatically be updated. If we need to add more kinds of fields, which we do by adding parameters to the macro, the llvm side (if not updated) would trigger a compilation error.
On AIX, when accessing mmap'ed memory associated to a file on NFS, a
SIGBUS might be raised at random.
The problem is still in open state with the OS team.
This PR teaches the profile runtime, under certain conditions, to avoid
the mmap when reading the profile file during online merging.
This PR has no effect on any platform other than AIX because I'm not
aware of this problem on other platforms.
Other platforms can easily opt-in to this functionality in the future.
The logic in function `is_local_filesystem` was copied from
[llvm/lib/Support/Unix/Path.inc](f388ca3d9d/llvm/lib/Support/Unix/Path.inc (L515))
(https://reviews.llvm.org/D58801), because it seems that the
compiler-rt/profile cannot reuse code from llvm except through
`InstrProfData.inc`.
Thanks to @hubert-reinterpretcast for substantial feedback downstream.
---------
Co-authored-by: Wael Yehia <wyehia@ca.ibm.com>
Co-authored-by: Hubert Tong <hubert.reinterpretcast@gmail.com>
High-entropy ASLR allows up to 16-bits of entropy (2**16 4KB pages ==
256MB; a bit more in practice because of implementation details), which
is a significant chunk of the user address space on 32-bit systems (4GB
or less). This, combined with ASan's shadow (512MB) and ASan's fixed
shadow offset (512MB), makes it possible for large binaries to fail to
map the shadow.
This patch changes ASan to do a one-time re-exec without ASLR if it
cannot map the shadow, thus reclaiming the ~256MB of address space.
Alternatives considered:
1) We don't lower ASan's fixed shadow offset, because that would limit
non-PIE binaries.
2) We don't switch to a dynamic shadow offset, because ASan for 32-bit
Linux relies on the compile-time constant offset to optimize its
instrumentation and compiler-rt.
This is loosely inspired by
https://github.com/llvm/llvm-project/pull/78351,
https://github.com/llvm/llvm-project/pull/85142, and
https://github.com/llvm/llvm-project/pull/85674, though those were
required because there were no static shadow mappings that could fully
shadow the range of user mappings; this is not the case for ASan.
This pull request is the third part of an ongoing effort to extends PGO
instrumentation to GPU device code and depends on
https://github.com/llvm/llvm-project/pull/93365. This PR makes the
following changes:
- Allows PGO flags to be supplied to GPU targets
- Pulls version global from device
- Modifies `__llvm_write_custom_profile` and `lprofWriteDataImpl` to
allow the PGO version to be overridden
