This patch works around:
compiler-rt/lib/tysan/../sanitizer_common/sanitizer_platform_limits_posix.h:604:3:
error: anonymous structs are a GNU extension
[-Werror,-Wgnu-anonymous-struct]
This patch introduces the runtime components for type sanitizer: a
sanitizer for type-based aliasing violations.
It is based on Hal Finkel's https://reviews.llvm.org/D32197.
C/C++ have type-based aliasing rules, and LLVM's optimizer can exploit
these given TBAA metadata added by Clang. Roughly, a pointer of given
type cannot be used to access an object of a different type (with, of
course, certain exceptions). Unfortunately, there's a lot of code in the
wild that violates these rules (e.g. for type punning), and such code
often must be built with -fno-strict-aliasing. Performance is often
sacrificed as a result. Part of the problem is the difficulty of finding
TBAA violations. Hopefully, this sanitizer will help.
For each TBAA type-access descriptor, encoded in LLVM's IR using
metadata, the corresponding instrumentation pass generates descriptor
tables. Thus, for each type (and access descriptor), we have a unique
pointer representation. Excepting anonymous-namespace types, these
tables are comdat, so the pointer values should be unique across the
program. The descriptors refer to other descriptors to form a type
aliasing tree (just like LLVM's TBAA metadata does). The instrumentation
handles the "fast path" (where the types match exactly and no
partial-overlaps are detected), and defers to the runtime to handle all
of the more-complicated cases. The runtime, of course, is also
responsible for reporting errors when those are detected.
The runtime uses essentially the same shadow memory region as tsan, and
we use 8 bytes of shadow memory, the size of the pointer to the type
descriptor, for every byte of accessed data in the program. The value 0
is used to represent an unknown type. The value -1 is used to represent
an interior byte (a byte that is part of a type, but not the first
byte). The instrumentation first checks for an exact match between the
type of the current access and the type for that address recorded in the
shadow memory. If it matches, it then checks the shadow for the
remainder of the bytes in the type to make sure that they're all -1. If
not, we call the runtime. If the exact match fails, we next check if the
value is 0 (i.e. unknown). If it is, then we check the shadow for the
remainder of the byes in the type (to make sure they're all 0). If
they're not, we call the runtime. We then set the shadow for the access
address and set the shadow for the remaining bytes in the type to -1
(i.e. marking them as interior bytes). If the type indicated by the
shadow memory for the access address is neither an exact match nor 0, we
call the runtime.
The instrumentation pass inserts calls to the memset intrinsic to set
the memory updated by memset, memcpy, and memmove, as well as
allocas/byval (and for lifetime.start/end) to reset the shadow memory to
reflect that the type is now unknown. The runtime intercepts memset,
memcpy, etc. to perform the same function for the library calls.
The runtime essentially repeats these checks, but uses the full TBAA
algorithm, just as the compiler does, to determine when two types are
permitted to alias. In a situation where access overlap has occurred and
aliasing is not permitted, an error is generated.
As a note, this implementation does not use the compressed shadow-memory
scheme discussed previously
(http://lists.llvm.org/pipermail/llvm-dev/2017-April/111766.html). That
scheme would not handle the struct-path (i.e. structure offset)
information that our TBAA represents. I expect we'll want to further
work on compressing the shadow-memory representation, but I think it
makes sense to do that as follow-up work.
This includes build fixes for Linux from Mingjie Xu.
Depends on #76260 (Clang support), #76259 (LLVM support)
PR: https://github.com/llvm/llvm-project/pull/76261
Avoid issues caused by `.subsections_via_symbols` directive, by using
numbered labels instead of named labels for the branch locations.
This reverts commit 4032ce3413d0230b0ccba1203536f9cb35e5c3b5.
Support platform-specific mangling to avoid the compiler emitting a call
to a function that is mangled differently than the definition in the
runtime library.
According to the conversation
[here](https://github.com/llvm/llvm-project/pull/119414#issuecomment-2536495859),
some platforms don't enable `__arm_cpu_features` with a global
constructor, but rather do so lazily when called from the FMV resolver.
PR #119414 removed the CMake guard to check to see if the targetted
platform is baremetal or supports sys/auxv. Without this check, the
routines rely on `__arm_cpu_features` being initialised when they may
not be, depending on the platform.
This PR simply avoids building the SME routines for those platforms for
now.
fd_set is defined by `sys/select.h`. On musl, this header is not
transitively included by the other headers.
Failure message:
```
compiler-rt/lib/rtsan/rtsan_interceptors_posix.cpp:761:37: error: unknown type name 'fd_set'; did you mean 'fd_t'?
761 | INTERCEPTOR(int, pselect, int nfds, fd_set *readfds, fd_set *writefds,
| ^~~~~~
| fd_t
```
When #92921 added the `__arm_get_current_vg` functionality, it used the
FMV feature bits mechanism rather than the mechanism that was previously
added for SME which called `getauxval` on Linux platforms or
`__aarch64_sme_accessible` required for baremetal libraries. It is
better to always use `__aarch64_cpu_features`.
For baremetal we still need to rely on `__arm_sme_accessible` to
initialise the struct.
**Related:** #117925
**About this PR:**
This PR performs 3 small but related fixes for ASan users on Windows:
1. It ensures that the `allocator_may_return_null` flag is honored when
set through the user function `__asan_default_options`. For more
details, please see: #117925
2. It adds a missing `AllocatorMayReturnNull()` check inside
`InternalAlloc` that's needed to avoid error'ing out when the allocator
_correctly_ returns `null` when `allocator_may_return_null` is set.
3. In `sanitizer_win`'s `ReturnNullptrOnOOMOrDie`, it allows returning
`null` when the last error is set to `ERROR_INVALID_PARAMETER` which may
be set by `VirtualAlloc` on WIndows when attempting to allocate
exceedingly large memory.
I've added test cases that should cover these new behaviors. Happy to
take on any feedback as well. Thank you :-)
---------
Co-authored-by: David Justo <dajusto@microsoft.com>
Add RISC-V support for XRay. The RV64 implementation has been tested in
both QEMU and in our hardware environment.
Currently this requires D and C extensions, but since both RV64GC and
RVA22/RVA23 are becoming mainstream, I don't think this requirement will
be a big problem.
Based on the previous work by @a-poduval :
https://reviews.llvm.org/D117929
---------
Co-authored-by: Ashwin Poduval <ashwin.poduval@gmail.com>
Add the `dump_at_exit` flag to control whether or not profiles should be
dumped when the program exits. Since we can call
`__memprof_profile_dump()` directly, we don't necessarily need to dump
profiles at exit.
This patch simplifies the code in two different ways:
* When SVE is available, return `cntd` directly to avoid the need for
bitfield insert.
* When SME is available, check the PSTATE.SM bit of `SVCR` directly
rather than calling `__arm_sme_state`.
* Finished the type and size verification
* Remove the TODO for checking if array size can be fit into LinkTy
because if there's a truncation happens, other DCHECK like offset
checking will catch the failure. In addition, it's supposed to be a rare
case.
The functions are not relevant for most sanitizers and only required for
MSan to see which regions have been written to. This eliminates a link
dependency for all other sanitizers and fixes#59007: while `-lresolv`
had been added for the static runtime in 6dce56b2a308, it wasn't added
to the shared runtimes.
Instead of just moving the interceptors, we adapt them to MSan
conventions:
* We don't skip intercepting when `msan_init_is_running` is true, but
directly call ENSURE_MSAN_INITED() like most other interceptors. It
seems unlikely that these functions are called during initialization.
* We don't unpoison `errno`, because none of the functions is specified
to use it.
Given that FMV support is required for the SME builtins to be built, the
FMV constructor as defined in:
compiler-rt/lib/builtins/cpu_model/aarch64.c
already initialises the feature bits, so there's no need to create
another one.
Based on the feedback from #118782, this switches most of the pointer
arithmetic in __llvm_profile_merge_from_buffer to work on uintptr_t
instead of const char *, only casting back to a pointer when performing
actual accesses.
This ensures that all the arithmetic is performed without any
assumptions about pointer overflow.
This re-applies 570ecdcf8b4, which was reverted in 74e8a37ff32 due to bot
failures. This commit renames sysv_resolve.cpp to resolve.cpp, which was the
cause of the config errors.
The unit tests may transiently depend on __orc_rt_log_error, which is usually
provided by an alias when loading the runtime through the JIT. The unit tests
statically link the runtime, so this patch provides a fixed definition.
This reapplies 570ecdcf8b4, which was reverted in 6073dd923b8 due to bot
failures.
The test failures on Linux were fixed by:
1. Removing an overly restrictive assertion (query dependence on a symbol no
longer implies a MaterializingInfo for that symbol)
2. Adding reentry and resolver files to the ORC runtime CMakeLists.txt for
Linux.
3. Adding the __orc_rt_reentry -> __orc_rt_sysv_reentry alias to ELFNixPlatform.
Since the sanitizer merge in commit r15-5164-gfa321004f3f628 of GCC
which entails LLVM commit 61a6439f35b6de28ff4aff4450d6fca970292fd5, GCCs
bootstrap is broken on s390 -m31. This is due to commit
ec68dc1ca4d967b599f1202855917d5ec9cae52f which introduces stricter type
checking which is why GCC bootstrap fails with
```
In file included from /gcc/src/libsanitizer/interception/interception.h:18,
from /gcc/src/libsanitizer/interception/interception_type_test.cpp:14:
/gcc/src/libsanitizer/interception/interception_type_test.cpp:30:61: error: static assertion failed
30 | COMPILER_CHECK((__sanitizer::is_same<::SSIZE_T, ::ssize_t>::value));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
/gcc/src/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:363:44: note: in definition of macro 'COMPILER_CHECK'
363 | #define COMPILER_CHECK(pred) static_assert(pred, "")
| ^~~~
make[8]: *** [Makefile:469: interception_type_test.lo] Error 1
```
The culprit seems to be that we don't check for equality of type sizes
anymore but rather whether the types are indeed the same. On s390 -m31
we have that `sizeof(int)==sizeof(long)` holds which is why previously
the checks succeeded. They fail now because
```
size_t => unsigned long
ssize_t => long
ptrdiff_t => int
::SSIZE_T => __sanitizer::sptr => int
::PTRDIFF_T => __sanitizer::sptr => int
```
This is fixed by mapping `SSIZE_T` to `long` in the end.
```
#if defined(__s390__) && !defined(__s390x__)
typedef long ssize;
#else
typedef sptr ssize;
#endif
#define SSIZE_T __sanitizer::ssize
```
For some targets uptr is mapped to unsigned int and size_t to unsigned
long and sizeof(int)==sizeof(long) holds. Still, these are distinct
types and type checking may fail. Therefore, replace uptr by
usize/SIZE_T wherever a size_t is expected.
Part of #116957
…d reentry.
These utilities provide new, more generic and easier to use support for
lazy compilation in ORC.
LazyReexportsManager is an alternative to LazyCallThroughManager. It
takes requests for lazy re-entry points in the form of an alias map:
lazy-reexports = {
( <entry point symbol #1>, <implementation symbol #1> ),
( <entry point symbol #2>, <implementation symbol #2> ),
...
( <entry point symbol #n>, <implementation symbol #n> )
}
LazyReexportsManager then:
1. binds the entry points to the implementation names in an internal
table.
2. creates a JIT re-entry trampoline for each entry point.
3. creates a redirectable symbol for each of the entry point name and
binds redirectable symbol to the corresponding reentry trampoline.
When an entry point symbol is first called at runtime (which may be on
any thread of the JIT'd program) it will re-enter the JIT via the
trampoline and trigger a lookup for the implementation symbol stored in
LazyReexportsManager's internal table. When the lookup completes the
entry point symbol will be updated (via the RedirectableSymbolManager)
to point at the implementation symbol, and execution will proceed to the
implementation symbol.
Actual construction of the re-entry trampolines and redirectable symbols
is delegated to an EmitTrampolines functor and the
RedirectableSymbolsManager respectively.
JITLinkReentryTrampolines.h provides a JITLink-based implementation of
the EmitTrampolines functor. (AArch64 only in this patch, but other
architectures will be added in the near future).
Register state save and reentry functionality is added to the ORC
runtime in the __orc_rt_sysv_resolve and __orc_rt_resolve_implementation
functions (the latter is generic, the former will need custom
implementations for each ABI and architecture to be supported, however
this should be much less effort than the existing OrcABISupport
approach, since the ORC runtime allows this code to be written as native
assembly).
The resulting system:
1. Works equally well for in-process and out-of-process JIT'd code.
2. Requires less boilerplate to set up.
Given an ObjectLinkingLayer and PlatformJD (JITDylib containing the ORC
runtime), setup is just:
```c++
auto RSMgr = JITLinkRedirectableSymbolManager::Create(OLL);
if (!RSMgr)
return RSMgr.takeError();
auto LRMgr = createJITLinkLazyReexportsManager(OLL, **RSMgr, PlatformJD);
if (!LRMgr)
return LRMgr.takeError();
```
after which lazy reexports can be introduced with:
```c++
JD.define(lazyReexports(LRMgr, <alias map>));
```
LazyObectLinkingLayer is updated to use this new method, but the LLVM-IR
level CompileOnDemandLayer will continue to use LazyCallThroughManager
and OrcABISupport until the new system supports a wider range of
architectures and ABIs.
The llvm-jitlink utility's -lazy option now uses the new scheme. Since
it depends on the ORC runtime, the lazy-link.ll testcase and associated
helpers are moved to the ORC runtime.
Add the `__memprof_default_options_str` variable, initialized via the
`-memprof-runtime-default-options` LLVM flag, to hold the default options string
for memprof. This allows us to set these options during compile time in
the clang invocation.
Also update the docs to describe the various ways to set these options.
These bounds checks work on the result of the pointer addition -- but
the pointer addition already asserts that no overflow may occur, so the
checks are optimized away after #118472. Avoid this by performing the
addition in a way that permits overflow.
warning: format specifies type 'void *' but the argument has type 'uptr'
(aka 'unsigned long long') [-Wformat] (observed at x86_64, in
AllocateTrampolineRegion)
warning: format specifies type 'char *' but the argument has type
'RVAPtr<char>' [-Wformat] (observed at x86_64, in
InternalGetProcAddress)
