Write uptime in real time seconds for every mem profile record.
Uptime is useful to make more sense out of the profile,
compare random lines, etc.
Depends on D110153.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110154
We do query it every 100ms now.
(GetRSS was fixed to not be dead slow IIRC)
Depends on D110152.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110153
BackgroundThread function is quite large,
move mem profile initialization into a separate function.
Depends on D110151.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110152
We allocate things from the internal allocator,
it's useful to know how much it consumes.
Depends on D110150.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110151
We currently query number of threads before reading /proc/self/smaps.
But reading /proc/self/smaps can take lots of time for huge processes
and it's retries several times with different buffer sizes.
Overall it can take tens of seconds. This can make number of threads
significantly inconsistent with the rest of the stats.
So query it after reading /proc/self/smaps.
Depends on D110149.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110150
Include info about MBlock/SyncObj memory consumption in the memory profile.
Depends on D110148.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110149
We account low and high ranges, but forgot abount the mid range.
Account mid range as well.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D110148
ScopedInterceptor::Enable/DisableIgnores is only used for some special cases.
Unline them from the common interceptor handling.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D110157
Switch Java heap move to the new scheme required for the new tsan runtime.
Instead of copying the shadow we reset the destination range.
The new v3 trace contains addresses of accesses, so we cannot simply copy the shadow.
This can lead to false negatives, but cannot lead to false positives.
Depends on D110159.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D110190
Pass -msse4.2 flag to the tests the same way we do for the runtime.
Layout of some structs in the runtime headers depends on the flag
(TSAN_VECTORIZE), so we need it to be consistent across the runtime
and tests.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D110192
CallUserSignalHandler function is quite large and complex.
Move errno spoiling reporting into a separate function.
No logical changes.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D110159
dlsym calls into dynamic linker which calls malloc and other things.
It's problematic to do it during the actual exit, because
it can happen from a singal handler or from within the runtime
after we reported the first bug, etc.
See https://github.com/google/sanitizers/issues/1440 for an example
(captured in the added test).
Initialize the callbacks during startup instead.
Depends on D110159.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D110166
When setting the report path, recursively create the directory as
needed. This brings the profile path support for memprof on par with
normal PGO. The code was largely cloned from __llvm_profile_recursive_mkdir
in compiler-rt/lib/profile/InstrProfilingUtil.c.
Differential Revision: https://reviews.llvm.org/D109794
To intercept the functions in Win11's ntdll.dll, we need to use the trampoline
technique because there are bytes other than 0x90 or 0xcc in the gaps between
exported functions. This patch adds more patterns that appear in ntdll's
functions.
Bug: https://bugs.llvm.org/show_bug.cgi?id=51721
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D109941
The current code writes the pc-table at the process startup,
which may happen before the common_flags() are initialized.
Move writing to the process end.
This is consistent with how we write the counters and avoids the problem with the uninitalized flags.
Add prints if verbosity>=1.
Reviewed By: kostik
Differential Revision: https://reviews.llvm.org/D110119
Currently we set thr->tctx after OnStarted callback
taking thread registry mutex again and searching for the context.
But OnStarted already runs under the thread registry mutex
and has access to the context, so set it in the OnStarted.
This makes code simpler and faster.
Depends on D110132.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D110133
Thread state functions are split into 2 parts:
tsan entry function (e.g. ThreadStart) and thread registry
state change callback (e.g. OnStart). Currently these
pairs of functions are located far from each other and
in reverse order. This makes it hard to read and follow the logic.
Reorder the code so that OnFoo directly follows ThreadFoo.
No other code changes.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D110132
Some of the DPrintf's currently produce -Wformat warnings if enabled.
Fix these format strings.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D110131
When running tests like SanitizerCommon-asan-x86_64-Linux :: Linux/crypt_r.cpp,
it may attempt to use the host header crypt.h rather than a sysroot header.
This is significant in the event where struct crypt_data defined on host is
different from the sysroot used to make the sanitizer runtime libraries. This
can result in logical differences between the expected size/layout of struct
crypt_data known by sanitizers and the strict crypt_data provided by the host crypt.h.
Since tests should still use the CMAKE_SYSROOT, this ensures that CMAKE_SYSROOT
is propagated to compiler-rt tests.
Differential Revision: https://reviews.llvm.org/D109796
We hit some undefined symbol errors to 128-bit floating point functions when linking this test.
ld.lld: error: undefined symbol: __multf3
>>> referenced by strtof128_l.o:(round_and_return) in archive /usr/lib/x86_64-linux-gnu/libc.a
>>> referenced by strtof128_l.o:(round_and_return) in archive /usr/lib/x86_64-linux-gnu/libc.a
>>> referenced by strtof128_l.o:(round_and_return) in archive /usr/lib/x86_64-linux-gnu/libc.a
>>> referenced 4 more times
>>> did you mean: __muldf3
>>> defined in: /usr/local/google/home/leonardchan/llvm-monorepo/llvm-build-1-master-fuchsia-toolchain/lib/clang/14.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a
Host libc expects these to be defined, and compiler-rt will only define these
for certain platforms (see definition for CRT_LDBL_128BIT). Since we likely
can't do anything about the host libc, we can at least restrict the test to
check that these functions are supported.
Differential Revision: https://reviews.llvm.org/D109709
On Fuchsia, killing or exiting a process that has a thread listening to its own process's debugger exception channel can hang. Zircon may kill all the threads, send a synthetic exceptions to debugger, and wait for the debugger to have received them. This means the thread listening to the debug exception channel may be killed even as Zircon is waiting for that thread to drain the exception channel, and the process can become stuck in a half-dead state.
This situation is "weird" as it only arises when a process is trying to debug itself. Unfortunately, this is exactly the scenario for libFuzzer on Fuchsia: FuzzerUtilFuchsia spawns a crash-handling thread that acts like a debugger in order to be able to rewrite the crashed threads stack and resume them into libFuzzer's usual POSIX signal handlers. In practice, approximately 25% of fuzzers appear to hang on exit, after generating output and artifacts. These processes hang around until the platform is torn done, which is typically a ClusterFuzz VM. Thus, real-world impact has been somewhat mitigated. The issue should still be resolved for local users, though.
This change improves the behavior of exit() in libFuzzer by adding an atexit handler which closes an event shared with the crash handling thread. This signals to the crash handler that it should close the exception channel and be joined before the process actually exits.
Reviewed By: charco
Differential Revision: https://reviews.llvm.org/D109258
This way, we do not need to set LLVM_CMAKE_PATH to LLVM_CMAKE_DIR when (NOT LLVM_CONFIG_FOUND)
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D107717
Previously we used a global Allocator-scope mutex to lock when adding a
deallocation to the MIB cache. This resulted in a lot of contention.
Instead add and use per-set mutexes.
Along with this, we now need to remove the global miss and access count
variables and instead utilize the per-set statistics to report the
overall miss rate.
Differential Revision: https://reviews.llvm.org/D109853
Previously we used the minimum deployment target used for the platform
(e.g. iOS is 9.0). Unfortunately this leads to ABI incompatibilities with
arm64e devices running newer OSs. In particular the following TSan test
cases that used libcxx would fail due to the ABI mismatch.
* Darwin/libcxx-shared-ptr-recursive.mm
* Darwin/libcxx-shared-ptr-stress.mm
* Darwin/libcxx-shared-ptr.mm
* libcxx/std_shared_ptr.cpp
Given that arm64e is not ABI stable we should ideally match the
deployment target for sanitizer runtimes and their tests cases to the
device when building for arm64e. Unfortunately having a mixed deployment
target (based on architecture) isn't currently supported by the build system
and is non-trivial to implement.
As a stop-gap measure this patch changes the sanitizer test suites (but not the
sanitizer runtimes themselves) to use a newer deployment target when
targetting arm64e.
The deployment target used for arm64e is the SDK version because this
"should" match the OS version running on the target device (it is a
configuration error to not match them).
rdar://83080611
9ee64c374605683ae80b9641d5312a72c2a67336 has started using
COMPILER_RT_HAS_OMIT_FRAME_POINTER_FLAG inside scudo. However,
the relevant CMake check was performed in builtin-config-ix.cmake,
so the definition was missing when builtins were not built. Move
the check to config-ix.cmake, so that it runs unconditionally of
the components being built.
Fixes PR#51847
Differential Revision: https://reviews.llvm.org/D109812
On x86_64-unknown-linux-gnu, `-m32` tests set LD_LIBRARY_PATH to
`config.compiler_rt_libdir` (`$build/lib/clang/14.0.0/lib/x86_64-unknown-linux-gnu`)
instead of i386-unknown-linux-gnu, so `-shared-libsan` executables
cannot find their runtime (e.g. `TestCases/replaceable_new_delete.cpp`).
Detect -m32 and -m64 in config.target_cflags, and adjust `config.compiler_rt_libdir`.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D108859
Some setjmp calls within libc cannot be intercepted while their matching
longjmp calls can be. This causes problems if our setjmp/longjmp
interceptors don't use the exact same format as libc for populating and
reading the jmp_buf.
We add a magic field to our jmp_buf and populate it in setjmp. This
allows our longjmp interceptor to notice when a libc jmp_buf is passed
to it.
See discussion on https://reviews.llvm.org/D109699 and
https://reviews.llvm.org/D69045.
Fixes https://github.com/google/sanitizers/issues/1244.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D109787
This updates llvm/utils/sysroot.py to include the "DIA SDK" folder in the
sysroot.
It also updates the build to look for the DIA SDK there if a sysroot is set.
This requires moving LLVM_WINSYSROOT to config-ix.cmake.
For the GN build, I chose to pass a qualified path to diaguids in libs instead
of pushing a config with a `/libpath:` flag. The former requires a GN with
https://gn-review.googlesource.com/c/gn/+/12200, the latter requires D109624.
The former is more like the cmake build, arguably a bit simpler, and it's
easier to check for the wrong GN revision and easier to update GN.
Differential Revision: https://reviews.llvm.org/D109708
This patch use the same way as the https://reviews.llvm.org/rGfe1fa43f16beac1506a2e73a9f7b3c81179744eb to handle the thread local variable.
It allocates 2 * pointerSize space in GOT to represent the thread key and data address. Instead of using the _tls_get_addr function, I customed a function __orc_rt_elfnix_tls_get_addr to get the address of thread local varible. Currently, this is a wip patch, only one TLS relocation R_X86_64_TLSGD is supported and I need to add the corresponding test cases.
To allocate the TLS descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a `Edge::Kind K` argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not suitable, I can think further to solve this problem.
Differential Revision: https://reviews.llvm.org/D109293
Add integration tests for dyld interposition: DYLD_LIBRARY_PATH and
DYLD_INSERT_LIBRARIES.
DYLD_INSERT_LIBRARIES is also relevant for TSan thread
finalization/destruction sequence in the presence of additional pthread
introspection hooks (libBacktraceRecording.dylib for Xcode 'Queue
Debugging' feature).
rdar://78739125
Differential Revision: https://reviews.llvm.org/D109332
Don't print stderr to commandline when configuring compiler-rt for
darwin platforms. NFC.
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D108156
