mirrors/llvm-project
0
0
Fork 0
mirror of https://github.com/llvm/llvm-project.git synced 2025-04-24 05:26:05 +00:00
Code Issues Projects Releases Wiki Activity
llvm-project/compiler-rt/lib/xray/xray_trampoline_x86_64.S

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

322 lines
7.8 KiB
ArmAsm
Raw Permalink Normal View History

[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
//===-- xray_trampoline_x86.s -----------------------------------*- ASM -*-===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 08:50:56 +00:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
//
//===----------------------------------------------------------------------===//
//
// This file is a part of XRay, a dynamic runtime instrumentation system.
//
// This implements the X86-specific assembler for the trampolines.
//
//===----------------------------------------------------------------------===//
Add NO_EXEC_STACK_DIRECTIVE to xray assembly files. Summary: Add NO_EXEC_STACK_DIRECTIVE to xray assembly files Reviewers: dberris, javed.absar Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30953 llvm-svn: 297894
2017-03-15 21:18:47 +00:00
#include "../builtins/assembly.h"
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
#include "../sanitizer_common/sanitizer_asm.h"
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
// XRay trampolines which are not produced by intrinsics are not System V AMD64
// ABI compliant because they are called with a stack that is always misaligned
// by 8 bytes with respect to a 16 bytes alignment. This is because they are
// called immediately after the call to, or immediately before returning from,
// the function being instrumented. This saves space in the patch point, but
// misaligns the stack by 8 bytes.
.macro ALIGN_STACK_16B
#if defined(__APPLE__)
subq $$8, %rsp
#else
subq $8, %rsp
#endif
CFI_ADJUST_CFA_OFFSET(8)
.endm
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
.macro RESTORE_STACK_ALIGNMENT
#if defined(__APPLE__)
addq $$8, %rsp
#else
addq $8, %rsp
#endif
CFI_ADJUST_CFA_OFFSET(-8)
.endm
Add NO_EXEC_STACK_DIRECTIVE to xray assembly files. Summary: Add NO_EXEC_STACK_DIRECTIVE to xray assembly files Reviewers: dberris, javed.absar Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30953 llvm-svn: 297894
2017-03-15 21:18:47 +00:00
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
// This macro should lower the stack pointer by an odd multiple of 8.
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
.macro SAVE_REGISTERS
[XRay][compiler-rt] Stash flags as well in x86_64 trampoline Summary: This change saves and restores the full flags register in x86_64 mode. This makes running instrumented signal handlers safer, and avoids flags set during the execution of the event handlers from polluting the instrumented call's flags state. Reviewers: kpw, eizan, jfb Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D51277 llvm-svn: 340812
2018-08-28 10:32:50 +00:00
pushfq
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
CFI_ADJUST_CFA_OFFSET(8)
[XRay][compiler-rt+llvm] Update XRay register stashing semantics Summary: This change expands the amount of registers stashed by the entry and `__xray_CustomEvent` trampolines. We've found that since the `__xray_CustomEvent` trampoline calls can show up in situations where the scratch registers are being used, and since we don't typically want to affect the code-gen around the disabled `__xray_customevent(...)` intrinsic calls, that we need to save and restore the state of even the scratch registers in the handling of these custom events. Reviewers: pcc, pelikan, dblaikie, eizan, kpw, echristo, chandlerc Reviewed By: echristo Subscribers: chandlerc, echristo, hiraditya, davide, dblaikie, llvm-commits Differential Revision: https://reviews.llvm.org/D40894 llvm-svn: 323940
2018-02-01 02:21:54 +00:00
subq $240, %rsp
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
CFI_ADJUST_CFA_OFFSET(240)
[XRay][compiler-rt+llvm] Update XRay register stashing semantics Summary: This change expands the amount of registers stashed by the entry and `__xray_CustomEvent` trampolines. We've found that since the `__xray_CustomEvent` trampoline calls can show up in situations where the scratch registers are being used, and since we don't typically want to affect the code-gen around the disabled `__xray_customevent(...)` intrinsic calls, that we need to save and restore the state of even the scratch registers in the handling of these custom events. Reviewers: pcc, pelikan, dblaikie, eizan, kpw, echristo, chandlerc Reviewed By: echristo Subscribers: chandlerc, echristo, hiraditya, davide, dblaikie, llvm-commits Differential Revision: https://reviews.llvm.org/D40894 llvm-svn: 323940
2018-02-01 02:21:54 +00:00
movq %rbp, 232(%rsp)
movupd %xmm0, 216(%rsp)
movupd %xmm1, 200(%rsp)
movupd %xmm2, 184(%rsp)
movupd %xmm3, 168(%rsp)
movupd %xmm4, 152(%rsp)
movupd %xmm5, 136(%rsp)
movupd %xmm6, 120(%rsp)
movupd %xmm7, 104(%rsp)
movq %rdi, 96(%rsp)
movq %rax, 88(%rsp)
movq %rdx, 80(%rsp)
movq %rsi, 72(%rsp)
movq %rcx, 64(%rsp)
movq %r8, 56(%rsp)
movq %r9, 48(%rsp)
movq %r10, 40(%rsp)
movq %r11, 32(%rsp)
movq %r12, 24(%rsp)
movq %r13, 16(%rsp)
movq %r14, 8(%rsp)
movq %r15, 0(%rsp)
xray: fix assembly macro definition projects/compiler-rt/lib/xray/xray_trampoline_x86_64.S:33:7: error: unexpected token in '.endm' directive .endm SAVE_REGISTERS ^ projects/compiler-rt/lib/xray/xray_trampoline_x86_64.S:52:7: error: unexpected token in '.endm' directive .endm RESTORE_REGISTERS ^ Remove the trailing name on the `.endm` which does not take the name of the macro. This should bring the compiler-rt build bot back into working state. llvm-svn: 289852
2016-12-15 18:54:27 +00:00
.endm
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
.macro RESTORE_REGISTERS
[XRay][compiler-rt+llvm] Update XRay register stashing semantics Summary: This change expands the amount of registers stashed by the entry and `__xray_CustomEvent` trampolines. We've found that since the `__xray_CustomEvent` trampoline calls can show up in situations where the scratch registers are being used, and since we don't typically want to affect the code-gen around the disabled `__xray_customevent(...)` intrinsic calls, that we need to save and restore the state of even the scratch registers in the handling of these custom events. Reviewers: pcc, pelikan, dblaikie, eizan, kpw, echristo, chandlerc Reviewed By: echristo Subscribers: chandlerc, echristo, hiraditya, davide, dblaikie, llvm-commits Differential Revision: https://reviews.llvm.org/D40894 llvm-svn: 323940
2018-02-01 02:21:54 +00:00
movq 232(%rsp), %rbp
movupd 216(%rsp), %xmm0
movupd 200(%rsp), %xmm1
movupd 184(%rsp), %xmm2
movupd 168(%rsp), %xmm3
movupd 152(%rsp), %xmm4
movupd 136(%rsp), %xmm5
movupd 120(%rsp) , %xmm6
movupd 104(%rsp) , %xmm7
movq 96(%rsp), %rdi
movq 88(%rsp), %rax
movq 80(%rsp), %rdx
movq 72(%rsp), %rsi
movq 64(%rsp), %rcx
movq 56(%rsp), %r8
movq 48(%rsp), %r9
movq 40(%rsp), %r10
movq 32(%rsp), %r11
movq 24(%rsp), %r12
movq 16(%rsp), %r13
movq 8(%rsp), %r14
movq 0(%rsp), %r15
addq $240, %rsp
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
CFI_ADJUST_CFA_OFFSET(-240)
[XRay][compiler-rt] Stash flags as well in x86_64 trampoline Summary: This change saves and restores the full flags register in x86_64 mode. This makes running instrumented signal handlers safer, and avoids flags set during the execution of the event handlers from polluting the instrumented call's flags state. Reviewers: kpw, eizan, jfb Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D51277 llvm-svn: 340812
2018-08-28 10:32:50 +00:00
popfq
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
CFI_ADJUST_CFA_OFFSET(-8)
[XRay][compiler-rt][x86_64] Align the stack before and after calling handlers Summary: This change fixes the XRay trampolines aside from the __xray_CustomEvent trampoline to align the stack to 16-byte boundaries before calling the handler. Before this change we've not been explicitly aligning the stack to 16-byte boundaries, which makes it dangerous when calling handlers that leave the stack in a state that isn't strictly 16-byte aligned after calling the handlers. We add a test that makes sure we can handle these cases appropriately after the changes, and prevents us from regressing the state moving forward. Fixes http://llvm.org/PR35294. Reviewers: pelikan, pcc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40004 llvm-svn: 318261
2017-11-15 03:35:42 +00:00
.endm
Revert "[XRay][darwin] Initial XRay in Darwin Support" This reverts r317875. llvm-svn: 317877
2017-11-10 07:00:55 +00:00
.text
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
#if !defined(__APPLE__)
.section .text
Speculative fix for xray assembler error on MachO since r349976 This .file directive wasn't doing anything on MachO, and now that's diagnosed as an error. llvm-svn: 350132
2018-12-28 18:53:00 +00:00
.file "xray_trampoline_x86.S"
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
#else
.section __TEXT,__text
#endif
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
.macro LOAD_HANDLER_ADDR handler
#if !defined(XRAY_PIC)
movq ASM_SYMBOL(\handler)(%rip), %rax
#else
movq ASM_SYMBOL(\handler)@GOTPCREL(%rip), %rax
movq (%rax), %rax
#endif
.endm
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
//===----------------------------------------------------------------------===//
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
.globl ASM_SYMBOL(__xray_FunctionEntry)
[xray] Use hidden symbol visibility for xray trampolines Summary: We load multiple copies of the trampolines into memory when instrumenting DSOs. Hidden visibility prevents conflicts in this scenario. Reviewers: MaskRay, dberris, johnislarry Subscribers: #sanitizers Tags: #sanitizers Differential Revision: https://reviews.llvm.org/D78593
2020-04-21 18:18:23 -04:00
ASM_HIDDEN(__xray_FunctionEntry)
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
.align 16, 0x90
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_TYPE_FUNCTION(__xray_FunctionEntry)
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-BEGIN __xray_FunctionEntry
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
ASM_SYMBOL(__xray_FunctionEntry):
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
CFI_STARTPROC
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
SAVE_REGISTERS
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
ALIGN_STACK_16B
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
// This load has to be atomic, it's concurrent with __xray_patch().
// On x86/amd64, a simple (type-aligned) MOV instruction is enough.
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray19XRayPatchedFunctionE
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
testq %rax, %rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
je LOCAL_LABEL(tmp0)
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
[XRay][compiler-rt+llvm] Update XRay register stashing semantics Summary: This change expands the amount of registers stashed by the entry and `__xray_CustomEvent` trampolines. We've found that since the `__xray_CustomEvent` trampoline calls can show up in situations where the scratch registers are being used, and since we don't typically want to affect the code-gen around the disabled `__xray_customevent(...)` intrinsic calls, that we need to save and restore the state of even the scratch registers in the handling of these custom events. Reviewers: pcc, pelikan, dblaikie, eizan, kpw, echristo, chandlerc Reviewed By: echristo Subscribers: chandlerc, echristo, hiraditya, davide, dblaikie, llvm-commits Differential Revision: https://reviews.llvm.org/D40894 llvm-svn: 323940
2018-02-01 02:21:54 +00:00
// The patched function prologue puts its xray_instr_map index into %r10d.
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
movl %r10d, %edi
xor %esi,%esi
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
callq *%rax
[XRay][compiler-rt][x86_64] Align the stack before and after calling handlers Summary: This change fixes the XRay trampolines aside from the __xray_CustomEvent trampoline to align the stack to 16-byte boundaries before calling the handler. Before this change we've not been explicitly aligning the stack to 16-byte boundaries, which makes it dangerous when calling handlers that leave the stack in a state that isn't strictly 16-byte aligned after calling the handlers. We add a test that makes sure we can handle these cases appropriately after the changes, and prevents us from regressing the state moving forward. Fixes http://llvm.org/PR35294. Reviewers: pelikan, pcc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40004 llvm-svn: 318261
2017-11-15 03:35:42 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(tmp0):
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
RESTORE_STACK_ALIGNMENT
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
RESTORE_REGISTERS
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
retq
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-END
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_SIZE(__xray_FunctionEntry)
CFI_ENDPROC
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
//===----------------------------------------------------------------------===//
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
.globl ASM_SYMBOL(__xray_FunctionExit)
[xray] Use hidden symbol visibility for xray trampolines Summary: We load multiple copies of the trampolines into memory when instrumenting DSOs. Hidden visibility prevents conflicts in this scenario. Reviewers: MaskRay, dberris, johnislarry Subscribers: #sanitizers Tags: #sanitizers Differential Revision: https://reviews.llvm.org/D78593
2020-04-21 18:18:23 -04:00
ASM_HIDDEN(__xray_FunctionExit)
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
.align 16, 0x90
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_TYPE_FUNCTION(__xray_FunctionExit)
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-BEGIN __xray_FunctionExit
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
ASM_SYMBOL(__xray_FunctionExit):
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
CFI_STARTPROC
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
ALIGN_STACK_16B
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
// Save the important registers first. Since we're assuming that this
// function is only jumped into, we only preserve the registers for
// returning.
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
subq $64, %rsp
CFI_ADJUST_CFA_OFFSET(64)
[XRay][compiler-rt] Cleanup CFI/CFA annotations on trampolines Summary: This is a follow-up to D32202. While the previous change (D32202) did fix the stack alignment issue, we were still at a weird state in terms of the CFI/CFA directives (as the offsets were wrong). This change cleans up the SAVE/RESTORE macros for the trampoline, accounting the stack pointer adjustments with less instructions and with some clearer math. We note that the offsets will be different on the exit trampolines, because we don't typically 'call' into this trampoline and we only ever jump into them (i.e. treated as a tail call that's patched in at runtime). Reviewers: eugenis, kpw, pelikan Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32214 llvm-svn: 300815
2017-04-20 03:26:04 +00:00
movq %rbp, 48(%rsp)
[XRay][compiler-rt] Fix up CFI annotations and stack alignment Summary: Previously, we had been very undisciplined about CFI annotations with the XRay trampolines. This leads to runtime crashes due to mis-alined stack pointers that some function implementations may run into (i.e. those using instructions that require properly aligned addresses coming from the stack). This patch attempts to clean that up, as well as more accurately use the correct amounts of space on the stack for stashing and un-stashing registers. Reviewers: eugenis, kcc Subscribers: kpw, llvm-commits Differential Revision: https://reviews.llvm.org/D32202 llvm-svn: 300660
2017-04-19 05:37:14 +00:00
movupd %xmm0, 32(%rsp)
movupd %xmm1, 16(%rsp)
movq %rax, 8(%rsp)
movq %rdx, 0(%rsp)
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray19XRayPatchedFunctionE
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
testq %rax,%rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
je LOCAL_LABEL(tmp2)
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
movl %r10d, %edi
movl $1, %esi
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
callq *%rax
[XRay][compiler-rt][x86_64] Align the stack before and after calling handlers Summary: This change fixes the XRay trampolines aside from the __xray_CustomEvent trampoline to align the stack to 16-byte boundaries before calling the handler. Before this change we've not been explicitly aligning the stack to 16-byte boundaries, which makes it dangerous when calling handlers that leave the stack in a state that isn't strictly 16-byte aligned after calling the handlers. We add a test that makes sure we can handle these cases appropriately after the changes, and prevents us from regressing the state moving forward. Fixes http://llvm.org/PR35294. Reviewers: pelikan, pcc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40004 llvm-svn: 318261
2017-11-15 03:35:42 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(tmp2):
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
// Restore the important registers.
[XRay][compiler-rt] Cleanup CFI/CFA annotations on trampolines Summary: This is a follow-up to D32202. While the previous change (D32202) did fix the stack alignment issue, we were still at a weird state in terms of the CFI/CFA directives (as the offsets were wrong). This change cleans up the SAVE/RESTORE macros for the trampoline, accounting the stack pointer adjustments with less instructions and with some clearer math. We note that the offsets will be different on the exit trampolines, because we don't typically 'call' into this trampoline and we only ever jump into them (i.e. treated as a tail call that's patched in at runtime). Reviewers: eugenis, kpw, pelikan Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32214 llvm-svn: 300815
2017-04-20 03:26:04 +00:00
movq 48(%rsp), %rbp
[XRay][compiler-rt] Fix up CFI annotations and stack alignment Summary: Previously, we had been very undisciplined about CFI annotations with the XRay trampolines. This leads to runtime crashes due to mis-alined stack pointers that some function implementations may run into (i.e. those using instructions that require properly aligned addresses coming from the stack). This patch attempts to clean that up, as well as more accurately use the correct amounts of space on the stack for stashing and un-stashing registers. Reviewers: eugenis, kcc Subscribers: kpw, llvm-commits Differential Revision: https://reviews.llvm.org/D32202 llvm-svn: 300660
2017-04-19 05:37:14 +00:00
movupd 32(%rsp), %xmm0
movupd 16(%rsp), %xmm1
movq 8(%rsp), %rax
movq 0(%rsp), %rdx
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
addq $64, %rsp
CFI_ADJUST_CFA_OFFSET(-64)
RESTORE_STACK_ALIGNMENT
[compiler-rt][XRay] re-submitting r276117, with fixes for build breakage due to extraneous and missing dependencies and attempts to build on unsupported OSes Summary: This is a fixed-up version of D21612, to address failure identified post-commit. Original commit description: This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting). Fixes include: - Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built - Including <cstddef> to fix std::size_t issue Reviewers: kcc, rnk, echristo Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22611 llvm-svn: 276251
2016-07-21 07:39:55 +00:00
retq
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-END
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_SIZE(__xray_FunctionExit)
CFI_ENDPROC
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
//===----------------------------------------------------------------------===//
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
.globl ASM_SYMBOL(__xray_FunctionTailExit)
[xray] Use hidden symbol visibility for xray trampolines Summary: We load multiple copies of the trampolines into memory when instrumenting DSOs. Hidden visibility prevents conflicts in this scenario. Reviewers: MaskRay, dberris, johnislarry Subscribers: #sanitizers Tags: #sanitizers Differential Revision: https://reviews.llvm.org/D78593
2020-04-21 18:18:23 -04:00
ASM_HIDDEN(__xray_FunctionTailExit)
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
.align 16, 0x90
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_TYPE_FUNCTION(__xray_FunctionTailExit)
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-BEGIN __xray_FunctionTailExit
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
ASM_SYMBOL(__xray_FunctionTailExit):
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
CFI_STARTPROC
[XRay][compiler-rt][NFC] Deduplicate code in x86-64 trampolines. Summary: The layout of all registers saved on stack shouldn't deviate and will be reused in future trampolines as well. While there, fix whitespace and clarify comments. Author: mpel Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27799 llvm-svn: 289789
2016-12-15 09:04:05 +00:00
SAVE_REGISTERS
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
ALIGN_STACK_16B
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray19XRayPatchedFunctionE
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
testq %rax,%rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
je LOCAL_LABEL(tmp4)
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
movl %r10d, %edi
[XRay][compiler-rt] Handle tail-call exits in the XRay runtime Summary: This change starts differentiating tail exits from normal exits. We also increase the version number of the "naive" log to version 2, which will be the starting version where these records start appearing. In FDR mode we treat the tail exits as normal exits, and are thus subject to the same treatment with regard to record unwriting. Updating the version number is important to signal older builds of the llvm-xray tool that do not deal with the tail exit records must fail early (and that users should only use the llvm-xray tool built after the support for tail exits to get accurate handling of these records). Depends on D37964. Reviewers: kpw, pelikan Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37965 llvm-svn: 313515
2017-09-18 06:18:03 +00:00
movl $2, %esi
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
callq *%rax
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(tmp4):
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
RESTORE_STACK_ALIGNMENT
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
RESTORE_REGISTERS
[XRay][x86_64] Define a tail exit trampoline. Summary: We define a new trampoline that's a hybrid between the exit and entry trampolines with the following properties: - Saves all of the callee-saved registers according to the x86_64 calling conventions. - Indicate to the log handler function being called that this is a function exit event. This fixes a bug that is a result of not saving enough of the register states, and that the log handler is clobbering registers that would be used by the function being tail-exited into manifesting as runtime errors. Reviewers: rSerge, echristo, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26020 llvm-svn: 285787
2016-11-02 04:11:29 +00:00
retq
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-END
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_SIZE(__xray_FunctionTailExit)
CFI_ENDPROC
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
//===----------------------------------------------------------------------===//
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
.globl ASM_SYMBOL(__xray_ArgLoggerEntry)
[xray] Use hidden symbol visibility for xray trampolines Summary: We load multiple copies of the trampolines into memory when instrumenting DSOs. Hidden visibility prevents conflicts in this scenario. Reviewers: MaskRay, dberris, johnislarry Subscribers: #sanitizers Tags: #sanitizers Differential Revision: https://reviews.llvm.org/D78593
2020-04-21 18:18:23 -04:00
ASM_HIDDEN(__xray_ArgLoggerEntry)
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
.align 16, 0x90
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_TYPE_FUNCTION(__xray_ArgLoggerEntry)
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-BEGIN __xray_ArgLoggerEntry
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
ASM_SYMBOL(__xray_ArgLoggerEntry):
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
CFI_STARTPROC
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
SAVE_REGISTERS
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
ALIGN_STACK_16B
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
// Again, these function pointer loads must be atomic; MOV is fine.
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray13XRayArgLoggerE
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
testq %rax, %rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
jne LOCAL_LABEL(arg1entryLog)
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
// If [arg1 logging handler] not set, defer to no-arg logging.
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray19XRayPatchedFunctionE
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
testq %rax, %rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
je LOCAL_LABEL(arg1entryFail)
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(arg1entryLog):
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
[XRay] [compiler-rt] FDR logging arg1 handler Summary: Write out records about logged function call first arguments. D32840 implements the reading of this in llvm-xray. Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32844 llvm-svn: 314378
2017-09-28 05:29:59 +00:00
// First argument will become the third
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
movq %rdi, %rdx
[XRay] [compiler-rt] FDR logging arg1 handler Summary: Write out records about logged function call first arguments. D32840 implements the reading of this in llvm-xray. Reviewers: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32844 llvm-svn: 314378
2017-09-28 05:29:59 +00:00
// XRayEntryType::LOG_ARGS_ENTRY into the second
mov $0x3, %esi
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
// 32-bit function ID becomes the first
movl %r10d, %edi
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
callq *%rax
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(arg1entryFail):
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
RESTORE_STACK_ALIGNMENT
[XRay][compiler-rt][x86_64] Preserve flags in x86_64 trampolines. (#89452) Previously, some xray trampolines would modify condition codes (before saving/after restoring flags) due to stack alignment instructions, which use add/sub. I am not aware of issues that this causes in practice (outside of the situation described in https://github.com/llvm/llvm-project/pull/89364, which is only problematic due to a different bug). Nevertheless, it seems nicer and less error-prone for xray instrumentation to be as unobstrusive/preserve as much state as possible.
2024-05-27 16:31:17 -07:00
RESTORE_REGISTERS
[XRay] [compiler-rt] Allow logging the first argument of a function call. Summary: Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled in a way to (optionally) pass their first call argument to your logging handler. For practical and performance reasons, only the first argument is supported, and only up to 64 bits. Reviewers: javed.absar, dberris Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29703 llvm-svn: 297000
2017-03-06 07:25:41 +00:00
retq
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-END
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_SIZE(__xray_ArgLoggerEntry)
CFI_ENDPROC
Add NO_EXEC_STACK_DIRECTIVE to xray assembly files. Summary: Add NO_EXEC_STACK_DIRECTIVE to xray assembly files Reviewers: dberris, javed.absar Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30953 llvm-svn: 297894
2017-03-15 21:18:47 +00:00
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
//===----------------------------------------------------------------------===//
[xray] Apply default visibility to __xray_CustomEvent/__xray_TypedEvent `__xray_customevent` and `__xray_typedevent` are built-in functions in Clang. With -fxray-instrument, they are lowered to `__xray_CustomEvent` (with 2 arguments) or `__xray_TypedEvent` (with 3 arguments). xray patching is supported for shared objects, but they may contain `__xray_customevent` and `__xray_typedevent` references that need to be satisfied by default visibility definitions exported by the executable. lld since df54f627fad789ccb11c72a9fddf116decbeba0e, like GNU ld, catches the scenario at link time.
2024-04-12 10:49:39 -07:00
// __xray_*Event have default visibility so that they can be referenced by user
// DSOs that do not link against the runtime.
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
.global ASM_SYMBOL(__xray_CustomEvent)
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
.align 16, 0x90
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_TYPE_FUNCTION(__xray_CustomEvent)
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-BEGIN __xray_CustomEvent
[sanitizer] Refactor how assembly files are handled This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target. Differential Revision: https://reviews.llvm.org/D40143 llvm-svn: 319339
2017-11-29 19:27:25 +00:00
ASM_SYMBOL(__xray_CustomEvent):
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
CFI_STARTPROC
[XRay][compiler-rt] Support sled versioning for custom event sleds Summary: This change introduces versions to the instrumentation map entries we emit for XRay instrumentaiton points. The status quo for the version is currently set to 0 (as emitted by the LLVM back-end), and versions will count up to 255 (unsigned char). This change is in preparation for supporting the newer version of the custom event sleds that will be emitted by the LLVM compiler. While we're here, we take the opportunity to stash more registers and align the stack properly in the __xray_CustomEvent trampoline. Reviewers: kpw, pcc, dblaikie Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D36816 llvm-svn: 311524
2017-08-23 04:42:37 +00:00
SAVE_REGISTERS
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
// We take two arguments to this trampoline, which should be in rdi and rsi
[XRay][compiler-rt+llvm] Update XRay register stashing semantics Summary: This change expands the amount of registers stashed by the entry and `__xray_CustomEvent` trampolines. We've found that since the `__xray_CustomEvent` trampoline calls can show up in situations where the scratch registers are being used, and since we don't typically want to affect the code-gen around the disabled `__xray_customevent(...)` intrinsic calls, that we need to save and restore the state of even the scratch registers in the handling of these custom events. Reviewers: pcc, pelikan, dblaikie, eizan, kpw, echristo, chandlerc Reviewed By: echristo Subscribers: chandlerc, echristo, hiraditya, davide, dblaikie, llvm-commits Differential Revision: https://reviews.llvm.org/D40894 llvm-svn: 323940
2018-02-01 02:21:54 +00:00
// already.
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray22XRayPatchedCustomEventE
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
testq %rax,%rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
je LOCAL_LABEL(customEventCleanup)
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
callq *%rax
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(customEventCleanup):
[XRay][compiler-rt] Support sled versioning for custom event sleds Summary: This change introduces versions to the instrumentation map entries we emit for XRay instrumentaiton points. The status quo for the version is currently set to 0 (as emitted by the LLVM back-end), and versions will count up to 255 (unsigned char). This change is in preparation for supporting the newer version of the custom event sleds that will be emitted by the LLVM compiler. While we're here, we take the opportunity to stash more registers and align the stack properly in the __xray_CustomEvent trampoline. Reviewers: kpw, pcc, dblaikie Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D36816 llvm-svn: 311524
2017-08-23 04:42:37 +00:00
RESTORE_REGISTERS
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
retq
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-END
[XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin This change is the first in a series of changes to get the XRay runtime building on macOS. This first allows us to build the minimal parts of XRay to get us started on supporting macOS development. These include: - CMake changes to allow targeting x86_64 initially. - Allowing for building the initialisation routines without `.preinit_array` support. - Use __sanitizer::SleepForMillis() to work around the lack of clock_nanosleep on macOS. - Deprecate the xray_fdr_log_grace_period_us flag, and introduce the xray_fdr_log_grace_period_ms flag instead, to use milliseconds across platforms. Reviewers: kubamracek Subscribers: llvm-commits, krytarowski, nglevin, mgorny Differential Review: https://reviews.llvm.org/D39114 llvm-svn: 319165
2017-11-28 11:49:22 +00:00
ASM_SIZE(__xray_CustomEvent)
CFI_ENDPROC
[XRay][compiler-rt] Runtime changes to support custom event logging Summary: This change implements support for the custom event logging sleds and intrinsics at runtime. For now it only supports handling the sleds in x86_64, with the implementations for other architectures stubbed out to do nothing. NOTE: Work in progress, uploaded for exposition/exploration purposes. Depends on D27503, D30018, and D33032. Reviewers: echristo, javed.absar, timshen Subscribers: mehdi_amini, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D30630 llvm-svn: 302857
2017-05-12 01:07:41 +00:00
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
//===----------------------------------------------------------------------===//
.global ASM_SYMBOL(__xray_TypedEvent)
.align 16, 0x90
ASM_TYPE_FUNCTION(__xray_TypedEvent)
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-BEGIN __xray_TypedEvent
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
ASM_SYMBOL(__xray_TypedEvent):
CFI_STARTPROC
SAVE_REGISTERS
// We pass three arguments to this trampoline, which should be in rdi, rsi
// and rdx without our intervention.
Reapply " [XRay] Add support for instrumentation of DSOs on x86_64 (#90959)" (#113548) This fixes remaining issues in my previous PR #90959. Changes: - Removed dependency on LLVM header in `xray_interface.cpp` - Fixed XRay patching for some targets due to missing changes in architecture-specific patching functions - Addressed some remaining compiler warnings that I missed in the previous patch - Formatting I have tested these changes on `x86_64` (natively), as well as `ppc64le`, `aarch64` and `arm32` (cross-compiled and emulated using qemu). **Original description:** This PR introduces shared library (DSO) support for XRay based on a revised version of the implementation outlined in [this RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000). The feature enables the patching and handling of events from DSOs, supporting both libraries linked at startup or explicitly loaded, e.g. via `dlopen`. This patch adds the following: - The `-fxray-shared` flag to enable the feature (turned off by default) - A small runtime library that is linked into every instrumented DSO, providing position-independent trampolines and code to register with the main XRay runtime - Changes to the XRay runtime to support management and patching of multiple objects These changes are fully backward compatible, i.e. running without instrumented DSOs will produce identical traces (in terms of recorded function IDs) to the previous implementation. Due to my limited ability to test on other architectures, this feature is only implemented and tested with x86_64. Extending support to other architectures is fairly straightforward, requiring only a position-independent implementation of the architecture-specific trampoline implementation (see `compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference). This patch does not include any functionality to resolve function IDs from DSOs for the provided logging/tracing modes. These modes still work and will record calls from DSOs, but symbol resolution for these functions in not available. Getting this to work properly requires recording information about the loaded DSOs and should IMO be discussed in a separate RFC, as there are mulitple feasible approaches. --------- Co-authored-by: Sebastian Kreutzer <sebastian.kreutzer@tu-darmstadt.de>
2024-10-25 09:15:25 +01:00
LOAD_HANDLER_ADDR _ZN6__xray21XRayPatchedTypedEventE
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
testq %rax,%rax
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
je LOCAL_LABEL(typedEventCleanup)
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
[XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines This patch modifies the x86_64 XRay trampolines to fix the CFI information generated by the assembler. One of the main issues in correcting the CFI directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on the alignment of the stack. However, this macro is not really necessary because some additional assumptions can be made on the alignment of the stack when the trampolines are called. The code has been written as if the stack is guaranteed to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8 bytes with respect to a 16-bytes alignment. For this reason, always moving the stack pointer by 8 bytes is sufficient to restore the appropriate alignment. Trampolines that are called from within a function as a result of the builtins `__xray_typedevent` and `__xray_customevent` are necessarely called with the stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be eliminated. Fixes https://bugs.llvm.org/show_bug.cgi?id=49060 Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
callq *%rax
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
[xray] Use L* instead of .L* for Mach-O Note: Mach-O support is not yet done and check-xray is not allowed yet.
2023-06-16 12:04:28 -07:00
LOCAL_LABEL(typedEventCleanup):
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
RESTORE_REGISTERS
retq
[XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC) This change adds some assembler comments to facilitate analysis with llvm-mca. In particular, we're interested in identifying and later optimising (reducing) the cost of the key functions in the XRay implementation using both static analysis (with llvm-mca, etc.) and dynamic analysis (perf profiling, etc.) of microbenchmarks. llvm-svn: 331711
2018-05-08 01:57:51 +00:00
# LLVM-MCA-END
Implement trampoline and handler for typed xray event tracing. Summary: Compiler-rt support first before defining the __xray_typedevent() lowering in llvm. I'm looking for some early feedback before I touch much more code. Reviewers: dberris Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D43668 llvm-svn: 330218
2018-04-17 21:28:53 +00:00
ASM_SIZE(__xray_TypedEvent)
CFI_ENDPROC
//===----------------------------------------------------------------------===//
Add NO_EXEC_STACK_DIRECTIVE to xray assembly files. Summary: Add NO_EXEC_STACK_DIRECTIVE to xray assembly files Reviewers: dberris, javed.absar Reviewed By: dberris Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30953 llvm-svn: 297894
2017-03-15 21:18:47 +00:00
NO_EXEC_STACK_DIRECTIVE
Reference in New Issue Copy Permalink