This getZExtOrTrunc seems to have been added when getPtrExtOrTrunc
was introduced. getPtrExtOrTrunc is currently equivalent to getZExtOrTrunc,
but could be changed for some target in the future.
Reviewed By: t.p.northover
Differential Revision: https://reviews.llvm.org/D149680
This patch-set aims to simplify the existing RVV segment load/store
intrinsics to use a type that represents a tuple of vectors instead.
To achieve this, first we need to relax the current limitation for an
aggregate type to be a target of load/store/alloca when the aggregate
type contains homogeneous scalable vector types. Then to adjust the
prolog of an LLVM function during lowering to clang. Finally we
re-define the RVV segment load/store intrinsics to use the tuple types.
The pull request under the RVV intrinsic specification is
riscv-non-isa/rvv-intrinsic-doc#198
---
This is the 1st patch of the patch-set. This patch is originated from
D98169.
This patch allows aggregate type (StructType) that contains homogeneous
scalable vector types to be a target of load/store/alloca. The RFC of
this patch was posted in LLVM Discourse.
https://discourse.llvm.org/t/rfc-ir-permit-load-store-alloca-for-struct-of-the-same-scalable-vector-type/69527
The main changes in this patch are:
Extend `StructLayout::StructSize` from `uint64_t` to `TypeSize` to
accommodate an expression of scalable size.
Allow `StructType:isSized` to also return true for homogeneous
scalable vector types.
Let `Type::isScalableTy` return true when `Type` is `StructType`
and contains scalable vectors
Extra description is added in the LLVM Language Reference Manual on the
relaxation of this patch.
Authored-by: Hsiangkai Wang <kai.wang@sifive.com>
Co-Authored-by: eop Chen <eop.chen@sifive.com>
Reviewed By: craig.topper, nikic
Differential Revision: https://reviews.llvm.org/D146872
The current implementation of -fsanitize=function places two words (the prolog
signature and the RTTI proxy) at the function entry, which makes the feature
incompatible with Intel Indirect Branch Tracking (IBT) that needs an ENDBR instruction
at the function entry. To allow the combination, move the two words before the
function entry, similar to -fsanitize=kcfi.
Armv8.5 Branch Target Identification (BTI) has a similar requirement.
Note: for IBT and BTI, whether a function gets a marker instruction at the entry
generally cannot be assumed (it can be disabled by a function attribute or
stronger LTO optimizations).
It is extremely unlikely for two words preceding a function entry to be
inaccessible. One way to achieve this is by ensuring that a function is
aligned at a page boundary and making the preceding page unmapped or
unreadable. This is not reasonable for application or library code.
(Think: the first text section has crt* code not instrumented by
-fsanitize=function.)
We use 0xc105cafe for all targets. .long 0xc105cafe disassembles to invalid
instructions on all architectures I have tested, except Power where it is
`lfs 8, -13570(5)` (Load Floating-Point with a weird offset, unlikely to be used in real code).
---
For the removed function in AsmPrinter.cpp, remove an assert: `mdconst::extract`
already asserts non-nullness.
For compiler-rt/test/ubsan/TestCases/TypeCheck/Function/function.cpp,
when the function doesn't have prolog/epilog (-O1 and above), after moving the two words,
the address of the function equals the address of ret instruction,
so symbolizing the function will additionally get a non-zero column number.
Adjust the test to allow an optional column number.
```
.long 3238382334
.long .L__llvm_rtti_proxy-_Z1fv
_Z1fv: // symbolizing here retrieves the line table entry from the second .loc
.file 0 ...
.loc 0 1 0
.cfi_startproc
.loc 0 2 1 prologue_end
retq
```
Reviewed By: peter.smith
Differential Revision: https://reviews.llvm.org/D148665
Fixes: https://github.com/llvm/llvm-project/issues/62725
This patch fixes an error in which a DBG_INSTR_REF referring to a DBG_PHI in a
block that is not directly reachable from the entry block results in a crash
during LiveDebugValues. Note that this fix prevents a crash from occurring, but
will give undef locations to users of these PHIs even if a valid location exists.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D150707
Not sure what this was originally intended for, but this seems to be
unused. It didn't seem to be used when it was first added in D64630
either.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D150606
Some metadata prettyprinting, including variable prettyprinting and
debug line info comments, is currently only supported for `DBG_VALUE`.
This allows `DBG_INSTR_REF` can be printed in the same way.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D150620
Previously, `CCState::AllocateStack` always allocated stack space by increasing
offsets. For targets with stack growing up (away from zero) it is more
convenient to allocate arguments by decreasing offsets, so that the first
argument is at the top of the stack. This is important when calling a function
with variable number of arguments: the callee does not know the size of the
stack, but must be able to access "fixed" arguments. For that to work, the
"fixed" arguments should have fixed offsets relative to the stack top, i.e. the
variadic arguments area should be at the stack bottom (at lowest addresses).
The in-tree target with stack growing up is AMDGPU, but it allocates
arguments by increasing addresses. It does not support variadic arguments.
A drive-by change is to promote stack size/offset to 64-bit integer.
This is what MachineFrameInfo expects.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D149575
The term "next stack offset" is misleading because the next argument is
not necessarily allocated at this offset due to alignment constrains.
It also does not make much sense when allocating arguments at negative
offsets (introduced in a follow-up patch), because the returned offset
would be past the end of the next argument.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D149566
This patch is a continuation of D150110. It separates the cases for
ADD and SUB into their own cases so that computeForAddSub can be
directly called and the NSW flag passed. This allows better
optimization when the NSW flag is enabled, and allows fixing up the
TODO that was there previously in SimplifyDemandedBits.
Differential Revision: https://reviews.llvm.org/D150769
The dwarf discriminator field for pseudo probes is not supposed to be used until the first FS discriminator pass. Unfortunately there are always corner cases that accidientally set this field. For example, the inliner could set this field for an inlined instruction if the instruction does not come with any debug information. While fixing all such spots is possible, but for future-proff I'd like to enforce a general cleanup before assigning probes any FS discriminator.
Reviewed By: wenlei
Differential Revision: https://reviews.llvm.org/D150741
Despite previous effort {D148569} to avoid screwing up existing disrminator field, I'm still seeing some call probes getting a non-zero discriminator eventually in non-FS mode. It could be related to callsite merge. While they are investigated I'm disabling discriminator emission for non-FS mode. This avoids breaking the compatiblity with older tools like llvm-profgen and bolt.
Reviewed By: wenlei
Differential Revision: https://reviews.llvm.org/D150625
When an instruction is determined to be divergent, not all its outputs are
divergent. The users of only divergent outputs should now be examined for
divergence.
Also, replaced a repeating pattern of "if new divergent instruction, then add to
worklist" by combining it into a single function. This does not cause any change
in functionality.
Reviewed By: foad, arsenm
Differential Revision: https://reviews.llvm.org/D150636
This is the implementation of D149782
The patch implements a helper function that matches and fold the following cases in the DAGCombiner:
1. `bswap(logic_op(x, bswap(y))) -> logic_op(bswap(x), y)`
2. `bswap(logic_op(bswap(x), y)) -> logic_op(x, bswap(y))`
3. `bswap(logic_op(bswap(x), bswap(y))) -> logic_op(x, y)` in multiuse case, which still reduces the number of instructions.
The helper function accepts SDValue with BSWAP and BITREVERSE opcode. This patch folds the BSWAP cases and remain the BITREVERSE optimization in the future
Reviewed By: RKSimon, goldstein.w.n
Differential Revision: https://reviews.llvm.org/D149783
The S_LPROC32_ID and S_GPROC32_ID CodeView Debug Symbols have a flags
field which LLVM has had the values for (in the ProcSymFlags enum) but
has never actually set.
These flags are used by Microsoft-internal tooling that leverages debug
information to do binary analysis.
Modified LLVM to set the correct flags:
- ProcSymFlags::HasOptimizedDebugInfo - always set, as this indicates that
debug info is present for optimized builds (if debug info is not emitted
for optimized builds, then LLVM won't emit a debug symbol at all).
- ProcSymFlags::IsNoReturn and ProcSymFlags::IsNoInline - set if the
function has the NoReturn or NoInline attributes respectively.
- ProcSymFlags::HasFP - set if the function requires a frame pointer (per
TargetFrameLowering::hasFP).
Per discussion in review, XFAIL'ing lldb test until someone working on
lldb has a chance to look at it.
Differential Revision: https://reviews.llvm.org/D148761
Define intersectWith and unionWith as two complementary ways of
combining KnownBits. The names are chosen for consistency with
ConstantRange.
Deprecate commonBits as a synonym for intersectWith.
Differential Revision: https://reviews.llvm.org/D150443
An instruction should be sunk (if otherwise legal and profitable) regardless
of if it has a dead def of a physreg or not. Physreg defs are checked in other
places and sinking is only done with dead defs of regs that are not live into
the target MBB.
Differential Revision: https://reviews.llvm.org/D150447
Reviewed By: sebastian-ne, arsenm
This patch fixes a potential crash due to RegAllocFast not rewriting virtual
registers. This essentially happens because of a call to
MachineInstr::addRegisterKilled() in the process of allocating a "killed" vreg.
The former can eventually delete implicit operands without RegAllocFast
noticing, leading to some operands being "skipped" and not rewritten to use
physical registers.
Note that I noticed this crash when working on a solution for tying a register
with one/multiple of its sub-registers within an instruction. (See problem
description here:
https://discourse.llvm.org/t/pass-to-tie-an-output-operand-to-a-subregister-of-an-input-operand/67184).
Aside from this fix, I believe there could be further improvements to the
RegAllocFast when it comes to instructions with multiple uses of a same virtual
register. You can see it in the added test where the implicit uses have been
re-written in a somewhat surprising way because of phase ordering. Ultimately,
when allocating vregs for an instruction, I believe we should iterate on the
vregs it uses (and then process all the operands that use this vregs), instead
of directly iterating on operands and somewhat assuming each operand uses a
different vreg. This would in the end be quite close to what
greedy+virtregrewriter does. If that makes sense, I would probably spin off
another patch (after I get more familiar with RegAllocFast).
Differential Revision: https://reviews.llvm.org/D145169
Summary: DWARF32 is not supported for XCOFF64 under non-integrated-as mode on AIX, because system assembler will fill the debug section lengths according to DWARF64 format. While in intergrated-as mode, XCOFF64 should be able to select the DWARF format.
Reviewed By: shchenz
Differential Revision: https://reviews.llvm.org/D150181
The MachineUA now queries the target to determine if a given register holds a
uniform value. This is determined using the corresponding register bank if
available, or by a combination of the register class and value type. This
assumes that the target is optimizing for performance by choosing registers, and
the target is responsible for any mismatch with the inferred uniformity.
For example, on AMDGPU, an SGPR is now treated as uniform, except if the
register bank is VCC (i.e., the register holds a wave-wide vector of 1-bit
values) or equivalently if it has a value type of s1.
- This does not always work with inline asm, where the register bank or the
value type might not be present. We assume that the SGPR is uniform, because
it is not expected to be s1 in the vast majority of cases.
- The pseudo branch instruction SI_LOOP is now hard-coded to be always
divergent, although its condition is an SGPR.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D150438
Add some debug output to `outline` to assist in debugging + understanding the
code.
This will say
- How many things we found worth turning into outlined functions
- Whether or not candidates were pruned via the outlining algorithm
- The function created (if it was created)
- Where the calls were inserted
- What instruction was used to create the call
Sample output below:
```
NUMBER OF POTENTIAL FUNCTIONS: 5
WALKING FUNCTION LIST
PRUNED: 0/2 candidates
OUTLINE: Expected benefit (12 B) > threshold (1 B)
NEW FUNCTION: OUTLINED_FUNCTION_0
CREATE OUTLINED CALLS
CALL: OUTLINED_FUNCTION_0 in bar:<unknown>
.. BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp
CALL: OUTLINED_FUNCTION_0 in bar:<unknown>
.. BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp
PRUNED: 2/2 candidates
SKIP: Expected benefit (0 B) < threshold (1 B)
PRUNED: 0/2 candidates
OUTLINE: Expected benefit (8 B) > threshold (1 B)
NEW FUNCTION: OUTLINED_FUNCTION_1
CREATE OUTLINED CALLS
CALL: OUTLINED_FUNCTION_1 in bar:<unknown>
.. BL @OUTLINED_FUNCTION_1, implicit-def $lr, implicit $sp
CALL: OUTLINED_FUNCTION_1 in bar:<unknown>
.. BL @OUTLINED_FUNCTION_1, implicit-def $lr, implicit $sp
PRUNED: 2/2 candidates
SKIP: Expected benefit (0 B) < threshold (1 B)
PRUNED: 2/2 candidates
SKIP: Expected benefit (0 B) < threshold (1 B)
```
At a cycle C with divergent exits, UA was using a naive traversal of the exiting
edges to locate blocks that may use values defined inside C. But this traversal
fails when it encounters a cycle. This is now replaced with a much simpler
propagation that iterates over every instruction in C and checks any uses that
are outside C. But such an iteration can be expensive when C is very large; the
original strategy may need to be reconsidered if there is a regression in
compilation times.
Also fixed lit tests that should have originally caught the missed propagation
of temporal divergence.
Reviewed By: foad
Differential Revision: https://reviews.llvm.org/D149646
Prior to this patch, for the DWARF frame base LLVM uses the frame pointer
register if available, otherwise the stack pointer register. If the stack
pointer register is being used and a call or other code modifies the stack
pointer during the body of the function this results in the locations being
wrong and the debugger displaying the wrong values for variables.
By using DW_OP_call_frame_cfa in these situations the emitted location for
the variable will automatically handle changes in the stack pointer.
The CFA needs to be adjusted for the offset between the frame pointer/stack
pointer to allow the variable locations themselves to remain unchanged by
this patch.
Reviewed By: #debug-info, scott.linder, jryans
Differential Revision: https://reviews.llvm.org/D143463
Previously, LegalizeVectorOps used the result VT while LegalizeDAG
used the operand VT. This patch makes them both use the operand VT.
This also makes it consistent with how the default cost model works.
I've hacked the AArch64 cost model to maintain old behavior for some
f16 vectors.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D149572
The current logic is pretty limitted unless the `Op` is a
constant. This at least covers more obvious cases.
Reviewed By: craig.topper, foad
Differential Revision: https://reviews.llvm.org/D149196
Both of these functions recursively call themselves so it makes sense
to limit that upper bound.
Differential Revision: https://reviews.llvm.org/D149195
This patch adds a heuristic to skip if-conversion if the condition has a
high chance of being predictable.
If the condition is in a loop, consider it predictable if the condition
itself or all its operands are loop-invariant. E.g. this considers a load
from a loop-invariant address predictable; we were unable to prove that it
doesn't alias any of the memory-writes in the loop, but it is likely to
read to same value multiple times.
This is a relatively crude heuristic, but it helps to prevent excessive
if-conversion in multiple workloads in practice.
Reviewed By: apostolakis
Differential Revision: https://reviews.llvm.org/D141639
While the original motivation for this patch (address space 7 on
AMDGPU) has been reworked and is not presently planned to reach IR
translation, the incorrect (by the spec) handling of index offset
width in IR translation and CodeGenPrepare is likely to trip someone
- possibly future AMD, since we have a p7:160:256:256:32 now, so we
convert to the other API now.
Reviewed By: aemerson, arsenm
Differential Revision: https://reviews.llvm.org/D143526
Correct the legality of i32 mul_lohi on AArch64.
Previously, AArch64 incorrectly reported i32 mul_lohi as Legal.
This allowed BuildUDIV/SDIV to use them. A later DAGCombiner would
replace them with MULHS/MULHU because only the high half was used.
This conversion does not check the legality of MULHS/MULHU under
the assumption that LegalizeDAG can turn it back into MUL_LOHI later.
After they are converted to MULHS/MULHU, DAGCombine ran and saw that
these operations aren't supported but an i64 MUL is. So they get
converted to that plus a shift. Without this, LegalizeDAG would
convert back MUL_LOHI and isel would fail to find a pattern.
This patch teaches BuildUDIV/SDIV to create the wide mul and shift
so that we can report the correct operation legality on AArch64. It
also enables div by constant folding for more cases on VE.
I don't know if VE wants this div by constant optimization or not. If they
don't want it, they can use the isIntDivCheap hook to disable it.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D150333
This commit implements SelectionDAG lowering of dbg.declare intrinsics targeting
swiftasync Arguments, by putting them in the MachineFunction's table of
variables whose location doesn't change throughout the function.
Depends on D149882
Differential Revision: https://reviews.llvm.org/D149883
This commit implements IRTranslator lowering of dbg.declare intrinsics targeting
swiftasync Arguments, by putting them in the MachineFunction's table of
variables whose location doesn't change throughout the function.
Depends on D149881
Differential Revision: https://reviews.llvm.org/D149882
This patch consumes the EntryValueObjects in a MachineFunction's table, using
them to emit the appropriate debug information for these variables.
Depends on D149880
Differential Revision: https://reviews.llvm.org/D149881
Looks like code assumes that it will be always set, but it's not true:
https://reviews.llvm.org/D150420. This is temporarily suppression to enabled
stricter msan on a bot.
We were missing any support for ISD::INTRINSIC_W_CHAIN/INTRINSIC_VOID
used for memory operations.
For ISD::PREFETCH and target memory nodes we didn't add the subclass
data.
This patch handles all MemIntrinsicSDNode in one place and adds the
missing subclass data.
Note. Unlike load/stores we don't add the memory VT in AddNodeIDCustom or getMemIntrinsicNode. Not sure why.
Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D150387
This patch adds a check for whether the memory operand is known to be
a jump table and, if so, allows shrinkwrapping to continue. In the
case that we are looking at a jump table, I believe it is safe to
assume that the access will not be to the stack (but please correct me
if I am wrong here).
In the test attached, this is helpful in that we are able to generate
only one instruction for each non-default case in the original switch
statement.
Differential Revision: https://reviews.llvm.org/D149886
This patch encapsulates the encoding and decoding logic of basic block metadata into the Metadata struct, and also reduces the decoded size of `SHT_LLVM_BB_ADDR_MAP` section.
The patch would've looked more readable if we could use designated initializer, but that is a c++20 feature.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D148360
This commit implements the serialization and deserialization of the Machine
Function's EntryValueObjects.
Depends on D149879, D149778
Differential Revision: https://reviews.llvm.org/D149880
