Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).
When searching for an intrinsic name in a target specific slice of the
intrinsic name table, skip over the target prefix. For such cases,
currently the first loop iteration in `lookupLLVMIntrinsicByName` does
nothing (i.e., `Low` and `High` stay unchanged and it does not shrink
down the search window), so we can skip this useless first iteration by
skipping over the target prefix.
These lists are quite static. Heavy use of macros is undesirable, and
not idiomatic in LLVM, so let's just use the naive switch cases.
Note that the first two fields in the CONSTRAINEDFP property were
utterly unused (aside from a C++ test).
In the same vien as https://github.com/llvm/llvm-project/pull/105551.
Once both changes have landed, we'll be left with _BINARYOP which needs
a bit of additional untangling, and the actual opcode mappings.
These lists are quite static and several of the parameters are actually
constant across all users. Heavy use of macros is undesirable, and not
idiomatic in LLVM, so let's just use the naive switch cases.
I'll probably continue with removing the other property macros. These
two just happened to be the two I actually had to figure out for an
unrelated change.
An overload of `llvm::promoteCallWithIfThenElse` that updates the contextual profile.
High-level, this is very simple: after creating the `if... then (direct call) else (indirect call)` structure, we instrument the new callsites and BBs (the instrumentation will help with tracking for other IPO transformations, and, ultimately, to match counter values before flattening to `MD_prof`).
In more detail:
- move the callsite instrumentation of the indirect call to the `else` BB, before the indirect call
- create a new callsite instrumentation for the direct call
- create instrumentation for both the `then` and `else` BBs - we could instrument just one (MST-style) but we're not running the binary with this instrumentation, and at most this would save some space (less counters tracked). For simplicity instrumenting both at this point
- update each context belonging to the caller by updating the counters, and moving the indirect callee to the new, direct callsite ID
Issue #89287
This patch refactors the handling of reduction to eliminate layering
violations.
* Introduced `getReductionIntrinsicID` in LoopUtils.h for mapping
recurrence kinds to llvm.vector.reduce.* intrinsic IDs.
* Updated `VectorBuilder::createSimpleTargetReduction` to accept
llvm.vector.reduce.* intrinsic directly.
* New function `VPIntrinsic::getForIntrinsic` for mapping intrinsic ID
to the same functional VP intrinsic ID.
Add the callsite intrinsic.
Structurally, it is very similar to the counter intrinsics, hence the inheritance relationship. We can probably rename `InstrProfCntrInstBase` to `InstrProfIndexedBase` later - because the "counting" aspect is really left to derived types of `InstrProfCntrInstBase`, and it only concerns itself with the index aspect (which is what we care about for `callsite`, too)
(Tracking Issue: #89287, RFC referenced there)
Helps avoid the crash in verifier when it tries to print the error.
`none` token might be produced by llvm-reduce, since it's a default
value, so by extension this also fixes llvm-reduce crash, allowing it to
just discard invalid IR.
---------
Co-authored-by: arpilipe <apilipenko@azul.com>
Part 1 of 3. This includes the LLVM back-end processing and profile
reading/writing components. compiler-rt changes are included.
Differential Revision: https://reviews.llvm.org/D138846
This seems to cause Clang to crash, see comments on the code review. Reverting
until the problem can be investigated.
> Part 1 of 3. This includes the LLVM back-end processing and profile
> reading/writing components. compiler-rt changes are included.
>
> Differential Revision: https://reviews.llvm.org/D138846
This reverts commit a50486fd736ab2fe03fcacaf8b98876db77217a7.
Part 1 of 3. This includes the LLVM back-end processing and profile
reading/writing components. compiler-rt changes are included.
Differential Revision: https://reviews.llvm.org/D138846
Some VP intrinsic definitions were missing the
VP_PROPERTY_FUNCTIONAL_INTRINSIC property. This patch fills them in, and
adds a static_assert that all VP intrinsics have an equivalent opcode or
intrinsic defined so we don't forget them in future.
Some VP intrinsics don't have an equivalent, namely merge and strided
load/store. For those, a new property was added to mark that they don't
have a non-VP equivalent.
This adds a helper method to get the ID of the functionally equivalent
intrinsic, similar to the existing getFunctionalOpcodeForVP and
getConstrainedIntrinsicIDForVP method.
VPIntrinsics with VP_PROPERTY_BINARYOP property should have the ability
to be queried with with VPBinOpIntrinsic::isVPBinOp, similiar to how
intrinsics with the VP_PROPERTY_REDUCTION property can be queried with
VPReductionIntrinsic::isVPReduction.
This will be used in #65706. In that PR the usage of this class is
tested.
This adds a helper method to get the ID of the functionally equivalent
intrinsic, similar to the existing getFunctionalOpcodeForVP and
getConstrainedIntrinsicIDForVP methods.
Not sure if it's notable or not, but I can't find any existing uses of
VP_PROPERTY_FUNCTIONAL_INTRINSIC?
It could potentially be used in #65706 to scalarize VP intrinsics.
There is no vp.fpclass after FCLASS_VL(D151176), try to support vp.fpclass.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D152993
The last time this function was updated DIArgLists were not supported for
dbg.assigns. Without this patch it's possible to now dereference an invalid
(the end) iterator. This occurs when the Address component gets replaced and
there's a DIArgList for the Value component (the contained values are
irrelevant).
The added unittest crashes without the code change in this patch.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D147922
Follow up to https://reviews.llvm.org/D146987.
Remove assertion that the Value must be a pointer type. This fires in
real-world examples e.g. by codegenprepare introducing ptrtoint conversions.
The buildbots have not caught up yet but without this change the test
compiler-rt/test/ubsan/TestCases/TypeCheck/vptr.cpp fails with an ICE.
RawLocationWrapper wraps the location operand of a debug intrinsic which may be
either (wrapped in MetadataAsValue) a DIArgList, ValueAsMetadata, or an empty
MDTuple. This class lets us avoid duplicating location handling code in a later
patch in this stack.
Reviewed By: StephenTozer
Differential Revision: https://reviews.llvm.org/D145909
The m_VScale() matcher is unusual in that it requires a DataLayout.
It is currently used to determine the size of the GEP type. However,
I believe it is sufficient to check for the canonical
<vscale x 1 x i8> form here -- I don't think there's a need to
recognize exotic variations like <vscale x 1 x i4> as a vscale
constant representation as well.
Differential Revision: https://reviews.llvm.org/D144566
Unlike D140903 this patch folds in treating an empty metadata address component
of a dbg.assign the same as undef because it was already being treated that way
in the AssignmentTrackingAnalysis pass.
Reviewed By: scott.linder
Differential Revision: https://reviews.llvm.org/D141125
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
This commit fixes LLVMAnalysis and its dependencies.
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Fix replaceVariableLocationOp unconditionally replacing the first operand of a
dbg.assign.
Reviewed By: jryans
Differential Revision: https://reviews.llvm.org/D138561
The Assignment Tracking debug-info feature is outlined in this RFC:
https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir
Add the llvm.dbg.assign intrinsic boilerplate. This updates the textual-bitcode
roundtrip test to also check that round-tripping with the intrinsic works.
The intrinsic marks the position of a source level assignment.
The llvm.dbg.assign interface looks like this (each parameter is wrapped in
MetadataAsValue, and Value * type parameters are first wrapped in
ValueAsMetadata):
void @llvm.dbg.assign(Value *Value,
DIExpression *ValueExpression,
DILocalVariable *Variable,
DIAssignID *ID,
Value *Address,
DIExpression *AddressExpression)
The first three parameters look and behave like an llvm.dbg.value. ID is a
reference to a store. The intrinsic is "linked to" instructions in the same
function that use the same ID as an attachment. That is mostly conceptual at
this point; the two-way link infrastructure will come in another patch. Address
is the destination address of the store and it is modified by
AddressExpression. LLVM currently encodes variable fragment information in
DIExpressions, so as an implementation quirk the FragmentInfo for Variable is
contained within ValueExpression only.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D132223
WinEHPrepare marks any function call from EH funclets as unreachable, if it's not a nounwind intrinsic or has no proper funclet bundle operand. This
affects ARC intrinsics on Windows, because they are lowered to regular function calls in the PreISelIntrinsicLowering pass. It caused silent binary truncations and crashes during unwinding with the GNUstep ObjC runtime: https://github.com/gnustep/libobjc2/issues/222
This patch adds a new function `llvm::IntrinsicInst::mayLowerToFunctionCall()` that aims to collect all affected intrinsic IDs.
* Clang CodeGen uses it to determine whether or not it must emit a funclet bundle operand.
* PreISelIntrinsicLowering asserts that the function returns true for all ObjC runtime calls it lowers.
* LLVM uses it to determine whether or not a funclet bundle operand must be propagated to inlined call sites.
Reviewed By: theraven
Differential Revision: https://reviews.llvm.org/D128190
Undef tokens may appear in unreached code as result of RAUW of some optimization,
and it should not be considered as bad IR.
Patch by Dmitry Bakunevich!
Differential Revision: https://reviews.llvm.org/D128904
Reviewed By: mkazantsev
This patch mostly follows up on D121292 which introduced the vp.fcmp
intrinsic.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D122729
This patch adds the first support for vector-predicated comparison
intrinsics, starting with vp.fcmp. It uses metadata to encode its
condition code, like the llvm.experimental.constrained.fcmp intrinsic.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D121292
This adds LLVMAnyPointerToElt to use instead of LLVMPointerToElt.
This allows us to preserve the address space as part of the type
overload for the intrinsic, but still require the vector element
type to match the pointer type.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D122042
VPIntrinsic::getStaticVectorLength infers the operational vector length
of a VPIntrinsic instance from a type that is used with the intrinsic.
The function used the mask operand before. Yet, vp.merge|select do not
have a mask operand (in the predicating sense that the other VP
intrinsics are using them - it is a selection mask for them). Fallback
to the return type to fix this.
Reviewed By: kaz7
Differential Revision: https://reviews.llvm.org/D121913
