mirrors/llvm-project
0
0
Fork 0
mirror of https://github.com/llvm/llvm-project.git synced 2025-05-07 16:16:07 +00:00
Code Issues Projects Releases Wiki Activity
llvm-project/llvm/lib/CodeGen/PreISelIntrinsicLowering.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

423 lines
15 KiB
C++
Raw Normal View History

[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
//===- PreISelIntrinsicLowering.cpp - Pre-ISel intrinsic lowering pass ----===//
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 08:50:56 +00:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
//
//===----------------------------------------------------------------------===//
//
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
// This pass implements IR lowering for the llvm.memcpy, llvm.memmove,
// llvm.memset, llvm.load.relative and llvm.objc.* intrinsics.
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
//
//===----------------------------------------------------------------------===//
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
#include "llvm/CodeGen/PreISelIntrinsicLowering.h"
[ObjC] Override TailCallKind when lowering objc intrinsics The tail-call-kind-ness is known by the ObjCARC analysis and can be enforced while lowering the intrinsics to calls. This allows us to get the requested tail calls at -O0 without trying to preserve the attributes throughout passes that change code even at -O0 ,like the Always Inliner, where the ObjCOpt pass doesn't run. Differential Revision: https://reviews.llvm.org/D69980
2019-11-06 17:09:55 -08:00
#include "llvm/Analysis/ObjCARCInstKind.h"
[ObjC][ARC] Replace uses of ObjC intrinsics that are arguments of operand bundle "clang.arc.attachedcall" with ObjC runtime functions The existing code only handles the case where the intrinsic being rewritten is used as the called function pointer of a call/invoke.
2021-11-08 21:19:07 -08:00
#include "llvm/Analysis/ObjCARCUtil.h"
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
#include "llvm/Analysis/TargetTransformInfo.h"
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
#include "llvm/CodeGen/Passes.h"
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
#include "llvm/CodeGen/TargetLowering.h"
#include "llvm/CodeGen/TargetPassConfig.h"
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
#include "llvm/IR/Function.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Instructions.h"
[WinEH] Apply funclet operand bundles to nounwind intrinsics that lower to function calls in the course of IR transforms WinEHPrepare marks any function call from EH funclets as unreachable, if it's not a nounwind intrinsic or has no proper funclet bundle operand. This affects ARC intrinsics on Windows, because they are lowered to regular function calls in the PreISelIntrinsicLowering pass. It caused silent binary truncations and crashes during unwinding with the GNUstep ObjC runtime: https://github.com/gnustep/libobjc2/issues/222 This patch adds a new function `llvm::IntrinsicInst::mayLowerToFunctionCall()` that aims to collect all affected intrinsic IDs. * Clang CodeGen uses it to determine whether or not it must emit a funclet bundle operand. * PreISelIntrinsicLowering asserts that the function returns true for all ObjC runtime calls it lowers. * LLVM uses it to determine whether or not a funclet bundle operand must be propagated to inlined call sites. Reviewed By: theraven Differential Revision: https://reviews.llvm.org/D128190
2022-07-26 11:40:48 +02:00
#include "llvm/IR/IntrinsicInst.h"
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
#include "llvm/IR/Module.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
#include "llvm/IR/Type.h"
Sink all InitializePasses.h includes This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-11-13 13:15:01 -08:00
#include "llvm/InitializePasses.h"
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
#include "llvm/Pass.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
#include "llvm/Support/Casting.h"
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
#include "llvm/Target/TargetMachine.h"
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
#include "llvm/Transforms/Utils/LowerMemIntrinsics.h"
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
using namespace llvm;
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
/// Threshold to leave statically sized memory intrinsic calls. Calls of known
/// size larger than this will be expanded by the pass. Calls of unknown or
/// lower size will be left for expansion in codegen.
static cl::opt<int64_t> MemIntrinsicExpandSizeThresholdOpt(
"mem-intrinsic-expand-size",
cl::desc("Set minimum mem intrinsic size to expand in IR"), cl::init(-1),
cl::Hidden);
namespace {
struct PreISelIntrinsicLowering {
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
const TargetMachine &TM;
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
const function_ref<TargetTransformInfo &(Function &)> LookupTTI;
/// If this is true, assume it's preferably to leave memory intrinsic calls
/// for replacement with a library call later. Otherwise this depends on
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
/// TargetLoweringInfo availability of the corresponding function.
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
const bool UseMemIntrinsicLibFunc;
explicit PreISelIntrinsicLowering(
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
const TargetMachine &TM_,
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
function_ref<TargetTransformInfo &(Function &)> LookupTTI_,
bool UseMemIntrinsicLibFunc_ = true)
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
: TM(TM_), LookupTTI(LookupTTI_),
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
UseMemIntrinsicLibFunc(UseMemIntrinsicLibFunc_) {}
static bool shouldExpandMemIntrinsicWithSize(Value *Size,
const TargetTransformInfo &TTI);
bool expandMemIntrinsicUses(Function &F) const;
bool lowerIntrinsics(Module &M) const;
};
} // namespace
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
static bool lowerLoadRelative(Function &F) {
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
if (F.use_empty())
return false;
bool Changed = false;
Type *Int32Ty = Type::getInt32Ty(F.getContext());
[CodeGen] Use make_early_inc_range (NFC)
2021-11-01 22:38:48 -07:00
for (Use &U : llvm::make_early_inc_range(F.uses())) {
auto CI = dyn_cast<CallInst>(U.getUser());
[IR] Replace all uses of CallBase::getCalledValue() with getCalledOperand(). This method has been commented as deprecated for a while. Remove it and replace all uses with the equivalent getCalledOperand(). I also made a few cleanups in here. For example, to removes use of getElementType on a pointer when we could just use getFunctionType from the call. Differential Revision: https://reviews.llvm.org/D78882
2020-04-27 20:15:59 -07:00
if (!CI || CI->getCalledOperand() != &F)
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
continue;
IRBuilder<> B(CI);
Value *OffsetPtr =
[IRBuilder] Add CreatePtrAdd() method (NFC) (#77582) This abstracts over the common pattern of creating a gep with i8 element type.
2024-01-12 14:21:21 +01:00
B.CreatePtrAdd(CI->getArgOperand(0), CI->getArgOperand(1));
[llvm] Drop some typed pointer handling/bitcasts Differential Revision: https://reviews.llvm.org/D157016
2023-08-01 12:15:14 +02:00
Value *OffsetI32 = B.CreateAlignedLoad(Int32Ty, OffsetPtr, Align(4));
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
[IRBuilder] Add CreatePtrAdd() method (NFC) (#77582) This abstracts over the common pattern of creating a gep with i8 element type.
2024-01-12 14:21:21 +01:00
Value *ResultPtr = B.CreatePtrAdd(CI->getArgOperand(0), OffsetI32);
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
CI->replaceAllUsesWith(ResultPtr);
CI->eraseFromParent();
Changed = true;
}
return Changed;
}
[ObjC] Override TailCallKind when lowering objc intrinsics The tail-call-kind-ness is known by the ObjCARC analysis and can be enforced while lowering the intrinsics to calls. This allows us to get the requested tail calls at -O0 without trying to preserve the attributes throughout passes that change code even at -O0 ,like the Always Inliner, where the ObjCOpt pass doesn't run. Differential Revision: https://reviews.llvm.org/D69980
2019-11-06 17:09:55 -08:00
// ObjCARC has knowledge about whether an obj-c runtime function needs to be
// always tail-called or never tail-called.
static CallInst::TailCallKind getOverridingTailCallKind(const Function &F) {
objcarc::ARCInstKind Kind = objcarc::GetFunctionClass(&F);
if (objcarc::IsAlwaysTail(Kind))
return CallInst::TCK_Tail;
else if (objcarc::IsNeverTail(Kind))
return CallInst::TCK_NoTail;
return CallInst::TCK_None;
}
Add nonlazybind to objc_retain/objc_release when converting from intrinsics. For performance reasons, clang set nonlazybind on these functions. Now that we are using intrinsics instead of runtime calls, we should set this attribute when creating the runtime functions. llvm-svn: 349558
2018-12-18 22:31:34 +00:00
static bool lowerObjCCall(Function &F, const char *NewFn,
bool setNonLazyBind = false) {
[WinEH] Apply funclet operand bundles to nounwind intrinsics that lower to function calls in the course of IR transforms WinEHPrepare marks any function call from EH funclets as unreachable, if it's not a nounwind intrinsic or has no proper funclet bundle operand. This affects ARC intrinsics on Windows, because they are lowered to regular function calls in the PreISelIntrinsicLowering pass. It caused silent binary truncations and crashes during unwinding with the GNUstep ObjC runtime: https://github.com/gnustep/libobjc2/issues/222 This patch adds a new function `llvm::IntrinsicInst::mayLowerToFunctionCall()` that aims to collect all affected intrinsic IDs. * Clang CodeGen uses it to determine whether or not it must emit a funclet bundle operand. * PreISelIntrinsicLowering asserts that the function returns true for all ObjC runtime calls it lowers. * LLVM uses it to determine whether or not a funclet bundle operand must be propagated to inlined call sites. Reviewed By: theraven Differential Revision: https://reviews.llvm.org/D128190
2022-07-26 11:40:48 +02:00
assert(IntrinsicInst::mayLowerToFunctionCall(F.getIntrinsicID()) &&
"Pre-ISel intrinsics do lower into regular function calls");
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
if (F.use_empty())
return false;
// If we haven't already looked up this function, check to see if the
// program already contains a function with this name.
Module *M = F.getParent();
[opaque pointer types] Add a FunctionCallee wrapper type, and use it. Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
2019-02-01 02:28:03 +00:00
FunctionCallee FCache = M->getOrInsertFunction(NewFn, F.getFunctionType());
Preserve the linkage for objc* intrinsics as clang will set them to weak_external in some cases Clang uses weak linkage for objc runtime functions when they are not available on the platform. The intrinsic has this linkage so we just need to pass that on to the runtime call. llvm-svn: 349559
2018-12-18 22:42:08 +00:00
[opaque pointer types] Add a FunctionCallee wrapper type, and use it. Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
2019-02-01 02:28:03 +00:00
if (Function *Fn = dyn_cast<Function>(FCache.getCallee())) {
Preserve the linkage for objc* intrinsics as clang will set them to weak_external in some cases Clang uses weak linkage for objc runtime functions when they are not available on the platform. The intrinsic has this linkage so we just need to pass that on to the runtime call. llvm-svn: 349559
2018-12-18 22:42:08 +00:00
Fn->setLinkage(F.getLinkage());
if (setNonLazyBind && !Fn->isWeakForLinker()) {
// If we have Native ARC, set nonlazybind attribute for these APIs for
// performance.
Add nonlazybind to objc_retain/objc_release when converting from intrinsics. For performance reasons, clang set nonlazybind on these functions. Now that we are using intrinsics instead of runtime calls, we should set this attribute when creating the runtime functions. llvm-svn: 349558
2018-12-18 22:31:34 +00:00
Fn->addFnAttr(Attribute::NonLazyBind);
Preserve the linkage for objc* intrinsics as clang will set them to weak_external in some cases Clang uses weak linkage for objc runtime functions when they are not available on the platform. The intrinsic has this linkage so we just need to pass that on to the runtime call. llvm-svn: 349559
2018-12-18 22:42:08 +00:00
}
}
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
[ObjC] Override TailCallKind when lowering objc intrinsics The tail-call-kind-ness is known by the ObjCARC analysis and can be enforced while lowering the intrinsics to calls. This allows us to get the requested tail calls at -O0 without trying to preserve the attributes throughout passes that change code even at -O0 ,like the Always Inliner, where the ObjCOpt pass doesn't run. Differential Revision: https://reviews.llvm.org/D69980
2019-11-06 17:09:55 -08:00
CallInst::TailCallKind OverridingTCK = getOverridingTailCallKind(F);
[llvm] Use make_early_inc_range (NFC)
2021-11-15 21:28:46 -08:00
for (Use &U : llvm::make_early_inc_range(F.uses())) {
auto *CB = cast<CallBase>(U.getUser());
[ObjC][ARC] Replace uses of ObjC intrinsics that are arguments of operand bundle "clang.arc.attachedcall" with ObjC runtime functions The existing code only handles the case where the intrinsic being rewritten is used as the called function pointer of a call/invoke.
2021-11-08 21:19:07 -08:00
if (CB->getCalledFunction() != &F) {
objcarc::ARCInstKind Kind = objcarc::getAttachedARCFunctionKind(CB);
(void)Kind;
assert((Kind == objcarc::ARCInstKind::RetainRV ||
[ObjCARC] Use "UnsafeClaimRV" to refer to unsafeClaim in enums. NFC. This matches the actual runtime function more closely. I considered also renaming both RetainRV/UnsafeClaimRV to end with "ARV", for AutoreleasedReturnValue, but there's less potential for confusion there.
2022-01-24 17:14:39 -08:00
Kind == objcarc::ARCInstKind::UnsafeClaimRV) &&
[ObjC][ARC] Replace uses of ObjC intrinsics that are arguments of operand bundle "clang.arc.attachedcall" with ObjC runtime functions The existing code only handles the case where the intrinsic being rewritten is used as the called function pointer of a call/invoke.
2021-11-08 21:19:07 -08:00
"use expected to be the argument of operand bundle "
"\"clang.arc.attachedcall\"");
[llvm] Use make_early_inc_range (NFC)
2021-11-15 21:28:46 -08:00
U.set(FCache.getCallee());
[ObjC][ARC] Replace uses of ObjC intrinsics that are arguments of operand bundle "clang.arc.attachedcall" with ObjC runtime functions The existing code only handles the case where the intrinsic being rewritten is used as the called function pointer of a call/invoke.
2021-11-08 21:19:07 -08:00
continue;
}
auto *CI = cast<CallInst>(CB);
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
assert(CI->getCalledFunction() && "Cannot lower an indirect call!");
[IR][NFC] Update IRBuilder to use InsertPosition (#96497) Uses the new InsertPosition class (added in #94226) to simplify some of the IRBuilder interface, and removes the need to pass a BasicBlock alongside a BasicBlock::iterator, using the fact that we can now get the parent basic block from the iterator even if it points to the sentinel. This patch removes the BasicBlock argument from each constructor or call to setInsertPoint. This has no functional effect, but later on as we look to remove the `Instruction *InsertBefore` argument from instruction-creation (discussed [here](https://discourse.llvm.org/t/psa-instruction-constructors-changing-to-iterator-only-insertion/77845)), this will simplify the process by allowing us to deprecate the InsertPosition constructor directly and catch all the cases where we use instructions rather than iterators.
2024-06-24 17:27:43 +01:00
IRBuilder<> Builder(CI->getIterator());
[CodeGen] Construct SmallVector with iterator ranges (NFC)
2020-12-31 09:39:11 -08:00
SmallVector<Value *, 8> Args(CI->args());
[WinEH] Apply funclet operand bundles to nounwind intrinsics that lower to function calls in the course of IR transforms WinEHPrepare marks any function call from EH funclets as unreachable, if it's not a nounwind intrinsic or has no proper funclet bundle operand. This affects ARC intrinsics on Windows, because they are lowered to regular function calls in the PreISelIntrinsicLowering pass. It caused silent binary truncations and crashes during unwinding with the GNUstep ObjC runtime: https://github.com/gnustep/libobjc2/issues/222 This patch adds a new function `llvm::IntrinsicInst::mayLowerToFunctionCall()` that aims to collect all affected intrinsic IDs. * Clang CodeGen uses it to determine whether or not it must emit a funclet bundle operand. * PreISelIntrinsicLowering asserts that the function returns true for all ObjC runtime calls it lowers. * LLVM uses it to determine whether or not a funclet bundle operand must be propagated to inlined call sites. Reviewed By: theraven Differential Revision: https://reviews.llvm.org/D128190
2022-07-26 11:40:48 +02:00
SmallVector<llvm::OperandBundleDef, 1> BundleList;
CI->getOperandBundlesAsDefs(BundleList);
CallInst *NewCI = Builder.CreateCall(FCache, Args, BundleList);
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
NewCI->setName(CI->getName());
[ObjC] Override TailCallKind when lowering objc intrinsics The tail-call-kind-ness is known by the ObjCARC analysis and can be enforced while lowering the intrinsics to calls. This allows us to get the requested tail calls at -O0 without trying to preserve the attributes throughout passes that change code even at -O0 ,like the Always Inliner, where the ObjCOpt pass doesn't run. Differential Revision: https://reviews.llvm.org/D69980
2019-11-06 17:09:55 -08:00
// Try to set the most appropriate TailCallKind based on both the current
// attributes and the ones that we could get from ObjCARC's special
// knowledge of the runtime functions.
//
// std::max respects both requirements of notail and tail here:
// * notail on either the call or from ObjCARC becomes notail
// * tail on either side is stronger than none, but not notail
CallInst::TailCallKind TCK = CI->getTailCallKind();
NewCI->setTailCallKind(std::max(TCK, OverridingTCK));
Reland "[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn." This reverts commit cb62f67088aaf79493350547f74870318b71acc5. Fixes: https://github.com/llvm/llvm-project/issues/69658
2023-10-20 09:46:03 -07:00
// Transfer the 'returned' attribute from the intrinsic to the call site.
// By applying this only to intrinsic call sites, we avoid applying it to
// non-ARC explicit calls to things like objc_retain which have not been
// auto-upgraded to use the intrinsics.
unsigned Index;
if (F.getAttributes().hasAttrSomewhere(Attribute::Returned, &Index) &&
Index)
NewCI->addParamAttr(Index - AttributeList::FirstArgIndex,
Attribute::Returned);
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
if (!CI->use_empty())
CI->replaceAllUsesWith(NewCI);
CI->eraseFromParent();
}
return true;
}
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
// TODO: Should refine based on estimated number of accesses (e.g. does it
// require splitting based on alignment)
bool PreISelIntrinsicLowering::shouldExpandMemIntrinsicWithSize(
Value *Size, const TargetTransformInfo &TTI) {
ConstantInt *CI = dyn_cast<ConstantInt>(Size);
if (!CI)
return true;
uint64_t Threshold = MemIntrinsicExpandSizeThresholdOpt.getNumOccurrences()
? MemIntrinsicExpandSizeThresholdOpt
: TTI.getMaxMemIntrinsicInlineSizeThreshold();
uint64_t SizeVal = CI->getZExtValue();
// Treat a threshold of 0 as a special case to force expansion of all
// intrinsics, including size 0.
return SizeVal > Threshold || Threshold == 0;
}
[PreISelIntrinsicLowering] Use TLI for correct function We should query the subtarget of the calling function, not of the intrinsic. This probably makes no functional difference (as libcalls are unlikely to vary across subtargets), but fixes minor compile-time regressions from unnecessary subtarget instantiations. Followup to D157567. Differential Revision: https://reviews.llvm.org/D157848
2023-08-14 10:39:05 +02:00
static bool canEmitLibcall(const TargetMachine &TM, Function *F,
RTLIB::Libcall LC) {
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
// TODO: Should this consider the address space of the memcpy?
[PreISelIntrinsicLowering] Use TLI for correct function We should query the subtarget of the calling function, not of the intrinsic. This probably makes no functional difference (as libcalls are unlikely to vary across subtargets), but fixes minor compile-time regressions from unnecessary subtarget instantiations. Followup to D157567. Differential Revision: https://reviews.llvm.org/D157848
2023-08-14 10:39:05 +02:00
const TargetLowering *TLI = TM.getSubtargetImpl(*F)->getTargetLowering();
return TLI->getLibcallName(LC) != nullptr;
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
}
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
// TODO: Handle atomic memcpy and memcpy.inline
// TODO: Pass ScalarEvolution
bool PreISelIntrinsicLowering::expandMemIntrinsicUses(Function &F) const {
Intrinsic::ID ID = F.getIntrinsicID();
bool Changed = false;
for (User *U : llvm::make_early_inc_range(F.users())) {
Instruction *Inst = cast<Instruction>(U);
switch (ID) {
case Intrinsic::memcpy: {
auto *Memcpy = cast<MemCpyInst>(Inst);
Function *ParentFunc = Memcpy->getFunction();
const TargetTransformInfo &TTI = LookupTTI(*ParentFunc);
if (shouldExpandMemIntrinsicWithSize(Memcpy->getLength(), TTI)) {
[PreISelIntrinsicLowering] Use TLI for correct function We should query the subtarget of the calling function, not of the intrinsic. This probably makes no functional difference (as libcalls are unlikely to vary across subtargets), but fixes minor compile-time regressions from unnecessary subtarget instantiations. Followup to D157567. Differential Revision: https://reviews.llvm.org/D157848
2023-08-14 10:39:05 +02:00
if (UseMemIntrinsicLibFunc &&
canEmitLibcall(TM, ParentFunc, RTLIB::MEMCPY))
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
break;
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
// TODO: For optsize, emit the loop into a separate function
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
expandMemCpyAsLoop(Memcpy, TTI);
Changed = true;
Memcpy->eraseFromParent();
}
break;
}
case Intrinsic::memmove: {
auto *Memmove = cast<MemMoveInst>(Inst);
Function *ParentFunc = Memmove->getFunction();
const TargetTransformInfo &TTI = LookupTTI(*ParentFunc);
if (shouldExpandMemIntrinsicWithSize(Memmove->getLength(), TTI)) {
[PreISelIntrinsicLowering] Use TLI for correct function We should query the subtarget of the calling function, not of the intrinsic. This probably makes no functional difference (as libcalls are unlikely to vary across subtargets), but fixes minor compile-time regressions from unnecessary subtarget instantiations. Followup to D157567. Differential Revision: https://reviews.llvm.org/D157848
2023-08-14 10:39:05 +02:00
if (UseMemIntrinsicLibFunc &&
canEmitLibcall(TM, ParentFunc, RTLIB::MEMMOVE))
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
break;
LowerMemIntrinsics: Handle inserting addrspacecast for memmove lowering We're missing a trivial non-AA way to check for non-aliasing address spaces.
2023-06-10 12:20:43 -04:00
if (expandMemMoveAsLoop(Memmove, TTI)) {
LowerMemIntrinsics: Skip memmove with different address spaces This is a quick fix for an assert when the source and dest have different address spaces. The pointer compare needs to have matching types, but we can't generically introduce addrspacecast and we don't know if the address spaces alias.
2023-06-10 11:58:09 -04:00
Changed = true;
Memmove->eraseFromParent();
}
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
}
break;
}
case Intrinsic::memset: {
auto *Memset = cast<MemSetInst>(Inst);
Function *ParentFunc = Memset->getFunction();
const TargetTransformInfo &TTI = LookupTTI(*ParentFunc);
if (shouldExpandMemIntrinsicWithSize(Memset->getLength(), TTI)) {
[PreISelIntrinsicLowering] Use TLI for correct function We should query the subtarget of the calling function, not of the intrinsic. This probably makes no functional difference (as libcalls are unlikely to vary across subtargets), but fixes minor compile-time regressions from unnecessary subtarget instantiations. Followup to D157567. Differential Revision: https://reviews.llvm.org/D157848
2023-08-14 10:39:05 +02:00
if (UseMemIntrinsicLibFunc &&
canEmitLibcall(TM, ParentFunc, RTLIB::MEMSET))
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
break;
expandMemSetAsLoop(Memset);
Changed = true;
Memset->eraseFromParent();
}
break;
}
default:
llvm_unreachable("unhandled intrinsic");
}
}
return Changed;
}
bool PreISelIntrinsicLowering::lowerIntrinsics(Module &M) const {
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
bool Changed = false;
for (Function &F : M) {
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
switch (F.getIntrinsicID()) {
default:
break;
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
case Intrinsic::memcpy:
case Intrinsic::memmove:
case Intrinsic::memset:
Changed |= expandMemIntrinsicUses(F);
break;
PreISelIntrinsicLowering: Identify load.relative with intrinsic id
2023-06-07 08:16:51 -04:00
case Intrinsic::load_relative:
Changed |= lowerLoadRelative(F);
break;
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
case Intrinsic::objc_autorelease:
Changed |= lowerObjCCall(F, "objc_autorelease");
break;
case Intrinsic::objc_autoreleasePoolPop:
Changed |= lowerObjCCall(F, "objc_autoreleasePoolPop");
break;
case Intrinsic::objc_autoreleasePoolPush:
Changed |= lowerObjCCall(F, "objc_autoreleasePoolPush");
break;
case Intrinsic::objc_autoreleaseReturnValue:
Changed |= lowerObjCCall(F, "objc_autoreleaseReturnValue");
break;
case Intrinsic::objc_copyWeak:
Changed |= lowerObjCCall(F, "objc_copyWeak");
break;
case Intrinsic::objc_destroyWeak:
Changed |= lowerObjCCall(F, "objc_destroyWeak");
break;
case Intrinsic::objc_initWeak:
Changed |= lowerObjCCall(F, "objc_initWeak");
break;
case Intrinsic::objc_loadWeak:
Changed |= lowerObjCCall(F, "objc_loadWeak");
break;
case Intrinsic::objc_loadWeakRetained:
Changed |= lowerObjCCall(F, "objc_loadWeakRetained");
break;
case Intrinsic::objc_moveWeak:
Changed |= lowerObjCCall(F, "objc_moveWeak");
break;
case Intrinsic::objc_release:
Add nonlazybind to objc_retain/objc_release when converting from intrinsics. For performance reasons, clang set nonlazybind on these functions. Now that we are using intrinsics instead of runtime calls, we should set this attribute when creating the runtime functions. llvm-svn: 349558
2018-12-18 22:31:34 +00:00
Changed |= lowerObjCCall(F, "objc_release", true);
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
break;
case Intrinsic::objc_retain:
Add nonlazybind to objc_retain/objc_release when converting from intrinsics. For performance reasons, clang set nonlazybind on these functions. Now that we are using intrinsics instead of runtime calls, we should set this attribute when creating the runtime functions. llvm-svn: 349558
2018-12-18 22:31:34 +00:00
Changed |= lowerObjCCall(F, "objc_retain", true);
Rewrite objc intrinsics to runtime methods in PreISelIntrinsicLowering instead of SDAG. SelectionDAG currently changes these intrinsics to function calls, but that won't work for other ISel's. Also we want to eventually support nonlazybind and weak linkage coming from the front-end which we can't do in SelectionDAG. llvm-svn: 349552
2018-12-18 22:20:03 +00:00
break;
case Intrinsic::objc_retainAutorelease:
Changed |= lowerObjCCall(F, "objc_retainAutorelease");
break;
case Intrinsic::objc_retainAutoreleaseReturnValue:
Changed |= lowerObjCCall(F, "objc_retainAutoreleaseReturnValue");
break;
case Intrinsic::objc_retainAutoreleasedReturnValue:
Changed |= lowerObjCCall(F, "objc_retainAutoreleasedReturnValue");
break;
case Intrinsic::objc_retainBlock:
Changed |= lowerObjCCall(F, "objc_retainBlock");
break;
case Intrinsic::objc_storeStrong:
Changed |= lowerObjCCall(F, "objc_storeStrong");
break;
case Intrinsic::objc_storeWeak:
Changed |= lowerObjCCall(F, "objc_storeWeak");
break;
case Intrinsic::objc_unsafeClaimAutoreleasedReturnValue:
Changed |= lowerObjCCall(F, "objc_unsafeClaimAutoreleasedReturnValue");
break;
case Intrinsic::objc_retainedObject:
Changed |= lowerObjCCall(F, "objc_retainedObject");
break;
case Intrinsic::objc_unretainedObject:
Changed |= lowerObjCCall(F, "objc_unretainedObject");
break;
case Intrinsic::objc_unretainedPointer:
Changed |= lowerObjCCall(F, "objc_unretainedPointer");
break;
case Intrinsic::objc_retain_autorelease:
Changed |= lowerObjCCall(F, "objc_retain_autorelease");
break;
case Intrinsic::objc_sync_enter:
Changed |= lowerObjCCall(F, "objc_sync_enter");
break;
case Intrinsic::objc_sync_exit:
Changed |= lowerObjCCall(F, "objc_sync_exit");
break;
}
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
}
return Changed;
}
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
namespace {
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
class PreISelIntrinsicLoweringLegacyPass : public ModulePass {
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
public:
static char ID;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
PreISelIntrinsicLoweringLegacyPass() : ModulePass(ID) {}
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<TargetTransformInfoWrapperPass>();
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
AU.addRequired<TargetPassConfig>();
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
}
bool runOnModule(Module &M) override {
auto LookupTTI = [this](Function &F) -> TargetTransformInfo & {
return this->getAnalysis<TargetTransformInfoWrapperPass>().getTTI(F);
};
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
const auto &TM = getAnalysis<TargetPassConfig>().getTM<TargetMachine>();
PreISelIntrinsicLowering Lowering(TM, LookupTTI);
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
return Lowering.lowerIntrinsics(M);
}
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
};
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
} // end anonymous namespace
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
char PreISelIntrinsicLoweringLegacyPass::ID;
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
INITIALIZE_PASS_BEGIN(PreISelIntrinsicLoweringLegacyPass,
"pre-isel-intrinsic-lowering",
"Pre-ISel Intrinsic Lowering", false, false)
INITIALIZE_PASS_DEPENDENCY(TargetPassConfig)
INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_END(PreISelIntrinsicLoweringLegacyPass,
"pre-isel-intrinsic-lowering",
"Pre-ISel Intrinsic Lowering", false, false)
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 313194
2017-09-13 21:15:20 +00:00
ModulePass *llvm::createPreISelIntrinsicLoweringPass() {
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
return new PreISelIntrinsicLoweringLegacyPass();
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
}
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
PreservedAnalyses PreISelIntrinsicLoweringPass::run(Module &M,
ModuleAnalysisManager &AM) {
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
auto &FAM = AM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
auto LookupTTI = [&FAM](Function &F) -> TargetTransformInfo & {
return FAM.getResult<TargetIRAnalysis>(F);
};
PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions We need a better mechanism for expressing which calls you are allowed to emit and which calls are recognized. This should be applied to the 17 branch.
2023-08-09 20:26:11 -04:00
PreISelIntrinsicLowering Lowering(TM, LookupTTI);
CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering Expand large or unknown size memory intrinsics into loops in the default lowering pipeline if the target doesn't have the corresponding libfunc. Previously AMDGPU had a custom pass which existed to call the expansion utilities. With a default no-libcall option, we can remove the libfunc checks in LoopIdiomRecognize for these, which never made any sense. This also provides a path to lifting the immarg restriction on llvm.memcpy.inline. There seems to be a bug where TLI reports functions as available if you use -march and not -mtriple.
2023-06-07 09:03:17 -04:00
if (!Lowering.lowerIntrinsics(M))
[PM] Port PreISelIntrinsicLowering to the new PM llvm-svn: 273713
2016-06-24 20:13:42 +00:00
return PreservedAnalyses::all();
else
return PreservedAnalyses::none();
Introduce llvm.load.relative intrinsic. This intrinsic takes two arguments, ``%ptr`` and ``%offset``. It loads a 32-bit value from the address ``%ptr + %offset``, adds ``%ptr`` to that value and returns it. The constant folder specifically recognizes the form of this intrinsic and the constant initializers it may load from; if a loaded constant initializer is known to have the form ``i32 trunc(x - %ptr)``, the intrinsic call is folded to ``x``. LLVM provides that the calculation of such a constant initializer will not overflow at link time under the medium code model if ``x`` is an ``unnamed_addr`` function. However, it does not provide this guarantee for a constant initializer folded into a function body. This intrinsic can be used to avoid the possibility of overflows when loading from such a constant. Differential Revision: http://reviews.llvm.org/D18367 llvm-svn: 267223
2016-04-22 21:18:02 +00:00
}
Reference in New Issue Copy Permalink