We can use Intrinsic::getDeclaration() here, we just have to pass
the correct arguments. This function accepts only the mangled types,
not all argument types.
With these three intrinsics it's probable faster to check the number of
arguments first and then check the names. We can also handle ctlz and
cttz in the same block.
This is an attempt at rebooting https://reviews.llvm.org/D28990
I've included AutoUpgrade changes to modify the data layout to satisfy the compatible layout check. But this does mean alloca, loads, stores, etc in old IR will automatically get this new alignment.
This should fix PR46320.
Reviewed By: echristo, rnk, tmgross
Differential Revision: https://reviews.llvm.org/D86310
Partial progress towards removing in-tree uses of `getPointerTo()`,
by employing the following options:
* Drop the call entirely if the sole purpose of it is to support a no-op
bitcast (remove the no-op bitcast as well).
* Replace with `PointerType::get()`/`PointerType::getUnqual()`
This is a NFC cleanup effort.
Reviewed By: barannikov88
Differential Revision: https://reviews.llvm.org/D155232
One of the main user of these kind of coroutines is swift. There yield-once (`retcon.once`) coroutines are used to temporary "expose" pointers to internal fields of various objects creating borrow scopes.
However, in some cases it might be useful also to allow these coroutines to produce a normal result, but there is no convenient way to represent this (as compared to switched-resume kind of coroutines where C++ `co_return`
is transformed to a member / callback call on promise object).
The extension is simple: we allow continuation function to have a non-void result and accept optional extra arguments via a special `llvm.coro.end.result` intrinsic that would essentially forward them as normal results.
The NVPTX intrinsics are under 'n'. Use the consume_front API, so fix
that. Refactor the helper function to group matchers on the first
component and check that first. Do similarly with the final set of
intrinsics, which have a lot of commonality in the matching. Finally
reorder the argument/return type checking wrt name checking -- the
former is going to be cheaper, so do that first before checking the
name.#
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D158445
Check for 'wasm.' prefix before proceeding, and a bit of common handling
for some of the intrinsics therein.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D158370
Consume the dbg. prefix before looking further. Also merge the matched
path.
Differential Revision: https://reviews.llvm.org/D158341
Reviewed By: nikic
It is more efficient to gate on the prefix before checking for the
individual cases. Also, the regexps weren't consistent about escaping '.'.
Differential Revision: https://reviews.llvm.org/D157923
Reviewed By: arsenm
There's a large amount of commonality in the riscv upgrader, make that
clearer. And check for a riscv prefix before diving in.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D157924
If the only purpose of the upgrade is a mangling change, this is
handled generically by intrinsic remangling, and does not require
any hand-written code.
The address-space intrinsics need detecting, and all have very similar
processing. Refactor to do just that.
Differential Revision: https://reviews.llvm.org/D157763
Reviewed By: jroelofs
The old syncscope parameter never really worked correctly, but
effectively gave "workgroup" scope. Use something faster than system
but more correct than before.
https://reviews.llvm.org/D157389
Previously we returned i32 on RV32 and i64 on RV64. The instructions
only consume 32 bits and only produce 32 bits. For RV64, the result
is sign extended to 64 bits like *W instructions.
This patch removes this detail from the interface to improve
portability and consistency. This matches the proposal for scalar
intrinsics here https://github.com/riscv-non-isa/riscv-c-api-doc/pull/44
I've included IR autoupgrade support as well.
I'll be doing this for other builtins/intrinsics that currently use
'long' in other patches.
Reviewed By: VincentWu
Differential Revision: https://reviews.llvm.org/D154647
Currently, bf16 has been scatteredly added to the PTX codegen. This patch aims to complete the set of instructions and code path required to support bf16 data type.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D144911
Co-authored-by: Artem Belevich <tra@google.com>
Move `AttributeMask` out of `llvm/IR/Attributes.h` to a new file
`llvm/IR/AttributeMask.h`. After doing this we can remove the
`#include <bitset>` and `#include <set>` directives from `Attributes.h`.
Since there are many headers including `Attributes.h`, but not needing
the definition of `AttributeMask`, this causes unnecessary bloating of
the translation units and slows down compilation.
This commit adds in the include directive for `llvm/IR/AttributeMask.h`
to the handful of source files that need to see the definition.
This reduces the total number of preprocessing tokens across the LLVM
source files in lib from (roughly) 1,917,509,187 to 1,902,982,273 - a
reduction of ~0.76%. This should result in a small improvement in
compilation time.
Differential Revision: https://reviews.llvm.org/D153728
This matches gcc. It also lets us fix a bug that the byteselect
predicate was not being evaluated in tablegen. We can't have i8
TImmLeaf in tablegen because i8 isn't a type for any register class.
I've added AutoUpgrade support for the IR intrinsics.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D152627
Re-land D145441 with data layout upgrade code fixed to not break OpenMP.
This reverts commit 3f2fbe92d0f40bcb46db7636db9ec3f7e7899b27.
Differential Revision: https://reviews.llvm.org/D149776
Per discussion at
https://discourse.llvm.org/t/representing-buffer-descriptors-in-the-amdgpu-target-call-for-suggestions/68798,
we define two new address spaces for AMDGCN targets.
The first is address space 7, a non-integral address space (which was
already in the data layout) that has 160-bit pointers (which are
256-bit aligned) and uses a 32-bit offset. These pointers combine a
128-bit buffer descriptor and a 32-bit offset, and will be usable with
normal LLVM operations (load, store, GEP). However, they will be
rewritten out of existence before code generation.
The second of these is address space 8, the address space for "buffer
resources". These will be used to represent the resource arguments to
buffer instructions, and new buffer intrinsics will be defined that
take them instead of <4 x i32> as resource arguments. ptr
addrspace(8). These pointers are 128-bits long (with the same
alignment). They must not be used as the arguments to getelementptr or
otherwise used in address computations, since they can have
arbitrarily complex inherent addressing semantics that can't be
represented in LLVM. Even though, like their address space 7 cousins,
these pointers have deterministic ptrtoint/inttoptr semantics, they
are defined to be non-integral in order to prevent optimizations that
rely on pointers being a [0, [addr_max]] value from applying to them.
Future work includes:
- Defining new buffer intrinsics that take ptr addrspace(8) resources.
- A late rewrite to turn address space 7 operations into buffer
intrinsics and offset computations.
This commit also updates the "fallback address space" for buffer
intrinsics to the buffer resource, and updates the alias analysis
table.
Depends on D143437
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D145441
Auto-upgrade can be expensive, especially UpgradeDebugInfo() since it runs the verifier.
With this patch, we can specify that the imported bitcode is built with the same revision of LLVM, meaning there's no need to run any auto-upgrading. For now, limit this to just debug info since projects like Rust want to support multiple versions of LLVM at the same time (at compiler build time, not at Rust source code build time) and run a subset of the autoupgrade functionality for simplicity.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D143229
Enabling opaque pointers has changed the mangled names of these two ARM
intrinsics:
`arm.mve.vldr.gather.offset.predicated.v2i64.p0i64.v2i64.v4i1`
`arm.mve.vstr.scatter.offset.predicated.p0i64.v2i64.v2i64.v4i1`
They are now spelled as:
`arm.mve.vldr.gather.offset.predicated.v2i64.p0.v2i64.v4i1`
`arm.mve.vstr.scatter.offset.predicated.p0.v2i64.v2i64.v4i1`
Upgrade intrinsic upgrading code to account for the change in names.
Differential Revision: https://reviews.llvm.org/D142900
Use deduction guides instead of helper functions.
The only non-automatic changes have been:
1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).
Per reviewers' comment, some useless makeArrayRef have been removed in the process.
This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.
Differential Revision: https://reviews.llvm.org/D140955
Address the inconsistency between FLT_ROUNDS_ and SET_ROUNDING SDAG
node. Rename FLT_ROUNDS_ to GET_ROUNDING and add llvm.get.rounding
intrinsic to replace flt.rounds.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D139507
The global constant arguments could be in a different address space
than the first argument, so we have to add another overloaded argument.
This patch was originally made for CHERI LLVM (where globals can be in
address space 200), but it also appears to be useful for in-tree targets
as can be seen from the test diffs.
Differential Revision: https://reviews.llvm.org/D138722
Almost all of the other SVE LLVM IR intrinsics take i32 values
for lane indices or other immediates. We should bring the bfloat
intrinsics in line with that. It will also make it easier to
add support for the SVE2.1 float intrinsics in future, since
they reuse the same underlying instruction classes.
I've maintained backwards compatibility with the old i64 variants
and used the autoupgrade mechanism.
Differential Revision: https://reviews.llvm.org/D138788
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
This was trying to auto-upgrade a read_register call with missing type
mangling. This first would break since getCalledFunction checks the
callee type is consistent, so this would assert there. After that,
the replacement code would die on the type mismatch. Be more
defensive and let the verifier code produce an error that the IR
is broken.
Although i32 type is illegal in the backend, RV64I has pretty good support for i32 types by using W instructions.
By adding n32 to the DataLayout string, middle end optimizations will consider i32 to be a native type. One known effect of this is enabling LoopStrengthReduce on loops with i32 induction variables. This can be beneficial because C/C++ code often has loops with i32 induction variables due to the use of `int` or `unsigned int`.
If this patch exposes performance issues, those are better addressed by tuning LSR or other passes.
Reviewed By: asb, frasercrmck
Differential Revision: https://reviews.llvm.org/D116735
This was remangling the old function rather than the new one, and
could result in failures when we were performing both a struct
return upgrade and an opaque pointer upgrade.
