During the transition from debug intrinsics to debug records, we used
several different command line options to customise handling: the
printing of debug records to bitcode and textual could be independent of
how the debug-info was represented inside a module, whether the
autoupgrader ran could be customised. This was all valuable during
development, but now that totally removing debug intrinsics is coming
up, this patch removes those options in favour of a single flag
(experimental-debuginfo-iterators), which enables autoupgrade, in-memory
debug records, and debug record printing to bitcode and textual IR.
We need to do this ahead of removing the
experimental-debuginfo-iterators flag, to reduce the amount of
test-juggling that happens at that time.
There are quite a number of weird test behaviours related to this --
some of which I simply delete in this commit. Things like
print-non-instruction-debug-info.ll , the test suite now checks for
debug records in all tests, and we don't want to check we can print as
intrinsics. Or the update_test_checks tests -- these are duplicated with
write-experimental-debuginfo=false to ensure file writing for intrinsics
is correct, but that's something we're imminently going to delete.
A short survey of curious test changes:
* free-intrinsics.ll: we don't need to test that debug-info is a zero
cost intrinsic, because we won't be using intrinsics in the future.
* undef-dbg-val.ll: apparently we pinned this to non-RemoveDIs in-memory
mode while we sorted something out; it works now either way.
* salvage-cast-debug-info.ll: was testing intrinsics-in-memory get
salvaged, isn't necessary now
* localize-constexpr-debuginfo.ll: was producing "dead metadata"
intrinsics for optimised-out variable values, dbg-records takes the
(correct) representation of poison/undef as an operand. Looks like we
didn't update this in the past to avoid spurious test differences.
* Transforms/Scalarizer/dbginfo.ll: this test was explicitly testing
that debug-info affected codegen, and we deferred updating the tests
until now. This is just one of those silent gnochange issues that get
fixed by RemoveDIs.
Finally: I've added a bitcode test, dbg-intrinsics-autoupgrade.ll.bc,
that checks we can autoupgrade debug intrinsics that are in bitcode into
the new debug records.
I observed that we have the boundary comments in the codebase like:
```
//===----------------------------------------------------------------------===//
// ...
//===----------------------------------------------------------------------===//
```
I also observed that there are incomplete boundary comments. The
revision is generated by a script that completes the boundary comments.
```
//===----------------------------------------------------------------------===//
// ...
...
```
Signed-off-by: hanhanW <hanhan0912@gmail.com>
This commit adds an additional overload to `replaceOpWithMultiple` that
accepts additional container types. This has been brought up by users of
the new `replaceOpWithMultiple` API.
In particular, one missing container type was
`SmallVector<SmallVector<Value>>`. The "default" `ArrayRef<ValueRange>`
container type can lead to use-after-scope errors in cases such as:
```c++
// Compute the replacement value ranges. Some replacements are single
// values, some are value ranges.
SmallVector<ValueRange> repl;
repl.push_back(someValueRange); // OK
for (...) {
// push_back(Value) triggers an implicit conversion to ValueRange,
// which does not own the range.
repl.push_back(someValue); // triggers use-after-scope later
}
rewriter.replaceOpWithMultiple(op, repl);
```
In this example, users should use `SmallVector<SmallVector<Value>>
repl;`.
This commit pulls apart the inherent attribute dependence of classes
like EnumAttrInfo and EnumAttrCase, factoring them out into simpler
EnumCase and EnumInfo variants. This allows specifying the cases of an
enum without needing to make the cases, or the EnumInfo itself, a
subclass of SignlessIntegerAttrBase.
The existing classes are retained as subclasses of the new ones, both
for backwards compatibility and to allow attribute-specific information.
In addition, the new BitEnum class changes its default printer/parser
behavior: cases when multiple keywords appear, like having both nuw and
nsw in overflow flags, will no longer be quoted by the operator<<, and
the FieldParser instance will now expect multiple keywords. All
instances of BitEnumAttr retain the old behavior.
This patch adds the `IsolatedFromAbove` trait as a dependent trait to
the `DataLayoutOpInterface` op interface.
The motivation behind this change comes from the implementation of the
`ptr` dialect, specifically the `ptr.type_offset` op. This op produces
an int-like value that equates to the size of a memory element. This is
useful for ptr arithmetic and indexing arrays. For example:
```mlir
%f32_off = ptr.type_offset f32 : index
%addr = ptr.ptradd %ptr, %f32_off : !ptr, index
%x = ptr.load %addr : !ptr -> f32 // Read ptr[1]
```
Without the `IsolatedFromAvobe` trait in the DL interface, the
`ptr.type_offset` cannot be `ConstantLike`. Why?
Take the example:
```mlir
op {DL1} {
%f32_off0 = ptr.type_offset f32 : index
op {DL2} {
%f32_off1 = ptr.type_offset f32 : index
}
}
```
If `ptr.type_offset` were to be `ConstantLike` then `canonicalize` would
hoist and unique the value. However, that could be wrong as DL2 could
have an entry to specify the size that's different from the size in DL1.
The best solution to the above problem is to make
`DataLayoutOpInterface` require the `IsolatedFromAbove` trait, as it
preserves the constness of values in the DL with respect to the
canonicalizer.
This gets the consumer fusion method in sync with the corresponding
producer fusion method `tileAndFuseProducerOfSlice`. Not taking this as
input required use of complicated analysis to retrieve the surrounding
loops which are very fragile. Just like the producer fusion method, the
loops need to be taken in as an argument, with typically the loops being
created by the tiling methods.
Some utilities are added to check that the loops passed in are perfectly
nested (in the case of an `scf.for` loop nest.
This is change 1 of N to simplify the implementation of tile and fuse
consumers.
---------
Signed-off-by: MaheshRavishankar <mahesh.ravishankar@gmail.com>
This patch decouples unrolling vector.gather and lowering vector.gather
to llvm.masked.gather.
This is consistent with how vector.load, vector.store,
vector.maskedload, vector.maskedstore lower to LLVM.
Some interesting test changes from this patch:
- 2D vector.gather lowering to llvm tests are deleted. This is
consistent with other memory load/store ops.
- There are still tests for 2D vector.gather, but the constant mask for
these test is modified. This is because with the updated lowering, one
of the unrolled vector.gather disappears because it is masked off (also
demonstrating why this is a better lowering path)
Overall, this makes vector.gather take the same consistent path for
lowering to LLVM as other load/store ops.
Discourse Discussion:
https://discourse.llvm.org/t/rfc-improving-gather-codegen-for-vector-dialect/85011/13
DenseSet, SmallPtrSet, SmallSet, SetVector, and StringSet recently
gained C++23-style insert_range. This patch replaces:
Dest.insert(Src.begin(), Src.end());
with:
Dest.insert_range(Src);
This patch does not touch custom begin like succ_begin for now.
This patch introduces the `MemorySpaceAttrInterface` interface. This
interface is responsible for handling the semantics of `ptr` operations.
For example, this interface can be used to create read-only memory
spaces, making any other operation other than a load a verification
error, see `TestConstMemorySpaceAttr` for a possible implementation of
this concept.
This patch also introduces Enum dependencies `AtomicOrdering`, and
`AtomicBinOp`, both enumerations are clones of the Enums with the same
name in the LLVM Dialect.
Also, see:
- [[RFC] `ptr` dialect & modularizing ptr ops in the LLVM
dialect](https://discourse.llvm.org/t/rfc-ptr-dialect-modularizing-ptr-ops-in-the-llvm-dialect/75142)
for rationale.
- https://github.com/llvm/llvm-project/pull/73057 for a prototype
implementation of the full change.
**Note: Ignore the first commit, that's being reviewed in
https://github.com/llvm/llvm-project/pull/86860 .**
This is a code cleanup. Update a few places in MLIR that should use
`hasSingleElement`/`getSingleElement`.
Note: `hasSingleElement` is faster than `.getSize() == 1` when it is
used with linked lists etc.
Depends on #131508.
This commit adds 1:N support to `SignatureConversion::remapInputs`. This
API allows users to replace a block argument with multiple replacement
values. (And the block argument is dropped.) The API already supported
"bbarg --> multiple bbargs" mappings, but "bbarg --> multiple SSA
values" was missing.
---------
Co-authored-by: Markus Böck <markus.boeck02@gmail.com>
Computing the bound of affine op
(ValueBoundsConstraintSet::computeBound) crashes due to the invalid dim
value given to the op. It is necessary for the pass to check the dim
attribute not to be greater than the rank of the input type.
Fixes https://github.com/llvm/llvm-project/issues/128807
This pattern to bubble up collapse shapes was missing in
`populateFoldReshapeOpsByCollapsingPatterns` .
Signed-off-by: Nirvedh Meshram <nirvedh@gmail.com>
205c5325b3
added a transform utility that moved all SSA dependences of an operation
before an insertion point. Similar to that, this PR adds a transform
utility function, `moveValueDefinitions` to move the slice of operations
that define all values in a `ValueRange` before the insertion point.
While very similar to `moveOperationDependencies`, this method differs
in a few ways
1. When computing the backward slice since the start of the slice is
value, the slice computed needs to be inclusive.
2. The combined backward slice needs to be sorted topologically before
moving them to avoid SSA use-def violations while moving individual ops.
The PR also adds a new transform op to test this new utility function.
---------
Signed-off-by: MaheshRavishankar <mahesh.ravishankar@gmail.com>
This patch adds a default memory space attribute to the DL and adds
methods to query the attribute. This is required as MLIR has no well
defined default memory space unlike LLVM which has 0. While `nullptr` is
a candidate for default memory space, the `ptr` dialect will remove the
possibility for `nullptr` memory spaces to avoid undefined semantics.
This patch also modifies the `DataLayoutTypeInterface::areCompatible` to
include the new DL spec and merged entries, as it is needed to query the default memory
space.
---------
Co-authored-by: Christian Ulmann <christianulmann@gmail.com>
*Update RescaleOp to use zero-point as operands instead of attributes.
*Check input_zp data type against the input and output_zp data type
against the output.
Signed-off-by: Peng Sun <peng.sun@arm.com>
Co-authored-by: Peng Sun <peng.sun@arm.com>
The added utility method moves all SSA values that an operation depends
upon before an insertion point. This is useful during transformations
where such movements might make transformations (like fusion) more
powerful.
To test the operation add a transform dialect op that calls the move
operation. To be able to capture the `notifyMatchFailure` messages from
the transformation and to report/check these in the test modify the
`ErrorCheckingTrackingListener` to capture the last match failure
notification.
---------
Signed-off-by: MaheshRavishankar <mahesh.ravishankar@gmail.com>
This reverts commit 32f543760c7f44c4c7d18bc00a3a1d8860c42bff.
Investigations showed that the unit test utilities were calling erase(),
causing a use-after-free. Fixed by rearranging checks in the test
When generating aliases from `OpAsm{Dialect,Type,Attr}Interface`, the
result would be sanitized and if the alias provided by the interface has
a trailing digit, AsmPrinter would attach an underscore to it to
presumably prevent confliction.
#### Motivation
There are two reasons to motivate the change from the old behavior to
the proposed behavior
1. If the type/attribute can generate unique alias from its content,
then the extra trailing underscore added by AsmPrinter will be strange
```mlir
func.func @add(%ct: !ct_L0_) -> !ct_L0_
%ct_0 = bgv.add %ct, %ct : (!ct_L0_, !ct_L0_) -> !ct_L0_
%ct_1 = bgv.add %ct_0, %ct_0 : (!ct_L0_, !ct_L0_) -> !ct_L0_
%ct_2 = bgv.add %ct_1, %ct_1 : (!ct_L0_, !ct_L0_) -> !ct_L0_
return %ct_2 : !ct_L0_
}
```
Which aesthetically would be better if we have `(!ct_L0, !ct_L0) ->
!ct_L0`
2. The Value name behavior is that, for the first instance, use no
suffix `_N`, which can be similarly applied to alias name. See the IR
above where the first one is called `%ct` and others are called `%ct_N`.
See `uniqueValueName` for detail.
#### Conflict detection
```mlir
!test.type<a = 3> // suggest !name0
!test.type<a = 4> // suggest !name0
!test.another<b = 3> // suggest !name0_
!test.another<b = 4> // suggest !name0_
```
The conflict detection is based on `nameCounts` in `initializeAliases`,
where
In the original way, the first two will get sanitized to `!name0_` and
`initializeAlias` can assign unique id `0, 1, 2, 3` to them.
In the current way, the `initializeAlias` uses `usedAliases` to track
which name has been used, and use such information to generate a suffix
id that will make the printed alias name unique.
The result for the above example is `!name0, !name0_1, !name0_,
!name0_2` now.
This commit adds builders of the form
```
static void build(..., [TypeRange resultTypes],
ValueRange operands, const Properties &properties,
ArrayRef<NamedAttribute> discardableAttributes = {},
[unsigned numRegions]);
```
to go alongside the existing
result/operands/[inherent + discardable attribute list] collective
builders.
This change is intended to support a refactor to the declarative rewrite
engine to make it populate the `Properties` struct instead of creating a
`DictionaryAttr`, thus enabling rewrite rules to handle non-`Attribute`
properties.
More generally, this means that generic code that would previously call
`getAttrs()` to blend together inherent and discardable attributes can
now use `getProperties()` and `getDiscardableAttrs()` separately, thus
removing the need to serialize everything into a temporary
`DictionaryAttr`.
* mlir::tosa::computeMultiplierAndShift returns a boolean, depending on
validity of the shift value
Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Co-authored-by: Tom Allsop <tom.allsop@arm.com>
Add mangling style as a spec entry to datalayout, and implemented
importing and exporting LLVM IR to MLIR (LLVM dialect).
Its represented as string as the scope of this PR is to preserve info
from LLVM IR, so client in MLIR still need to map deduce the meaning of
the string, like "e" means ELF, "o" for Mach-O, etc.
it addresses one of issues mentioned in this
[issue](https://github.com/llvm/llvm-project/issues/126046)
Previously, the input_unsigned and output_unsigned attributes on the
RESCALE op were optional. This commit updates them to be required,
ensuring compliance with the TOSA V1.0 Specification.
Signed-off-by: Peng Sun <peng.sun@arm.com>
Co-authored-by: James Ward <james.ward@arm.com>
Many types don't have a preferred alignment, but often specifying an ABI
alignment is required to implement APIs on top of data layouts. Default
the preferred alignment to `getABIAlignment` to simplify things.
When `resolveOperands` reports an error and no valid `SMLoc` was
provided, report the error at the beginning of the op instead of
crashing.
```
Assert `Ptr >= BufStart && Ptr <= Buffer->getBufferEnd()' in llvm/lib/Support/SourceMgr.cpp:llvm::SourceMgr::SrcBuffer::getLineNumberSpecialized failed
```
E.g., this is currently the case when parsing the following op with a
type but without any operands:
```
let assemblyFormat = "$str (`,` $args^)? attr-dict (`:` type($args)^)?";
```
Reported error (with this PR):
```
within split at mlir/test/IR/invalid-ops.mlir:122 offset :4:1: error: custom op 'test.variadic_args_types_split' number of operands and types do not match: got 0 operands and 1 types
test.variadic_args_types_split "hello_world" : i32
^
```
In the ODS-generated C++, the `SMLoc` is populated when parsing the
optional group containing `$args`. However, this group is missing in the
test case.
There are likely additional hand-written parsers that suffer from the
same problem.
Note: I tried emitting a second `SMLoc` for the optional type group in
the `OpFormatGen.cpp`, but this adds quite a bit of complexity in the
code base for little improvement in user experience.
TOSA MLIR profile-based validation is designed to identify the
profile/extension requirements for each operation in TOSA MLIR graph,
ensuring that TOSA operators conform to the profiles and extensions
enabled by the target implementation.
The available profiles/extensions are reflected in the availability
property attached to each TOSA operator in the dialect. The design of
availability, the profile/extension classes, and their interface, is
inspired by the SPIRV implementation.
This patch includes the following changes:
- Introduces profile and extension knowledge within the dialect and
establishes an interface to query this information.
- Implements profile-based validation logic in the pass.
- Adds a TargetEnv class that represents the capabilities enabled in the
target implementation, such as profiles, extensions, and levels.
- Adds a set of tests to ensure that profile and extension requirements
are properly attached to the operations and that validation correctly
verifies the requirements of a given operation against the target
implementation.
See
https://discourse.llvm.org/t/rfc-introduce-opasm-type-attr-interface-for-pretty-print-in-asmprinter/83792
for detailed introduction.
This is a follow up PR of #121187, by integrating OpAsmTypeInterface
with AsmPrinter. There are a few conditions when OpAsmTypeInterface
comes into play
* There is no OpAsmOpInterface
* Or OpAsmOpInterface::getAsmResultName/getBlockArgumentName does not
invoke `setName` (i.e. the default impl)
* All results have OpAsmTypeInterface (otherwise we can not handle
result grouping behavior)
Cc @River707 @jpienaar @ftynse for review.
Some loops cannot be unrolled by affine-loop-unroll pass. After running
lower-affine pass, they can be unrolled in scf.To enable conversion of
vector Ops in scf to llvm dialect, unroll-full option was added.
---------
Co-authored-by: Oleksandr "Alex" Zinenko <ftynse@gmail.com>
Moves `PackOp` and `UnPackOp` from the Tensor dialect to Linalg. This change
was discussed in the following RFC:
* https://discourse.llvm.org/t/rfc-move-tensor-pack-and-tensor-unpack-into-linalg
This change involves significant churn but only relocates existing code - no new
functionality is added.
**Note for Downstream Users**
Downstream users must update references to `PackOp` and `UnPackOp` as follows:
* Code: `s/tensor::(Up)PackOp/linalg::(Un)PackOp/g`
* Tests: `s/tensor.(un)pack/linalg.(un)pack/g`
No other modifications should be required.
`let constructor` is deprecated since the table gen backend emits most
of the glue logic to build a pass. This PR retires the td method for
most (I need another pass) passes in the Conversion directory.
Enable ops with only read side effects in scf.for to be hoisted with a
scf.if guard that checks against the trip count
This patch takes a step towards a less conservative LICM in MLIR as
discussed in the following discourse thread:
[Speculative LICM?](https://discourse.llvm.org/t/speculative-licm/80977)
This patch in particular does the following:
1. Relaxes the original constraint for hoisting that only hoists ops
without any side effects. This patch also allows the ops with only read
side effects to be hoisted into an scf.if guard only if every op in the
loop or its nested regions is side-effect free or has only read side
effects. This scf.if guard wraps the original scf.for and checks for
**trip_count > 0**.
2. To support this, two new interface methods are added to
**LoopLikeInterface**: _wrapInTripCountCheck_ and
_unwrapTripCountCheck_. Implementation starts with wrapping the scf.for
loop into scf.if guard using _wrapInTripCountCheck_ and if there is no
op hoisted into the this guard after we are done processing the
worklist, it unwraps the guard by calling _unwrapTripCountCheck_.
Another follow up fix to
https://github.com/llvm/llvm-project/pull/123910 to fix a build failure
that sometimes happens in shared library builds:
https://lab.llvm.org/buildbot/#/builders/50/builds/9724
In file included from
/home/tcwg-buildbot/worker/flang-aarch64-dylib/llvm-project/mlir/test/lib/Transforms/TestInlining.cpp:16:
/home/tcwg-buildbot/worker/flang-aarch64-dylib/llvm-project/mlir/test/lib/Transforms/../Dialect/Test/TestOps.h:148:10:
fatal error: 'TestOps.h.inc' file not found
148 | #include "TestOps.h.inc"
| ^~~~~~~~~~~~~~~
1 error generated.
This is a test library which is not part of libMLIR, so it should use
normal LINK_LIBS instead of mlir_target_link_libraries.
This fixes an issue introduced in #123910 and follows up on the fix in
#125004, which added the library to DEPENDS, which is not sufficient.
This PR adds the UB to LLVM/SPIR-V conversion pass to some pipelines and
tests. This is in preparation to introducing the generation of
`ub.poison` in Vector dialect transformations (first one in https://github.com/llvm/llvm-project/pull/125613).
It should effectively be NFC at this point.
This patch shares core interface methods dealing with argument and
result attributes from CallableOpInterface with the CallOpInterface and
makes them mandatory to gives more consistent guarantees about concrete
operations using these interfaces.
This allows adding argument attributes on call like operations, which is
sometimes required to get proper ABI, like with llvm.call (and llvm.invoke).
The patch adds optional `arg_attrs` and `res_attrs` attributes to operations using
these interfaces that did not have that already.
They can then re-use the common "rich function signature"
printing/parsing helpers if they want (for the LLVM dialect, this is
done in the next patch).
Part of RFC: https://discourse.llvm.org/t/mlir-rfc-adding-argument-and-result-attributes-to-llvm-call/84107
With the removal of mlir-vulkan-runner (as part of #73457) in
e7e3c45bc70904e24e2b3221ac8521e67eb84668, this pass no longer has to be
public (previously it had to be so the runner could use it). This commit
makes it instead only available for use by mlir-opt.
This is a recommit of 058d183980a2f334d085a46c32abded0557aa789 (#124301)
which had been reverted in 4573c857da88b3210d497d9a88a89351a74b5964 due
to a missing linker dependency on MLIRSPIRVTransforms in
mlir/test/lib/Pass/CMakeLists.txt (fixed in this commit).
