533175 Commits

Author SHA1 Message Date
Sergei Barannikov
fe95d6435f [ValueTracking] Add CharWidth argument to getConstantStringInfo (NFC)
The method assumes that host chars and target chars have the same width.
Add a CharWidth argument so that it can bail out if the requested char
width differs from the host char width.

Alternatively, the check could be done at call sites, but this is more
error-prone.

In the future, this method will be replaced with a different one that
allows host/target chars to have different widths. The prototype will
be the same except that StringRef is replaced with something that is
byte width agnostic. Adding CharWidth argument now reduces the future
diff.
2025-04-06 20:20:20 +03:00
Sergei Barannikov
e75ad5f9a0 [IR] Account for byte width in m_PtrAdd
The method has few uses yet, so just pass DL argument to it. The change
follows m_PtrToIntSameSize, and I don't see a better way of delivering
the byte width to the method.
2025-04-06 20:16:16 +03:00
Sergei Barannikov
b8c20416dc [IRBuilder] Add getByteTy and use it in CreatePtrAdd
The change requires DataLayout instance to be available, which, in turn,
requires insertion point to be set. In-tree tests detected only one case
when the function was called without setting an insertion point, it was
changed to create a constant expression directly.
2025-04-06 20:16:15 +03:00
Sergei Barannikov
555e531fde [ValueTracking] Make isBytewiseValue byte width agnostic
This is a simple change to show how easy it can be to support unusual
byte widths in the middle end.
2025-04-06 20:16:13 +03:00
Sergei Barannikov
97cce999ea [IR] Make @llvm.memset prototype byte width dependent
This patch changes the type of the value argument of @llvm.memset and
similar intrinsics from i8 to iN, where N is the byte width specified
in data layout string.
Note that the argument still has fixed type (not overloaded), but type
checker will complain if the type does not match the byte width.

Ideally, the type of the argument would be dependent on the address
space of the pointer argument. It is easy to do this (and I did it
downstream as a PoC), but since data layout string doesn't currently
allow different byte widths for different address spaces, I refrained
from doing it now.
2025-04-06 20:12:52 +03:00
Sergei Barannikov
40489908db [DataLayout] Add byte specification
This patch adds byte specification to data layout string.
The specification is `b:<size>`, where `<size>` is the size of a byte
in bits (later referred to as "byte width").

Limitations:
* The only values allowed for byte width are 8, 16, and 32.
16-bit bytes are popular, and my downstream target has 32-bit bytes.
These are the widths I'm going to add tests for in follow-up patches,
so this restriction only exists because other widths are untested.
* It is assumed that bytes are the same in all address spaces.
Supporting different byte widths in different address spaces would
require adding an address space argument to all DataLayout methods
that query ABI / preferred alignments because they return *byte*
alignments, and those will be different for different address spaces.
This is too much effort, but it can be done in the future if the need
arises, the specification reserves address space number before ':'.
2025-04-06 20:12:52 +03:00
Sergei Barannikov
eb70253fcb
[LangRef/DataLayout] Spell out requirements for alignment values (#104705)
For 'p' the added wording matches the implementation.

For 'i', 'f', 'v' the implementation also allows 0 for `<pref>`
component, making 'i16:16:0' valid, for example. 'Fi0', 'Fn0' and 'S0'
are also currently accepted. This is likely unintentional. There are no
tests in the codebase that rely on this behavior, so the added wording
prohibits zero alignments for these specifications.

For 'a', the implementation currently allows 0 for both `<abi>` and
`<pref>` components. The added wording prohibits specifying zero for
`<pref>` with the same justification as above. Zero `<abi>` is used in
tests, and the example at the end of the section suggests that this is
valid, so that's left unchanged.

This effectively prohibits zero alignments everywhere except for the
`<abi>` component of aggregate specification.
2025-04-06 20:08:52 +03:00
Florian Hahn
48441cb8a2
[Matrix] Properly set Changed status when optimizing transposes.
Currently Changed is not updated properly when transposes are optimized,
causing missing analysis invalidation. Update optimizeTransposes to
indicate if changes have been made.
2025-04-06 17:36:56 +01:00
Florian Hahn
449e2f5d66
[LV] Remove more DT updates from legacy code path (NFCI).
Remove some legacy DT updates. Those should already be handled when
updating the DT during VPlan execution.
2025-04-06 14:35:21 +01:00
Benjamin Kramer
0defd832eb [AArch64] Avoid unused variable warnings in release builds
This used to be under !NDEBUG before
0a1742708ddc3a2b31d65479aaad143b5f7562b2, so just put that back. The
code only consists of assertions.
2025-04-06 14:56:33 +02:00
Florian Hahn
ba3fa39b63
[EarlyCSE] Re-generate checks for intrinsics.ll. 2025-04-06 12:20:31 +01:00
Florian Hahn
283a78a088
Reapply "[LV] Don't add blocks to loop in GeneratedRTChecks (NFC)."
This reverts commit 46a2f4174a051f29a09dbc3844df763571c67309.

Recommits 2fd6f8fb5e3a with corresponding VPlan change to ensure
LoopInfo is updated for all blocks during VPlan execution if needed.
2025-04-06 12:18:11 +01:00
Benjamin Kramer
3e08dcd767 [mlir][inliner] Move callback types from InlinerConfig -> InlinerInterface. NFC.
The proper layering here is that Inliner depends on InlinerUtils, and
not the other way round. Maybe it's time to give InliningUtils a less
terrible file name.
2025-04-06 13:02:42 +02:00
Sergei Barannikov
0a1742708d
[SelectionDAG] Wire up -gen-sdnode-info TableGen backend (#125358)
This patch introduces SelectionDAGGenTargetInfo and SDNodeInfo classes,
which provide methods for accessing the generated SDNode descriptions.

Pull Request: https://github.com/llvm/llvm-project/pull/125358
Draft PR: https://github.com/llvm/llvm-project/pull/119709
RFC: https://discourse.llvm.org/t/rfc-tablegen-erating-sdnode-descriptions
2025-04-06 13:14:37 +03:00
Hui
7013b51548
[libc++] Implement std::flat_multiset (#128363)
fixes https://github.com/llvm/llvm-project/issues/105193
2025-04-06 10:50:55 +01:00
Florian Hahn
8d71a2a905
[VPlan] Use ExitBocks to check in VPlan::isExitBlock (NFC).
Exit blocks of the VPlan are now hold in ExitBlocks. Use it to check if
a block is an exit block. Otherwise we currently mis-classify the scalar
loop header also as exit block, as it is not explicitly connected to the
exit blocks.

NFC at the moment, as the helper currently is never queried with the
scalar header, but that will change in the future.
2025-04-06 09:48:52 +01:00
Fangrui Song
53e1c8b118 AArch64MCExpr: Remove unused getSpecifier 2025-04-06 00:38:09 -07:00
Fangrui Song
c0b4a8edfe MCValue: Replace getRefKind with getSpecifier 2025-04-06 00:12:45 -07:00
Fangrui Song
e7dc05ebcf MCValue: Make print private
This is a debug-only feature (the relocation specifier is
target-specific and cannot be printed without backend support) and not
supposed to be used externally. PowerPC imappropriated used it
(removed by 7cb66ff4648a15741a1908658dfef5cb3d4a9199).
2025-04-06 00:09:30 -07:00
Fangrui Song
7cb66ff464 [PowerPC] Use reportError
Report a proper error and fix de1dc9c98f9ce74d38aceb44e00d258370d1bb34
2025-04-06 00:03:48 -07:00
Fangrui Song
8fa5b6cc02 MCValue: Replace getAccessVariant with getSpecifier
Commit 52eb11f925ddeba4e1b3840fd636ee87387f3ada temporarily introduced
getSymSpecifier to prepare for "MCValue: Replace MCSymbolRefExpr members
with MCSymbol" (d5893fc2a7e1191afdb4940469ec9371a319b114). The
refactoring is now complete.
2025-04-05 23:55:35 -07:00
Fangrui Song
e592393610 MCValue: Replace getSymSpecifier with getSpecifier
Commit 52eb11f925ddeba4e1b3840fd636ee87387f3ada temporarily introduced
getSymSpecifier to prepare for "MCValue: Replace MCSymbolRefExpr members
with MCSymbol" (d5893fc2a7e1191afdb4940469ec9371a319b114). The
refactoring is now complete.
2025-04-05 23:44:57 -07:00
Fangrui Song
803fbdd1fa [PowerPC] Report proper error for invalid relocation specifier
Generalize the test from https://reviews.llvm.org/D83255

Replace getAccessVariant with MCValue::getSpecifier
Simplify code after MCValue improvement 94821ce45fe93aa78cc5ea03cd9deac91b7af127
2025-04-05 22:13:55 -07:00
Fangrui Song
d5893fc2a7 MCValue: Replace MCSymbolRefExpr members with MCSymbol
Commit 0999cbd0b9ed8aa893cce10d681dec6d54b200ad (2014) introduced
`MCValue::RefKind` for AArch64 ELF as a clean approach to encode the
relocation specifier.

Following numerous migration commits, direct references to getSymA and
getSymB have been eliminated. This allows us to seamlessly update SymA
and SymB, replacing MCSymbolRefExpr with MCSymbol.

Removeing reliance on a MCAssembler::evaluateFixup hack
(`if (Target.SymSpecifier || SA.isUndefined()) {` (previosuly
`if (A->getKind() != MCSymbolRefExpr::VK_None || SA.isUndefined()) {`))
requires 38c3ad36be1facbe6db2dede7e93c0f12fb4e1dc and 4182d2dcb5ecbfc34d41a6cd11810cd36844eddb

Revert the temporary RISCV/LoongArch workaround
(7e62715e0cd433ed97749549c6582c4e1aa689a3) during migration.

MCAssembler::evaluateFixup needs an extra `!Add->isAbsolute()` case
to support `movq abs@GOTPCREL(%rip), %rax; abs = 42` in llvm/test/MC/ELF/relocation-alias.s
(ELFObjectWriter::isSymbolRefDifferenceFullyResolvedImpl asserts if
called on an absolute symbol).
2025-04-05 21:02:08 -07:00
Fangrui Song
4182d2dcb5 [ARM,PowerPC] shouldForceRelocation: check MCValue::Specifier
Follow-up to 38c3ad36be1facbe6db2dede7e93c0f12fb4e1dc

Removes reliance on a MCAssembler::evaluateFixup hack
`if (Target.SymSpecifier || SA.isUndefined()) {` (previosuly
`if (A->getKind() != MCSymbolRefExpr::VK_None || SA.isUndefined()) {`)

Target.SymSpecifier will soon go away when MCValue replaces
MCSymbolRefExpr members with MCSymbol and removes the temporary
`SymSpecifier` workaround.
2025-04-05 20:04:07 -07:00
Fangrui Song
38c3ad36be Define MCAsmBackend::shouldForceRelocation
Return true if the MCValue has a specifier. When a relocation specifier
is specified, GNU Assembler will generate a relocation unless the
specifier can be optimized due to target-specific reasons (e.g. PPC `@l`
`@ha`).

This reduces targets' reliance on a MCAssembler::evaluateFixup hack
`if (Target.SymSpecifier || SA.isUndefined()) {`, previosuly
`if (A->getKind() != MCSymbolRefExpr::VK_None || SA.isUndefined()) {`

llvm/test/MC/SystemZ/fixups.s is known to rely on this hack.
2025-04-05 19:01:16 -07:00
Fangrui Song
94821ce45f MCValue: Store SymA specifier at Specifier
The relocation specifier should be accessed via MCValue::Specifier.
However, some targets encode the relocation specifier within SymA using
MCSymbolRefExpr::SubclassData and access it via getAccessVariant(), though
this method is now deprecated.

This change stores the SymA specifier at Specifier as well, unifying the
two code paths.

* CSKY: GOT- and PLT- relocations now suppress the STT_SECTION
  conversion.
* AArch64: https://reviews.llvm.org/D156505 added `getRefkind` check to
  prevent folding. This is a hack and is now removed.

MCValue: Unify relocation specifier storage by storing SymA specifier at Specifier

The relocation specifier is accessed via MCValue::Specifier, but some
targets encoded it within SymA using MCSymbolRefExpr::SubclassData and
retrieved it through the now-deprecated getAccessVariant() method. This
commit unifies the two approaches by storing the SymA specifier at
`Specifier` as well.

Additional changes:

- CSKY: GOT- and PLT- relocations now suppress STT_SECTION conversion.
- AArch64: Removed the `getRefkind` check hack (introduced in https://reviews.llvm.org/D156505) that prevented folding.
  Removed the assertion from `getRelocType`.
- RISCV: Removed the assertion from `getRelocType`.

Future plans:

- Replace MCSymbolRefExpr members with MCSymbol within MCValue.
- Remove `getSymSpecifier` (added for migration).
2025-04-05 17:16:54 -07:00
Owen Pan
aaaeb86ace
[clang-format] Merge inline short functions for BS_Whitesmiths (#134473)
Fix #126747
2025-04-05 17:03:55 -07:00
weiwei chen
55ff96abfa
[X86][Test] Reorder PassMgrF and OS. (#134481)
Reordering `OS` and `PassMgrF` should fix the asan failure that's caused
by OS being destroyed before `PassMgrF` deletes the AsmPrinter.

As shown in[ this asan run
](https://lab.llvm.org/buildbot/#/builders/52/builds/7340/steps/12/logs/stdio)

```
  This frame has 15 object(s):
    [32, 48) 'PassMgrF' (line 154)
    [64, 1112) 'Buf' (line 155)
    [1248, 1304) 'OS' (line 156) <== Memory access at offset 1280 is inside this variable
```
which indicates an ordering problem. 

This should help to fix all the sanitizer failures caused by the test
`X86MCInstLowerTest.cpp` that's introduced by [this
PR](https://github.com/llvm/llvm-project/pull/133352#issuecomment-2780173791).
2025-04-05 19:41:52 -04:00
Alan
7833107993
[OCaml] Make OCaml MetadataKind type consistent with C API (#134507)
Fixes breakage of OCaml API introduced by commit 6894734.
2025-04-05 18:21:38 -04:00
Fangrui Song
f8f9be3241 [ARM,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 14:41:47 -07:00
Fangrui Song
590d2a3ca5 [Mips,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 14:25:06 -07:00
Fangrui Song
d9a767cfc8 [Mips,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.

Create a MipsMCExpr::create overload that takes MCSymbol as an argument.
We use the order preferred by other targets.
2025-04-05 14:23:01 -07:00
Fangrui Song
0431fea88a [AMDGPU,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 14:04:00 -07:00
Fangrui Song
acca419685 [WebAssembly,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 14:01:05 -07:00
Fangrui Song
7e62715e0c [RISCV,LoongArch,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.

Creating a MCSymbolRefExpr in *AsmBackend::handleAddSubRelocations is
not efficient, but it is temporary and will be replaced when MCValue no
longer uses MCSymbolRefExpr.
2025-04-05 13:58:16 -07:00
junfengd-nv
aeec94500a
[mlir][inliner] Add doClone and canHandleMultipleBlocks callbacks to Inliner Config (#131226)
Current inliner disables inlining when the caller is in a region with
single block trait, while the callee function contains multiple blocks.
the SingleBlock trait is used in operations such as do/while loop, for
example fir.do_loop, fir.iterate_while and fir.if. Typically, calls within
loops are good candidates for inlining. However, functions with multiple
blocks are also common. for example, any function with "if () then
return" will result in multiple blocks in MLIR.

This change gives the flexibility of a customized inliner to handle such
cases.
doClone: clones instructions and other information from the callee
function into the caller function. .
canHandleMultipleBlocks: checks if functions with multiple blocks can be
inlined into a region with the SingleBlock trait.

The default behavior of the inliner remains unchanged.

---------

Co-authored-by: jeanPerier <jean.perier.polytechnique@gmail.com>
Co-authored-by: Mehdi Amini <joker.eph@gmail.com>
2025-04-05 22:56:55 +02:00
Fangrui Song
70f5632cad [PowerPC,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 13:53:38 -07:00
Fangrui Song
7cf8a6201a [AArch64,MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 13:51:45 -07:00
Florian Hahn
46a2f4174a
Revert "[LV] Don't add blocks to loop in GeneratedRTChecks (NFC)."
This reverts commit 2fd6f8fb5e3a52e901276d97c285b8de66742985.

This missed a possible case, causing buildbot failures.
2025-04-05 21:47:14 +01:00
Fangrui Song
0c84d71eda [MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 13:40:04 -07:00
Owen Pan
d71ee7d230
[clang-format] Set C11 instead of C17 for LK_C (#134472)
Fix #134453
2025-04-05 13:35:45 -07:00
Fangrui Song
b1cd3cb3f4 [MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 13:34:24 -07:00
Fangrui Song
086af83688 [MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 13:23:14 -07:00
Fangrui Song
7ccdc3d5ca [MC] Replace getSymA()->getSymbol() with getAddSym. NFC
We will replace the MCSymbolRefExpr member in MCValue with MCSymbol.
This change reduces dependence on MCSymbolRefExpr.
2025-04-05 13:16:25 -07:00
Florian Hahn
2fd6f8fb5e
[LV] Don't add blocks to loop in GeneratedRTChecks (NFC).
Blocks will get added to parent loops as needed during VPlan execution.
2025-04-05 21:10:26 +01:00
Fangrui Song
52eb11f925 [MC] Replace getSpecifier(Target.getSymA()) with Target.getSymSpecifier()
Add MCValue::getSymSpecifier as a workaround for targets that encode the
relocation specifier on SymA. This function asserts that SymA is not
null.
2025-04-05 13:08:19 -07:00
Aiden Grossman
582b1b2ac9 [CI] Use env variable to enable pip breaking system packages
This patch uses an env variable instead of the --break-system-packages
flag. This enables the heterogenous configuration between the old and
new premerge systems as the old premerge container does not recognize
the --break-system-packages flag. An env variable will work on new
premerge and have no impact on old premerge.
2025-04-05 20:04:04 +00:00
Fangrui Song
ed2b82fb8b MCValue: Change getAccessVariant to return uint16_t
Some targets encode the relocation specifier within SymA using
MCSymbolRefExpr::SubclassData. They will cast the specifier
to *MCExpr::Specifier.

Migrate away from the confusing MCSymbolRefExpr::VariantKind.

Note: getAccessVariant is a deprecated method to get the relocation
specifier.
2025-04-05 12:12:11 -07:00
Fangrui Song
5e8f43811a MCValue: Make getSymB private and improve documentation 2025-04-05 12:03:59 -07:00