Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change focuses on the
option's prefixed names and the array of prefixes. These are present in
every option and the dominant source of dynamic relocations for PIE or
PIC users of LLVM and Clang tooling. In some cases, 100s or 1000s of
them for the Clang driver which has a huge number of options.
This PR addresses this by building a string table and a prefixes table
that can be referenced with indices rather than pointers that require
dynamic relocations. This removes almost 7k dynmaic relocations from the
`clang` binary, roughly 8% of the remaining dynmaic relocations outside
of vtables. For busy-boxing use cases where many different option tables
are linked into the same binary, the savings add up a bit more.
The string table is a straightforward mechanism, but the prefixes
required some subtlety. They are encoded in a Pascal-string fashion with
a size followed by a sequence of offsets. This works relatively well for
the small realistic prefixes arrays in use.
Lots of code has to change in order to land this though: both all the
option library code has to be updated to use the string table and
prefixes table, and all the users of the options library have to be
updated to correctly instantiate the objects.
Some follow-up patches in the works to provide an abstraction for this
style of code, and to start using the same technique for some of the
other strings here now that the infrastructure is in place.
Remove support for obfuscated bitcode in dsymutil and the DWARF linker.
We no longer support bitcode submissions and the obfuscation support has
been removed from the rest of the compiler.
rdar://123863918
Remove support for obfuscated bitcode in dsymutil and the DWARF linker.
We no longer support bitcode submissions and the obfuscation support has
been removed from the rest of the compiler.
rdar://123863918
This patch is extracted from #74725.
The DwarfStreamer interface looks overcomplicated and has unnecessary
dependencies. This patch avoids creation of DwarfStreamer by DWARFLinker and
simplifies interface.
This patch renames values of dsymutil/llvm-dwarfutil options:
--linker apple -> --linker classic
--linker llvm -> --linker parallel
The purpose to rename options is to avoid using vendor names and to
match with library names. It should be safe to rename options at current
stage as they are not seemed widely used(we may not preserve backward
compatibility).
It was noted that new DWARFLinker libraries do not follow naming
agreement -
https://github.com/llvm/llvm-project/pull/75925#issuecomment-1883301659
This patch rename libraries to match with the agreement.
Rename LLVMDWARFLinkerBase library into the LLVMDWARFLinker. Rename
LLVMDWARFLinker library into the LLVMDWARFLinkerClassic. Correct include
path according to the new directory structure.
This patch creates DWARFLinkerBase library, places DWARFLinker code into
DWARFLinker\Classic, places DWARFLinkerParallel into DWARFLinker\Parallel.
updates BOLT to use new library. This patch is NFC.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
This reverts commit 122c89b271af30b86536cad7bac64ea9c56615ed. Change does not build, with errors such as:
In file included from ../llvm-project/llvm/tools/dsymutil/DebugMap.h:24,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:60:17: error: declaration of ‘llvm::dsymutil::SymbolMapping llvm::dsymutil::ValidReloc::SymbolMapping’ changes meaning of ‘SymbolMapping’ [-fpermissive]
60 | SymbolMapping SymbolMapping;
| ^~~~~~~~~~~~~
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:36:8: note: ‘SymbolMapping’ declared here as ‘struct llvm::dsymutil::SymbolMapping’
36 | struct SymbolMapping {
| ^~~~~~~~~~~~~
In file included from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/DebugMap.h:198:32: error: declaration of ‘std::optional<llvm::dsymutil::RelocationMap> llvm::dsymutil::DebugMapObject::RelocationMap’ changes meaning of ‘RelocationMap’ [-fpermissive]
198 | std::optional<RelocationMap> RelocationMap;
| ^~~~~~~~~~~~~
In file included from ../llvm-project/llvm/tools/dsymutil/DebugMap.h:24,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:76:7: note: ‘RelocationMap’ declared here as ‘class llvm::dsymutil::RelocationMap’
76 | class RelocationMap {
| ^~~~~~~~~~~~~
This adds support in dsymutil for mergeable libraries [1].
dsymutil reads a new stab emitted by ld, allowing it to operate on
dynamic libraries instead of object files. It also now loads the DWARF
files associated to the libraries, and build the debug map for each
binary from the list of symbols exported by the library. For each Debug
Map Object, there is a new associated Relocation Map which is serialized
from the information retrieved in the original debug_info (or
debug_addr) section of the .o file.
The final DWARF file has multiple compile units, so the offsets
information of the relocations are adjusted relatively to the compile
unit they will end up belonging to, inside the final linked DWARF file.
[1] https://developer.apple.com/documentation/xcode/configuring-your-project-to-use-mergeable-libraries
Differential revision: https://reviews.llvm.org/D158124
These files satisfy all of the following:
- misc-include-cleaner indicates that these files do not need
Endian.h.
- They do not mention "endian" anywhere.
- They do not include any *.inc or *.def, which could need
llvm::support::endian.
Remove the paper trail warning from dsymutil and the DWARF linker. The
original purpose of this functionality was to leave a paper trail in the
output artifact about missing object files. The current implementation
however has diverged and is the source of a pretty serious bug:
- In the debug map parser, missing object files are not the only
warnings we emit. When paper trail warnings are enabled, all of them end
up in the dSYM which wasn't the goal.
- When warnings are associated with a object file in the debug map, it
is skipped by the DWARF linker. This only makes sense if the object file
is missing and is obviously incorrect for any other type of warning
(such as a missing symbol).
The combination of the two means that we can generate broken DWARF when
the feature is enabled. AFAIK it was only used by Apple and nobody I
spoke to has relied on it, so rather than fixing the broken behavior I
propose we remove it.
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Reland2: temporarily disabled call to "--linker llvm" for tls-variable.test
and location-expression.test as it does not work properly on bigendian
architecture.
Differential Revision: https://reviews.llvm.org/D153268
This reverts commit 0229dd0626b2538c78ebbd2b1bd44c31fbf1cdec.
This introduces two test failures on s390x.
tools/dsymutil/X86/location-expression.test:
warning: cann't load line table.
note: while processing CU1
/builddir/build/BUILD/llvm-18.0.0.src/test/tools/dsymutil/X86/location-expression.test:20:10: error: CHECK: expected string not found in input
# CHECK: DW_AT_name{{.*}}"CU1"
^
<stdin>:34:32: note: scanning from here
0x0000000b: DW_TAG_compile_unit [1] *
^
<stdin>:37:2: note: possible intended match here
DW_AT_name [DW_FORM_strp] ( .debug_str[0x09000000] = )
^
tools/dsymutil/X86/tls-variable.test:
warning: cann't load line table.
note: while processing CU1
/builddir/build/BUILD/llvm-18.0.0.src/test/tools/dsymutil/X86/tls-variable.test:19:10: error: CHECK: expected string not found in input
# CHECK: DW_AT_name{{.*}}"CU1"
^
<stdin>:26:32: note: scanning from here
0x0000000b: DW_TAG_compile_unit
^
<stdin>:29:2: note: possible intended match here
DW_AT_name ()
^
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Differential Revision: https://reviews.llvm.org/D153268
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Differential Revision: https://reviews.llvm.org/D153268
This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lldb. I've also
renamed OptionVisibility::Default to "DefaultVis" to avoid ambiguity
since the undecorated name has to be available anywhere Options.inc is
included.
Original message follows:
This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.
We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.
Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.
Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.
Differential Revision: https://reviews.llvm.org/D157149
This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.
We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.
Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.
Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.
Differential Revision: https://reviews.llvm.org/D157149
All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D157028
This patch simplifies line table generation. It removes global
array of all units ranges(RangesTy &getValidAddressRanges()).
The comment says that global array of all units ranges is necessary
to handle corner cases inside line table rows. Removing that
special handling shows that its current usage is handling of
"end of range case" which is already handled correctly
(without special handling). .debug_line tables for clang binary
built with and without this patch are equal.
Differential Revision: https://reviews.llvm.org/D154288
This patch creates skeleton implementation for the DWARFLinkerParallel.
It also integrates DWARFLinkerParallel into dsymutil and llvm-dwarfutil,
so that empty DWARFLinker::link() can be called. To do this new command
line option is added "--linker apple/llvm". Additionally it changes
existing DWARFLinker interfaces/implementations to be compatible:
use Error for error reporting for the DWARFStreamer, make DWARFFile to
owner of referenced resources, other small refactorings.
Differential Revision: https://reviews.llvm.org/D147952
This patch creates skeleton implementation for the DWARFLinkerParallel.
It also integrates DWARFLinkerParallel into dsymutil and llvm-dwarfutil,
so that empty DWARFLinker::link() can be called. To do this new command
line option is added "--linker apple/llvm". Additionally it changes
existing DWARFLinker interfaces/implementations to be compatible:
use Error for error reporting for the DWARFStreamer, make DWARFFile to
owner of referenced resources, other small refactorings.
Differential Revision: https://reviews.llvm.org/D147952
This patch adds handling of DW_OP_addrx and DW_OP_constx expression operands.
In --update case these operands are preserved as is. Otherwise they are
converted into the DW_OP_addr and DW_OP_const[*]u correspondingly.
Differential Revision: https://reviews.llvm.org/D147066
This reduces dependencies on `llvm-tblgen` so much.
`CodeGenTypes` depends on `Support` at the moment.
Be careful to append deps on this, since Targets' tablegens
depend on this.
Depends on D149024
Differential Revision: https://reviews.llvm.org/D148769
This is rework of;
- D30046 (LLT)
Since I have introduced `llvm-min-tblgen` as D146352, `llvm-tblgen`
may depend on `CodeGen`.
`LowLevlType.h` originally belonged to `CodeGen`. Almost all userse are
still under `CodeGen` or `Target`. I think `CodeGen` is the right place
to put `LowLevelType.h`.
`MachineValueType.h` may be moved as well. (later, D149024)
I have made many modules depend on `CodeGen`. It is consistent but
inefficient. It will be split out later, D148769
Besides, I had to isolate MVT and LLT in modmap, since
`llvm::PredicateInfo` clashes between `TableGen/CodeGenSchedule.h`
and `Transforms/Utils/PredicateInfo.h`.
(I think better to introduce namespace llvm::TableGen)
Depends on D145937, D146352, and D148768.
Differential Revision: https://reviews.llvm.org/D148767
This patch makes interface of AddressManager from DWARFLinker
to be compatible with AddressesMap from DWARFLinkerParallel.
This makes both linkers to be interchangeable.
Differential Revision: https://reviews.llvm.org/D147455
As a preparation for implementing DWARFv5 address ranges generation,
this patch refactors cloneAddressAttribute() method. It has special
handling for addresses which can be relocated in some unrelated value,
for applying relocations twice, for indexed addresses. Instead of
all these special handlings this patch uses general handling:
Read attribute value from InputDIE and apply PCOffset.
Another thing is that current handling of DW_FORM_addrx misses the
fact that relocations might be applied twice in some cases. This
patch fixes this problem also.
Differential Revision: https://reviews.llvm.org/D143269
This patch is extracted from D96035. It creates an empty library DWARFLinkerParallel.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D140787
generation code from DWARFLinker. It adds command line option:
--build-accelerator [none,DWARF]
Build accelerator tables(default: none)
=none - Do not build accelerators
=DWARF - Build accelerator tables according to the resulting DWARF version
DWARFv4: .debug_pubnames and .debug_pubtypes
DWARFv5: .debug_names
Differential Revision: https://reviews.llvm.org/D139638
Use deduction guides instead of helper functions.
The only non-automatic changes have been:
1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).
Per reviewers' comment, some useless makeArrayRef have been removed in the process.
This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.
Differential Revision: https://reviews.llvm.org/D140955
This avoids recomputing string length that is already known at compile time.
It has a slight impact on preprocessing / compile time, see
https://llvm-compile-time-tracker.com/compare.php?from=3f36d2d579d8b0e8824d9dd99bfa79f456858f88&to=e49640c507ddc6615b5e503144301c8e41f8f434&stat=instructions:u
This a recommit of e953ae5bbc313fd0cc980ce021d487e5b5199ea4 and the subsequent fixes caa713559bd38f337d7d35de35686775e8fb5175 and 06b90e2e9c991e211fecc97948e533320a825470.
The above patchset caused some version of GCC to take eons to compile clang/lib/Basic/Targets/AArch64.cpp, as spotted in aa171833ab0017d9732e82b8682c9848ab25ff9e.
The fix is to make BuiltinInfo tables a compilation unit static variable, instead of a private static variable.
Differential Revision: https://reviews.llvm.org/D139881
Revert "Fix lldb option handling since e953ae5bbc313fd0cc980ce021d487e5b5199ea4 (part 2)"
Revert "Fix lldb option handling since e953ae5bbc313fd0cc980ce021d487e5b5199ea4"
GCC build hangs on this bot https://lab.llvm.org/buildbot/#/builders/37/builds/19104
compiling CMakeFiles/obj.clangBasic.dir/Targets/AArch64.cpp.d
The bot uses GNU 11.3.0, but I can reproduce locally with gcc (Debian 12.2.0-3) 12.2.0.
This reverts commit caa713559bd38f337d7d35de35686775e8fb5175.
This reverts commit 06b90e2e9c991e211fecc97948e533320a825470.
This reverts commit e953ae5bbc313fd0cc980ce021d487e5b5199ea4.
