Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change focuses on the
option's prefixed names and the array of prefixes. These are present in
every option and the dominant source of dynamic relocations for PIE or
PIC users of LLVM and Clang tooling. In some cases, 100s or 1000s of
them for the Clang driver which has a huge number of options.
This PR addresses this by building a string table and a prefixes table
that can be referenced with indices rather than pointers that require
dynamic relocations. This removes almost 7k dynmaic relocations from the
`clang` binary, roughly 8% of the remaining dynmaic relocations outside
of vtables. For busy-boxing use cases where many different option tables
are linked into the same binary, the savings add up a bit more.
The string table is a straightforward mechanism, but the prefixes
required some subtlety. They are encoded in a Pascal-string fashion with
a size followed by a sequence of offsets. This works relatively well for
the small realistic prefixes arrays in use.
Lots of code has to change in order to land this though: both all the
option library code has to be updated to use the string table and
prefixes table, and all the users of the options library have to be
updated to correctly instantiate the objects.
Some follow-up patches in the works to provide an abstraction for this
style of code, and to start using the same technique for some of the
other strings here now that the infrastructure is in place.
Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld
-flavor`, would not longer work on Windows, when these tools were linked
as part of `llvm-driver`. This was caused by `InitLLVM` which was part
of the `*_main()` function of these tools, which in turn calls
`windows::GetCommandLineArguments`. That function completly replaces
argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by
`llvm-driver` prior to calling these tools was reset.
`InitLLVM` is now called by the `llvm-driver`. Any tool that
participates in (or is part of) the `llvm-driver` doesn't call
`InitLLVM` anymore.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
Summary:
llvm-ar is symlinked as llvm-ranlib and will act as ranlib when invoked in that mode. llvm-ar since [[ 4f2cfbe531 | compiler/llvm-project@4f2cfbe ]] supports the -X options, but doesn't seem to accept them when running as llvm-ranlib.
In AIX OS , according to https://www.ibm.com/docs/en/aix/7.2?topic=r-ranlib-command
-X mode Specifies the type of object file ranlib should examine. The mode must be one of the following:
32
Processes only 32-bit object files
64
Processes only 64-bit object files
32_64, any
Processes both 32-bit and 64-bit object files
The default is to process 32-bit object files (ignore 64-bit objects). The mode can also be set with the OBJECT_MODE environment variable. For example, OBJECT_MODE=64 causes ranlib to process any 64-bit objects and ignore 32-bit objects. The -X flag overrides the OBJECT_MODE variable.
Reviewers: James Henderson, MaskRay, Stephen Peckham
Differential Revision: https://reviews.llvm.org/D142660
All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D157028
Switch the parse of command line options fromllvm::cl to OptTable.
The motivation for this change is to continue adding llvm based tools
to the llvm driver multicall.
Differential Revision: https://reviews.llvm.org/D153665
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
component into a new LLVM Component called "TargetParser". This
potentially enables using tablegen to maintain this information, as
is shown in https://reviews.llvm.org/D137517. This cannot currently
be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
information in the TargetParser:
- `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
the current Host machine for info about it, primarily to support
getting the host triple, but also for `-mcpu=native` support in e.g.
Clang. This is fairly tightly intertwined with the information in
`X86TargetParser.h`, so keeping them in the same component makes
sense.
- `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
the target triple parser and representation. This is very intertwined
with the Arm target parser, because the arm architecture version
appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.
And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM
Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.
If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.
Differential Revision: https://reviews.llvm.org/D137838
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
This was done as a test for D137302 and it makes sense to push these changes
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D137493
When using llvm-libtool-darwin as a drop in replacement for cctools
libtool, Xcode expects you to create a dependency info file. This file
is a very simple format describing the input files, the output files,
and the version of the tool. This logic is mirrored from that of
ld64.lld, which supports creating this file as well. Ideally we could
extract it, but I don't think we want to throw this into one of the
grab-bag libraries given how small the logic is.
Differential Revision: https://reviews.llvm.org/D134322
cctools libtool allows you to link dynamic libraries by passing through
a number of arguments to ld64. Because of this the default arguments
libtool receives in Xcode contains arguments that only matter in that
case. This change ignores this argument, at least until we ever support
that dynamic use case, so that you can use llvm-libtool-darwin as a
drop-in replacement in Xcode for cctools libtool. There are more
arguments we could ignore for this case, but we can probably add those
as the use case comes up.
Differential Revision: https://reviews.llvm.org/D134309
libtool can currently produce 2 warnings:
1. No symbols were in the object file
2. An object file with the same basename was specified multiple times
The first warning here is often harmless and may just mean you have some
translation units with no symbols for the target you're building for.
The second warning can lead to real issues like those mentioned in
https://reviews.llvm.org/D113130 where ODR violations can slip in.
This introduces a new -warnings_as_errors flag that can be used by build
systems that want to verify they never hit these warnings. For example
with bazel the libtool caller first uniques names to make sure the
duplicate base name case is not possible, but if that doesn't work as
expected, having it fail would be preferred.
It's also worth noting that llvm-libtool-darwin works around an issue
that cctools libtool experiences related to debug info and duplicate
basenames, the workaround is described here:
30baa5d2a4/llvm/lib/Object/ArchiveWriter.cpp (L424-L465)
And it avoids this bug:
f0cbbb1c37/DuplicateBasenameIssue
Differential Revision: https://reviews.llvm.org/D118931
When using llvm-libtool-darwin with LTO building llvm itself, it crashed
on a file with an asm module in the bitcode. This fixes that by
correctly registering the targets for this.
Differential Revision: https://reviews.llvm.org/D118575
This more closely mirrors apple's libtool, and also potentially makes it
clearer for multi-arch archives where the issue lies.
Differential Revision: https://reviews.llvm.org/D118867
Loosen the test to make it pass on Windows.
This reapplies commit 4993eff3e253a1c04e1a1a2fa5d68f6b33423419.
This reverts commit 96c66040125e87808b23410632b1a6a34547b4e3.
This reverts commit 4993eff3e253a1c04e1a1a2fa5d68f6b33423419.
Tests are failing on Windows: http://45.33.8.238/win/52360/step_11.txt.
We'll need to account for forward slashes vs. backslashes.
Print a warning if `llvm-libtool-darwin` if any of the object
files provided by the user have the same file name.
The tool will now print a warning if there is a name collision across:
* Two object files
* An object file and an object file from within a static library
* Two object files from different static libraries
Here is an example of the error:
```
$ llvm-libtool-darwin -static -o archive.a out.o out.o
error: file 'out.o' was specified multiple times.
in: out.o
in: out.o
$ llvm-libtool-darwin -static -o archive.a out.o
$ llvm-libtool-darwin -static -o combined.a archive.a out.o
error: file 'out.o' was specified multiple times.
in: archive.a
in: out.o
```
This change mimics apple's cctools libtool's behavior which always shows a warning in such cases.
Reviewed By: smeenai
Differential Revision: https://reviews.llvm.org/D113130
Here we are refactoring the code to encapsulate data into classes. This allows
us to avoid passing the same objects through many functions that don't directly
use them. Now, functions that need to access data can do so from the class
state.
Reviewed By: jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D113127
The existing code has unnecessary logic to indirectly pass
errors through function calls. This diff gets rid of the fluff.
Test Plan: Existing unit tests
Reviewed By: jhenderson, drodriguez, alexander-shaposhnikov
Differential Revision: https://reviews.llvm.org/D113301
We can build it with -Werror=global-constructors now. This helps
in situation where libSupport is embedded as a shared library,
potential with dlopen/dlclose scenario, and when command-line
parsing or other facilities may not be involved. Avoiding the
implicit construction of these cl::opt can avoid double-registration
issues and other kind of behavior.
Reviewed By: lattner, jpienaar
Differential Revision: https://reviews.llvm.org/D105959
We can build it with -Werror=global-constructors now. This helps
in situation where libSupport is embedded as a shared library,
potential with dlopen/dlclose scenario, and when command-line
parsing or other facilities may not be involved. Avoiding the
implicit construction of these cl::opt can avoid double-registration
issues and other kind of behavior.
Reviewed By: lattner, jpienaar
Differential Revision: https://reviews.llvm.org/D105959
We can build it with -Werror=global-constructors now. This helps
in situation where libSupport is embedded as a shared library,
potential with dlopen/dlclose scenario, and when command-line
parsing or other facilities may not be involved. Avoiding the
implicit construction of these cl::opt can avoid double-registration
issues and other kind of behavior.
Reviewed By: lattner, jpienaar
Differential Revision: https://reviews.llvm.org/D105959
CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.
(Already remarked by jhenderson on D70769.)
No behavior change.
Differential Revision: https://reviews.llvm.org/D100957
TextAPI/ELF has moved out into InterfaceStubs, so theres no longer a
need to seperate out TextAPI between formats.
Reviewed By: ributzka, int3, #lld-macho
Differential Revision: https://reviews.llvm.org/D99811
In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.
```
static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFile(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true, bool IsVolatile = false);
static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFileOrSTDIN(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true);
static ErrorOr<std::unique_ptr<MB>>
getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
bool IsText, bool RequiresNullTerminator, bool IsVolatile);
static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
getFile(const Twine &Filename, bool IsVolatile = false);
```
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D99182
1. Emit warnings for files without symbols.
2. Add -no_warning_for_no_symbols.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D95843
No longer rely on an external tool to build the llvm component layout.
Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.
These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.
Differential Revision: https://reviews.llvm.org/D90848
This diff adds support for LLVM bitcode objects to llvm-libtool-darwin.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D88722
This patch performs a minor cleanup of the class Slice:
static methods and constructors which take a pointer but assume that
it's not null now take the argument by reference.
NFC.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D88320
The -V option in cctools' libtool prints out the version number and
performs any specified operation. Add this option to LLVM's version.
cctools is more forgiving of invalid command lines when -V is specified,
but I think it's better to give errors instead of silently producing no
output.
Unfortunately, when -V is present, options that would otherwise be
required aren't anymore, so we need to perform some manual argument
validation.
Reviewed By: alexshap
Differential Revision: https://reviews.llvm.org/D86359
Add support for passing in libraries via `-l` and `-L` options to
`llvm-libtool-darwin`.
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D85540
Add support for producing universal binaries containing archives when
`llvm-libtool-darwin` is given inputs of multiple architectures.
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D85334
Add support for -arch_only option for llvm-libtool-darwin. This diff
also adds support for accepting universal files as input and flattening
them to create the required static library. Supports input universal
files contaning both Mach-O object files or archives.
Differences from cctools' libtool:
- `-arch_only` can be specified multiple times
- archives containing universal files are considered invalid (libtool
allows such archives)
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D84770
