Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld
-flavor`, would not longer work on Windows, when these tools were linked
as part of `llvm-driver`. This was caused by `InitLLVM` which was part
of the `*_main()` function of these tools, which in turn calls
`windows::GetCommandLineArguments`. That function completly replaces
argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by
`llvm-driver` prior to calling these tools was reset.
`InitLLVM` is now called by the `llvm-driver`. Any tool that
participates in (or is part of) the `llvm-driver` doesn't call
`InitLLVM` anymore.
Binutils 2.36 had a somewhat controversial change in how the
--preprocessor option was handled in GNU windres; previously, the option
was interpreted as a part of the command string, potentially containing
multiple arguments (which even was hinted at in the documentation).
In Binutils 2.36, this was changed to interpret the --preprocessor
argument as one argument (possibly containing spaces) pointing at the
preprocessor executable.
The existing behaviour where implicit arguments like -E -xc -DRC_INVOKED
are dropped if --preprocessor is specified, was kept.
This was a breaking change for some users of GNU windres, see
https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=21c33bcbe36377abf01614fb1b9be439a3b6de20,
https://sourceware.org/bugzilla/show_bug.cgi?id=27594, and
https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=5edb8e3f5ad8d74a83fc0df7f6e4514eed0aa77f.
As multiple years have passed since, the behaviour change seems to be
here to stay, and any users of the previous form of the option have been
forced to avoid this construct. Thus update llvm-windres to match the
new way Binutils of handling this option.
One construct for specifying the path to the preprocessor, which works
both before and after binutils 2.36 (and this change in llvm-windres) is
to specify options like this:
--preprocessor path/to/executable --preprocessor-arg -E
--preprocessor-arg -xc -DRC_INVOKED
The llvm::sys::ExecuteAndWait function doesn't resolve the file to be
executed from $PATH - i.e. it is similar to execv(), not execvp().
Due to this, specifying a --preprocessor argument to llvm-windres only
worked if it specified an absolute path to the preprocessor executable.
This was observed as one of the issues in
https://github.com/msys2/MINGW-packages/pull/19157.
Before d2fa6b694c2052cef1ddd507f6569bc84e3bbe35, this usage of
--preprocessor seemed to work, because the first argument of Args[] was
ignored and llvm-windres just executed the autodetected clang executable
regardless.
Also improve the error messages printed if preprocessing failed. (If the
preprocessor executable was started but itself returned an error, we
don't get any error string.)
If passing the windres option --preprocessor, the default arguments "-E
-xc -DRC_INVOKED" aren't passed. If these are passed explicitly by the
user via --preprocessor-arg instead, we need to make sure that "-xc" is
passed before the input filename, as this compiler/preprocessor option
only has an effect on input files that follow it.
This fixes one of the issues with llvm-windres observed in
https://github.com/msys2/MINGW-packages/pull/19157.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
GNU windres supports this, while MS rc.exe doesn't.
MS rc.exe only supports treating consecutive string literals as if they
were fused into one in a few fixed locations (most of which are already
supported), while GNU windres supports this essentially anywhere in any
string. See
b989fcbae6f179ad887d19ceef83ace1c00b87cc for one recent change that
extended support for this in one specific resource.
A reasonable use case for multiple concatenated string literals that GNU
windres accepts is `1 ICON DIR "/name.ico"`, where the directory is
provided via the preprocessor, expanding to another string literal; this
is https://github.com/llvm/llvm-project/issues/51286.
Extend the parser to try to consume all consecutive string tokens,
whenever reading a filename. Adjust the handling of user data resources
read from a file to use the readFilename() helper.
While this probably doesn't cover every single case where GNU windres
might accept concatenated string literals, this is the primary missing
case that has been reported so far.
Note that llvm::support::endianness has been renamed to
llvm::endianness while becoming an enum class as opposed to an
enum. This patch replaces support::{big,little,native} with
llvm::endianness::{big,little,native}.
Whether a temp file or a pipe is used for preprocessing is an
internal detail, this flag has a notable effect on the preprocessing
in GNU windres. Without this flag, GNU windres passes command
arguments as-is to popen(), which means they get evaluated by a
shell without being re-escaped for this case. To mimic this,
llvm-windres has manually tried to unescape arguments.
When GNU windres is given the --use-temp-file flag, it uses a
different API for invoking the preprocessor, and this API takes care
of preserving special characters in the command line arguments.
For users of GNU windres, this means that by using --use-temp-file,
they don't need to do the (quite terrible) double escaping of
quotes/spaces etc.
The xz project uses the --use-temp-file flag when invoking
GNU windres, see
6b117d3b1f.
However as llvm-windres didn't implement this flag and just
assumed the GNU windres popen() behaviour, they had to use a
different codepath for llvm-windres.
That separate codepath for llvm-windres broke later when llvm-windres
got slightly more accurate unescaping of lone quotes in
0f4c6b120f21d582ab7c5c4f2b2a475086c34938 /
https://reviews.llvm.org/D146848 (fixing a discrepancy to GNU
windres as found in https://github.com/llvm/llvm-project/issues/57334),
and this was reported in
https://github.com/mstorsjo/llvm-mingw/issues/363.
Not touching the implementation of the --preprocessor option
with respect to the --use-temp-file flag; that option is doubly
tricky as GNU windres changed its behaviour in a backwards incompatible
way recently (and llvm-windres currently matches the old behaviour).
(See
https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=21c33bcbe36377abf01614fb1b9be439a3b6de20,
https://sourceware.org/bugzilla/show_bug.cgi?id=27594 and
https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=5edb8e3f5ad8d74a83fc0df7f6e4514eed0aa77f;hp=3abbafc2aacc6706fea3e3e326e2f08d107c3672
for the behaviour change.)
Differential Revision: https://reviews.llvm.org/D159223
In internal google builds, MainExecPath doesn't go to the directory with `clang`.
Fall back to using Argv0 if MainExecPath doesn't find any clangs.
Differential Revision: https://reviews.llvm.org/D158901
This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lldb. I've also
renamed OptionVisibility::Default to "DefaultVis" to avoid ambiguity
since the undecorated name has to be available anywhere Options.inc is
included.
Original message follows:
This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.
We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.
Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.
Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.
Differential Revision: https://reviews.llvm.org/D157149
This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.
We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.
Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.
Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.
Differential Revision: https://reviews.llvm.org/D157149
The llvm-rc tool tries to locate a suitable Clang executable to
use for preprocessing. For this purpose, it first checks within
the same directory as the llvm-rc tool, checking with a couple
different names, followed by checking all of $PATH for another
couple names.
On Windows, the InitLLVM() function always sets up Argv[0] with the
full path to the executable, while on Unix, Argv[0] is kept as is.
Therefore, call getMainExecutable to try to resolve the directory of
the executable before looking for colocated Clang executables.
This makes 282744a9ce18120dc0a6eceb02693b36980d9498 actually have
the desired effect.
Differential Revision: https://reviews.llvm.org/D157241
All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D157028
This more clearly identifies the tool as llvm-rc.
This should hopefully allow Meson to check for parts of these strings
in the output of "$CMD /?" when detecting the kind of resource compiler
tool, to allow Meson to recognize llvm-rc.
Differential Revision: https://reviews.llvm.org/D154545
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
This patch migrates uses of StringRef::{starts,ends}with_insensitive
to StringRef::{starts,ends}_with_insensitive so that we can use names
similar to those used in std::string_view. I'm planning to deprecate
StringRef::{starts,ends}with_insensitive once the migration is
complete across the code base.
Differential Revision: https://reviews.llvm.org/D150426
Some background context: GNU windres invokes the preprocessor in
a subprocess. Some windres options are passed through to the
preproocessor, e.g. -D options for predefining defines.
When GNU windres passes these options onwards, it takes the options
in exact the form they are received (in argv or similar) and
assembles them into a single preprocessor command string which gets
interpreted by a shell (IIRC via the popen() function, or similar).
When LLVM invokes subprocesses, it does so via APIs that take
properly split argument vectors, to avoid needing to worry about
shell quoting/escaping/unescaping. But in the case of LLVM windres,
we have to emulate the effect of the shell parsing done by popen().
Most of the relevant cases are already taken care of here, but this
patch fixes an uncommon case encountered in
https://github.com/llvm/llvm-project/issues/57334.
(This case is uncommon since it doesn't do what one would want to;
the quotes need to be escaped more to work as intended through
the popen() shell).
Differential Revision: https://reviews.llvm.org/D146848
When preprocessing was integrated to llvm-rc in 2021, this was a
new requirement (previously one could execute llvm-rc without a
suitable preprocessing tool to be available).
As a transitional helper, llvm-rc fell back on skipping preprocessing
if no suitable tool was found (with a warning printed), but users
could pass an llvm-rc specific option to silence the warning, if they
explicitly want to run the tool without preprocessing.
Now 2 years later, remove the transitional helper - error out if
preprocessing failed. The option for disabling preprocessing remains.
Differential Revision: https://reviews.llvm.org/D146797
This was the original option name from the first iteration of the patch
that added the feature, but during review, a different name was suggested
and preferred - but the reference in the helpful message was missed.
Differential Revision: https://reviews.llvm.org/D146796
In some cases, there's no adjacent executable named "clang" or
"clang-cl", but one name "clang-<major>". This logic doesn't
cover every possible deployment setup of course, but should
cover more fairly common/reasonable cases.
See
caaae171ac (commitcomment-105808524)
for discussion about a case where this would have been helpful.
Differential Revision: https://reviews.llvm.org/D146794
The arguments passed in this option were passed onto the child
process, but we still blindly used the clang binary that we had
found to sys::ExecuteAndWait as the intended executable to run.
If the user hasn't specified any custom --preprocessor command,
Args[0] is equal to the variable Clang.
This doesn't affect any tests, since the tests only print the
arguments it would try to execute (but not the first parameter to
sys::ExecuteAndWait), but there's no testes for executing it
(and validating that it did execute the right thing).
Differential Revision: https://reviews.llvm.org/D146793
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
Use deduction guides instead of helper functions.
The only non-automatic changes have been:
1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).
Per reviewers' comment, some useless makeArrayRef have been removed in the process.
This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.
Differential Revision: https://reviews.llvm.org/D140955
This avoids recomputing string length that is already known at compile time.
It has a slight impact on preprocessing / compile time, see
https://llvm-compile-time-tracker.com/compare.php?from=3f36d2d579d8b0e8824d9dd99bfa79f456858f88&to=e49640c507ddc6615b5e503144301c8e41f8f434&stat=instructions:u
This a recommit of e953ae5bbc313fd0cc980ce021d487e5b5199ea4 and the subsequent fixes caa713559bd38f337d7d35de35686775e8fb5175 and 06b90e2e9c991e211fecc97948e533320a825470.
The above patchset caused some version of GCC to take eons to compile clang/lib/Basic/Targets/AArch64.cpp, as spotted in aa171833ab0017d9732e82b8682c9848ab25ff9e.
The fix is to make BuiltinInfo tables a compilation unit static variable, instead of a private static variable.
Differential Revision: https://reviews.llvm.org/D139881
Revert "Fix lldb option handling since e953ae5bbc313fd0cc980ce021d487e5b5199ea4 (part 2)"
Revert "Fix lldb option handling since e953ae5bbc313fd0cc980ce021d487e5b5199ea4"
GCC build hangs on this bot https://lab.llvm.org/buildbot/#/builders/37/builds/19104
compiling CMakeFiles/obj.clangBasic.dir/Targets/AArch64.cpp.d
The bot uses GNU 11.3.0, but I can reproduce locally with gcc (Debian 12.2.0-3) 12.2.0.
This reverts commit caa713559bd38f337d7d35de35686775e8fb5175.
This reverts commit 06b90e2e9c991e211fecc97948e533320a825470.
This reverts commit e953ae5bbc313fd0cc980ce021d487e5b5199ea4.
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
component into a new LLVM Component called "TargetParser". This
potentially enables using tablegen to maintain this information, as
is shown in https://reviews.llvm.org/D137517. This cannot currently
be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
information in the TargetParser:
- `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
the current Host machine for info about it, primarily to support
getting the host triple, but also for `-mcpu=native` support in e.g.
Clang. This is fairly tightly intertwined with the information in
`X86TargetParser.h`, so keeping them in the same component makes
sense.
- `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
the target triple parser and representation. This is very intertwined
with the Arm target parser, because the arm architecture version
appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.
And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM
Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.
If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.
Differential Revision: https://reviews.llvm.org/D137838
The llvm-driver, enabled with LLVM_TOOL_LLVM_DRIVER_BUILD combines many llvm executables
into one to save overall toolchain size. This patch adds a few more llvm tools to the
llvm-driver.
Differential Revision: https://reviews.llvm.org/D135281
