269 Commits

Author SHA1 Message Date
Sam Clegg
3c5845703c
[lld][WebAssembly] Move input vectors from symtab to ctx. NFC (#78640)
Also convert from std::vector to SmallVector.

This matches the ELF linker where these were moved into the ctx object
in 9a572164d592e and converted to SmallVector in ba948c5a9c524b.
2024-01-18 15:53:13 -08:00
Sam Clegg
184c22dd3a
[lld][WebAssembly] Move linker global state in to context object. NFC (#78629)
See lld/ELF/Config.h
2024-01-18 15:01:21 -08:00
Sam Clegg
f268495914
[lld][WebAssembly] Rename fetch() to extract() to match ELF linker. NFC (#78625) 2024-01-18 14:39:38 -08:00
SingleAccretion
b2cdf3cc4c
[lld][WebAssembly] Add an --initial-heap option (#75594)
It is beneficial to preallocate a certain number of pages in the linear
memory (i. e. use the "minimum" field of WASM memories) so that fewer
"memory.grow"s are needed at startup.

So far, the way to do that has been to pass the "--initial-memory"
option to the linker. It works, but has the very significant downside of
requiring the user to know the size of static data beforehand, as it
must not exceed the number of bytes passed-in as "--initial-memory".

The new "--initial-heap" option avoids this downside by simply appending
the specified number of pages to static data (and stack), regardless of
how large they already are.

Ref: https://github.com/emscripten-core/emscripten/issues/20888.
2023-12-15 10:16:38 -08:00
Sam Clegg
97b25d91df
[lld][WebAssembly] Don't set importUndefined when -shared is used. NFC (#75241)
`importUndefined` is only used a couple of places and both of those
already handle `isPIC` separately.
2023-12-12 13:19:42 -08:00
Sam Clegg
89d5635f0a [lld][WebAssembly] Add --keep-section flag
This flag causes wasm-ld preserve a section even in the face of
`--strip-all`.  This is useful, for example, to preserve the
target_features section in the ase of clang (which can run wasm-opt
after linking), and emcc (which performs a bunch of post-link work).

Fixes: https://github.com/llvm/llvm-project/issues/60613
Fixes: https://github.com/llvm/llvm-project/issues/55781

Differential Revision: https://reviews.llvm.org/D149917
2023-11-02 14:23:45 -07:00
Sam Clegg
93adcb770b [lld][WebAssembly] Add --table-base setting
This is similar to `--global-base` but determines where to place the
table segments rather than that data segments.

See https://github.com/emscripten-core/emscripten/issues/20097

Differential Revision: https://reviews.llvm.org/D158892
2023-08-25 16:10:38 -07:00
Sam Clegg
8e44f037dc [lld][WebAssembly] Add support for -soname
This change writes the module name to the name section of the wasm
binary.  We use the `-soname` argument to determine the name and we
default the output file basename if this option is not specified.

In the future we will likely want to embed the soname in the dylink
section too, but this the first step in supporting `-soname`.

Differential Revision: https://reviews.llvm.org/D158001
2023-08-15 18:33:45 -07:00
Justin Bogner
dcb6d212fd Reapply "[Option] Add "Visibility" field and clone the OptTable APIs to use it"
This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lldb. I've also
renamed OptionVisibility::Default to "DefaultVis" to avoid ambiguity
since the undecorated name has to be available anywhere Options.inc is
included.

Original message follows:

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149
2023-08-15 01:16:58 -07:00
Justin Bogner
4e3b89483a Revert "[Option] Add "Visibility" field and clone the OptTable APIs to use it"
this is failing on bots, reverting to investigate.

This reverts commit a16104e6da6f36f3d72dbf53d10ba56495a0d65a.
2023-08-14 13:31:02 -07:00
Justin Bogner
a16104e6da [Option] Add "Visibility" field and clone the OptTable APIs to use it
This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149
2023-08-14 13:24:54 -07:00
Sam Clegg
7a8ee92fa8 [lld][WebAssembly] Process stub libraries in a loop
When stub libraries trigger the fetching of new object files we can
potentially introduce new undefined symbols so process the stub in
loop until no new objects are pulled in.

Differential Revision: https://reviews.llvm.org/D153466
2023-08-11 11:07:40 -07:00
Jan Svoboda
3f092f37b7 [llvm] Extract common OptTable bits into macros
All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D157028
2023-08-04 13:57:13 -07:00
Alexandre Ganea
6f2e92c10c Re-land [LLD] Allow usage of LLD as a library
This reverts commit aa495214b39d475bab24b468de7a7c676ce9e366.

As discussed in https://github.com/llvm/llvm-project/issues/53475 this patch
allows for using LLD-as-a-lib. It also lets clients link only the drivers that
they want (see unit tests).

This also adds the unit test infra as in the other LLVM projects. Among the
test coverage, I've added the original issue from @krzysz00, see:
https://github.com/ROCmSoftwarePlatform/D108850-lld-bug-reproduction

Important note: this doesn't allow (yet) linking in parallel. This will come a
bit later hopefully, in subsequent patches, for COFF at least.

Differential revision: https://reviews.llvm.org/D119049
2023-06-19 07:35:11 -04:00
Leonard Chan
aa495214b3 Revert "[LLD] Allow usage of LLD as a library"
This reverts commit 2700da5fe28d8b17c66e5c960d2188276a6ced39.

Reverting since this causes some test failures on our builders: https://ci.chromium.org/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8778372807208184913/overview
2023-06-14 20:36:27 +00:00
Alexandre Ganea
2700da5fe2 [LLD] Allow usage of LLD as a library
As discussed in https://github.com/llvm/llvm-project/issues/53475 this patch allows using LLD-as-a-lib. It also lets clients link only the drivers that they want (see unit tests).

This also adds the unit test infra as in the other LLVM projects. Among the test coverage, I've added the original issue from @krzysz00, see: https://github.com/ROCmSoftwarePlatform/D108850-lld-bug-reproduction

Important note: this doesn't allow (yet) linking in parallel. This will come a bit later, in subsequent patches, for COFF at last.

Differential revision: https://reviews.llvm.org/D119049
2023-06-13 16:22:59 -04:00
Fangrui Song
8d85c96e0e [lld] StringRef::{starts,ends}with => {starts,ends}_with. NFC
The latter form is now preferred to be similar to C++20 starts_with.
This replacement also removes one function call when startswith is not inlined.
2023-06-05 14:36:19 -07:00
Sam Clegg
d43f0889dd [lld][WebAssembly] stub objects: Fix handling of LTO libcall dependencies
This actually simplifies the code by performs a pre-pass of the stub
objects prior to LTO.

This should be the final change needed before we can make the switch
on the emscripten side: https://github.com/emscripten-core/emscripten/pull/18905

Differential Revision: https://reviews.llvm.org/D148287
2023-04-13 20:32:05 -07:00
Sam Clegg
aca110f9dd [lld][WebAssembly] Trace export of symbols when specified with --trace-symbol. NFC
Differential Revision: https://reviews.llvm.org/D148190
2023-04-13 10:36:27 -07:00
Sam Clegg
d9d840cdaf [lld][WebAssembly] Fix stub library parsing with windows line endings
Also, fix checking of first line in ::parse.  We can't use the
::getLines helper here since that already does comment stripping
internally.

Differential Revision: https://reviews.llvm.org/D147548
2023-04-04 11:55:01 -07:00
Sam Clegg
2ea8a3a56a [lld][WebAssembly] Process stub libraries before performing LTO
There are cases where stub library processing can trigger new exports
which might require them to be included at LTO time.

Specifically `processStubLibraries` marks symbols as `forceExports`
which even effect the LTO process.

And since the LTO process can generate new undefined symbols
(specifically libcall function) we need to also process the stub
libraries after LTO.

Differential Revision: https://reviews.llvm.org/D147190
2023-03-30 14:27:19 -07:00
Sam Clegg
3111784ff7 [lld][WebAssembly] Initial support for stub libraries
See the docs in lld/docs/WebAssembly.rst for more on this.

This feature unlocks a lot of simplification in the emscripten toolchain
since we can represent the JS libraries to wasm-ld as stub libraries.

See https://github.com/emscripten-core/emscripten/issues/18875

Differential Revision: https://reviews.llvm.org/D145308
2023-03-23 14:26:27 -07:00
Congcong Cai
467cf15428 [NFC] Fix typo lld::wasm in comment 2023-03-23 15:40:55 +08:00
Sam Clegg
d32f71a91a [lld][WebAssembly] Use C++17 nested namespace syntax in most places. NFC
Like D131405, but for wasm-ld.

Differential Revision: https://reviews.llvm.org/D145399
2023-03-09 18:37:32 -08:00
Sam Clegg
8aef04fa69 [lld][WebAssembly] Implement --why-extract flag from the ELF backend
See https://reviews.llvm.org/D109572 for the original ELF version.

Differential Revision: https://reviews.llvm.org/D145431
2023-03-06 17:52:49 -08:00
Derek Schuff
c7af9ae577 Reapply: [WebAssembly] Implement build-id feature
Implement the --build-id flag similarly to ELF, and generate a
build_id section according to the WebAssembly tool convention
specified in https://github.com/WebAssembly/tool-conventions/pull/183

The default style ("fast" aka "tree") hashes the contents of the
output and (unlike ELF) generates a v5 UUID based on the hash (using a
random namespace). It also supports generating a random v4 UUID, a
sha1 hash, and a user-specified string (as ELF does).

Differential Revision: https://reviews.llvm.org/D107662

Fix MSVC build by std::copy on the underying buffer rather than
directly from std::array to llvm::MutableArrayRef
2023-03-03 17:15:55 -08:00
Derek Schuff
c8e5354f00 Revert "[WebAssembly] Implement build-id feature"
This reverts commit 41e31466af6a7feab82742bb01af43f4f3ae4ede
due to a build failure on Windows.
2023-03-02 15:28:56 -08:00
Derek Schuff
41e31466af [WebAssembly] Implement build-id feature
Implement the --build-id flag similarly to ELF, and generate a build_id
section according to the WebAssembly tool convention specified in
https://github.com/WebAssembly/tool-conventions/pull/183

The default style ("fast" aka "tree") hashes the contents of the output
and (unlike ELF) generates a v5 UUID based on the hash (using a random
namespace).
It also supports generating a random v4 UUID, a sha1 hash,
and a user-specified string (as ELF does).

Differential Revision: https://reviews.llvm.org/D107662
2023-03-02 14:19:06 -08:00
Scott Linder
45ee0a9afc [LLD] Add --lto-CGO[0-3] option
Allow controlling the CodeGenOpt::Level independent of the LTO
optimization level in LLD via new options for the COFF, ELF, MachO, and
wasm frontends to lld. Most are spelled as --lto-CGO[0-3], but COFF is
spelled as -opt:lldltocgo=[0-3].

See D57422 for discussion surrounding the issue of how to set the CG opt
level. The ultimate goal is to let each function control its CG opt
level, but until then the current default means it is impossible to
specify a CG opt level lower than 2 while using LTO. This option gives
the user a means to control it for as long as it is not handled on a
per-function basis.

Reviewed By: MaskRay, #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D141970
2023-02-15 17:34:35 +00:00
Archibald Elliott
d768bf994f [NFC][TargetParser] Replace uses of llvm/Support/Host.h
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
2023-02-10 09:59:46 +00:00
Nico Weber
a2ae9e3a24 [lld] fix comment typos to cycle bots 2023-02-04 15:23:30 -05:00
serge-sans-paille
07bb29d8ff
[OptTable] Precompute OptTable prefixes union table through tablegen
This avoid rediscovering this table when reading each options, providing
a sensible 2% speedup when processing and empty file, and a measurable
speedup on typical workloads, see:

This is optional, the legacy, on-the-fly, approach can still be used
through the GenericOptTable class, while the new one is used through
PrecomputedOptTable.

https://llvm-compile-time-tracker.com/compare.php?from=4da6cb3202817ee2897d6b690e4af950459caea4&to=19a492b704e8f5c1dea120b9c0d3859bd78796be&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D140800
2023-01-12 12:08:06 +01:00
Kazu Hirata
b9ef5648b5 [lld] Use std::optional instead of llvm::Optional (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-02 18:29:04 -08:00
Fangrui Song
6b9a80de49 [lld] Fix iwyu problems after 83d59e05b201760e3f364ff6316301d347cbad95
The commit transitively includes lld/include/lld/Common/ErrorHandler.h into
lld/include/lld/Common/Driver.h, which is not intended.
2022-12-28 10:46:45 -08:00
serge-sans-paille
d9ab3e82f3
[clang] Use a StringRef instead of a raw char pointer to store builtin and call information
This avoids recomputing string length that is already known at compile time.

It has a slight impact on preprocessing / compile time, see

https://llvm-compile-time-tracker.com/compare.php?from=3f36d2d579d8b0e8824d9dd99bfa79f456858f88&to=e49640c507ddc6615b5e503144301c8e41f8f434&stat=instructions:u

This a recommit of e953ae5bbc313fd0cc980ce021d487e5b5199ea4 and the subsequent fixes caa713559bd38f337d7d35de35686775e8fb5175 and 06b90e2e9c991e211fecc97948e533320a825470.

The above patchset caused some version of GCC to take eons to compile clang/lib/Basic/Targets/AArch64.cpp, as spotted in aa171833ab0017d9732e82b8682c9848ab25ff9e.
The fix is to make BuiltinInfo tables a compilation unit static variable, instead of a private static variable.

Differential Revision: https://reviews.llvm.org/D139881
2022-12-27 09:55:19 +01:00
Vitaly Buka
aa171833ab Revert "[clang] Use a StringRef instead of a raw char pointer to store builtin and call information"
Revert "Fix lldb option handling since e953ae5bbc313fd0cc980ce021d487e5b5199ea4 (part 2)"
Revert "Fix lldb option handling since e953ae5bbc313fd0cc980ce021d487e5b5199ea4"

GCC build hangs on this bot https://lab.llvm.org/buildbot/#/builders/37/builds/19104
compiling CMakeFiles/obj.clangBasic.dir/Targets/AArch64.cpp.d

The bot uses GNU 11.3.0, but I can reproduce locally with gcc (Debian 12.2.0-3) 12.2.0.

This reverts commit caa713559bd38f337d7d35de35686775e8fb5175.
This reverts commit 06b90e2e9c991e211fecc97948e533320a825470.
This reverts commit e953ae5bbc313fd0cc980ce021d487e5b5199ea4.
2022-12-25 23:12:47 -08:00
serge-sans-paille
e953ae5bbc
[clang] Use a StringRef instead of a raw char pointer to store builtin and call information
This avoids recomputing string length that is already known at compile
time.

It has a slight impact on preprocessing / compile time, see

https://llvm-compile-time-tracker.com/compare.php?from=3f36d2d579d8b0e8824d9dd99bfa79f456858f88&to=e49640c507ddc6615b5e503144301c8e41f8f434&stat=instructions:u

This is a recommit of 719d98dfa841c522d8d452f0685e503538415a53 that into
account a GGC issue (probably
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92181) when dealing with
intiailizer_list and constant expressions.

Workaround this by avoiding initializer list, at the expense of a
temporary plain old array.

Differential Revision: https://reviews.llvm.org/D139881
2022-12-24 10:25:06 +01:00
serge-sans-paille
07d9ab9aa5
Revert "[clang] Use a StringRef instead of a raw char pointer to store builtin and call information"
There are still remaining issues with GCC 12, see for instance

https://lab.llvm.org/buildbot/#/builders/93/builds/12669

This reverts commit 5ce4e92264102de21760c94db9166afe8f71fcf6.
2022-12-23 13:29:21 +01:00
serge-sans-paille
5ce4e92264
[clang] Use a StringRef instead of a raw char pointer to store builtin and call information
This avoids recomputing string length that is already known at compile
time.

It has a slight impact on preprocessing / compile time, see

https://llvm-compile-time-tracker.com/compare.php?from=3f36d2d579d8b0e8824d9dd99bfa79f456858f88&to=e49640c507ddc6615b5e503144301c8e41f8f434&stat=instructions:u

This is a recommit of 719d98dfa841c522d8d452f0685e503538415a53 with a
change to llvm/utils/TableGen/OptParserEmitter.cpp to cope with GCC bug
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108158

Differential Revision: https://reviews.llvm.org/D139881
2022-12-23 12:48:17 +01:00
serge-sans-paille
b7065a31b5
Revert "[clang] Use a StringRef instead of a raw char pointer to store builtin and call information"
Failing builds: https://lab.llvm.org/buildbot#builders/9/builds/19030
This is GCC specific and has been reported upstream: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108158

This reverts commit 719d98dfa841c522d8d452f0685e503538415a53.
2022-12-23 11:36:56 +01:00
serge-sans-paille
719d98dfa8
[clang] Use a StringRef instead of a raw char pointer to store builtin and call information
This avoids recomputing string length that is already known at compile
time.

It has a slight impact on preprocessing / compile time, see

https://llvm-compile-time-tracker.com/compare.php?from=3f36d2d579d8b0e8824d9dd99bfa79f456858f88&to=e49640c507ddc6615b5e503144301c8e41f8f434&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D139881
2022-12-23 10:31:47 +01:00
serge-sans-paille
6a35815c73
Store OptTable::Info::Name as a StringRef
This is a recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e,
with a few cleanups.

This avoids implicit conversion to StringRef at several points, which in
turns avoid redundant calls to strlen.

As a side effect, this greatly simplifies the implementation of
StrCmpOptionNameIgnoreCase.

It also eventually gives a consistent, humble speedup in compilation
time (timing updated since original commit).

https://llvm-compile-time-tracker.com/compare.php?from=de4b6a1bc64db33643f001ad45fae7b92b4a4688&to=c23a93d1292052b4be2fbe8c586fa31143d0c7ed&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D139274
2022-12-08 10:28:56 +01:00
Kazu Hirata
c68af42fa8 [lld] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 23:12:36 -08:00
Sam Clegg
3c28a6d2cb [lld][WebAssembly] Add support for LLD_REPRODUCE
This is feature of the ELF and COFF linker that we were missing until
now.

Differential Revision: https://reviews.llvm.org/D137377
2022-11-03 16:55:37 -07:00
Florian Mayer
2390bb2347 [NFC] use has_value instead of hasValue 2022-10-31 16:43:14 -07:00
Dan Gohman
d4c8a0edca [wasm-ld] Allow importing/exporting the output module's memory with arbitrary names
This adds an `--export-memory` option to wasm-ld which allows passing
a name to give to the exported memory, and extends `--import-memory` to
allow passing a <module>,<name> pair specifying where the memory should
be imported from.

This is based on https://reviews.llvm.org/D131376, with the main
difference being that it only supports exporting memory by one name.

Differential Revision: https://reviews.llvm.org/D135898
2022-10-31 13:59:46 -07:00
Sam Clegg
6912ed7b8f [lld][WebAssembly] Don't allow --global-base to be specified in -share/-pie or --relocatable modes
Add some checks around this combination of flags

Also, honor `--global-base` when specified in `--stack-first` mode
rather than ignoring it.  But error out if the specified base preseeds
the end of the stack.

Differential Revision: https://reviews.llvm.org/D136117
2022-10-18 17:19:26 -07:00
Dan Gohman
4b24e9be17 [wasm-ld] Define a __heap_end symbol marking the end of allocated memory.
Define a `__heap_end` symbol that marks the end of the memory region
that starts at `__heap_base`. This will allow malloc implementations to
know how much memory they can use at `__heap_base` even if someone has
done a `memory.grow` before they can initialize their state.

Differential Revision: https://reviews.llvm.org/D136110
2022-10-17 16:39:06 -07:00
Sam Clegg
1532be98f9 [lld][WebAssembly] Add symbols marking start/end of stack region
Currently emscripten is make assumptions about that memory layout,
assuming the stack is between `__data_end` and `__heap_base`:

af961ad5c4/system/lib/compiler-rt/stack_limits.S (L42-L61)

With this change we can be more precise:

https://github.com/emscripten-core/emscripten/pull/18057

Differential Revision: https://reviews.llvm.org/D135910
2022-10-13 14:52:17 -07:00
Sam Clegg
c07e838130 [lld][WebAssembly] Add --extra-features flag to add addional features
This flag acts just like the existing `--features` flag but instead
of replacing the set of inferred features it adds to it.

This is useful for example if you want to `--export` a mutable global
but none of the input of object were built with mutable global support.
In that case you can do `--extra-features=mutable-globals` to avoid the
linker error that would otherwise be generated in this case:

wasm-ld: error: mutable global exported but 'mutable-globals' feature not present in inputs: `__stack_pointer`. Use --no-check-features to suppress.

Differential Revision: https://reviews.llvm.org/D135831
2022-10-13 09:25:02 -07:00