348 Commits

Author SHA1 Message Date
Sam Clegg
39d32b238d
[WebAssembly] Use 64-bit table when targeting wasm64 (#92042)
See https://github.com/WebAssembly/memory64/issues/51
2024-05-23 18:25:58 -07:00
SingleAccretion
cb4f94db83
[lld][WebAssembly] Add --no-growable-memory (#82890)
We recently added `--initial-heap` - an option that allows one to up the
initial memory size without the burden of having to know exactly how
much is needed.

However, in the process of implementing support for this in Emscripten
(https://github.com/emscripten-core/emscripten/pull/21071), we have
realized that `--initial-heap` cannot support the use-case of
non-growable memories by itself, since with it we don't know what to set
`--max-memory` to.

We have thus agreed to move the above work forward by introducing
another option to the linker (see
https://github.com/emscripten-core/emscripten/pull/21071#discussion_r1491755616),
one that would allow users to explicitly specify they want a
non-growable memory.

This change does this by introducing `--no-growable-memory`: an option
that is mutally exclusive with `--max-memory` (for simplicity - we can
also decide that it should override or be overridable by `--max-memory`.
In Emscripten a similar mix of options results in `--no-growable-memory`
taking precedence). The option specifies that the maximum memory size
should be set to the initial memory size, effectively disallowing memory
growth.

Closes #81932.
2024-02-25 08:43:11 -08:00
Sam Clegg
3c5845703c
[lld][WebAssembly] Move input vectors from symtab to ctx. NFC (#78640)
Also convert from std::vector to SmallVector.

This matches the ELF linker where these were moved into the ctx object
in 9a572164d592e and converted to SmallVector in ba948c5a9c524b.
2024-01-18 15:53:13 -08:00
Sam Clegg
184c22dd3a
[lld][WebAssembly] Move linker global state in to context object. NFC (#78629)
See lld/ELF/Config.h
2024-01-18 15:01:21 -08:00
SingleAccretion
b2cdf3cc4c
[lld][WebAssembly] Add an --initial-heap option (#75594)
It is beneficial to preallocate a certain number of pages in the linear
memory (i. e. use the "minimum" field of WASM memories) so that fewer
"memory.grow"s are needed at startup.

So far, the way to do that has been to pass the "--initial-memory"
option to the linker. It works, but has the very significant downside of
requiring the user to know the size of static data beforehand, as it
must not exceed the number of bytes passed-in as "--initial-memory".

The new "--initial-heap" option avoids this downside by simply appending
the specified number of pages to static data (and stack), regardless of
how large they already are.

Ref: https://github.com/emscripten-core/emscripten/issues/20888.
2023-12-15 10:16:38 -08:00
Sam Clegg
93adcb770b [lld][WebAssembly] Add --table-base setting
This is similar to `--global-base` but determines where to place the
table segments rather than that data segments.

See https://github.com/emscripten-core/emscripten/issues/20097

Differential Revision: https://reviews.llvm.org/D158892
2023-08-25 16:10:38 -07:00
Sam Clegg
cb5bc75680 [lld][WebAssembly] Always include bss segments when --emit-relocs is used
If we don't do this then we end up with symbols that refer to
non-existent segments.

Differential Revision: https://reviews.llvm.org/D158025
2023-08-15 17:03:40 -07:00
Fangrui Song
8db61ed29d [WebAssembly] Stabilize custom section order
It currently depends on the StringMap iteration order, which is not
guaranteed to be deterministic. Use MapVector to stabilize the order.
2023-07-22 09:37:22 -07:00
Fangrui Song
a3622ac80b [wasm-ld] Switch to xxh3_64bits
Similar to recent changes to ELF (e.g., D154813), Mach-O, and COFF to
improve hashing performance.

Reviewed By: dschuff

Differential Revision: https://reviews.llvm.org/D155752
2023-07-20 10:47:47 -07:00
Fangrui Song
8d85c96e0e [lld] StringRef::{starts,ends}with => {starts,ends}_with. NFC
The latter form is now preferred to be similar to C++20 starts_with.
This replacement also removes one function call when startswith is not inlined.
2023-06-05 14:36:19 -07:00
Kazu Hirata
d5a50b02e6 Fix warnings
This patch fixes:

  mlir/lib/Dialect/Linalg/TransformOps/LinalgTransformOps.cpp:1334:51:
  warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]

  lld/wasm/Writer.cpp:250:39: warning: suggest parentheses around ‘&&’
  within ‘||’ [-Wparentheses]
2023-04-10 10:11:41 -07:00
Sam Clegg
3111784ff7 [lld][WebAssembly] Initial support for stub libraries
See the docs in lld/docs/WebAssembly.rst for more on this.

This feature unlocks a lot of simplification in the emscripten toolchain
since we can represent the JS libraries to wasm-ld as stub libraries.

See https://github.com/emscripten-core/emscripten/issues/18875

Differential Revision: https://reviews.llvm.org/D145308
2023-03-23 14:26:27 -07:00
Heejin Ahn
4e844a1498 [WebAssembly] Replace Bugzilla links with Github issues
Reviewed By: dschuff, asb

Differential Revision: https://reviews.llvm.org/D145966
2023-03-17 20:13:00 -07:00
Sam Clegg
d32f71a91a [lld][WebAssembly] Use C++17 nested namespace syntax in most places. NFC
Like D131405, but for wasm-ld.

Differential Revision: https://reviews.llvm.org/D145399
2023-03-09 18:37:32 -08:00
Derek Schuff
c7af9ae577 Reapply: [WebAssembly] Implement build-id feature
Implement the --build-id flag similarly to ELF, and generate a
build_id section according to the WebAssembly tool convention
specified in https://github.com/WebAssembly/tool-conventions/pull/183

The default style ("fast" aka "tree") hashes the contents of the
output and (unlike ELF) generates a v5 UUID based on the hash (using a
random namespace). It also supports generating a random v4 UUID, a
sha1 hash, and a user-specified string (as ELF does).

Differential Revision: https://reviews.llvm.org/D107662

Fix MSVC build by std::copy on the underying buffer rather than
directly from std::array to llvm::MutableArrayRef
2023-03-03 17:15:55 -08:00
Derek Schuff
c8e5354f00 Revert "[WebAssembly] Implement build-id feature"
This reverts commit 41e31466af6a7feab82742bb01af43f4f3ae4ede
due to a build failure on Windows.
2023-03-02 15:28:56 -08:00
Derek Schuff
41e31466af [WebAssembly] Implement build-id feature
Implement the --build-id flag similarly to ELF, and generate a build_id
section according to the WebAssembly tool convention specified in
https://github.com/WebAssembly/tool-conventions/pull/183

The default style ("fast" aka "tree") hashes the contents of the output
and (unlike ELF) generates a v5 UUID based on the hash (using a random
namespace).
It also supports generating a random v4 UUID, a sha1 hash,
and a user-specified string (as ELF does).

Differential Revision: https://reviews.llvm.org/D107662
2023-03-02 14:19:06 -08:00
Sam Clegg
9369b7d307 [lld][WebAssembly] Limit size of shared 64-bit memories of 2^^34
This is current limit in v8.  See
https://github.com/WebAssembly/memory64/issues/33 how we might change
this in the future.

Differential Revision: https://reviews.llvm.org/D143783
2023-02-13 12:21:32 -08:00
Sam Clegg
a0495e6b00 [lld][WebAssembly] Apply relocations to TLS data
This is only needed in the case of dynamic linking and pthreads.
Previously these relocations were simply not being applied.

Verified that this works using a more real world emscripten test:
https://github.com/emscripten-core/emscripten/pull/18641

Differential Revision: https://reviews.llvm.org/D143020
2023-01-31 15:50:44 -08:00
Sam Clegg
ff8e0ed930 [lld][WebAssembly] Fix memory.fill argument in 64-bit mode
This only effects folks building with wasm64 + shared memory which
is not currently a supported configuration in emscripten or any other
wasm toolchain.

Differential Revision: https://reviews.llvm.org/D141005
2023-01-10 03:11:05 -08:00
Kazu Hirata
b9ef5648b5 [lld] Use std::optional instead of llvm::Optional (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-02 18:29:04 -08:00
Guillaume Chatelet
1565317287 Revert D139181 "[lld][Alignment][NFC] Use Align instead of log2 of alignment in Wasm Sections"
As discussed on the patch the Align type is probably not a good fit for
linkers.
This reverts commit cfe77f23d6f190d54763a7575cee95aceb9216bc.
2022-12-20 10:58:56 +00:00
Fangrui Song
d88e8dcc72 [lld] llvm::Optional::value => operator*/operator->
std::optional::value() has undesired exception checking semantics and is
unavailable in some older Xcode. The call sites block std::optional migration.
2022-12-17 05:03:57 +00:00
Guillaume Chatelet
cfe77f23d6 [lld][Alignment][NFC] Use Align instead of log2 of alignment in Wasm Sections
I intend to slowly upgrade all alignments to the Align type in lld as well.
Some places talk about alignment in Bytes while other specify them as Log2(Bytes).
Let's make sure all of this is coherent.

Differential Revision: https://reviews.llvm.org/D139181
2022-12-02 12:47:41 +00:00
Fangrui Song
640d9b3296 [lld] Fix duplicate word typos. NFC
Based on lld/ part of D137338 but reflowed comments.
2022-11-08 17:28:04 -08:00
Florian Mayer
2390bb2347 [NFC] use has_value instead of hasValue 2022-10-31 16:43:14 -07:00
Dan Gohman
d4c8a0edca [wasm-ld] Allow importing/exporting the output module's memory with arbitrary names
This adds an `--export-memory` option to wasm-ld which allows passing
a name to give to the exported memory, and extends `--import-memory` to
allow passing a <module>,<name> pair specifying where the memory should
be imported from.

This is based on https://reviews.llvm.org/D131376, with the main
difference being that it only supports exporting memory by one name.

Differential Revision: https://reviews.llvm.org/D135898
2022-10-31 13:59:46 -07:00
Sam Clegg
6912ed7b8f [lld][WebAssembly] Don't allow --global-base to be specified in -share/-pie or --relocatable modes
Add some checks around this combination of flags

Also, honor `--global-base` when specified in `--stack-first` mode
rather than ignoring it.  But error out if the specified base preseeds
the end of the stack.

Differential Revision: https://reviews.llvm.org/D136117
2022-10-18 17:19:26 -07:00
Dan Gohman
4b24e9be17 [wasm-ld] Define a __heap_end symbol marking the end of allocated memory.
Define a `__heap_end` symbol that marks the end of the memory region
that starts at `__heap_base`. This will allow malloc implementations to
know how much memory they can use at `__heap_base` even if someone has
done a `memory.grow` before they can initialize their state.

Differential Revision: https://reviews.llvm.org/D136110
2022-10-17 16:39:06 -07:00
Sam Clegg
1532be98f9 [lld][WebAssembly] Add symbols marking start/end of stack region
Currently emscripten is make assumptions about that memory layout,
assuming the stack is between `__data_end` and `__heap_base`:

af961ad5c4/system/lib/compiler-rt/stack_limits.S (L42-L61)

With this change we can be more precise:

https://github.com/emscripten-core/emscripten/pull/18057

Differential Revision: https://reviews.llvm.org/D135910
2022-10-13 14:52:17 -07:00
Sam Clegg
c07e838130 [lld][WebAssembly] Add --extra-features flag to add addional features
This flag acts just like the existing `--features` flag but instead
of replacing the set of inferred features it adds to it.

This is useful for example if you want to `--export` a mutable global
but none of the input of object were built with mutable global support.
In that case you can do `--extra-features=mutable-globals` to avoid the
linker error that would otherwise be generated in this case:

wasm-ld: error: mutable global exported but 'mutable-globals' feature not present in inputs: `__stack_pointer`. Use --no-check-features to suppress.

Differential Revision: https://reviews.llvm.org/D135831
2022-10-13 09:25:02 -07:00
Nico Weber
cd7ffa2e52 lld: Include name of output file in "failed to write output" diag
Differential Revision: https://reviews.llvm.org/D133110
2022-09-14 14:57:47 -04:00
Sam Clegg
113b568829 [lld][WebAssembly] Rename SymbolTable::getSymbols to match ELF backend. NFC
The ELF backend originally used `getSymbols()` but went though a
sequence of changes that resulted in this method being called
`symbols()`.

d8f8abbd4a2823f223bd7bc56445541fb221b512 replaced `getSymbols()` with
`forEachSymbol`.

a2fc96441788fba1e4709d63677f34ed8e321dae replaced `forEachSymbol` with
`llvm::iterator_range`.

e9262edf0d11a907763098d8e101219ccd9c43e9 replaced `llvm::iterator_range`
with `symbols()`.

Differential Revision: https://reviews.llvm.org/D131284
2022-08-31 14:33:45 -07:00
Kazu Hirata
5cff5142a8 Use value instead of getValue (NFC) 2022-07-15 20:03:13 -07:00
Kazu Hirata
e5f568a49f Use has_value instead of hasValue (NFC) 2022-07-13 01:58:03 -07:00
Sam Clegg
53217ecb88 [lld][WebAssembly] Don't apply data relocations at static constructor time
Instead, export `__wasm_apply_data_relocs` and `__wasm_call_ctors`
separately.

This is required since user code in a shared library (such as static
constructors) should not be run until relocations have been applied to
all loaded libraries.

See: https://github.com/emscripten-core/emscripten/issues/17295

Differential Revision: https://reviews.llvm.org/D128515
2022-06-27 15:50:02 -07:00
Kazu Hirata
3b7c3a654c Revert "Don't use Optional::hasValue (NFC)"
This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.
2022-06-25 11:56:50 -07:00
Kazu Hirata
aa8feeefd3 Don't use Optional::hasValue (NFC) 2022-06-25 11:55:57 -07:00
Kazu Hirata
064a08cd95 Don't use Optional::hasValue (NFC) 2022-06-20 20:05:16 -07:00
Kazu Hirata
5413bf1bac Don't use Optional::hasValue (NFC) 2022-06-20 11:33:56 -07:00
Nico Weber
8c589939f5 fix comment typos to cycle bots 2022-06-19 18:34:12 -04:00
Kazu Hirata
757d9d22cd [lld] Use value_or instead of getValueOr (NFC) 2022-06-19 00:29:41 -07:00
Sam Clegg
457f38a7b0 [lld][WebAssembly] Revert moving of data relocations to start function
Back in https://reviews.llvm.org/D117412 we moved the application of
data reloctions to the wasm start function.

However, because the dynamic linker doesn't know the final addresses
at module instantiation time, this proved to be too early and the
relocations could be applied with the wrong values.

Fixes: https://github.com/emscripten-core/emscripten/issues/17150

Differential Revision: https://reviews.llvm.org/D127333
2022-06-09 17:49:35 -07:00
Sam Clegg
87099a0438 [lld][WebAssembly] Remove unnecessary accessor methods. NFC
This is less code, and matches more closely the ELF linker.

Differential Revision: https://reviews.llvm.org/D126979
2022-06-03 11:43:44 -07:00
Sam Clegg
198815e18d [lld][WebAssembly] Avoid importing/exporting hidden symbols in shared libraries
We have some special handling for weakly defined symbols where we both
import and export them, but this is not needed for hidden symbols which
should never be imported or exported.

See https://github.com/emscripten-core/emscripten/pull/16972

This should also help with:
https://github.com/emscripten-core/emscripten/issues/15487

Differential Revision: https://reviews.llvm.org/D126491
2022-05-26 13:43:25 -07:00
Sam Clegg
74f9841977 [lld][WebAssembly] Allow use of statically allocated TLS region.
It turns out we were already allocating static address space for TLS
data along with the non-TLS static data, but this space was going
unused/ignored.

With this change, we include the TLS segment in `__wasm_init_memory`
(which does the work of loading the passive segments into memory when a
module is first loaded).  We also set the `__tls_base` global to point
to the start of this segment.

This means that the runtime can use this static copy of the TLS data for
the first/primary thread if it chooses, rather than doing a runtime
allocation prior to calling `__wasm_init_tls`.

Practically speaking, this will allow emscripten to avoid dynamic
allocation of TLS region on the main thread.

Differential Revision: https://reviews.llvm.org/D126107
2022-05-23 17:27:17 -07:00
Sam Clegg
a04a507714 [lld][WebAssembly] Fix crash accessing non-live __tls_base symbol
In programs that don't otherwise depend on `__tls_base` it won't
be marked as live.  However this symbol is used internally in
a couple of places do we need to mark it as live explictily in
those places.

Fixes: #54386

Differential Revision: https://reviews.llvm.org/D121931
2022-03-17 13:59:45 -07:00
Sam Clegg
4690bf2ed3 [lld][WebAssembly] Take advantage of extended const expressions when available
In particular we use these in two places:

1. When building PIC code we no longer need to combine output segments
   into a single segment that can be initialized at `__memory_base`.
   Instead each segment can encode its offset from `__memory_base` in
   its initializer.  e.g.

```
(i32.add (global.get __memory_base) (i32.const offset)
```

2. When building PIC code we no longer need to relocation internalized
   global addresses.  We can just initialize them with their correct
   offsets.

Differential Revision: https://reviews.llvm.org/D121420
2022-03-15 17:50:05 -07:00
Sam Clegg
86c90f9bfd [lld][WebAssembly] Add --unresolved-symbols=import-dynamic
This is a new mode for handling unresolved symbols that allows all
symbols to be imported in the same that they would be in the case of
`-fpie` or `-shared`, but generting an otherwise fixed/non-relocatable
binary.

Code linked in this way should still be compiled with `-fPIC` so that
data symbols can be resolved via imports.

This essentially allows the building of static binaries that have
dynamic imports.  See:
https://github.com/emscripten-core/emscripten/issues/12682

As with other uses of the experimental dynamic linking ABI, this
behaviour will produce a warning unless run with `--experimental-pic`.

Differential Revision: https://reviews.llvm.org/D91577
2022-03-15 15:10:21 -07:00
Sam Clegg
dabbab6861 [lld][WebAssembly] Apply global relocs before data relocs
Since the code for apply data relocations can sometimes use
the values stored in he globals, they need to be relocated
before the data relocations can be run.

Fixes: https://github.com/emscripten-core/emscripten/issues/13398

Differential Revision: https://reviews.llvm.org/D119666
2022-02-16 14:30:39 -08:00