Fangrui Song da68d2164e [ELF] Cap parallel::strategy to 16 threads when --threads= is unspecified
When --threads= is unspecified, we set it to
`parallel::strategy.compute_thread_count()`, which uses
sched_getaffinity (Linux)/cpuset_getaffinity (FreeBSD)/std:🧵:hardware_concurrency (others).
With extensive testing on many machines (many configurations from
{aarch64,x86-64} x {Linux,FreeBSD,Windows} x allocators(native,mimalloc,rpmalloc) combinations)
with varying workloads, we discovered that when the concurrency is larger than
16, the linking process is slower than using --threads=16 due to parallelism
overhead outweighs optimizations. This is particularly harmful for machines with
many cores or when the link job competes with other jobs.

Cap parallel::strategy when --threads= is unspecified.
For some workloads changing the concurrency from 8 to 16 has nearly no improvement.

--thinlto-jobs= is unchanged since ThinLTO backend compiles are embarrassingly
parallel.

Link: https://discourse.llvm.org/t/avoidable-overhead-from-threading-by-default/69160

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D147493
2023-04-12 13:13:38 -07:00
..
2023-02-15 17:34:35 +00:00
2023-04-10 10:11:41 -07:00

LLVM Linker (lld)

This directory and its subdirectories contain source code for the LLVM Linker, a modular cross platform linker which is built as part of the LLVM compiler infrastructure project.

lld is open source software. You may freely distribute it under the terms of the license agreement found in LICENSE.txt.

Benchmarking

In order to make sure various developers can evaluate patches over the same tests, we create a collection of self contained programs.

It is hosted at https://s3-us-west-2.amazonaws.com/linker-tests/lld-speed-test.tar.xz

The current sha256 is 10eec685463d5a8bbf08d77f4ca96282161d396c65bd97dc99dbde644a31610f.