16145 Commits

Author SHA1 Message Date
Michał Górny
23c665371a [compiler-rt] Switch from llvm-config to find_package(LLVM)
Replace the use of the deprecated `llvm-config` tool with LLVM's CMake
files for detecting LLVM in standalone builds.

Differential Revision: https://reviews.llvm.org/D137024
2022-11-02 07:17:42 +01:00
Vitaly Buka
e02110e2ab Revert "[Sanitizers] Modified __aarch64__ to use the 64 bit version of the allocator."
We need to land asan_allocator.h patches first.

This reverts commit 75a8cdbc0659dc20746f31721cc48ef00c49f746.
2022-11-01 17:04:10 -07:00
Kirill Stoimenov
75a8cdbc06 [Sanitizers] Modified __aarch64__ to use the 64 bit version of the allocator.
This change will switch SizeClassAllocator32 to SizeClassAllocator64 on ARM. This might potentially affect ARM platforms with 39-bit address space. This addresses [[ https://github.com/google/sanitizers/issues/703  | issues/703  ]], but unlike [[ https://reviews.llvm.org/D60243 | D60243 ]] it defaults to 64 bit allocator.

Reviewed By: vitalybuka, MaskRay

Differential Revision: https://reviews.llvm.org/D137136
2022-11-01 20:48:42 +00:00
Roy Sundahl
37a25b48e1 [asan] Add missing __asan_set_shadow_0x() calls.
Add new calls introduced in https://reviews.llvm.org/D136197 to weak_symbols.txt.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D137178
2022-11-01 11:15:33 -07:00
Youling Tang
cb84df210a [sanitizer] Fix build error with current LoongArch Clang
Fix the following build failures:
```
In file included from /home/loongson/llvm-work/llvm-project/compiler-rt/lib/sanitizer_common/sanit>
llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_syscall_linux_loongarch64.inc:27:23: error>
  register u64 a7 asm("a7") = nr;
                      ^
llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_syscall_linux_loongarch64.inc:28:23: error>
  register u64 a0 asm("a0");                      ^
```

The non-prefix `$` syntax in inline assembly is not supported in clang
yet (it is supported by gcc), add prefix `$` to solve the problem.

Differential Revision: https://reviews.llvm.org/D137145
2022-11-01 20:48:53 +08:00
Youling Tang
867ff7c994 [asan][test] Fix build errors in loongarch64
- Add get_bits_for_arch for loongarch64 detection, fix the following error,
```
$ make check-asan
-- sanitizer_common tests on "Linux" will run against "asan"
CMake Error at test/asan/CMakeLists.txt:22 (message):
  Unknown target architecture: loongarch64
Call Stack (most recent call first):
  test/asan/CMakeLists.txt:49 (get_bits_for_arch)
```

- Do not use __builtin_longjmp, fix the following error,
```
llvm-project/compiler-rt/lib/asan/tests/asan_test.cpp:635:3: error: __builtin_longjmp is not supported for the current target
  __builtin_longjmp((void**)buf, 1);
```

Differential Revision: https://reviews.llvm.org/D137012
2022-11-01 20:21:56 +08:00
Youling Tang
5b27578fc5 [builtins][LoongArch] Port __clear_cache to LoongArch Linux
There are two failures in the current builtins,
Failed Tests (2):
  Builtins-loongarch64-linux :: clear_cache_test.c
  Builtins-loongarch64-linux :: enable_execute_stack_test.c

It is caused by __clear_cache not being implemented and triggering `abort`.

"__clear_cache" is implemented in the same way as "clear_cache" in gcc (
in gcc/config/loongarch/loongarch.md)

Differential Revision: https://reviews.llvm.org/D136921
2022-11-01 20:15:09 +08:00
Youling Tang
6e6704b0dc [compiler-rt][builtins] Support builtins for LoongArch
Initial builtins for LoongArch.
Add loongarch64 to ALL_CRT_SUPPORTED_ARCH list.
Support fe_getround and fe_raise_inexact in builtins.

Differential Revision: https://reviews.llvm.org/D136338
2022-11-01 20:09:32 +08:00
Michał Górny
a05810e8f2 Revert "[compiler-rt] Switch from llvm-config to find_package(LLVM)"
This reverts commit 37acf9bdd4686397a6d37000e8d3fe35f456a860.  It broke
a buildbot.
2022-11-01 05:49:50 +01:00
Michał Górny
37acf9bdd4 [compiler-rt] Switch from llvm-config to find_package(LLVM)
Replace the use of the deprecated `llvm-config` tool with LLVM's CMake
files for detecting LLVM in standalone builds.

Differential Revision: https://reviews.llvm.org/D137024
2022-11-01 04:19:31 +01:00
Patrick Walton
c5ea9b8a51 [test][asan] Make the printf-5.c test case actually emit a volatile memcpy.
The current test in printf-5.c appears to try to emit a volatile memcpy for the
format string, but it doesn't because the volatile qualifier is implicitly
casted away. Using a string literal instead preserves the volatile qualifier.

This is a follow-up to D137031 and is a prerequisite for D136822, which elides
memcpys in more instances and would otherwise break this test.

Differential Revision: https://reviews.llvm.org/D137042
2022-10-30 13:20:36 -07:00
Chia-hung Duan
9dc5b322d5 Revert "Revert "[scudo] Fix the calculating of memory group usage""
This reverts commit 69fe7abb393ba7d6ee9c8ff1429316845b5bad37.

Fixed the arguments order while calling batchGroupBase()

Differential Revision: https://reviews.llvm.org/D136995
2022-10-29 06:26:50 +00:00
Chia-hung Duan
69fe7abb39 Revert "[scudo] Fix the calculating of memory group usage"
This reverts commit 6130d67f70ad2e063b4407b6ee31c7db19464fee.
2022-10-28 22:15:50 +00:00
Chia-hung Duan
6130d67f70 [scudo] Fix the calculating of memory group usage
In SizeClassAllocator64, the boundary of a memory group may not align to
the region begin. Which means the begin addr of a memory group may
smaller than region begin. This leads to wrong judgement of memory group
usage.

Differential Revision: https://reviews.llvm.org/D136898
2022-10-28 20:29:17 +00:00
Chia-hung Duan
a1e325ce7c [scudo] Lazy initialize the PageMap while page releasing
We allocate the page map before knowing if there're groups can be
released. This may result in many redundant map()/unmap() operations if
there's no page to release.

Make the page map be lazy initialized.

Differential Revision: https://reviews.llvm.org/D136873
2022-10-28 20:29:17 +00:00
Advenam Tacet
dd1b7b797a [1b/3][ASan][compiler-rt] API for annotating objects memory
This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in std::vector, to std::string and std::deque collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for std::deque, or between the size and capacity bounds for std::string).

The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took iter1_begin, iter1_end, iter2_begin iterators (with a custom comparison function). When object iter1 was longer than iter2, read out-of-bounds on iter2 could happen. Container sanitization would detect it.

This revision extends a compiler-rt ASan sanitization API function sanitizer_annotate_contiguous_container used to sanitize/annotate containers like std::vector to support different allocators and situations when granules are shared between objects. Those changes are necessary to support annotating objects' self memory (in contrast to annotating memory allocated by an object) like short std::basic_string (with short string optimization). That also allows use of non-standard memory allocators, as alignment requirement is no longer necessary.

This also updates an API function to verify if a double ended contiguous container is correctly annotated (__sanitizer_verify_contiguous_container).

If you have any questions, please email:
advenam.tacet@trailofbits.com
disconnect3d@trailofbits.com

Reviewed By: #sanitizers, vitalybuka

Differential Revision: https://reviews.llvm.org/D132522
2022-10-27 23:29:43 -07:00
Wael Yehia
c8eb932ce6 [PGO] Simplify InstrProfilingRuntime.cpp
Differential Revision: https://reviews.llvm.org/D136192
2022-10-28 05:12:46 +00:00
Chia-hung Duan
4383aa0336 [scudo] Fix size of MaxNumCachedHint Trusty Config
Also add a test for TrustyConfig to ensure it'll be awared of any scudo
configuration change.

Differential Revision: https://reviews.llvm.org/D136732
2022-10-26 18:07:22 +00:00
Alex Brachet
443e2a10f6 Reland "[PGO] Make emitted symbols hidden"
This was reverted because it was breaking when targeting Darwin which
tried to export these symbols which are now hidden. It should be safe
to just stop attempting to export these symbols in the clang driver,
though Apple folks will need to change their TAPI allow list described
in the commit where these symbols were originally exported
f538018562

Then reverted again because it broke tests on MacOS, they should be
fixed now.

Bug: https://github.com/llvm/llvm-project/issues/58265

Differential Revision: https://reviews.llvm.org/D135340
2022-10-26 17:13:05 +00:00
Wael Yehia
aceb67eeb5 Add a testcase for D136192.
Differential Revision: https://reviews.llvm.org/D136192
2022-10-26 14:41:06 +00:00
YunQiang Su
76b92df0d6 [sanitizer] Update struct_kernel_stat_sz for mips32 and mipsn32
On mips32 _LARGEFILE_SOURCE and _FILE_OFFSET_BITS == 64 is used (compiler-rt/cmake/base-config-ix.cmake),
thus the correct struct_kernel_stat_sz should be 160 instead of 144.

This value is also updated for N32, since we will use
_LARGEFILE_SOURCE and _FILE_OFFSET_BITS == 64 in future.

Fix https://github.com/llvm/llvm-project/issues/55499

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D135553
2022-10-25 19:00:38 -07:00
Nico Weber
76745d2b58 Revert "[PGO] Make emitted symbols hidden"
This reverts commit 04877284b4592e9286cab43467662c1b4ff81861.
Looks like this is still breaking the test
Profile-x86_64 :: instrprof-darwin-dead-strip.c
(see comment on https://reviews.llvm.org/D135340).
2022-10-25 08:54:47 -04:00
Enna1
345b0587a3 [memprof] Respect COMPILER_RT_BUILD_MEMPROF when install memprof headers
When COMPILER_RT_BUILD_MEMPROF is disabled, the memprof headers should not be installed.

Reviewed By: mgorny, tejohnson

Differential Revision: https://reviews.llvm.org/D136550
2022-10-25 09:35:22 +08:00
zijunzhao
1d31ea68c0 Fix LazyInitialization in tsan
In Android, further initialization is always necessary whether preinit_array can be used.
LazyInitialize is needed regardless of .preinit_array support on platforms where runtime is loaded as dynamic library, e.g. Android.

Reviewed By: dvyukov, vitalybuka

Differential Revision: https://reviews.llvm.org/D135925
2022-10-25 00:59:03 +00:00
Roy Sundahl
d39486ddd5 [ASAN] Remove asserts introduced in https://reviews.llvm.org/D136197
Additional calls were introduced for outlining (opposite of inlining)
in https://reviews.llvm.org/D136197 which contain asserts that partial
poisoning of a single byte wouldn't happen consecutively but this is
too strong and actually does occur in Windows. Removing those asserts
as they are unnecessary

Differential Revision: https://reviews.llvm.org/D136645
2022-10-24 17:33:40 -07:00
Roy Sundahl
0c35b6165c [ASAN] Don't inline when -asan-max-inline-poisoning-size=0
When -asan-max-inline-poisoning-size=0, all shadow memory access should be
outlined (through asan calls). This was not occuring when partial poisoning
was required on the right side of a variable's redzone. This diff contains
the changes necessary to implement and utilize  __asan_set_shadow_01() through
__asan_set_shadow_07(). The change is necessary for the full abstraction of
the asan implementation and will enable experimentation with alternate strategies.

Differential Revision: https://reviews.llvm.org/D136197
2022-10-24 14:17:59 -07:00
Alex Brachet
04877284b4 [PGO] Make emitted symbols hidden
This was reverted because it was breaking when targeting Darwin which
tried to export these symbols which are now hidden. It should be safe
to just stop attempting to export these symbols in the clang driver,
though Apple folks will need to change their TAPI allow list described
in the commit where these symbols were originally exported
f538018562

Bug: https://github.com/llvm/llvm-project/issues/58265

Differential Revision: https://reviews.llvm.org/D135340
2022-10-24 19:05:10 +00:00
Kazu Hirata
3f8d2c917c Ensure newlines at the end of files (NFC) 2022-10-22 09:29:40 -07:00
Chia-hung Duan
b525392a17 Revert "Revert "[scudo] Support partial page releasing""
This reverts commit 1cf1b36112b3d475d443909881b02799b548e6b8.

Reviewed By: cferris

Differential Revision: https://reviews.llvm.org/D136027
2022-10-21 22:29:34 +00:00
Chia-hung Duan
c0f91856a3 Reland "[scudo] Manage free blocks in BatchGroup."
This is not a pure revert of c929bcb7d85700494217f3a2148549f8757e0eed.
It also includes a bug fix.

Differential Revision: https://reviews.llvm.org/D136029
2022-10-21 19:57:28 +00:00
Wael Yehia
461a1836d3 [PGO][AIX] Improve dummy var retention and allow -bcdtors:csect linking.
1) Use a static array of pointer to retain the dummy vars.
2) Associate liveness of the array with that of the runtime hook variable
   __llvm_profile_runtime.
3) Perform the runtime initialization through the runtime hook variable.
4) Preserve the runtime hook variable using the -u linker flag.

Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D136192
2022-10-21 16:32:42 +00:00
Enna1
30f6f2c09d [memprof] Support installation of memprof headers
This change allows users manually calling memprof public C API (e.g. __memprof_profile_dump).

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D136067
2022-10-20 20:31:45 +08:00
Vitaly Buka
859b614837 [docs] Update compiler-rt/CODE_OWNERS.TXT
Reviewed By: compnerd, kcc, lhames, phosek, tejohnson, dvyukov, MaskRay

Differential Revision: https://reviews.llvm.org/D135617
2022-10-19 17:28:48 -07:00
Arthur Eubanks
97196a2d92 [test][asan][Darwin] Pass -mlinker-version=133 to linker invocation in odr-lto.cpp
When building clang with lld, we don't get a default mlinker-version [1]. This causes us to not pass -lto_library to ld64 [2].
Explicitly pass -mlinker-version=133 so we properly pass -lto_library to ld64 and don't get LLVM bitcode version mismatches due to LTO.

[1] 55ae180a4c/clang/CMakeLists.txt (L345)
[2] 55ae180a4c/clang/lib/Driver/ToolChains/Darwin.cpp (L262-L270)
2022-10-19 16:35:43 -07:00
Florian Mayer
b96cbbd901 [sanitizer] Let internal symbolizer use toupper and tolower 2022-10-18 16:19:11 -07:00
Arthur Eubanks
aa89ceab48 [win][compiler-rt] Make tests use lld-link instead of link
Git bash ships with a link.exe. We try to add git bash to the beginning
of PATH (see D84380). These tests end up executing the wrong link.exe.

As a workaround, use lld-link. Note that `REQUIRES: lld-available` tests currently aren't running, see D128567. I did manually verify that these tests pass with lld-link.

Reviewed By: rnk, hans

Differential Revision: https://reviews.llvm.org/D136108
2022-10-18 08:46:23 -07:00
Arthur Eubanks
db98c06f11 [ubsan][test] Make some tests have shorter file names
We're hitting path size limits on Windows on these tests. As a
workaround, make the file names shorter.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D136113
2022-10-18 08:45:27 -07:00
YunQiang Su
9221aa6d64 [CMake] Fix MIPSr6 build for compiler-rt
The current version pass -mips64r2 or -mips32r2 options,
which make it failed to build on r6 platform.

In this patch: we detect whether we are MIPSr6 by
    _MIPS_ARCH_MIPS32R6/_MIPS_ARCH_MIPS64R6
The out and install path is set to the default triple instead of
hardcoded one, since the clang ask for it.

Differential Revision: https://reviews.llvm.org/D135735
2022-10-18 04:49:25 +00:00
Chia-hung Duan
5de73d27bd [scudo] Change region size from 1 MB to 2 MB in tests
In SizeClassAllocator64, the RegionBeg is determined by RegionBase +
random offset. The offset is n pages, where n is a random number less or
equal to 16. However, on certain platforms which have large page size,
it may end up immediately OOM without mapping any block pages. For
example,

PageSize = 64 KB, RegionSize = 1 MB

Suppose the random number n is 16, then the random offset will be
64 * 16 = 1024 KB which is equal to the RegionSize.

On most platforms we don't have such large page size and we have
different PRNG(pseudo random number generator) behaviors, thus we didn't
hit any failures before. Given that this now only affects the tests,
only increase the region size is enough.

Will revisit the logic of calculating the random offset.

Differential Revision: https://reviews.llvm.org/D136025
2022-10-17 20:22:33 +00:00
David Spickett
d231efe1ee [compiler-rt] Relax pthread_getaffinity test to account for cgroups/docker
Fixes #58283

When running in a docker container you can have fewer cores assigned
to you than get_nrpoc would suggest.

Since the test just wants to know that interception worked, allow
any result > 0 and <= the global core count.

Reviewed By: MaskRay, vitalybuka

Differential Revision: https://reviews.llvm.org/D135677
2022-10-17 10:45:30 +00:00
Chia-hung Duan
3ead26e65b [scudo] Fix implicitly narrow casting (NFC)
u16 may be promoted to int by arithmetic type conversion. Do an explicit
cast to avoid certain compiler's warning. This fixes the problem
introduced by 0fb2aeef5310eaba2915b30810464a744a80da15

Differential Revision: https://reviews.llvm.org/D135985
2022-10-15 03:54:53 +00:00
Kamau Bridgeman
c929bcb7d8 Revert "[scudo] Manage free blocks in BatchGroup."
This reverts commit cf9d7f55d3bec7640fa8b2f8ec1d9c1268233caa.
2022-10-14 15:11:57 -05:00
Kamau Bridgeman
1cf1b36112 Revert "[scudo] Support partial page releasing"
This reverts commit 9c26f51f5e178ac0fda98419e3a61d205d3b58b1.
2022-10-14 15:11:44 -05:00
Kamau Bridgeman
5fe6f3e0bc Revert "[scudo] Fix implicitly narrow casting (NFC)"
This reverts commit fd7c7ad4fe0138314b922ea0db1691d5a679cc75.
2022-10-14 15:11:34 -05:00
Chia-hung Duan
fd7c7ad4fe [scudo] Fix implicitly narrow casting (NFC)
u16 may be promoted to int by arithmetic type conversion. Do an explicit
cast to avoid certain compiler's warning.

Differential Revision: https://reviews.llvm.org/D135945
2022-10-14 18:59:08 +00:00
Chia-hung Duan
9c26f51f5e [scudo] Support partial page releasing
Block grouping enables us doing partial page releasing so that we can
release the pages in a finer granularity. Which means we don't need to
visit all blocks to determine which pages are unused. Besides, this
means we can do incremental page releasing depends on the number fo free
blocks.

Reviewed By: cryptoad, cferris

Differential Revision: https://reviews.llvm.org/D134226
2022-10-13 23:35:07 +00:00
Chia-hung Duan
cf9d7f55d3 [scudo] Manage free blocks in BatchGroup.
Scudo is supposed to allocate any blocks across the entired mapped
pages and each page is equally likely to be selected. Which means Scudo
is leaning to touch as many pages as possible. This brings better
security but it also sacrifices the chance of releasing dirty pages.

To alleviate the unmanagable footprint growing, this CL introduces the
BatchGroup concept. Each blocks will be classified into a BatchGroup
according to its address. While allocation, we are leaning to allocate
blocks in the same group first. Note that the blocks selected from a
group is still random over several pages. At the same time, we have
better prediction of dirty page growing speed. Besides, we are able to
do partial page releasing by examing part of BatchGroups.

Reviewed By: cryptoad, cferris

Differential Revision: https://reviews.llvm.org/D133897
2022-10-13 23:35:06 +00:00
Chia-hung Duan
56dafd024c [scudo] Add PageReleaseContext to convey page usage status.
PageReleaseContext contains all the information needed for determing if
a page can be released. Splitting out the context increases the flexibility
of heterogenous free lists in the future. Also rename PackedCounterArray to
PageMap.

Reviewed By: cryptoad, cferris

Differential Revision: https://reviews.llvm.org/D133895
2022-10-13 23:35:06 +00:00
Chia-hung Duan
0fb2aeef53 Use u16 to store Count/MaxCount
The Count/MaxCount used in TransferBatch and PerClass can be fit in u16 in
current configurations and it's also reasonable to have a u16 limit. The
spare 16 bits will be used for additional status like pages mapping
status in a TransferBatch.

Reviewed By: cryptoad, cferris, vitalybuka

Differential Revision: https://reviews.llvm.org/D133145
2022-10-13 23:35:06 +00:00
Wiktor Garbacz
e851f7dbca Fix LSan build
This change fixes a build bug introduced in
39db491957dcf095936d81bed89c2b4edae2a1e7

Differential Revision: https://reviews.llvm.org/D135860
2022-10-13 10:44:27 -07:00