allocation ring buffer is allowed to be zero. Update the logic in the
test so that on the platform that disables it won't fail this case.
Reviewed By: fmayer
Differential Revision: https://reviews.llvm.org/D144055
This CL adds the proper thread-safety annotations for most of the
functions and variables. However, given the restriction of the current
architecture, in some cases, we may not be able to use the annotations
easily. The followings are two exceptions,
1. enable()/disable(): Many structures in scudo are enabled/disabled by
acquiring the lock in each instance. This makes those structure act
like a `lock`. We can't mark those functions with ACQUIRE()/RELEASE()
because that makes the entire allocator become another `lock`. In the
end, that implies we need to *acquire* the `allocator` before each
malloc et al. request. Therefore, adding a variable to tell the
status of those structures may be a better way to cooperate with
thread-safety annotation.
2. TSD/TSD shared/TSD exclusive: These three have simiar restrictions as
mentioned above. In addition, they don't always need to be released
if it's a thread local instance. However, thread-safety analysis
doesn't support conditional branch. Which means we can't mark the
proper annotations around the uses of TSDs. We may consider to make
it consistent and which makes the code structure simpler.
This CL is supposed to introduce the annotations with the least code
refactoring. So only trivial thread safety issues will be addressed
here. For example, lacking of acquiring certain lock before accessing
certain variables will have the ScopedLock inserted. Other than that,
they are supposed to be done in the later changes.
Reviewed By: cferris
Differential Revision: https://reviews.llvm.org/D140706
Scudo is expected to call __scudo_allocate_hook on allocations, and
__scudo_deallocate_hook on deallocations, but it's behavior is not
clear on reallocations. Currently, non-trivial reallocations call
__scudo_allocate_hook but never __scudo_deallocate_hook. We should
prefer either calling both, none, or a dedicated
hook (__scudo_reallocate_hook, for instance).
This patch implements the former, and adds a unit test to enforce
those expectations.
Reviewed By: Chia-hungDuan
Differential Revision: https://reviews.llvm.org/D141407
Because of lock contention, we temporarily disabled the printing of
regions' status when it's exhausted. Given that it's useful when the
Region OOM happens, this CL brings it back without lock contention.
Differential Revision: https://reviews.llvm.org/D141955
We didn't acquire the mutex while accessing those lock protected data,
this CL fixes it and now we don't need to disable the allocator while
reading its states.
Differential Revision: https://reviews.llvm.org/D142149
clang-cl doesn't support -dumpmachine directly, so we need to
preface it with /clang: in order to get this probing function
to work.
This is needed in order to run cmake directly on the runtimes
directory.
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D143557
I found a few Python files not using Python 3 syntax in the tree
when trying to apply reformatting to them. This was converted with
2to3 and the changes seemed sane.
Reviewed By: MaskRay, browneee
Differential Revision: https://reviews.llvm.org/D143701
Emit all constant integers produced by SanitizerBinaryMetadata as
ULEB128 to further reduce binary space used. Increasing the version is
not necessary given this change depends on (and will land) along with
the bump to v2.
To support this, the !pcsections metadata format is extended to allow
for per-section options, encoded in the first MD operator which must
always be a string and contain the section: "<section>!<options>".
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D143484
Optimize the encoding of "covered" metadata by:
1. Reducing feature mask from 4 bytes to 1 byte (needs increase once we
reach more than 8 features).
2. Only emitting UAR stack args size if it is non-zero, saving 4 bytes
in the common case.
One caveat is that the emitted metadata for function PC (offset), size,
and UAR size (if enabled) are no longer aligned to 4 bytes.
SanitizerBinaryMetadata version base is increased to 2, since the change
is backwards incompatible.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D143482
If you are missing the iOS SDK on your macOS (for example you don't have
full Xcode but just CommandLineTools) then CMake currently errors
out without a helpful message. This patch disables iOS support in
compiler-rt if the iOS SDK is not found. This can be overriden by
passing -DCOMPILER_RT_ENABLE_IOS=ON.
Reviewed By: delcypher, thetruestblue
Differential Revision: https://reviews.llvm.org/D133273
Depends on https://reviews.llvm.org/D142861.
Alternative to https://reviews.llvm.org/D137601.
xxHash is much faster than djbHash. This makes a simple Rust test case with a large constant string 10% faster to compile.
Previous attempts at changing this hash function (e.g. https://reviews.llvm.org/D97396) had to be reverted due to breaking tests that depended on iteration order.
No additional tests fail with this patch compared to `main` when running `check-all` with `-DLLVM_ENABLE_PROJECTS="all"` (on a Linux host), so I hope I found everything that needs to be changed.
Differential Revision: https://reviews.llvm.org/D142862
The fix only affects Darwin, but to write the test I had to modify
the MemoryMappingLayout class which is used by all OSes,
to allow for mocking of image header (this change should be NFC). Hence no [Darwin] in the subject
so I can get more eyes on it.
While looking for a memory gap to put the shadow area into, the sanitizer code
scans through the loaded images, and for each image it scans through its
loader command to determine the occupied memory ranges.
While doing so, if the 'segment load' (kLCSegment) loader comand is encountered, the command scanning function
returns success (true), but does not decrement the command list iterator counter.
The result is that the function is called again and again, with the iterator counter
now being too high. The command scanner keeps updating the loader command pointer,
by using the command size field.
If the loop counter is too high, the command pointer
lands into unintended area ( beyond
+sizeof(mac_header64)+header->sizeofcmds ),
and result depends on the random content found there.
The random content interpreted as loader command might contain a large integer value in the
cmdsize field - this value is added to the current loader command pointer,
which might now point to an inaccessible memory address. It can occasionally result
in a crash if it happens to run beyond the mapped memory segment.
Note that when the area after the loader command list
contains zeros or small integers only, the loop will end normally and the problem
will go unnoticed. So it happened until now since having a some big value
after the header area, falling into command size field is a pretty rare situation.
The fix makes sure that the iterator counter gets updated when the segment load (kLCSegment)
loader command is found too, and in the same code location so the updates will always go together.
Undo the changes in the sanitizer_procmaps_mac.cpp to see the test failing.
rdar://101161047
rdar://102819707
Differential Revision: https://reviews.llvm.org/D142164
After D143438 there is no point in this check as the size will never 0. Also this keeps it consistent with ASAN where there is not size check in LsanMetadata::allocated.
Reviewed By: MaskRay, vitalybuka
Differential Revision: https://reviews.llvm.org/D143442
Without this, if hardening measures like FORTIFY_SOURCE are are in
/etc/clang/*.cfg, many sanitizer tests will die before the sanitizer
can trap the problem being tested, because e.g. the _chk variants
of common functions will abort first.
This gets the number of failing tests down from 42->3 for me (and the
remaining 3 are unrelated).
See: 52ce6776cf98e993c6ec04ae54b52e1354fff917
See: 136f77805fd89cd30e69b3d1204fbf7efedd9a12
Closes: https://github.com/llvm/llvm-project/issues/60394
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D143322
The fix only affects Darwin, but to write the test I had to modify
the MemoryMappingLayout class which is used by all OSes,
to allow for mocking of image header (this change should be NFC). Hence no [Darwin] in the subject
so I can get more eyes on it.
While looking for a memory gap to put the shadow area into, the sanitizer code
scans through the loaded images, and for each image it scans through its
loader command to determine the occupied memory ranges.
While doing so, if the 'segment load' (kLCSegment) loader comand is encountered, the command scanning function
returns success (true), but does not decrement the command list iterator counter.
The result is that the function is called again and again, with the iterator counter
now being too high. The command scanner keeps updating the loader command pointer,
by using the command size field.
If the loop counter is too high, the command pointer
lands into unintended area ( beyond <header addr>+sizeof(mac_header64)+header->sizeofcmds ),
and result depends on the random content found there.
The random content interpreted as loader command might contain a large integer value in the
cmdsize field - this value is added to the current loader command pointer,
which might now point to an inaccessible memory address. It can occasionally result
in a crash if it happens to run beyond the mapped memory segment.
Note that when the area after the loader command list
contains zeros or small integers only, the loop will end normally and the problem
will go unnoticed. So it happened until now since having a some big value
after the header area, falling into command size field is a pretty rare situation.
The fix makes sure that the iterator counter gets updated when the segment load (kLCSegment)
loader command is found too, and in the same code location so the updates will always go together.
Undo the changes in the sanitizer_procmaps_mac.cpp to see the test failing.
rdar://101161047
rdar://102819707
Differential Revision: https://reviews.llvm.org/D142164
On 32-bit glibc>=2.34 systems using 64bit time_t build fails because
_FILE_OFFSET_BITS is undefined here but _TIME_BITS is still set to 64
Fixes
```
/usr/include/features-time64.h:26:5: error: "_TIME_BITS=64 is allowed
only with _FILE_OFFSET_BITS=64"
| # error "_TIME_BITS=64 is allowed only with _FILE_OFFSET_BITS=64"
| ^
| 1 error generated.
```
Reviewed By: thesamesam, MaskRay
Differential Revision: https://reviews.llvm.org/D140812
For atomics metadata, we can make data race analysis more efficient by
entirely ignoring functions that include memory accesses but which only
access non-escaping (non-shared) and/or non-mutable memory. Such
functions will not be considered to be covered by "atomics" metadata,
resulting in the following benefits:
1. reduces "covered" metadata; and
2. allows data race analysis to skip such functions.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D143159
It's better and easier for us to just have threads contend against each
other in the tests if it's more than the maximum supported number of
hardware threads available.
Specifically, the recoverable test fails on Android because the
GTEST_SKIP in a called function, and it only properly works from the
TEST_* harness function. Android tests run on cuttlefish, which can be a
single core with two hyperthreads.
Reviewed By: fmayer
Differential Revision: https://reviews.llvm.org/D143221
For HWASAN this would be the tagged address. It is the same pointer when pointer tagging is not used. Coincidently this also fixes some test which rely on comparing pointers.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D143121
They were being initialized anyway, I believe, but the logic was a bit
convoluted for the Clang warnings to detect so we were getting "variable 'EBX'
may be uninitialized when used here" later on.
Older GNU assemblers generate R_X86_64_PC32 relocation against
defined non-weak global branch targets with default visibility. A
linker may issue an error when building a shared library. Add a local
alias, .Linterceptor_sigsetjmp, to __interceptor_sigsetjmp to avoid
R_X86_64_PC32 relocation for "jmp __interceptor_sigsetjmp" with older
GNU assemblers.
Fixes: https://github.com/llvm/llvm-project/issues/60426
Differential Revision: https://reviews.llvm.org/D142995
Loosen up the matching so that occasional cpu migrations don't break the
test. This showed up occasionally in internal testing.
Differential Revision: https://reviews.llvm.org/D143000
The orginal single folder layout produced libraries in the form:
lib/linux/<libname>-<archname>.a
That archname for Arm depended on whether you had hard or soft float.
This is sometimes shown as "arm" (soft) vs. "armhf" (hard).
If this is set in a triple we do it in the final portion, the ABI.
"gnueabi" for soft float, "gnueabihf" for hard float.
Meaning that the expected triple is:
arm-unknown-linux-gnueabihf
Not this:
armhf-unknown-linux-gnueabihf
For the per target layout I have decided to rely on the ABI portion
of the triple instead of the arch name used for the old layout
(doing that would produce the invalid triple above).
This means that building with triple:
armv8l-unknown-linux-gnueabihf
Will result in libraries in:
lib/arm-unknown-linux-gnueabihf/
And clang will now know to look for "arm" instead of "armv8l".
Meaning that we can share libraries between an armv8 and armv7 build
as we did with the previous layout. In addition to handling spelling
differences e.g. "armv8l" with an "l" on some Linux distros.
compiler-rt will autodetect that the "armhf" and/or "arm" architecture
can be built. We then replace the given triple's architecture with that.
Then if the triple's float ABI doesn't match, we change that. That new
triple is then used as the folder name.
If you select to build only the given triple, with COMPILER_RT_DEFAULT_TARGET_ONLY,
compiler-rt will not autodetect the architecture and for that I assume you
know what you're doing. In that case the library path will use the unomdified triple.
From what I can tell, this is how most large builds e.g Android and
Arm's Embedded Toolchain for llvm do things. I assume that big endian "armeb"
builds would end up doing this too.
Bare metal builds will not be using per target runtime dirs so they
remain as they were.
Depends on D139536
Reviewed By: MaskRay, phosek
Differential Revision: https://reviews.llvm.org/D140011
A lot of tests are disabled by using UNSUPPORTED. The plan is to remove UNSUPPORTED for tests that are fixed.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D142676
This broke the sanitizer tests on Mac, see e.g.
https://green.lab.llvm.org/green/job/clang-stage1-RA/32739/ and comment on the
code review.
> A lot of tests are disabled by using UNSUPPORTED. The plan is to remove UNSUPPORTED for tests that are fixed.
>
> Reviewed By: vitalybuka
>
> Differential Revision: https://reviews.llvm.org/D142676
This reverts commit f9a01630988716f1b52afe6727f34fe86c07c58a.
and follow-up commit bf47ffaa76fbda1ba96d41ee2681e45d2445be1e
(https://reviews.llvm.org/D142812).
