There are some programs that try to handle all malloc failures.
Let's allow testing of such programs.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D144374
This reverts commit 37eb9d13f891f7656f811516e765b929b169afe0.
Test failures have been fixed:
- ubsan failure fixed by 72eac42f21c0f45a27f3eaaff9364cbb5189b9e4
- warn-unsafe-buffer-usage-fixits-local-var-span.cpp fixed by
03cc52dfd1dbb4a59b479da55e87838fb93d2067 (wasn't related)
- test-output-format.ll failure was spurious, build failed at
https://lab.llvm.org/buildbot/#/builders/54/builds/3545 (b4431b2d945b6fc19b1a55ac6ce969a8e06e1e93)
but passed at
https://lab.llvm.org/buildbot/#/builders/54/builds/3546 (5ae99be0377248c74346096dc475af254a3fc799)
which is before my revert
b4431b2d94...5ae99be037
Original commit message:
Depends on https://reviews.llvm.org/D142861.
Alternative to https://reviews.llvm.org/D137601.
xxHash is much faster than djbHash. This makes a simple Rust test case with a large constant string 10% faster to compile.
Previous attempts at changing this hash function (e.g. https://reviews.llvm.org/D97396) had to be reverted due to breaking tests that depended on iteration order.
No additional tests fail with this patch compared to `main` when running `check-all` with `-DLLVM_ENABLE_PROJECTS="all"` (on a Linux host), so I hope I found everything that needs to be changed.
Differential Revision: https://reviews.llvm.org/D142862
Allocation actions may run JIT'd code, which isn't permitted in -noexec mode.
Testcases that depend on actions running should be moved to the ORC runtime.
This caused false positives because the existing logic was not taking into account that pointers could have a tag in them.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D144305
Since binary ID lookup makes CLI object arguments optional, it should be
possible to pass a list of source files without a binary. Unfortunately,
the current syntax will always interpret the first source file as a
binary. This change adds a `-sources` option to cause all later
positional arguments to be considered sources.
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D144207
Looks like builders were failing for the sized variants of operator
delete. These are hidden behind the __cpp_sized_deallocation macro
which can be defined with -fsized-deallocation.
The implementations of those functions require the rounding target to be
power-of-two. It's better to add a debugging check to avoid misuse.
Besides, add a general verion of those three to accommadate non
power-of-two cases.
Also change the name to roundUp/roundDown/isAligned
Reviewed By: cferris, cryptoad
Differential Revision: https://reviews.llvm.org/D142658
Ideally, we want to assert that all the operations on
Cache/QuarantineCache always have the `Mutex` acquired. However,
the current architecture of accessing TSD is not easy to cooperate
with the thread-safety analysis because of pointer aliasing. In
alternative, we add the getters for accessing TSD member and attach
proper thread-safety annotations on them.
Reviewed By: cferris
Differential Revision: https://reviews.llvm.org/D142151
allocation ring buffer is allowed to be zero. Update the logic in the
test so that on the platform that disables it won't fail this case.
Reviewed By: fmayer
Differential Revision: https://reviews.llvm.org/D144055
This CL adds the proper thread-safety annotations for most of the
functions and variables. However, given the restriction of the current
architecture, in some cases, we may not be able to use the annotations
easily. The followings are two exceptions,
1. enable()/disable(): Many structures in scudo are enabled/disabled by
acquiring the lock in each instance. This makes those structure act
like a `lock`. We can't mark those functions with ACQUIRE()/RELEASE()
because that makes the entire allocator become another `lock`. In the
end, that implies we need to *acquire* the `allocator` before each
malloc et al. request. Therefore, adding a variable to tell the
status of those structures may be a better way to cooperate with
thread-safety annotation.
2. TSD/TSD shared/TSD exclusive: These three have simiar restrictions as
mentioned above. In addition, they don't always need to be released
if it's a thread local instance. However, thread-safety analysis
doesn't support conditional branch. Which means we can't mark the
proper annotations around the uses of TSDs. We may consider to make
it consistent and which makes the code structure simpler.
This CL is supposed to introduce the annotations with the least code
refactoring. So only trivial thread safety issues will be addressed
here. For example, lacking of acquiring certain lock before accessing
certain variables will have the ScopedLock inserted. Other than that,
they are supposed to be done in the later changes.
Reviewed By: cferris
Differential Revision: https://reviews.llvm.org/D140706
Scudo is expected to call __scudo_allocate_hook on allocations, and
__scudo_deallocate_hook on deallocations, but it's behavior is not
clear on reallocations. Currently, non-trivial reallocations call
__scudo_allocate_hook but never __scudo_deallocate_hook. We should
prefer either calling both, none, or a dedicated
hook (__scudo_reallocate_hook, for instance).
This patch implements the former, and adds a unit test to enforce
those expectations.
Reviewed By: Chia-hungDuan
Differential Revision: https://reviews.llvm.org/D141407
Because of lock contention, we temporarily disabled the printing of
regions' status when it's exhausted. Given that it's useful when the
Region OOM happens, this CL brings it back without lock contention.
Differential Revision: https://reviews.llvm.org/D141955
We didn't acquire the mutex while accessing those lock protected data,
this CL fixes it and now we don't need to disable the allocator while
reading its states.
Differential Revision: https://reviews.llvm.org/D142149
clang-cl doesn't support -dumpmachine directly, so we need to
preface it with /clang: in order to get this probing function
to work.
This is needed in order to run cmake directly on the runtimes
directory.
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D143557
I found a few Python files not using Python 3 syntax in the tree
when trying to apply reformatting to them. This was converted with
2to3 and the changes seemed sane.
Reviewed By: MaskRay, browneee
Differential Revision: https://reviews.llvm.org/D143701
Emit all constant integers produced by SanitizerBinaryMetadata as
ULEB128 to further reduce binary space used. Increasing the version is
not necessary given this change depends on (and will land) along with
the bump to v2.
To support this, the !pcsections metadata format is extended to allow
for per-section options, encoded in the first MD operator which must
always be a string and contain the section: "<section>!<options>".
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D143484
Optimize the encoding of "covered" metadata by:
1. Reducing feature mask from 4 bytes to 1 byte (needs increase once we
reach more than 8 features).
2. Only emitting UAR stack args size if it is non-zero, saving 4 bytes
in the common case.
One caveat is that the emitted metadata for function PC (offset), size,
and UAR size (if enabled) are no longer aligned to 4 bytes.
SanitizerBinaryMetadata version base is increased to 2, since the change
is backwards incompatible.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D143482
If you are missing the iOS SDK on your macOS (for example you don't have
full Xcode but just CommandLineTools) then CMake currently errors
out without a helpful message. This patch disables iOS support in
compiler-rt if the iOS SDK is not found. This can be overriden by
passing -DCOMPILER_RT_ENABLE_IOS=ON.
Reviewed By: delcypher, thetruestblue
Differential Revision: https://reviews.llvm.org/D133273
Depends on https://reviews.llvm.org/D142861.
Alternative to https://reviews.llvm.org/D137601.
xxHash is much faster than djbHash. This makes a simple Rust test case with a large constant string 10% faster to compile.
Previous attempts at changing this hash function (e.g. https://reviews.llvm.org/D97396) had to be reverted due to breaking tests that depended on iteration order.
No additional tests fail with this patch compared to `main` when running `check-all` with `-DLLVM_ENABLE_PROJECTS="all"` (on a Linux host), so I hope I found everything that needs to be changed.
Differential Revision: https://reviews.llvm.org/D142862
The fix only affects Darwin, but to write the test I had to modify
the MemoryMappingLayout class which is used by all OSes,
to allow for mocking of image header (this change should be NFC). Hence no [Darwin] in the subject
so I can get more eyes on it.
While looking for a memory gap to put the shadow area into, the sanitizer code
scans through the loaded images, and for each image it scans through its
loader command to determine the occupied memory ranges.
While doing so, if the 'segment load' (kLCSegment) loader comand is encountered, the command scanning function
returns success (true), but does not decrement the command list iterator counter.
The result is that the function is called again and again, with the iterator counter
now being too high. The command scanner keeps updating the loader command pointer,
by using the command size field.
If the loop counter is too high, the command pointer
lands into unintended area ( beyond
+sizeof(mac_header64)+header->sizeofcmds ),
and result depends on the random content found there.
The random content interpreted as loader command might contain a large integer value in the
cmdsize field - this value is added to the current loader command pointer,
which might now point to an inaccessible memory address. It can occasionally result
in a crash if it happens to run beyond the mapped memory segment.
Note that when the area after the loader command list
contains zeros or small integers only, the loop will end normally and the problem
will go unnoticed. So it happened until now since having a some big value
after the header area, falling into command size field is a pretty rare situation.
The fix makes sure that the iterator counter gets updated when the segment load (kLCSegment)
loader command is found too, and in the same code location so the updates will always go together.
Undo the changes in the sanitizer_procmaps_mac.cpp to see the test failing.
rdar://101161047
rdar://102819707
Differential Revision: https://reviews.llvm.org/D142164
After D143438 there is no point in this check as the size will never 0. Also this keeps it consistent with ASAN where there is not size check in LsanMetadata::allocated.
Reviewed By: MaskRay, vitalybuka
Differential Revision: https://reviews.llvm.org/D143442
Without this, if hardening measures like FORTIFY_SOURCE are are in
/etc/clang/*.cfg, many sanitizer tests will die before the sanitizer
can trap the problem being tested, because e.g. the _chk variants
of common functions will abort first.
This gets the number of failing tests down from 42->3 for me (and the
remaining 3 are unrelated).
See: 52ce6776cf98e993c6ec04ae54b52e1354fff917
See: 136f77805fd89cd30e69b3d1204fbf7efedd9a12
Closes: https://github.com/llvm/llvm-project/issues/60394
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D143322
The fix only affects Darwin, but to write the test I had to modify
the MemoryMappingLayout class which is used by all OSes,
to allow for mocking of image header (this change should be NFC). Hence no [Darwin] in the subject
so I can get more eyes on it.
While looking for a memory gap to put the shadow area into, the sanitizer code
scans through the loaded images, and for each image it scans through its
loader command to determine the occupied memory ranges.
While doing so, if the 'segment load' (kLCSegment) loader comand is encountered, the command scanning function
returns success (true), but does not decrement the command list iterator counter.
The result is that the function is called again and again, with the iterator counter
now being too high. The command scanner keeps updating the loader command pointer,
by using the command size field.
If the loop counter is too high, the command pointer
lands into unintended area ( beyond <header addr>+sizeof(mac_header64)+header->sizeofcmds ),
and result depends on the random content found there.
The random content interpreted as loader command might contain a large integer value in the
cmdsize field - this value is added to the current loader command pointer,
which might now point to an inaccessible memory address. It can occasionally result
in a crash if it happens to run beyond the mapped memory segment.
Note that when the area after the loader command list
contains zeros or small integers only, the loop will end normally and the problem
will go unnoticed. So it happened until now since having a some big value
after the header area, falling into command size field is a pretty rare situation.
The fix makes sure that the iterator counter gets updated when the segment load (kLCSegment)
loader command is found too, and in the same code location so the updates will always go together.
Undo the changes in the sanitizer_procmaps_mac.cpp to see the test failing.
rdar://101161047
rdar://102819707
Differential Revision: https://reviews.llvm.org/D142164
On 32-bit glibc>=2.34 systems using 64bit time_t build fails because
_FILE_OFFSET_BITS is undefined here but _TIME_BITS is still set to 64
Fixes
```
/usr/include/features-time64.h:26:5: error: "_TIME_BITS=64 is allowed
only with _FILE_OFFSET_BITS=64"
| # error "_TIME_BITS=64 is allowed only with _FILE_OFFSET_BITS=64"
| ^
| 1 error generated.
```
Reviewed By: thesamesam, MaskRay
Differential Revision: https://reviews.llvm.org/D140812
For atomics metadata, we can make data race analysis more efficient by
entirely ignoring functions that include memory accesses but which only
access non-escaping (non-shared) and/or non-mutable memory. Such
functions will not be considered to be covered by "atomics" metadata,
resulting in the following benefits:
1. reduces "covered" metadata; and
2. allows data race analysis to skip such functions.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D143159
It's better and easier for us to just have threads contend against each
other in the tests if it's more than the maximum supported number of
hardware threads available.
Specifically, the recoverable test fails on Android because the
GTEST_SKIP in a called function, and it only properly works from the
TEST_* harness function. Android tests run on cuttlefish, which can be a
single core with two hyperthreads.
Reviewed By: fmayer
Differential Revision: https://reviews.llvm.org/D143221
