llvm-project/libcxx/utils/ci/run-buildbot

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

842 lines
28 KiB
Plaintext
Raw Normal View History

#!/usr/bin/env bash
# ===----------------------------------------------------------------------===##
#
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
#
# ===----------------------------------------------------------------------===##
set -e
set -o pipefail
unset LANG
unset LC_ALL
unset LC_COLLATE
PROGNAME="$(basename "${0}")"
function usage() {
cat <<EOF
Usage:
${PROGNAME} [options] <BUILDER>
[-h|--help] Display this help and exit.
--llvm-root <DIR> Path to the root of the LLVM monorepo. By default, we try
to figure it out based on the current working directory.
--build-dir <DIR> The directory to use for building the library. By default,
this is '<llvm-root>/build/<builder>'.
Environment variables
CC The C compiler to use, this value is used by CMake. This
variable is optional.
CXX The C++ compiler to use, this value is used by CMake. This
variable is optional.
CMAKE The CMake binary to use. This variable is optional.
CLANG_FORMAT The clang-format binary to use when generating the format
ignore list.
EOF
}
if [[ $# == 0 ]]; then
usage
exit 0
fi
while [[ $# -gt 0 ]]; do
case ${1} in
-h|--help)
usage
exit 0
;;
--llvm-root)
MONOREPO_ROOT="${2}"
shift; shift
;;
--build-dir)
BUILD_DIR="${2}"
shift; shift
;;
*)
BUILDER="${1}"
shift
;;
esac
done
MONOREPO_ROOT="${MONOREPO_ROOT:="$(git rev-parse --show-toplevel)"}"
BUILD_DIR="${BUILD_DIR:=${MONOREPO_ROOT}/build/${BUILDER}}"
INSTALL_DIR="${BUILD_DIR}/install"
# If we can find Ninja/CMake provided by Xcode, use those since we know their
# version will generally work with the Clang shipped in Xcode (e.g. if Clang
# knows about -std=c++20, the CMake bundled in Xcode will probably know about
# that flag too).
if xcrun --find ninja &>/dev/null; then
NINJA="$(xcrun --find ninja)"
elif which ninja &>/dev/null; then
# The current implementation of modules needs the absolute path to the ninja
# binary.
# TODO MODULES Is this still needed when CMake has libc++ module support?
NINJA="$(which ninja)"
else
NINJA="ninja"
fi
if [ -z "${CMAKE}" ]; then
if xcrun --find cmake &>/dev/null; then
CMAKE="$(xcrun --find cmake)"
else
CMAKE="cmake"
fi
fi
function step() {
endstep
set +x
if [[ ! -z ${GITHUB_ACTIONS+x} ]]; then
echo "::group::$1"
export IN_GROUP=1
else
echo "--- $1"
fi
set -x
}
function endstep() {
set +x
if [[ ! -z ${GITHUB_ACTIONS+x} ]] && [[ ! -z ${IN_GROUP+x} ]]; then
echo "::endgroup::"
unset IN_GROUP
fi
set -x
}
function error() {
echo "::error::$1"
}
function clean() {
rm -rf "${BUILD_DIR}"
}
function generate-cmake-base() {
step "Generating CMake"
${CMAKE} \
-S "${MONOREPO_ROOT}/runtimes" \
-B "${BUILD_DIR}" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
-DLIBCXX_ENABLE_WERROR=YES \
-DLIBCXXABI_ENABLE_WERROR=YES \
-DLIBUNWIND_ENABLE_WERROR=YES \
-DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
"${@}"
}
function generate-cmake() {
generate-cmake-base \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \
-DLIBCXX_CXX_ABI=libcxxabi \
"${@}"
}
function generate-cmake-libcxx-win() {
generate-cmake-base \
-DLLVM_ENABLE_RUNTIMES="libcxx" \
-DCMAKE_C_COMPILER=clang-cl \
-DCMAKE_CXX_COMPILER=clang-cl \
"${@}"
}
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
function generate-cmake-android() {
generate-cmake-base \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" \
-DLIBCXX_CXX_ABI=libcxxabi \
"${@}"
}
function check-runtimes() {
step "Building libc++ test dependencies"
${NINJA} -vC "${BUILD_DIR}" cxx-test-depends
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}" check-cxx
step "Running the libc++abi tests"
${NINJA} -vC "${BUILD_DIR}" check-cxxabi
step "Running the libunwind tests"
${NINJA} -vC "${BUILD_DIR}" check-unwind
}
# TODO: The goal is to test this against all configurations. We should also move
# this to the Lit test suite instead of being a separate CMake target.
function check-abi-list() {
step "Running the libc++ ABI list test"
${NINJA} -vC "${BUILD_DIR}" check-cxx-abilist || (
error "Generating the libc++ ABI list after failed check"
${NINJA} -vC "${BUILD_DIR}" generate-cxx-abilist
false
)
}
function test-armv7m-picolibc() {
clean
# To make it easier to get this builder up and running, build picolibc
# from scratch. Anecdotally, the build-picolibc script takes about 16 seconds.
# This could be optimised by building picolibc into the Docker container.
step "Building picolibc from source"
${MONOREPO_ROOT}/libcxx/utils/ci/build-picolibc.sh \
--build-dir "${BUILD_DIR}" \
--install-dir "${INSTALL_DIR}" \
--target armv7m-none-eabi
step "Generating CMake for compiler-rt"
flags="--sysroot=${INSTALL_DIR}"
${CMAKE} \
-S "${MONOREPO_ROOT}/compiler-rt" \
-B "${BUILD_DIR}/compiler-rt" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
-DCMAKE_C_FLAGS="${flags}" \
-DCMAKE_CXX_FLAGS="${flags}" \
-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON \
"${@}"
step "Generating CMake for libc++"
generate-cmake \
-DLIBCXX_TEST_CONFIG="armv7m-picolibc-libc++.cfg.in" \
-DLIBCXXABI_TEST_CONFIG="armv7m-picolibc-libc++abi.cfg.in" \
-DLIBUNWIND_TEST_CONFIG="armv7m-picolibc-libunwind.cfg.in" \
-DCMAKE_C_FLAGS="${flags}" \
-DCMAKE_CXX_FLAGS="${flags}" \
"${@}"
step "Installing compiler-rt"
${NINJA} -vC "${BUILD_DIR}/compiler-rt" install
# Prior to clang 19, armv7m-none-eabi normalised to armv7m-none-unknown-eabi.
# clang 19 changed this to armv7m-unknown-none-eabi. So for as long as 18.x
# is supported, we have to ask clang what the triple will be.
NORMALISED_TARGET_TRIPLE=$(${CC-cc} --target=armv7m-none-eabi -print-target-triple)
# Without this step linking fails later in the build.
mv "${BUILD_DIR}/install/lib/${NORMALISED_TARGET_TRIPLE}"/* "${BUILD_DIR}/install/lib"
check-runtimes
}
# Print the version of a few tools to aid diagnostics in some cases
step "Diagnose tools in use"
${CMAKE} --version
${NINJA} --version
if [ ! -z "${CXX}" ]; then ${CXX} --version; fi
case "${BUILDER}" in
check-generated-output)
# `! foo` doesn't work properly with `set -e`, use `! foo || false` instead.
# https://stackoverflow.com/questions/57681955/set-e-does-not-respect-logical-not
clean
generate-cmake
# Reject patches that forgot to re-run the generator scripts.
step "Making sure the generator scripts were run"
set +x # Printing all the commands below just creates extremely confusing output
${NINJA} -vC "${BUILD_DIR}" libcxx-generate-files
git diff | tee ${BUILD_DIR}/generated_output.patch
git ls-files -o --exclude-standard | tee ${BUILD_DIR}/generated_output.status
! grep -q '^--- a' ${BUILD_DIR}/generated_output.patch || false
if [ -s ${BUILD_DIR}/generated_output.status ]; then
echo "It looks like not all the generator scripts were run,"
echo "did you forget to build the libcxx-generate-files target?"
echo "Did you add all new files it generated?"
false
fi
# This depends on LC_COLLATE set at the top of this script.
step "Reject patches that introduce non-ASCII characters or hard tabs."
! grep -rn '[^ -~]' libcxx/include libcxx/src libcxx/test \
--exclude '*.dat' \
--exclude '*unicode*.cpp' \
--exclude '*print*.sh.cpp' \
--exclude 'escaped_output.*.pass.cpp' \
--exclude 'format_tests.h' \
--exclude 'format.functions.tests.h' \
--exclude 'formatter.*.pass.cpp' \
--exclude 'grep.pass.cpp' \
--exclude 'locale-specific_form.pass.cpp' \
--exclude 'ostream.pass.cpp' \
--exclude 'transcoding.pass.cpp' \
--exclude 'underflow.pass.cpp' \
|| false
;;
#
# Various Standard modes
#
generic-cxx03)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx03.cmake"
check-runtimes
check-abi-list
;;
generic-cxx11)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx11.cmake"
check-runtimes
check-abi-list
;;
generic-cxx14)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx14.cmake"
check-runtimes
check-abi-list
;;
generic-cxx17)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx17.cmake"
check-runtimes
check-abi-list
;;
generic-cxx20)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx20.cmake"
check-runtimes
check-abi-list
;;
generic-cxx23)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx23.cmake"
check-runtimes
check-abi-list
;;
generic-cxx26)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx26.cmake"
check-runtimes
check-abi-list
;;
#
# Other compiler support
#
generic-gcc)
clean
generate-cmake -DLIBCXX_ENABLE_WERROR=NO \
-DLIBCXXABI_ENABLE_WERROR=NO \
-DLIBUNWIND_ENABLE_WERROR=NO
check-runtimes
;;
generic-gcc-cxx11)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-cxx11.cmake" \
-DLIBCXX_ENABLE_WERROR=NO \
-DLIBCXXABI_ENABLE_WERROR=NO \
-DLIBUNWIND_ENABLE_WERROR=NO
check-runtimes
;;
#
# Sanitizers
#
generic-asan)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-asan.cmake"
check-runtimes
;;
generic-msan)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-msan.cmake"
check-runtimes
;;
generic-tsan)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-tsan.cmake"
check-runtimes
;;
generic-ubsan)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-ubsan.cmake"
check-runtimes
;;
#
# Various build configurations
#
bootstrapping-build)
clean
step "Generating CMake"
${CMAKE} \
-S "${MONOREPO_ROOT}/llvm" \
-B "${BUILD_DIR}" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_CXX_COMPILER_LAUNCHER="ccache" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
-DLLVM_ENABLE_PROJECTS="clang;lldb" \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \
-DLLVM_RUNTIME_TARGETS="$(${CXX} --print-target-triple)" \
-DLLVM_HOST_TRIPLE="$(${CXX} --print-target-triple)" \
-DLLVM_TARGETS_TO_BUILD="host" \
-DRUNTIMES_BUILD_ALLOW_DARWIN=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests"
step "Running the LLDB libc++ data formatter tests"
${NINJA} -vC "${BUILD_DIR}" lldb-api-test-deps
${BUILD_DIR}/bin/llvm-lit -sv --param dotest-args='--category libc++' "${MONOREPO_ROOT}/lldb/test/API"
step "Running the libc++ and libc++abi tests"
${NINJA} -vC "${BUILD_DIR}" check-runtimes
step "Installing libc++ and libc++abi to a fake location"
${NINJA} -vC "${BUILD_DIR}" install-runtimes
ccache -s
;;
generic-static)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-static.cmake"
check-runtimes
;;
generic-merged)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-merged.cmake" \
-DLIBCXX_TEST_CONFIG="llvm-libc++-shared.cfg.in" \
-DLIBCXXABI_TEST_CONFIG="llvm-libc++abi-merged.cfg.in" \
-DLIBUNWIND_TEST_CONFIG="llvm-libunwind-merged.cfg.in"
check-runtimes
;;
generic-hardening-mode-fast)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-hardening-mode-fast.cmake"
check-runtimes
check-abi-list
;;
generic-hardening-mode-fast-with-abi-breaks)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-hardening-mode-fast-with-abi-breaks.cmake"
check-runtimes
# Not checking ABI list since we purposefully enable ABI breaking changes
;;
generic-hardening-mode-extensive)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-hardening-mode-extensive.cmake"
check-runtimes
check-abi-list
;;
generic-hardening-mode-debug)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-hardening-mode-debug.cmake"
check-runtimes
check-abi-list
;;
#
# Module builds
#
generic-modules)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-modules.cmake"
check-runtimes
check-abi-list
;;
generic-modules-lsv)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-modules-lsv.cmake"
check-runtimes
check-abi-list
;;
#
# Parts removed
#
generic-no-threads)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-threads.cmake"
check-runtimes
;;
generic-no-filesystem)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-filesystem.cmake"
check-runtimes
;;
generic-no-random_device)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-random_device.cmake"
check-runtimes
;;
generic-no-localization)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-localization.cmake"
check-runtimes
;;
generic-no-terminal)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-terminal.cmake"
check-runtimes
;;
generic-no-unicode)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-unicode.cmake"
check-runtimes
;;
generic-no-wide-characters)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-wide-characters.cmake"
check-runtimes
;;
generic-no-tzdb)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-tzdb.cmake"
check-runtimes
;;
generic-no-experimental)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-experimental.cmake"
check-runtimes
check-abi-list
;;
generic-no-exceptions)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-exceptions.cmake"
check-runtimes
check-abi-list
;;
generic-no-rtti)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-no-rtti.cmake"
check-runtimes
;;
#
# Other miscellaneous jobs
#
generic-abi-unstable)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-abi-unstable.cmake"
check-runtimes
;;
generic-optimized-speed)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-optimized-speed.cmake"
check-runtimes
;;
apple-configuration)
clean
step "Installing libc++ with the Apple system configuration"
arch="$(uname -m)"
xcrun --sdk macosx \
${MONOREPO_ROOT}/libcxx/utils/ci/apple-install-libcxx.sh \
--llvm-root ${MONOREPO_ROOT} \
--build-dir ${BUILD_DIR} \
--install-dir ${INSTALL_DIR} \
--symbols-dir "${BUILD_DIR}/symbols" \
--architectures "${arch}" \
--version "999.99"
step "Running tests against Apple-configured libc++"
# TODO: It would be better to run the tests against the fake-installed version of libc++ instead
xcrun --sdk macosx ninja -vC "${BUILD_DIR}/${arch}" check-cxx check-cxxabi check-cxx-abilist
;;
apple-system-hardened)
clean
arch="$(uname -m)"
version="$(sw_vers --productVersion)"
params="target_triple=${arch}-apple-macosx${version}"
params+=";hardening_mode=fast"
# In the Apple system configuration, we build libc++ and libunwind separately.
step "Installing libc++ and libc++abi in Apple-system configuration"
${CMAKE} \
-S "${MONOREPO_ROOT}/runtimes" \
-B "${BUILD_DIR}/cxx" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}/cxx" \
-DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" \
-DLIBCXX_CXX_ABI=libcxxabi \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/Apple.cmake" \
-DLIBCXX_TEST_CONFIG="apple-libc++-system.cfg.in" \
-DLIBCXXABI_TEST_CONFIG="apple-libc++abi-system.cfg.in" \
-DLIBCXX_TEST_PARAMS="${params}" \
-DLIBCXXABI_TEST_PARAMS="${params}"
step "Installing libunwind in Apple-system configuration"
${CMAKE} \
-S "${MONOREPO_ROOT}/runtimes" \
-B "${BUILD_DIR}/unwind" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}/unwind" \
-DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
-DLLVM_ENABLE_RUNTIMES="libunwind" \
-DLIBUNWIND_TEST_CONFIG="apple-libunwind-system.cfg.in" \
-DLIBUNWIND_TEST_PARAMS="${params}" \
-DCMAKE_INSTALL_NAME_DIR="/usr/lib/system"
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}/cxx" check-cxx
step "Running the libc++abi tests"
${NINJA} -vC "${BUILD_DIR}/cxx" check-cxxabi
step "Running the libunwind tests"
${NINJA} -vC "${BUILD_DIR}/unwind" check-unwind
;;
apple-system)
clean
arch="$(uname -m)"
version="$(sw_vers --productVersion)"
params="target_triple=${arch}-apple-macosx${version}"
# In the Apple system configuration, we build libc++ and libunwind separately.
step "Installing libc++ and libc++abi in Apple-system configuration"
${CMAKE} \
-S "${MONOREPO_ROOT}/runtimes" \
-B "${BUILD_DIR}/cxx" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}/cxx" \
-DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" \
-DLIBCXX_CXX_ABI=libcxxabi \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/Apple.cmake" \
-DLIBCXX_TEST_CONFIG="apple-libc++-system.cfg.in" \
-DLIBCXXABI_TEST_CONFIG="apple-libc++abi-system.cfg.in" \
-DLIBCXX_TEST_PARAMS="${params}" \
-DLIBCXXABI_TEST_PARAMS="${params}"
step "Installing libunwind in Apple-system configuration"
${CMAKE} \
-S "${MONOREPO_ROOT}/runtimes" \
-B "${BUILD_DIR}/unwind" \
-GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}/unwind" \
-DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
-DLLVM_ENABLE_RUNTIMES="libunwind" \
-DLIBUNWIND_TEST_CONFIG="apple-libunwind-system.cfg.in" \
-DLIBUNWIND_TEST_PARAMS="${params}" \
-DCMAKE_INSTALL_NAME_DIR="/usr/lib/system"
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}/cxx" check-cxx
step "Running the libc++abi tests"
${NINJA} -vC "${BUILD_DIR}/cxx" check-cxxabi
step "Running the libunwind tests"
${NINJA} -vC "${BUILD_DIR}/unwind" check-unwind
;;
aarch64)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/AArch64.cmake"
check-runtimes
;;
aarch64-no-exceptions)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/AArch64.cmake" \
-DLIBCXX_ENABLE_EXCEPTIONS=OFF \
-DLIBCXXABI_ENABLE_EXCEPTIONS=OFF
check-runtimes
;;
# Aka Armv8 32 bit
armv8)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Armv8Arm.cmake"
check-runtimes
;;
armv8-no-exceptions)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Armv8Thumb-no-exceptions.cmake"
check-runtimes
;;
# Armv7 32 bit. One building Arm only one Thumb only code.
armv7)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Armv7Arm.cmake"
check-runtimes
;;
armv7-no-exceptions)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Armv7Thumb-no-exceptions.cmake"
check-runtimes
;;
armv7m-picolibc)
test-armv7m-picolibc \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/Armv7M-picolibc.cmake"
;;
armv7m-picolibc-no-exceptions)
test-armv7m-picolibc \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/Armv7M-picolibc.cmake" \
-DLIBCXXABI_ENABLE_EXCEPTIONS=OFF \
-DLIBCXXABI_ENABLE_STATIC_UNWINDER=OFF \
-DLIBCXX_ENABLE_EXCEPTIONS=OFF \
-DLIBCXX_ENABLE_RTTI=OFF
;;
clang-cl-dll)
clean
# TODO: Currently, building with the experimental library breaks running
# tests (the test linking look for the c++experimental library with the
# wrong name, and the statically linked c++experimental can't be linked
# correctly when libc++ visibility attributes indicate dllimport linkage
# anyway), thus just disable the experimental library. Remove this
# setting when cmake and the test driver does the right thing automatically.
generate-cmake-libcxx-win -DLIBCXX_TEST_PARAMS="enable_experimental=False"
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}" check-cxx
;;
clang-cl-static)
clean
generate-cmake-libcxx-win -DLIBCXX_ENABLE_SHARED=OFF
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}" check-cxx
;;
clang-cl-no-vcruntime)
clean
# Building libc++ in the same way as in clang-cl-dll above, but running
# tests with -D_HAS_EXCEPTIONS=0, which users might set in certain
# translation units while using libc++, even if libc++ is built with
# exceptions enabled.
generate-cmake-libcxx-win -DLIBCXX_TEST_PARAMS="enable_experimental=False" \
-DLIBCXX_TEST_CONFIG="llvm-libc++-shared-no-vcruntime-clangcl.cfg.in"
step "Running the libc++ tests"
[libcxx] [test] Fix running tests with Clang-cl in Debug mode When building in debug mode, the debug version of the MSVC CRT gets linked in. (This is the default in CMake in general. In the case of libcxx, we manually link the CRT though - and in debug mode, we pick the debug version of the CRT.) When building the tests, we need to use the same version of the CRT as was used for building the library. Additionally; the debug CRT defaults to pop up a dialog box when asserts fail, which blocks running tests. By including the set_windows_crt_report_mode.h helper header, we change the assert behaviour back to that of release mode - printing a message and exiting immediately. This was supported by the old libcxx test system, where support for it was added in 7e3ee09ad24cbca3ea7687c50b53be5269127fb1. When porting over to the newer test setup, this mechanism wasn't brought over (and the old test infrastructure was removed in a48f018bb7d8fadc67c08e71409c31713daa0071). Thus: In debug mode, link against the debug versions of msvcrt and msvcprt, define _DEBUG (enabling CRT debug mode code patterns), and include the set_windows_crt_report_mode.h header. Based on a patch by Andrew Ng. Linking of the debug version of the CRT can also be done by using the new -fms-runtime-lib= Clang option. However that option was added in Clang 16, and libcxx only requires Clang 15 for now; therefore doing the CRT linking entirely manually for now (just as before). Additionally, adjust set_windows_crt_report_mode.h to avoid including the body of the file when building in C mode or in C++03 mode. This fixes the following two tests: libcxx/include_as_c.sh.cpp libcxx/selftest/dsl/dsl.sh.py The former test is built in C mode. The latter tries compiling things as C++03. Some of the vcruntime headers that we include break in C++03 mode when MS CRT debug mode is enabled. Differential Revision: https://reviews.llvm.org/D155554
2023-07-14 12:16:23 +03:00
${NINJA} -vC "${BUILD_DIR}" check-cxx
;;
clang-cl-debug)
clean
generate-cmake-libcxx-win -DLIBCXX_TEST_PARAMS="enable_experimental=False" \
-DCMAKE_BUILD_TYPE=Debug
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}" check-cxx
;;
clang-cl-static-crt)
clean
# Test linking a static libc++ with the static CRT ("MultiThreaded" denotes
# the static CRT, as opposed to "MultiThreadedDLL" which is the default).
generate-cmake-libcxx-win -DLIBCXX_ENABLE_SHARED=OFF \
-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded
step "Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}" check-cxx
;;
mingw-dll)
clean
generate-cmake \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/MinGW.cmake"
check-runtimes
;;
mingw-static)
clean
generate-cmake \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/MinGW.cmake" \
-DLIBCXX_ENABLE_SHARED=OFF \
-DLIBUNWIND_ENABLE_SHARED=OFF
check-runtimes
;;
mingw-dll-i686)
clean
generate-cmake \
-DCMAKE_C_COMPILER=i686-w64-mingw32-clang \
-DCMAKE_CXX_COMPILER=i686-w64-mingw32-clang++ \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/MinGW.cmake"
check-runtimes
;;
mingw-incomplete-sysroot)
# When bringing up a new cross compiler from scratch, we build
# libunwind/libcxx in a setup where the toolchain is incomplete and
# unable to perform the normal linker checks; this requires a few
# special cases in the CMake files.
#
# Building in an incomplete setup requires setting CMAKE_*_COMPILER_WORKS,
# as CMake fails to probe the compiler. This case also requires
# setting CMAKE_CXX_COMPILER_TARGET, as LLVM's heuristics for setting
# the triple fails when CMake hasn't been able to probe the environment.
# (This is what one has to do when building the initial libunwind/libcxx
# for a new toolchain.)
clean
generate-cmake \
-DCMAKE_C_COMPILER_WORKS=TRUE \
-DCMAKE_CXX_COMPILER_WORKS=TRUE \
-DCMAKE_C_COMPILER_TARGET=x86_64-w64-windows-gnu \
-DCMAKE_CXX_COMPILER_TARGET=x86_64-w64-windows-gnu \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/MinGW.cmake"
# Only test that building succeeds; there's not much extra value in running
# the tests here, as it would be equivalent to the mingw-dll config above.
step "Building the runtimes"
${NINJA} -vC "${BUILD_DIR}"
;;
aix)
clean
generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/AIX.cmake" \
-DLIBCXX_TEST_CONFIG="ibm-libc++-shared.cfg.in" \
-DLIBCXXABI_TEST_CONFIG="ibm-libc++abi-shared.cfg.in" \
-DLIBUNWIND_TEST_CONFIG="ibm-libunwind-shared.cfg.in"
check-abi-list
check-runtimes
;;
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
android-ndk-*)
clean
ANDROID_EMU_IMG="${BUILDER#android-ndk-}"
. "${MONOREPO_ROOT}/libcxx/utils/ci/vendor/android/emulator-functions.sh"
if ! validate_emu_img "${ANDROID_EMU_IMG}"; then
error "android-ndk suffix must be a valid emulator image (${ANDROID_EMU_IMG})" >&2
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
exit 1
fi
ARCH=$(arch_of_emu_img ${ANDROID_EMU_IMG})
# Use the Android compiler by default.
export CC=${CC:-/opt/android/clang/clang-current/bin/clang}
export CXX=${CXX:-/opt/android/clang/clang-current/bin/clang++}
# The NDK libc++_shared.so is always built against the oldest supported API
# level. When tests are run against a device with a newer API level, test
# programs can be built for any supported API level, but building for the
# newest API (i.e. the system image's API) is probably the most interesting.
PARAMS="executor=${MONOREPO_ROOT}/libcxx/utils/adb_run.py;target_triple=$(triple_of_arch ${ARCH})$(api_of_emu_img ${ANDROID_EMU_IMG})"
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
generate-cmake-android -C "${MONOREPO_ROOT}/runtimes/cmake/android/Arch-${ARCH}.cmake" \
-C "${MONOREPO_ROOT}/libcxx/cmake/caches/AndroidNDK.cmake" \
-DCMAKE_SYSROOT=/opt/android/ndk/sysroot \
-DLIBCXX_TEST_PARAMS="${PARAMS}" \
-DLIBCXXABI_TEST_PARAMS="${PARAMS}"
check-abi-list
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
${NINJA} -vC "${BUILD_DIR}" install-cxx install-cxxabi
# Start the emulator and make sure we can connect to the adb server running
# inside of it.
"${MONOREPO_ROOT}/libcxx/utils/ci/vendor/android/start-emulator.sh" ${ANDROID_EMU_IMG}
trap "${MONOREPO_ROOT}/libcxx/utils/ci/vendor/android/stop-emulator.sh" EXIT
. "${MONOREPO_ROOT}/libcxx/utils/ci/vendor/android/setup-env-for-emulator.sh"
# Create adb_run early to avoid concurrent `mkdir -p` of common parent
# directories.
adb shell mkdir -p /data/local/tmp/adb_run
adb push "${BUILD_DIR}/lib/libc++_shared.so" /data/local/tmp/libc++/libc++_shared.so
step "Running the libc++ tests"
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
${NINJA} -vC "${BUILD_DIR}" check-cxx
step "Running the libc++abi tests"
[libc++][Android] Support libc++ testing on Android (#69274) I could probably break this commit into more pieces. --- This patch adds libc++ support for Android L (Android 5.0+) and up, tested using the Android team's current compiler, a recent version of the AOSP sysroot, and the x86[-64] Android Emulator. CMake and Lit Configuration: Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake to cross-compile to Android without using CMake's built-in NDK support (which only works with an actual packaged NDK). Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++ (and libc++abi) for Android. This file configures libc++ to match what the NDK distributes, e.g.: - libc++_shared.so (includes libc++abi objects, there is no libc++abi.so). libunwind is linked statically but not exported. - libc++_static.a (does not include libc++abi) and libc++abi.a - `std::__ndk1` namespace - All the libraries are built with `__ANDROID_API__=21`, even when they are linked to something targeting a higher API level. (However, when the Android LLVM team builds these components, they do not use these CMake cache files. Instead they use Python scripts to configure the builds. See https://android.googlesource.com/toolchain/llvm_android/.) Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android NDK's libc++_shared.so. These files can target old or new Android devices. The Android LLVM team uses these test files to test libc++ for both arm/arm64 and x86/x86_64 architectures. The Android testing mode works by setting %{executor} to adb_run.py, which uses `adb push` and `adb shell` to run tests remotely. adb_run.py always runs tests as the "shell" user even on an old emulator where "adb unroot" doesn't work. The script has workarounds for old Android devices. The script uses a Unix domain socket on the host (--job-limit-socket) to restrict concurrent adb invocations. Compiling the tests is a major part of libc++ testing run-time, so it's desirable to exploit all the host cores without overburdening the test devices, which can have far fewer cores. BuildKite CI: Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang and an Android sysroot to build libc++, then starts an Android emulator container to run tests. Run the emulator and an adb server in a separate Docker container (libcxx-ci-android-emulator), and create a separate Docker image for each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the container's adb server. Running the only adb server inside the container makes cleanup more reliable between test runs, e.g. the adb client doesn't create a `~/.android` directory and the adb server can be restarted along with the emulator using docker stop/run. (N.B. The emulator insists on connecting to an adb server and will start one itself if it can't connect to one.) The suffix to the android-ndk-* job is a label that concisely specifies an Android SDK emulator image. e.g.: - "system-images;android-21;default;x86" ==> 21-def-x86 - "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64 Fixes: https://github.com/llvm/llvm-project/issues/69270 Differential Revision: https://reviews.llvm.org/D139147
2023-10-19 13:58:30 -07:00
${NINJA} -vC "${BUILD_DIR}" check-cxxabi
;;
#################################################################
# Insert vendor-specific internal configurations below.
#
# This allows vendors to extend this file with their own internal
# configurations without running into merge conflicts with upstream.
#################################################################
#################################################################
*)
echo "${BUILDER} is not a known configuration"
exit 1
;;
esac
endstep # Make sure we close any still-open output group