This patch removes LLDB from a list of projects that are excluded from
building and testing on pre-merge CI on Linux.
Windows environment needs to be prepared in order to test LLDB
(https://github.com/llvm/llvm-project/pull/94208#issuecomment-2146256857),
but we don't have enough maintenance resources to do that at the moment.
Because LLDB has been in the list of projects that need to be tested on
Clang changes, this PR make this happen on Linux. This seems to be the
consensus in the discussion of this PR.
Flang triggers some OOM on Windows CI right now. This is disruptive to
MLIR and LLVM changes that don't touch Flang, as such we disable
building Flang on Windows only for these PR that don't touch flang. The
testing on Linux is unchanged, and the post-merge Windows testing is
still fully covering here.
Following the discussion in
https://github.com/llvm/llvm-project/pull/93233#issuecomment-2127920882,
this patch merges `clang-ci` pipeline into main `GitHub Pull Requests`
pipeline. `clang-ci` enables additional test coverage for Clang by
compiling it, and then using it to compile and test libc++, libc++abi,
and libunwind in C++03, C++26, and Clang Modules modes.
Additional work we skip and total time savings we should see:
1. Checking out the repo to generate the clang-ci pipeline (2 minutes)
2. Building Clang (3.5 minutes)
3. Uploading the artifacts once, then downloading them 3 times and
unpacking 3 times (0.5 minutes)
Note that because previously-split jobs for each mode are now under a
single Linux job, it now takes around 8 minutes more see the Linux CI
results despite total time savings.
The primary goal of this patch is to reduce the load of CI by removing
duplicated work. I consider this goal achieved. I could keep the job
parallelism we had (3 libc++ jobs depending on a main Linux job), but I
don't consider it worth the effort and opportunity cost, because
parallelism is not helping once the pool of builders is fully
subscribed.
This fixes all shellcheck warnings we have in `monolithic-linux.sh` and
`monolithic-windows.sh`.
All of them have to do with
[SC2086](https://www.shellcheck.net/wiki/SC2086) - Double quote to
prevent globbing and word splitting.
Clang has a custom separate pipeline integrated with libc++ that only
runs in release mode. It means that changes which touches only clang
won't run the clang tests in the configuration used by LLVM premerge and
will break it unknowingly.
#84828 added `-k 0` to pre-merge CI so that if one job fails the others
would continue building. This pull request fixes the location of `-k 0`
in the ninja command line.
Resolves#84842 and #83371
The -k option allows to continue the build after failures as much as
possible. This is useful here because when we run
> ninja check-llvm check-clang
we would like the clang tests to run even if there is a failure in a
llvm tests.
The downside is that a build failure in one file that would prevent from
running any test does not prevent from building more targets, wasting
build resources potentially.
Fixes#83371
We should diff against the base branch, not always against `main`. This
allows the BuildKite pre-commit CI to work properly when we target other
branches, such as `release/18.x`.
The "scheduled build" pipeline on BuildKite had been disabled for months
and doesn't exist anymore, so this script is effectively dead code. When
we set up a cron-activated build again, we should do it using Github
actions (which could trigger a BK pipeline if needed).
Keeping this script around just creates additional confusion about
what's used and what's not used for doing CI.
PR for: https://discourse.llvm.org/t/add-check-mlir-python-to-the-mlir-pre-commit-tests/74041
It’s easy to forget about the Python bindings and not build/test them
locally. It’s also easy to change something that’ll break the python
binding tests and not find out till after you’ve committed your change.
These tests seem to run quickly and don’t require much extra setup, so
let's add them to the general MLIR pre-merge tests.
we tried to generate a full diff against main in
ec9d80e but it resulted in wrong diffs.
It seems that the issue was that 'main' was not
updated after agent restart and diff main...HEAD kept growing.
Not enabling diff main...HEAD just yet and will check logs for new PRs
first.
Since we moved to Github PRs, the workflow has changed a bit and folks
often merge `main` back into their PR branch. This is fine, except the
previous way of determining modified files for pre-commit CI would use
the content modified just in the latest commit, whatever it is. This
means that in case someone merged main back into their PR branch, we'd
think that the files in the merge commit were modified by the PR, and
we'd spuriously trigger a CI run. This should fix this issue.
The downside is that the merge target is hardcoded to `main`, which
might not always be what we want. I still think this is an improvement
over the status quo.
- fixed build for linux (clang was missing)
- removed /monolithic-.. from build directory - it does not add anything
and makes path longer for windows which is not great;
- added env-based configuration to control cache and agent targeting;
- print (s)ccache stats to file not to pullute normal log.
This basically inlines the logic that was previously located in
https://github.com/google/llvm-premerge-checks so it is part of
the monorepo. This has the benefit of making it extremely easy
for individual projects to understand and modify this logic for
their own needs, unlike the current model where this logic lives
in a separate non-LLVM repository.
It also allows testing changes to the CI configuration using a simple
Phabricator review, since the code that defines the CI pipeline is taken
from the patch under review.
This (or something equivalent) is necessary if we want to retain the
current monolithic pre-commit CI throughout the GitHub PR transition.
Since triggering the monolithic CI is currently attached to the system
we use for triggering CI pipelines from Phabricator, we will lose that
part of the CI when we move to GitHub PRs if we don't do anything.
I've decided to rewrite the code as a shell script because the logic
was fairly simple and it seemed a lot easier than figuring out how to
pull only the relevant parts of llvm-premerge-checks into the monorepo.
Furthermore, I think we should strive to move away from the monolithic
CI altogether since sub-projects should understand, own and maintain the
tests that are relevant for them to run in the CI (with LLVM providing
the infrastructure). Hence, this is somewhat of a temporary solution
until monolithic CI is removed entirely.
Differential Revision: https://reviews.llvm.org/D158863
This simply moves existing CI-related scripts for Clang outside of libc++
to make the two projects independent with respect to CI.
The reason for making this change is that libc++'s CI pipeline history
was full of Clang runs, making it a lot harder to figure out information
about libc++'s own jobs. I created a pipeline for Clang and this patch
will run pre-commit CI for Clang inside that pipeline instead.
Differential Revision: https://reviews.llvm.org/D155078
