When --check-globals none, we skipped all the globals in check lines.
However, we are still checking the meta info in function defintion.
The generated checks are still sensitive to metadata changes.
This is to scrub the meta info and match them with {{.*}} instead.
Labels are matched using a regexp of the form '^(pattern):', which
requires the addition of a "suffix" concept to NamelessValue.
Aside from that, the key challenge is that block labels are values, and
we typically capture values including the prefix '%'. However, when
labels appear at the start of a basic block, the prefix '%' is not
included, so we must capture block label values *without* the prefix
'%'.
We don't know ahead of time whether an IR value is a label or not. In
most cases, they are prefixed by the word "label" (their type), but this
isn't the case in phi nodes. We solve this issue by leveraging the
two-phase nature of variable generalization: the first pass finds all
occurences of a variable and determines whether the '%' prefix can be
included or not. The second pass does the actual substitution.
This change also unifies the generalization path for assembly with that
for IR and analysis, in the hope that any future changes avoid diverging
those cases future.
I also considered the alternative of trying to detect the phi node case
using more regular expression special cases but ultimately decided
against that because it seemed more fragile, and perhaps the approach of
keeping a tentative prefix that may later be discarded could also be
eventually applied to some metadata and attribute cases.
Note that an early version of this change was reviewed as
https://reviews.llvm.org/D142452, before version numbers were
introduced. This is a substantially updated version of that change.
Resubmitting this after previous revert with the following changes:
- Split table into table_rhs_idx and table_candidate_idx so that
bisect.bisect_left can be used without the `key` argument, which
was introduced in Python 3.10
- Remove a re.Pattern type annotation
Original commit message:
Prior to this change, running UTC on larger tests, especially tests
with unnamed IR values, often resulted in a spuriously large diff
because e.g. TMPnn variables in the CHECK lines were renumbered. This
change attempts to reduce the diff by keeping those variable names the
same.
There are cases in which this "drift" of variable names can end up being
more confusing. The old behavior can be re-enabled with the
--reset-variable-names command line argument.
The improvement may not be immediately apparent in the diff of this change.
The point is that the diff of stable_ir_values.ll against
stable_ir_values.ll.expected after this change is smaller.
Ideally, we'd also keep meta variables for "global" objects stable, e.g.
for attributes (#nn) and metadata (!nn). However, that would require a
much more substantial refactoring of how we generate check lines, so I
left it for future work.
Prior to this change, running UTC on larger tests, especially tests
with unnamed IR values, often resulted in a spuriously large diff
because e.g. TMPnn variables in the CHECK lines were renumbered. This
change attempts to reduce the diff by keeping those variable names the
same.
There are cases in which this "drift" of variable names can end up being
more confusing. The old behavior can be re-enabled with the
--reset-variable-names command line argument.
The improvement may not be immediately apparent in the diff of this change.
The point is that the diff of stable_ir_values.ll against
stable_ir_values.ll.expected after this change is smaller.
Ideally, we'd also keep meta variables for "global" objects stable, e.g.
for attributes (#nn) and metadata (!nn). However, that would require a
much more substantial refactoring of how we generate check lines, so I
left it for future work.
When removing only lines that are global value CHECK lines, a related
CHECK-SAME line could be left dangling without a previous line to belong
to.
Resolves#78517
Recommits the changes from https://reviews.llvm.org/D148216.
Explicitly named globals are now matched literally, instead of emitting
a capture group for the name. This resolves#70047.
Metadata and annotations, on the other hand, are captured and matched
against by default, since their identifiers are not stable.
The reasons for revert (#63746) have been fixed:
The first issue, that of duplicated checkers, has already been resolved
in #70050.
This PR resolves the second issue listed in #63746, regarding the order
of named and unnamed globals. This is fixed by recording the index of
substrings containing global values, and sorting the checks according to
that index before emitting them. This results in global value checks
being emitted in the order they were seen instead of being grouped
separately.
This prevents update_cc_tests.py from emitting hard-coded identifiers
for metadata (global variable checkers still check hard-coded
identifiers). Instead it emits regex checkers that match even if the
identifiers change. Also adds a new mode for --check-globals: instead of
simply being on or off, it now has the options 'none', 'smart' and
'all', with 'none' and 'all' corresponding to the previous modes.
The 'smart' mode only emits checks for global definitions referenced
in the IR or other metadata that itself has a definition checker
emitted, making the rule transitive. It does not emit checks for
attribute sets, since that is better checked by --check-attributes. This
mode is made the new default. To make the change in default mode
backwards compatible a version bump is introduced (to v3), and the
default remains 'none' in v1 & v2.
This will result in metadata checks being emitted more often, so filters
are added to not check absolute file paths and compiler version git
hashes.
rdar://105239218
If --function-signature is used with --version 2, then also include
the return type/attributes in the check lines. This is the
implementation of D133943 rebased on the --version mechanism from
D142473.
This doesn't bump the default version yet, because I'd like to do
that together with D140212 (which enables --function-signature by
default), as these changes seem closely related. For now this
functionality can be accessed by explicitly passing --version 2
to UTC.
Fixes https://github.com/llvm/llvm-project/issues/61058.
Differential Revision: https://reviews.llvm.org/D144963
It's not clear to me how to write a test for this. The tests run in an
environment where the tools may not be in PATH, and so the existing
custom-tool.test needs to use --tool-binary when invoking
update_test_checks.py. But it can't do so without also using --tool.
This change does fix a problem though with using
update_any_test_checks.py in an environment where the tools *are*
available in PATH.
Differential Revision: https://reviews.llvm.org/D142450
We have a downstream project with a command-line utility that operates
pretty much exactly like `opt`. So it would make sense for us to
maintain tests with update_test_checks.py with our custom tool
substituted for `opt`, as this change allows.
Differential Revision: https://reviews.llvm.org/D136329
There is at least one Clang test (clang/test/CodeGen/arm_acle.c) which
has functions guarded by #if's that cause those functions to be compiled
only for a subset of RUN lines.
This results in a case where one RUN line has a body for the function
and another doesn't. Treat this case as a conflict for any prefixes that
the two RUN lines have in common.
This change exposed a bug where functions with '$' in the name weren't
properly recognized in ARM assembly (despite there being a test case
that was supposed to catch the problem!). This bug is fixed as well.
Differential Revision: https://reviews.llvm.org/D130089
Manually modifying the result of update_test_checks.py is discouraged,
we prefer unmodified check lines where possible. The output is also
considered authoritative nowadays, at least for tests targeting core
middle-end components, where not using it is an automatic review
rejection.
Differential Revision: https://reviews.llvm.org/D129259
Re-add filtering options with fixes for failed tests. We were not passing the
is_filtered argument in all check generator calls in update_cc_test_checks.py
Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions. The --filter option will emit only tool output
lines matching the given regular expression while the --filter-out option will
emit only tools output lines not matching the given regular expression. Filters
are applied in order of appearance on the command line (or in UTC_ARGS) and the
first matching filter terminates the search.
This allows test authors to create more focused tests by removing irrelevant
tool output and checking only the pieces of output necessary to test the desired
functionality.
Differential Revision: https://reviews.llvm.org/D117694
Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions. The --filter option will emit only tool output
lines matching the given regular expression while the --filter-out option will
emit only tools output lines not matching the given regular expression. Filters
are applied in order of appearance on the command line (or in UTC_ARGS) and the
first matching filter terminates the search.
This allows test authors to create more focused tests by removing irrelevant
tool output and checking only the pieces of output necessary to test the desired
functionality.
Differential Revision: https://reviews.llvm.org/D117694
When checking if two prefixes can be merged for a function,
update_llc_test_checks.py removed IR comments before comparing
llc outputs of different RUN lines.
This means, if one RUN line emited lines starting with ';' and another
RUN line emited the same lines except the ones starting with ';', both
RUNs would be merged (if they share a prefix).
However, CHECK-NEXT lines check the comments, otherwise they fail, so
the script should not merge RUNs if they contain different comments.
Differential Revision: https://reviews.llvm.org/D101312
Now that FileCheck eagerly complains when prefixes are unused,
the update script does the same, and is becoming very common
to need to drop some prefixes, yet figuring out the file
it complains about isn't obvious unless it actually tells us.
This has been rather useful in our downstream CHERI target where we want
to run tests both with addrspace(0) and addrspace(200) pointers.
With this patch we can prefix the opt command with
`sed -e 's/addrspace(200)/addrspace(0)/g' -e 's/-A200-P200-G200//g'` to
test both cases using the same IR input.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D95137
This allows to check for various globals (metadata/attributes/...) and
also resolves problems with globals (metadata/attributes/...) being
reused across different prefixes.
Reviewed By: sstefan1
Differential Revision: https://reviews.llvm.org/D94741
This allows us to have shared logic over multiple test runs, e.g. do we
have unused prefixes, or which function bodies have conflicting outputs
for a prefix appearing in different RUN lines.
This patch is just wrapping existing functionality, and replacing its uses.
A subsequent patch would then fold the current functionality into the newly
introduced class.
Differential Revision: https://reviews.llvm.org/D93413
Add the --include-generated-funcs option to update_cc_test_checks.py so that any
functions created by the compiler that don't exist in the source will also be
checked.
We need to maintain the output order of generated function checks so that
CHECK-LABEL works properly. To do so, maintain a list of functions output for
each prefix in the order they are output. Use this list to output checks for
generated functions in the proper order.
Differential Revision: https://reviews.llvm.org/D83004
With this patch we will match most *uses* of "temporary" named things in
the IR via regular expressions, not their name at creation time. The new
"values" we match are:
- "unnamed" globals: `@[0-9]+`
- debug metadata: `!dbg ![0-9]+`
- loop metadata: `!loop ![0-9]+`
- tbaa metadata: `!tbaa ![0-9]+`
- range metadata: `!range ![0-9]+`
- generic metadata: `metadata ![0-9]+`
- attributes groups: `#[0-9]`
We still don't match the declarations but that can be done later. This
patch can introduce churn when existing check lines contain the old
hardcoded versions of the above "values". We can add a flag to opt-out,
or opt-in, if necessary.
Reviewed By: arichardson, MaskRay
Differential Revision: https://reviews.llvm.org/D85099
Summary:
This introduces new flag to the update_test_checks and
update_cc_test_checks that allows for function attributes
to be checked in a check-line. If the flag is not set,
the behavior should remain the same.
Reviewers: jdoerfert
Subscribers: arichardson, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D83629
I intend to reuse this to add UTC_ARGS support for update_llc_test_checks.py
and update_cc_test_checks.py in D78478.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D78618
Remove the local versions of the IR_FUNCTION_RE matcher (they weren't doing anything different), and ensure all the function name matchers accept '.' and '-'.
We don't need to use '\.' inside python regex sets either, or '\-' as long as thats at the end of the set.
Update test scripts were limited because they performed a single action
on the entire file and if that action was controlled by arguments, like
the one introduced in D68819, there was no record of it.
This patch introduces the capability of changing the arguments passed to
the script "on-the-fly" while processing a test file. In addition, an
"on/off" switch was added so that processing can be disabled for parts
of the file where the content is simply copied. The last extension is a
record of the invocation arguments in the auto generated NOTE. These
arguments are also picked up in a subsequent invocation, allowing
updates with special options enabled without user interaction.
To change the arguments the string `UTC_ARGS:` has to be present in a
line, followed by "additional command line arguments". That is
everything that follows `UTC_ARGS:` will be added to a growing list
of "command line arguments" which is reparsed after every update.
Reviewed By: arichardson
Differential Revision: https://reviews.llvm.org/D69701
Attribute annotations on calls, e.g., #0, are not useful on their own.
This patch adds a flag to update_test_checks.py to scrub them.
Reviewed By: lebedev.ri
Differential Revision: https://reviews.llvm.org/D68851
'python' means Python 2 on some platforms while Python 3 on others.
'python3' is Python 3 only. Python 2.7 End of Life is set to January 1,
2020. Getting rid of Python 2 support reduces maintenance burden.
Reviewed By: lebedev.ri
Differential Revision: https://reviews.llvm.org/D70730
Summary:
This commit also introduces a common.debug() function to avoid many
`if args.verbose:` statements. Depends on D70428.
Reviewers: xbolva00, MaskRay, jdoerfert
Reviewed By: MaskRay
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D70432
Summary:
Add a function common.parse_commandline_args() that adds options common
to all tools (--verbose and --update-only) and returns the parsed
commandline arguments. I plan to use the shared parsing of --verbose in a
follow-up commit to remove most of the `if args.verbose:` checks in the
scripts.
Reviewers: xbolva00, MaskRay
Reviewed By: MaskRay
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D70428
With D69701, the options used when running the script on a file will be
recorded and reused on a rerun. This allows us to hide new features
behind flags, starting with the "define" that was introduced in D68819.
Summary:
This adds a switch to the update_test_checks that triggers arguments and
other function annotations, e.g., personality, to be present in the
check line. If not set, the behavior should be the same as before.
If arguments are recorded, their names are scrubbed from the IR to allow
merging.
This patch includes D68153.
Reviewers: lebedev.ri, greened, spatel, xbolva00, RKSimon, mehdi_amini
Subscribers: bollu, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D68819
Add a --preserve-names option to tell the script not to replace IR names.
Sometimes tests want those names. For example if a test is looking for a
modification to an existing instruction we'll want to make the names.
Differential Revision: https://reviews.llvm.org/D68081
llvm-svn: 373912
D64572 / rL365818 changed the way that the file paths were collected, which meant we lost the file pattern expansion necessary when working with DOS command prompt
llvm-svn: 373062
Summary:
Port of new feature introduced https://reviews.llvm.org/D65610 to other update scripts.
- update_*_checks.py: add an alias -u for --update-only
- port --update-only to other update_*_test_checks.py scripts
- update script aborts if the test file was generated by another update_*_test_checks.py utility
Reviewers: lebedev.ri, RKSimon, MaskRay, reames, gbedwell
Reviewed By: MaskRay
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D65793
llvm-svn: 368174
Intended use case is:
./utils/update_test_checks.py test/Transform/PassDir/* --update-only
(i.e. rapidly be able to see changes in autogened filed, before handing non-autogened tests individually)
Differential Revision: https://reviews.llvm.org/D65610
llvm-svn: 367900
Summary:
The script is silent for the following issue:
FileCheck %s -check-prefix=CHECK,POPCOUNT
FileCheck will catch it later, but I think we can warn here too.
Now it warns:
./update_llc_test_checks.py file.ll
WARNING: Supplied prefix 'CHECK,POPCOUNT' is invalid. Prefix must contain only alphanumeric characters, hyphens and underscores. Did you mean --check-prefixes=CHECK,POPCOUNT?
Reviewers: lebedev.ri, spatel, RKSimon, craig.topper, nikic, gbedwell
Reviewed By: RKSimon
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64589
llvm-svn: 367244
Summary:
Recently I ran into the following issue:
./update_test_checks.py /path/not-existing-file.ll
The script was silent and I was suprised why the real test file hadn't been updated.
Solution:
Emit warning if we detect this problem.
Reviewers: lebedev.ri, spatel, jdoerfert, nikic
Reviewed By: lebedev.ri, spatel, jdoerfert, nikic
Subscribers: jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64572
llvm-svn: 365818
