mirrors/llvm-project
0
0
Fork 0
mirror of https://github.com/llvm/llvm-project.git synced 2025-05-15 06:16:10 +00:00
Code Issues Projects Releases Wiki Activity
llvm-project/llvm/utils/lit/tests/shtest-shell.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

600 lines
17 KiB
Python
Raw Normal View History

[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# Check the internal shell handling component of the ShTest format.
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# FIXME: this test depends on order of tests
# RUN: rm -f %{inputs}/shtest-shell/.lit_test_times.txt
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# RUN: not %{lit} -j 1 -v %{inputs}/shtest-shell > %t.out
[lit] Dump some FileCheck inputs to try to debug some failing tests llvm-svn: 309400
2017-07-28 16:24:18 +00:00
# FIXME: Temporarily dump test output so we can debug failing tests on
# buildbots.
# RUN: cat %t.out
Revert r374652: "[lit] Fix internal diff's --strip-trailing-cr and use it" This series of patches still breaks a Windows bot. llvm-svn: 374679
2019-10-12 18:51:51 +00:00
# RUN: FileCheck --input-file %t.out %s
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
#
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# Test again in non-UTF shell to catch potential errors with python 2 seen
# on stdout-encoding.txt
[lit] Sort test start times based on prior test timing data Lit as it exists today has three hacks that allow users to run tests earlier: 1) An entire test suite can set the `is_early` boolean. 2) A very recently introduced "early_tests" feature. 3) The `--incremental` flag forces failing tests to run first. All of these approaches have problems. 1) The `is_early` feature was until very recently undocumented. Nevertheless it still lacks testing and is a imprecise way of optimizing test starting times. 2) The `early_tests` feature requires manual updates and doesn't scale. 3) `--incremental` is undocumented, untested, and it requires modifying the *source* file system by "touching" the file. This "touch" based approach is arguably a hack because it confuses editors (because it looks like the test was modified behind the back of the editor) and "touching" the test source file doesn't work if the test suite is read only from the perspective of `lit` (via advanced filesystem/build tricks). This patch attempts to simplify and address all of the above problems. This patch formalizes, documents, tests, and defaults lit to recording the execution time of tests and then reordering all tests during the next execution. By reordering the tests, high core count machines run faster, sometimes significantly so. This patch also always runs failing tests first, which is a positive user experience win for those that didn't know about the hidden `--incremental` flag. Finally, if users want, they can _optionally_ commit the test timing data (or a subset thereof) back to the repository to accelerate bots and first-time runs of the test suite. Reviewed By: jhenderson, yln Differential Revision: https://reviews.llvm.org/D98179
2021-03-10 10:19:15 -05:00
# FIXME: lit's testing sets source_root == exec_root which complicates running lit more than once per test.
# RUN: rm -f %{inputs}/shtest-shell/.lit_test_times.txt
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# RUN: env PYTHONIOENCODING=ascii not %{lit} -j 1 -a %{inputs}/shtest-shell > %t.ascii.out
# FIXME: Temporarily dump test output so we can debug failing tests on
# buildbots.
# RUN: cat %t.ascii.out
# RUN: FileCheck --input-file %t.ascii.out %s
#
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# END.
# CHECK: -- Testing:
[lit] Implement 'cat' command for internal shell Fixes PR36449 Patch by Chamal de Silva Differential Revision: https://reviews.llvm.org/D43501 llvm-svn: 328563
2018-03-26 18:05:12 +00:00
# CHECK: FAIL: shtest-shell :: cat-error-0.txt
# CHECK: *** TEST 'shtest-shell :: cat-error-0.txt' FAILED ***
# CHECK: $ "cat" "-b" "temp1.txt"
# CHECK: # command stderr:
# CHECK: Unsupported: 'cat': option -b not recognized
# CHECK: error: command failed with exit status: 1
# CHECK: ***
# CHECK: FAIL: shtest-shell :: cat-error-1.txt
# CHECK: *** TEST 'shtest-shell :: cat-error-1.txt' FAILED ***
# CHECK: $ "cat" "temp1.txt"
# CHECK: # command stderr:
# CHECK: [Errno 2] No such file or directory: 'temp1.txt'
# CHECK: error: command failed with exit status: 1
# CHECK: ***
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
[lit] Report line number for failed RUN command (Relands r333584, reverted in 333592.) When debugging test failures with -vv (or -v in the case of the internal shell), this makes it easier to locate the RUN line that failed. For example, clang's test/Driver/linux-ld.c has 892 total RUN lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines after concatenation for line continuations. When reading the generated shell script, this also makes it easier to locate the RUN line that produced each command. To support reporting RUN line numbers in the case of the internal shell, this patch extends the internal shell to support the null command, ":", except pipelines are not supported. To support reporting RUN line numbers in the case of windows cmd.exe as the external shell, this patch extends -vv to set "echo on" instead of "echo off" in bat files. (Support for windows cmd.exe as a lit external shell will likely be dropped later, but I found out too late.) Reviewed By: delcypher, asmith, stella.stamenova, jmorse, lebedev.ri, rnk Differential Revision: https://reviews.llvm.org/D44598 llvm-svn: 333614
2018-05-31 00:55:32 +00:00
# CHECK: FAIL: shtest-shell :: colon-error.txt
# CHECK: *** TEST 'shtest-shell :: colon-error.txt' FAILED ***
# CHECK: $ ":"
# CHECK: # command stderr:
# CHECK: Unsupported: ':' cannot be part of a pipeline
# CHECK: error: command failed with exit status: 127
# CHECK: ***
[lit] Fix internal diff newlines for -w/-b For example, without this patch: ``` $ python $LIT_BUILTINS/diff.py -b foo.txt bar.txt *** /tmp/foo.txt --- /tmp/bar.txt *************** *** 1,2 **** 1! 2--- 1,2 ---- 1! 20 ``` With this patch: ``` $ python $LIT_BUILTINS/diff.py -b foo.txt bar.txt *** /tmp/foo.txt --- /tmp/bar.txt *************** *** 1,2 **** 1 ! 2 --- 1,2 ---- 1 ! 20 ``` Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D71577
2019-12-17 10:23:11 -05:00
# CHECK: PASS: shtest-shell :: dev-null.txt
# CHECK: FAIL: shtest-shell :: diff-b.txt
# CHECK: *** TEST 'shtest-shell :: diff-b.txt' FAILED ***
# CHECK: $ "diff" "-b" "{{[^"]*}}.0" "{{[^"]*}}.1"
# CHECK: # command output:
# CHECK: 1,2
# CHECK-NEXT: {{^ }}f o o
# CHECK-NEXT: ! b a r
# CHECK-NEXT: ---
# CHECK-NEXT: {{^ }}f o o
# CHECK-NEXT: ! bar
# CHECK-EMPTY:
# CHECK: error: command failed with exit status: 1
# CHECK: ***
[lit] Clean up internal diff's encoding handling As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For Python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Avoid failures for Python 2.7 used on some Windows bots by transforming diff output with `lit.util.to_string` before writing it to stdout. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 375018
2019-10-16 17:21:24 +00:00
# CHECK: FAIL: shtest-shell :: diff-encodings.txt
# CHECK: *** TEST 'shtest-shell :: diff-encodings.txt' FAILED ***
# CHECK: $ "diff" "-u" "diff-in.bin" "diff-in.bin"
# CHECK-NOT: error
# CHECK: $ "diff" "-u" "diff-in.utf16" "diff-in.bin"
# CHECK: # command output:
# CHECK-NEXT: ---
# CHECK-NEXT: +++
# CHECK-NEXT: @@
# CHECK-NEXT: {{^ .f.o.o.$}}
# CHECK-NEXT: {{^-.b.a.r.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^\+.b.a.r.}}
[lit] Clean up internal diff's encoding handling As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For Python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Avoid failures for Python 2.7 used on some Windows bots by transforming diff output with `lit.util.to_string` before writing it to stdout. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 375018
2019-10-16 17:21:24 +00:00
# CHECK-NEXT: {{^ .b.a.z.$}}
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-u" "diff-in.utf8" "diff-in.bin"
# CHECK: # command output:
# CHECK-NEXT: ---
# CHECK-NEXT: +++
# CHECK-NEXT: @@
# CHECK-NEXT: -foo
# CHECK-NEXT: -bar
# CHECK-NEXT: -baz
# CHECK-NEXT: {{^\+.f.o.o.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^\+.b.a.r.}}
[lit] Clean up internal diff's encoding handling As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For Python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Avoid failures for Python 2.7 used on some Windows bots by transforming diff output with `lit.util.to_string` before writing it to stdout. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 375018
2019-10-16 17:21:24 +00:00
# CHECK-NEXT: {{^\+.b.a.z.$}}
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-u" "diff-in.bin" "diff-in.utf8"
# CHECK: # command output:
# CHECK-NEXT: ---
# CHECK-NEXT: +++
# CHECK-NEXT: @@
# CHECK-NEXT: {{^\-.f.o.o.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^\-.b.a.r.}}
[lit] Clean up internal diff's encoding handling As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For Python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Avoid failures for Python 2.7 used on some Windows bots by transforming diff output with `lit.util.to_string` before writing it to stdout. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 375018
2019-10-16 17:21:24 +00:00
# CHECK-NEXT: {{^\-.b.a.z.$}}
# CHECK-NEXT: +foo
# CHECK-NEXT: +bar
# CHECK-NEXT: +baz
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
[lit] Extend internal diff to support `-` argument When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-17 14:03:06 +00:00
# CHECK: $ "cat" "diff-in.bin"
# CHECK-NOT: error
# CHECK: $ "diff" "-u" "-" "diff-in.bin"
# CHECK-NOT: error
# CHECK: $ "cat" "diff-in.bin"
# CHECK-NOT: error
# CHECK: $ "diff" "-u" "diff-in.bin" "-"
# CHECK-NOT: error
# CHECK: $ "cat" "diff-in.bin"
# CHECK-NOT: error
# CHECK: $ "diff" "-u" "diff-in.utf16" "-"
# CHECK: # command output:
# CHECK-NEXT: ---
# CHECK-NEXT: +++
# CHECK-NEXT: @@
# CHECK-NEXT: {{^ .f.o.o.$}}
# CHECK-NEXT: {{^-.b.a.r.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^\+.b.a.r.}}
[lit] Extend internal diff to support `-` argument When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-17 14:03:06 +00:00
# CHECK-NEXT: {{^ .b.a.z.$}}
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "cat" "diff-in.bin"
# CHECK-NOT: error
# CHECK: $ "diff" "-u" "diff-in.utf8" "-"
# CHECK: # command output:
# CHECK-NEXT: ---
# CHECK-NEXT: +++
# CHECK-NEXT: @@
# CHECK-NEXT: -foo
# CHECK-NEXT: -bar
# CHECK-NEXT: -baz
# CHECK-NEXT: {{^\+.f.o.o.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^\+.b.a.r.}}
[lit] Extend internal diff to support `-` argument When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-17 14:03:06 +00:00
# CHECK-NEXT: {{^\+.b.a.z.$}}
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-u" "-" "diff-in.utf8"
# CHECK: # command output:
# CHECK-NEXT: ---
# CHECK-NEXT: +++
# CHECK-NEXT: @@
# CHECK-NEXT: {{^\-.f.o.o.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^\-.b.a.r.}}
[lit] Extend internal diff to support `-` argument When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-17 14:03:06 +00:00
# CHECK-NEXT: {{^\-.b.a.z.$}}
# CHECK-NEXT: +foo
# CHECK-NEXT: +bar
# CHECK-NEXT: +baz
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
[lit] Clean up internal diff's encoding handling As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For Python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Avoid failures for Python 2.7 used on some Windows bots by transforming diff output with `lit.util.to_string` before writing it to stdout. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 375018
2019-10-16 17:21:24 +00:00
# CHECK: $ "false"
# CHECK: ***
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: FAIL: shtest-shell :: diff-error-1.txt
# CHECK: *** TEST 'shtest-shell :: diff-error-1.txt' FAILED ***
# CHECK: $ "diff" "-B" "temp1.txt" "temp2.txt"
# CHECK: # command stderr:
# CHECK: Unsupported: 'diff': option -B not recognized
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
# CHECK: error: command failed with exit status: 1
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: ***
# CHECK: FAIL: shtest-shell :: diff-error-2.txt
# CHECK: *** TEST 'shtest-shell :: diff-error-2.txt' FAILED ***
# CHECK: $ "diff" "temp.txt"
# CHECK: # command stderr:
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
# CHECK: Error: missing or extra operand
# CHECK: error: command failed with exit status: 1
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: ***
# CHECK: FAIL: shtest-shell :: diff-error-3.txt
# CHECK: *** TEST 'shtest-shell :: diff-error-3.txt' FAILED ***
# CHECK: $ "diff" "temp.txt" "temp1.txt"
# CHECK: # command stderr:
# CHECK: Error: 'diff' command failed
# CHECK: error: command failed with exit status: 1
# CHECK: ***
# CHECK: FAIL: shtest-shell :: diff-error-4.txt
# CHECK: *** TEST 'shtest-shell :: diff-error-4.txt' FAILED ***
# CHECK: Exit Code: 1
# CHECK: # command output:
# CHECK: diff-error-4.txt.tmp
# CHECK: diff-error-4.txt.tmp1
# CHECK: *** 1 ****
# CHECK: ! hello-first
# CHECK: --- 1 ----
# CHECK: ! hello-second
# CHECK: ***
# CHECK: FAIL: shtest-shell :: diff-error-5.txt
# CHECK: *** TEST 'shtest-shell :: diff-error-5.txt' FAILED ***
# CHECK: $ "diff"
# CHECK: # command stderr:
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
# CHECK: Error: missing or extra operand
# CHECK: error: command failed with exit status: 1
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: ***
# CHECK: FAIL: shtest-shell :: diff-error-6.txt
# CHECK: *** TEST 'shtest-shell :: diff-error-6.txt' FAILED ***
# CHECK: $ "diff"
# CHECK: # command stderr:
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
# CHECK: Error: missing or extra operand
# CHECK: error: command failed with exit status: 1
# CHECK: ***
# CHECK: FAIL: shtest-shell :: diff-pipes.txt
# CHECK: *** TEST 'shtest-shell :: diff-pipes.txt' FAILED ***
# CHECK: $ "diff" "{{[^"]*}}.foo" "{{[^"]*}}.foo"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "FileCheck"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "diff" "-u" "{{[^"]*}}.foo" "{{[^"]*}}.bar"
# CHECK: note: command had no output on stdout or stderr
# CHECK: error: command failed with exit status: 1
# CHECK: $ "FileCheck"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "true"
[lit] Extend internal diff to support `-` argument When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-17 14:03:06 +00:00
# CHECK: $ "cat" "{{[^"]*}}.foo"
# CHECK: $ "diff" "-u" "-" "{{[^"]*}}.foo"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "cat" "{{[^"]*}}.foo"
# CHECK: $ "diff" "-u" "{{[^"]*}}.foo" "-"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "cat" "{{[^"]*}}.bar"
# CHECK: $ "diff" "-u" "{{[^"]*}}.foo" "-"
# CHECK: # command output:
# CHECK: @@
# CHECK-NEXT: -foo
# CHECK-NEXT: +bar
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "cat" "{{[^"]*}}.bar"
# CHECK: $ "diff" "-u" "-" "{{[^"]*}}.foo"
# CHECK: # command output:
# CHECK: @@
# CHECK-NEXT: -bar
# CHECK-NEXT: +foo
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "cat" "{{[^"]*}}.foo"
# CHECK: $ "diff" "-" "{{[^"]*}}.foo"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "FileCheck"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "cat" "{{[^"]*}}.bar"
# CHECK: $ "diff" "-u" "{{[^"]*}}.foo" "-"
# CHECK: note: command had no output on stdout or stderr
# CHECK: error: command failed with exit status: 1
# CHECK: $ "FileCheck"
# CHECK-NOT: note
# CHECK-NOT: error
# CHECK: $ "true"
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
# CHECK: $ "false"
Reland r374388: [lit] Make internal diff work in pipelines To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D66574 llvm-svn: 374648
2019-10-12 11:56:57 +00:00
# CHECK: ***
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: FAIL: shtest-shell :: diff-r-error-0.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-0.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: Only in {{.*}}dir1: dir1unique
# CHECK: Only in {{.*}}dir2: dir2unique
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-1.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-1.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: *** {{.*}}dir1{{.*}}subdir{{.*}}f01
# CHECK: --- {{.*}}dir2{{.*}}subdir{{.*}}f01
# CHECK: 12345
# CHECK: 00000
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-2.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-2.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: Only in {{.*}}dir2: extrafile
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-3.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-3.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: Only in {{.*}}dir1: extra_subdir
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-4.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-4.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: File {{.*}}dir1{{.*}}extra_subdir is a directory while file {{.*}}dir2{{.*}}extra_subdir is a regular file
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-5.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-5.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: Only in {{.*}}dir1: extra_subdir
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-6.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-6.txt' FAILED ***
[NFCI][lit] Unbreak more lit self-tests after D98179 All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-22 14:44:30 +03:00
# CHECK: $ "diff" "-r"
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: # command output:
# CHECK: File {{.*}}dir1{{.*}}extra_file is a regular empty file while file {{.*}}dir2{{.*}}extra_file is a directory
# CHECK: error: command failed with exit status: 1
[lit] Extend internal diff to support `-` argument When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-17 14:03:06 +00:00
# CHECK: FAIL: shtest-shell :: diff-r-error-7.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-7.txt' FAILED ***
# CHECK: $ "diff" "-r" "-" "{{[^"]*}}"
# CHECK: # command stderr:
# CHECK: Error: cannot recursively compare '-'
# CHECK: error: command failed with exit status: 1
# CHECK: FAIL: shtest-shell :: diff-r-error-8.txt
# CHECK: *** TEST 'shtest-shell :: diff-r-error-8.txt' FAILED ***
# CHECK: $ "diff" "-r" "{{[^"]*}}" "-"
# CHECK: # command stderr:
# CHECK: Error: cannot recursively compare '-'
# CHECK: error: command failed with exit status: 1
[lit] Implement "-r" option for builtin "diff" command + a test using that. Summary: That would allow to recursively compare directories in tests using "diff -r" on Windows in a similar way as it can be done on Linux or Mac. Reviewers: zturner, morehouse, vsk Reviewed By: zturner Subscribers: kcc, llvm-commits Differential Revision: https://reviews.llvm.org/D41776 llvm-svn: 322102
2018-01-09 18:23:34 +00:00
# CHECK: PASS: shtest-shell :: diff-r.txt
[lit] Fix internal diff's --strip-trailing-cr and use it Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before a `\n` at the end of a line. Without this patch, lit's internal diff only removes `\r` if it appears as the last character. That seems useless. This patch fixes that. This patch also adds `--strip-trailing-cr` to some tests that fail on Windows bots when D68664 is applied. Based on what I see in the bot logs, I think the following is happening. In each test there, lit diff is comparing a file with `\r\n` line endings to a file with `\n` line endings. Without D68664, lit diff reads those files in text mode, which in Windows causes `\r\n` to be replaced with `\n`. However, with D68664, lit diff reads the files in binary mode instead and thus reports that every line is different, just as GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr` to those tests restores the previous behavior while permitting the behavior of lit diff to be more like GNU diff. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68839 llvm-svn: 375020
2019-10-16 17:21:57 +00:00
# CHECK: FAIL: shtest-shell :: diff-strip-trailing-cr.txt
# CHECK: *** TEST 'shtest-shell :: diff-strip-trailing-cr.txt' FAILED ***
# CHECK: $ "diff" "-u" "diff-in.dos" "diff-in.unix"
# CHECK: # command output:
# CHECK: @@
# CHECK-NEXT: -In this file, the
# CHECK-NEXT: -sequence "\r\n"
# CHECK-NEXT: -terminates lines.
# CHECK-NEXT: +In this file, the
# CHECK-NEXT: +sequence "\n"
# CHECK-NEXT: +terminates lines.
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-u" "diff-in.unix" "diff-in.dos"
# CHECK: # command output:
# CHECK: @@
# CHECK-NEXT: -In this file, the
# CHECK-NEXT: -sequence "\n"
# CHECK-NEXT: -terminates lines.
# CHECK-NEXT: +In this file, the
# CHECK-NEXT: +sequence "\r\n"
# CHECK-NEXT: +terminates lines.
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-u" "--strip-trailing-cr" "diff-in.dos" "diff-in.unix"
# CHECK: # command output:
# CHECK: @@
# CHECK-NEXT: In this file, the
# CHECK-NEXT: -sequence "\r\n"
# CHECK-NEXT: +sequence "\n"
# CHECK-NEXT: terminates lines.
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-u" "--strip-trailing-cr" "diff-in.unix" "diff-in.dos"
# CHECK: # command output:
# CHECK: @@
# CHECK-NEXT: In this file, the
# CHECK-NEXT: -sequence "\n"
# CHECK-NEXT: +sequence "\r\n"
# CHECK-NEXT: terminates lines.
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "false"
# CHECK: ***
[lit] Extend internal diff to support -U When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff -U1 file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-U` as a command-line option. This patch adds `-U` support. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68668 llvm-svn: 374814
2019-10-14 19:59:30 +00:00
# CHECK: FAIL: shtest-shell :: diff-unified.txt
# CHECK: *** TEST 'shtest-shell :: diff-unified.txt' FAILED ***
# CHECK: $ "diff" "-u" "{{[^"]*}}.foo" "{{[^"]*}}.bar"
# CHECK: # command output:
# CHECK: @@ {{.*}} @@
# CHECK-NEXT: 3
# CHECK-NEXT: 4
# CHECK-NEXT: 5
# CHECK-NEXT: -6 foo
# CHECK-NEXT: +6 bar
# CHECK-NEXT: 7
# CHECK-NEXT: 8
# CHECK-NEXT: 9
# CHECK-EMPTY:
# CHECK-NEXT: error: command failed with exit status: 1
# CHECK-NEXT: $ "true"
# CHECK: $ "diff" "-U" "2" "{{[^"]*}}.foo" "{{[^"]*}}.bar"
# CHECK: # command output:
# CHECK: @@ {{.*}} @@
# CHECK-NEXT: 4
# CHECK-NEXT: 5
# CHECK-NEXT: -6 foo
# CHECK-NEXT: +6 bar
# CHECK-NEXT: 7
# CHECK-NEXT: 8
# CHECK-EMPTY:
# CHECK-NEXT: error: command failed with exit status: 1
# CHECK-NEXT: $ "true"
# CHECK: $ "diff" "-U4" "{{[^"]*}}.foo" "{{[^"]*}}.bar"
# CHECK: # command output:
# CHECK: @@ {{.*}} @@
# CHECK-NEXT: 2
# CHECK-NEXT: 3
# CHECK-NEXT: 4
# CHECK-NEXT: 5
# CHECK-NEXT: -6 foo
# CHECK-NEXT: +6 bar
# CHECK-NEXT: 7
# CHECK-NEXT: 8
# CHECK-NEXT: 9
# CHECK-NEXT: 10
# CHECK-EMPTY:
# CHECK-NEXT: error: command failed with exit status: 1
# CHECK-NEXT: $ "true"
# CHECK: $ "diff" "-U0" "{{[^"]*}}.foo" "{{[^"]*}}.bar"
# CHECK: # command output:
# CHECK: @@ {{.*}} @@
# CHECK-NEXT: -6 foo
# CHECK-NEXT: +6 bar
# CHECK-EMPTY:
# CHECK-NEXT: error: command failed with exit status: 1
# CHECK-NEXT: $ "true"
[lit] Make internal diff work in pipelines When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-17 14:02:42 +00:00
# CHECK: $ "diff" "-U" "30.1" "{{[^"]*}}" "{{[^"]*}}"
# CHECK: # command stderr:
# CHECK: Error: invalid '-U' argument: 30.1
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
# CHECK: $ "diff" "-U-1" "{{[^"]*}}" "{{[^"]*}}"
# CHECK: # command stderr:
# CHECK: Error: invalid '-U' argument: -1
# CHECK: error: command failed with exit status: 1
# CHECK: $ "true"
[lit] Extend internal diff to support -U When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff -U1 file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-U` as a command-line option. This patch adds `-U` support. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68668 llvm-svn: 374814
2019-10-14 19:59:30 +00:00
# CHECK: $ "false"
# CHECK: ***
[lit] Fix internal diff newlines for -w/-b For example, without this patch: ``` $ python $LIT_BUILTINS/diff.py -b foo.txt bar.txt *** /tmp/foo.txt --- /tmp/bar.txt *************** *** 1,2 **** 1! 2--- 1,2 ---- 1! 20 ``` With this patch: ``` $ python $LIT_BUILTINS/diff.py -b foo.txt bar.txt *** /tmp/foo.txt --- /tmp/bar.txt *************** *** 1,2 **** 1 ! 2 --- 1,2 ---- 1 ! 20 ``` Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D71577
2019-12-17 10:23:11 -05:00
# CHECK: FAIL: shtest-shell :: diff-w.txt
# CHECK: *** TEST 'shtest-shell :: diff-w.txt' FAILED ***
# CHECK: $ "diff" "-w" "{{[^"]*}}.0" "{{[^"]*}}.1"
# CHECK: # command output:
# CHECK: 1,3
# CHECK-NEXT: {{^ }}foo
# CHECK-NEXT: {{^ }}bar
# CHECK-NEXT: ! baz
# CHECK-NEXT: ---
# CHECK-NEXT: {{^ }}foo
# CHECK-NEXT: {{^ }}bar
# CHECK-NEXT: ! bat
# CHECK-EMPTY:
# CHECK: error: command failed with exit status: 1
# CHECK: ***
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# CHECK: FAIL: shtest-shell :: error-0.txt
# CHECK: *** TEST 'shtest-shell :: error-0.txt' FAILED ***
[lit] Improve readability of failing scripts. - This only applies to scripts executed by the _internal_ shell script interpreter. - This patch reworks the log to look more like a shell transcript, and be less verbose (but in the interest of calling attention to the important parts). Here is an example of the new format, for commands with/without failures and with/without output: ``` $ true $ echo hi hi $ false note: command had no output on stdout or stderr error: command failed with exit status 1 ``` llvm-svn: 271610
2016-06-02 23:49:42 +00:00
# CHECK: $ "not-a-real-command"
# CHECK: # command stderr:
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# CHECK: 'not-a-real-command': command not found
[lit] Improve readability of failing scripts. - This only applies to scripts executed by the _internal_ shell script interpreter. - This patch reworks the log to look more like a shell transcript, and be less verbose (but in the interest of calling attention to the important parts). Here is an example of the new format, for commands with/without failures and with/without output: ``` $ true $ echo hi hi $ false note: command had no output on stdout or stderr error: command failed with exit status 1 ``` llvm-svn: 271610
2016-06-02 23:49:42 +00:00
# CHECK: error: command failed with exit status: 127
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# CHECK: ***
# FIXME: The output here sucks.
#
# CHECK: FAIL: shtest-shell :: error-1.txt
# CHECK: *** TEST 'shtest-shell :: error-1.txt' FAILED ***
[lit] Terminate ": RUN at line N" with ";" not "&&" This fixes projects/compiler-rt/test/fuzzer/sigusr.test, which was broken by r333614. The trouble was that "&&" changes the command for which "$!" gives the pid. llvm-svn: 333620
2018-05-31 03:40:37 +00:00
# CHECK: shell parser error on: ': \'RUN: at line 3\'; echo "missing quote'
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# CHECK: ***
# CHECK: FAIL: shtest-shell :: error-2.txt
# CHECK: *** TEST 'shtest-shell :: error-2.txt' FAILED ***
# CHECK: Unsupported redirect:
# CHECK: ***
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: FAIL: shtest-shell :: mkdir-error-0.txt
# CHECK: *** TEST 'shtest-shell :: mkdir-error-0.txt' FAILED ***
# CHECK: $ "mkdir" "-p" "temp"
# CHECK: # command stderr:
# CHECK: Unsupported: 'mkdir' cannot be part of a pipeline
# CHECK: error: command failed with exit status: 127
# CHECK: ***
# CHECK: FAIL: shtest-shell :: mkdir-error-1.txt
# CHECK: *** TEST 'shtest-shell :: mkdir-error-1.txt' FAILED ***
# CHECK: $ "mkdir" "-p" "-m" "777" "temp"
# CHECK: # command stderr:
# CHECK: Unsupported: 'mkdir': option -m not recognized
# CHECK: error: command failed with exit status: 127
# CHECK: ***
# CHECK: FAIL: shtest-shell :: mkdir-error-2.txt
# CHECK: *** TEST 'shtest-shell :: mkdir-error-2.txt' FAILED ***
# CHECK: $ "mkdir" "-p"
# CHECK: # command stderr:
# CHECK: Error: 'mkdir' is missing an operand
# CHECK: error: command failed with exit status: 127
# CHECK: ***
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# CHECK: PASS: shtest-shell :: redirects.txt
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: FAIL: shtest-shell :: rm-error-0.txt
# CHECK: *** TEST 'shtest-shell :: rm-error-0.txt' FAILED ***
# CHECK: $ "rm" "-rf" "temp"
# CHECK: # command stderr:
# CHECK: Unsupported: 'rm' cannot be part of a pipeline
# CHECK: error: command failed with exit status: 127
# CHECK: ***
# CHECK: FAIL: shtest-shell :: rm-error-1.txt
# CHECK: *** TEST 'shtest-shell :: rm-error-1.txt' FAILED ***
# CHECK: $ "rm" "-f" "-v" "temp"
# CHECK: # command stderr:
# CHECK: Unsupported: 'rm': option -v not recognized
# CHECK: error: command failed with exit status: 127
# CHECK: ***
# CHECK: FAIL: shtest-shell :: rm-error-2.txt
# CHECK: *** TEST 'shtest-shell :: rm-error-2.txt' FAILED ***
# CHECK: $ "rm" "-r" "hello"
# CHECK: # command stderr:
# CHECK: Error: 'rm' command failed
# CHECK: error: command failed with exit status: 1
# CHECK: ***
# CHECK: FAIL: shtest-shell :: rm-error-3.txt
# CHECK: *** TEST 'shtest-shell :: rm-error-3.txt' FAILED ***
# CHECK: Exit Code: 1
# CHECK: ***
Add Support for Creating and Deleting Unicode Files and Directories in Lit This enables lit to work with unicode file names via mkdir, rm, and redirection. Lit still uses utf-8 internally, but converts to utf-16 on Windows, or just utf-8 bytes on everything else. Committed on behalf of Jason Mittertreiner Differential Revision: https://reviews.llvm.org/D56754 llvm-svn: 355122
2019-02-28 19:16:17 +00:00
# CHECK: PASS: shtest-shell :: rm-unicode-0.txt
[lit] Add a test for internal shell execution behaviors. llvm-svn: 174102
2013-01-31 22:15:20 +00:00
# CHECK: PASS: shtest-shell :: sequencing-0.txt
# CHECK: XFAIL: shtest-shell :: sequencing-1.txt
[lit] Don't fail when printing test output with special chars This addresses a UnicodeEncodeError when using Python 3.6.5 in Windows 10. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D69207
2019-10-18 12:53:45 -04:00
# CHECK: FAIL: shtest-shell :: stdout-encoding.txt
# CHECK: *** TEST 'shtest-shell :: stdout-encoding.txt' FAILED ***
# CHECK: $ "cat" "diff-in.bin"
# CHECK: # command output:
# CHECK-NEXT: {{^.f.o.o.$}}
[lit] Prevent hang when lit sees non-ASCII characters As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-07-14 18:41:05 +01:00
# CHECK-NEXT: {{^.b.a.r.}}
[lit] Don't fail when printing test output with special chars This addresses a UnicodeEncodeError when using Python 3.6.5 in Windows 10. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D69207
2019-10-18 12:53:45 -04:00
# CHECK-NEXT: {{^.b.a.z.$}}
# CHECK-NOT: error
# CHECK: $ "false"
# CHECK: ***
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 llvm-svn: 319528
2017-12-01 09:54:27 +00:00
# CHECK: PASS: shtest-shell :: valid-shell.txt
[lit] Improve naming of test result categories Improve consistency when printing test results: Previously we were using different labels for group names (the header for the list of, e.g., failing tests) and summary count lines. For example, "Failing Tests"/"Unexpected Failures". This commit changes lit to label things consistently. Improve wording of labels: When talking about individual test results, the first word in "Unexpected Failures", "Expected Passes", and "Individual Timeouts" is superfluous. Some labels contain the word "Tests" and some don't. Let's simplify the names. Before: ``` Failing Tests (1): ... Expected Passes : 3 Unexpected Failures: 1 ``` After: ``` Failed Tests (1): ... Passed: 3 Failed: 1 ``` Reviewed By: ldionne Differential Revision: https://reviews.llvm.org/D77708
2020-04-07 22:48:39 -07:00
# CHECK: Failed Tests (35)
Reference in New Issue Copy Permalink