This is an ongoing series of commits that are reformatting our
Python code.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: #libc, kwk, Mordante
Differential Revision: https://reviews.llvm.org/D150763
On Windows, the PATH env variable is used for locating dynamically
linked librarys, akin to LD_LIBRARY_PATH on Linux.
The tests that run with a dynamically linked libc++ used "--env
PATH=%{lib}" in the test config. This had the unfortunate side effect
of making other tools from PATH unavailable during the runtime of the
tests; in particular, it caused the "executor-has-no-bash" flag to be
set for all those Windows test configs (with the clang-cl static config
being the only one lacking it).
Thus, this increases the number of tests actually included in the
clang-cl dll and all mingw test configs by 9 tests.
The clang-cl static test configuration has been executing those tests
since the "--env PATH=%{lib}" was removed from that test config in
e78223e79efc886ef6f0ea5413deab3737d6d63b. (For mingw we haven't had a
need to split the test config between shared and static, which means
that the mingw static test config previously ran with --env PATH
needlessly.)
This increases the test coverage for patches like D146398 which
can't be executed in the executor-has-no-bash configs.
Change the default value of the arg.env to an empty array; when we do
pass values to the option, they get passed as an array of strings,
so make sure the variable behaves consistently when no arguments
have been passed.
Differential Revision: https://reviews.llvm.org/D148324
No test actually does this, but this makes the option behave like
the corresponding one in run.py.
This was broken by commit b8b23aa80eefe84187d6ba364d06496c90c53bdb
(https://reviews.llvm.org/D99242) which introduced quoting; instead
of quoting the whole space separated list, quote each individual
argument.
Differential Revision: https://reviews.llvm.org/D149319
We don't use Python 2 anymore, so let us do the recommended fix instead
of using the workaround made for Python 2.
Differential Revision: https://reviews.llvm.org/D107715
This safeguards against cases if some of the env vars contain chars
that are problematic for shells, e.g. if called with --env "X=Y;Z".
(In cases of cross testing for windows, the PATH variable can end up
specified with semicolon separators - even if specifying a PATH when
cross testing in such differing environments might not make sense or
do anything - but this makes ssh.py not break on such a variable.)
Differential Revision: https://reviews.llvm.org/D99242
If cross testing on Windows via WSL (at least with WSL 1), the Windows
executables can't be executed if they are in WSL specific directories
(like /tmp).
Differential Revision: https://reviews.llvm.org/D98028
In our CHERI Jenkins CI we need to pass `-F <custom_config_file>` to each
ssh/scp command to set various arguments such as the localhost port, usage
of controlmaster, etc. to speed up connections to our emulated QEMU systems.
For our specific use-case I could have also added a single --ssh-config-file
argument that can be used for both the scp and ssh commands, but being able
to pass arbitrary extra flags for both commands seems more flexible.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D84097
Adding a positional argparse.ONE_OR_MORE arguments will correctly remove
the "--" separator after --env and parse only the command. This also has
the advantage that misspelled flags raise an argparse error rather than
silently being added to the command to be executed.
I discovered this while adding a new commandline option to ssh.py to allow
passing additional arguments to the scp/ssh commands since this is required
for our CHERI CI where we need to pass `-F <custom_config_file>` to each
ssh/scp command to set various arguments such as the localhost port, usage
of controlmaster, etc. to speed up connections to our emulated QEMU systems.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D84096
Instead of passing file dependencies individually, assume that the
whole content of the unique test directory is a dependency. This
simplifies the test harness significantly, by making %T the directory
that contains everything required to run a test. This also removes the
need for the %{file_dependencies} substitution, which is removed by this
patch.
Furthermore, this patch also changes the harness to execute tests locally
inside %T, so as to avoid creating a separate directory for no purpose.
Running `export` when there is no environment variable to export will
cause the environment on the remote host to be printed. We don't want
that, so don't run any `export` command on the host when there's no env.
This was originally committed as f8452ddfcc33 and reverted in 7cb1aa9d9368.
The issue was that shell builtins were being escaped too, and apparently
Bash won't execute a builtin when it is quoted e.g. '!'. Instead, it
thinks it's a command and it can't find it.
Re-committing the change with that issue fixed.
On Windows, we must make sure to close the temporary tar file before we
try to scp it.
This is an alternative approach to https://reviews.llvm.org/D77500.
The benefit of doing this is that we can now handle directories that
contain symlinks and other arbitrary things, such as the static_test_env
required by filesystem tests.
As a fly-by fix, we also accumulate several commands to perform over SSH
and execute them at once instead of SSHing several times. This should be
faster on average.
If a ShTest has for example another command in front of the test
executable it wants to execute, ssh.py needs to properly translate
the path of that test executable to the executable on the remote host.
For example, running '%{exec} ! %t.exe', we can't assume that the
test-executable is the first argument after '%{exec}'.
