This patch adds the ability to get a thread at a give index, based on
insertion order, for SBSaveCore Options. This is primarily to benefit
scripts using SBSaveCore, and remove the need to have both options and a
second collection if your script is tracking what threads need to be
saved. Such as if you want to collect the source of all the threads to
be saved after the Core is generated.
Introduces a `member` property to `SBValue`. This property provides pythonic access to a
value's members, by name. The expression `value.member["name"]` will be an alternate
form form of writing `value.GetChildMemberWithName("name")`.
This PR fixes a simple SWIG issue with SBMemoryRegionInfoList not being
iterable out-of-the-box. This is mostly because of limitations to the
`lldb_iter` function, which doesn't allow for specifying arguments to
the size / iter functions passed.
Before:
```
(lldb) script
Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.
>>> for region in lldb.process.GetMemoryRegions():
... print(region)
...
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/opt/llvm/stable/Toolchains/llvm-sand.xctoolchain/usr/lib/python3.10/site-packages/lldb/__init__.py", line 114, in lldb_iter
yield elem(i)
TypeError: SBMemoryRegionInfoList.GetMemoryRegionAtIndex() missing 1 required positional argument: 'region_info'
```
After:
```
(lldb) script
Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.
>>> for region in lldb.process.GetMemoryRegions():
... print(region)
...
[0x0000000000200000-0x00000000002cf000 R--]
[0x00000000002cf000-0x0000000000597000 R-X]
[0x0000000000597000-0x00000000005ad000 R--]
[0x00000000005ad000-0x00000000005b1000 RW-]
[0x00000000005b1000-0x0000000000b68000 RW-]
[0x000000007fff7000-0x000000008fff7000 RW-]
[0x000002008fff7000-0x000010007fff8000 RW-]
[0x0000503000000000-0x0000503000010000 RW-]
[0x0000503e00000000-0x0000503e00010000 RW-]
[0x0000504000000000-0x0000504000010000 RW-]
[0x0000504e00000000-0x0000504e00010000 RW-]
[0x000050d000000000-0x000050d000010000 RW-]
[0x000050de00000000-0x000050de00010000 RW-]
[0x000050e000000000-0x000050e000010000 RW-]
[0x000050ee00000000-0x000050ee00010000 RW-]
[0x0000511000000000-0x0000511000010000 RW-]
[0x0000511e00000000-0x0000511e00010000 RW-]
[0x0000513000000000-0x0000513000010000 RW-]
...
```
The tests were using the variable directly to get the dwarf version used
for the test. That's only the overridden value, and won't be set if
we're using the compiler default. I also put a comment by the variable
to make sure people don't make the same mistake in the future.
When `FileAction` opens file with write access, it doesn't clear the
file nor append to the end of the file if it already exists. Instead, it
writes from cursor index 0.
For example, by using the settings `target.output-path` and
`target.error-path`, lldb will redirect process stdout/stderr to files.
It then calls this function to write to the files which the above
symptoms appear.
## Test
- Added unit test checking the file flags
- Added 2 api tests checking
- File content overwritten if the file path already exists
- Stdout and stderr redirection to the same file doesn't change its
behavior
A memory region can be relatively large. Searching for a value in the
entire region is time-consuming, especially when running tests against a
remote target, because the memory data is transferred in small chunks
over a relatively slow GDB Remote Protocol. The patch limits the address
range to be searched to 2K, which seems sufficient for these tests. In
my setup, for local runs, these tests now take half the time they did
before the patch. For a remote target, the improvement is even more
significant.
Line ending policies were changed in the parent, dccebddb3b80. To make
it easier to resolve downstream merge conflicts after line-ending
policies are adjusted this is a separate whitespace-only commit. If you
have merge conflicts as a result, you can simply `git add --renormalize
-u && git merge --continue` or `git add --renormalize -u && git rebase
--continue` - depending on your workflow.
`SBDebugger().Create()` returns a debugger with only the host platform
in its platform list. If the test suite is running for a remote
platform, it should be explicitly added and selected in the new debugger
created within the test, otherwise, the test will fail because the host
platform may not be able to launch the built binary.
The function should use the by-ref SBError argument instead of creating
a new one. This code has been here since ~forever, and was probably
copied from methods which return an SBError result (where one needs to
create a local variable).
This fix is based on a problem with cxx_compiler and cxx_linker macros
on Windows.
There was an issue with compiler detection in paths containing "icc". In
such case, Makefile.rules thought it was provided with icc compiler.
To solve that, utilities detection has been rewritten in Python.
The last element of compiler's path is separated, taking into account
the platform path delimiter, and compiler type is extracted, with regard
of possible cross-toolchain prefix.
---------
Co-authored-by: Pavel Labath <pavel@labath.sk>
Reapply #100443 and #101770. These were originally reverted due to a
test failure and an MSAN failure. I changed the test attribute to
restrict to x86 (following the other existing tests). I could not
reproduce the test or the MSAN failure and no repo steps were provided.
This patch loosen the parsing requirement to allow parsing not only
JSON dictionaries but also valid JSON type (integer, float, string,
bool, array, null).
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In #98403 I enabled the SBSaveCoreOptions object, which allows users via
the scripting API to define what they want saved into their core file.
As the first option I've added a threadlist, so users can scan and
identify which threads and corresponding stacks they want to save.
In order to support this, I had to add a new method to `Process.h` on
how we identify which threads are to be saved, and I had to change the
book keeping in minidump to ensure we don't double save the stacks.
Important to @jasonmolenda I also changed the MachO coredump to accept
these new APIs.
This PR adds `SBSaveCoreOptions`, which is a container class for options
when LLDB is taking coredumps. For this first iteration this container
just keeps parity with the extant API of `file, style, plugin`. In the
future this options object can be extended to allow users to take a
subset of their core dumps.
A new file was added to the python_api/events test, but I forgot to
git add it before making the PR. The commit was:
44d9692e6a657ec46e98e4912ac56417da67cfee
This fixes a bug where Process events were being delivered to secondary
listeners when the Private state thread listener was processing the
event. That meant the secondary listener could get an event before the
Primary listener did. That in turn meant the state when the secondary
listener got the event wasn't right yet. Plus it meant that the
secondary listener saw more events than the primary (not all events get
forwarded from the private to the public Process listener.)
This bug became much more evident when we had a stop hook that did some
work, since that delays the Primary listener event delivery. So I also
added a stop-hook to the test, and put a little delay in as well.
On Windows the function name is the full prototype including
the calling convention, all we care about is that the last part
is correct.
This also reverts the xfail added by 07b3e2c0c68b93a3d4d89426dc7fd14cc31ca6be.
In async mode, the test terminates sooner than it should
(`run_to_source_breakpoint` does not work in this mode), and then the
test crashes due to #76057. Most of the time, the test does not fail
because its already XFAILed, but the crash still registers as a failure.
TestThreadAPI.py test_StepInstruction started failing after #97493
Following assertion fails but I am not sure if test will pass after
changing the test.
AssertionError: 'void __cdecl call_me(bool)' != 'call_me(bool)'
I have marked it as xfail I ll run it on a Windows machine to find
an appropriate fix.
https://lab.llvm.org/buildbot/#/builders/141/builds/476
This was just a typo, none of the external execution control functions
should discard other plans. In particular, it means if you stop in a
hand-called function and step an instruction, the function call thread
plan gets unshipped, popping all the function call frames.
I also added a test that asserts the correct behavior. I tested all the
stepping operations even though only StepInstruction was wrong.
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
This is a second attempt to land #95007
Test Plan:
llvm-lit
llvm-project/lldb/test/API/python_api/find_in_memory/TestFindInMemory.py
llvm-project/lldb/test/API/python_api/find_in_memory/TestFindRangesInMemory.py
Reviewers: clayborg
Tasks: lldb
…andOverrideCallback (#94518)"
This reverts commit 7cff05ada05e87408966d56b4c1675033187ff5c. The API
test that was added erroneously imports a module that isn't needed and
wouldn't be found which causes a test failures. This reversion removes
that import.
`SBCommandInterpreter::CommandOverrideCallback` was not being exposed to
the Python API and has no coverage in the
API test suite, so this commits exposes and adds a test for it. Doing
this involves also adding a typemap for the callback used for this
function so that it matches the functionality of other callback
functions that are exposed to Python.
This is useful if you have a transcript of a user session and want to
rerun those commands with RunCommandInterpreter. The same functionality
is also useful in testing.
I'm adding it primarily for the second reason. In a subsequent patch,
I'm adding the ability to Python based commands to provide their
"auto-repeat" command. Among other things, that will allow potentially
state destroying user commands to prevent auto-repeat. Testing this with
Shell or pexpect tests is not nearly as accurate or convenient as using
RunCommandInterpreter, but to use that I need to allow auto-repeat.
I think for consistency's sake, having interactive sessions always do
auto-repeats is the right choice, though that's a lightly held
opinion...
# Changes
1. Changes to the structured transcript.
1. Add fields `commandName` and `commandArguments`. They will hold the
name and the arguments string of the expanded/executed command (e.g.
`breakpoint set` and `-f main.cpp -l 4`). This is not to be confused
with the `command` field, which holds the user input (e.g. `br s -f
main.cpp -l 4`).
2. Add field `timestampInEpochSeconds`. It will hold the timestamp when
the command is executed.
3. Rename field `seconds` to `durationInSeconds`, to improve
readability, especially since `timestampInEpochSeconds` is added.
2. When transcript is available and the newly added option
`--transcript` is present, add the transcript to the output of
`statistics dump`, as a JSON array under a new field `transcript`.
3. A few test name and comment changes.
But one made in a situation where that's impossible might only have an
error, and no symbol context, so that's not necessarily true. Check for
the target's validity before using it.
Fixes issue #93313
Test llvm-project/lldb/test/API/python_api/address_range/TestAddressRange.py is failing on Windows due adding a carriage return character at the end of line. Original PR is #93836.
This adds new SB API calls and classes to allow a user of the SB API to obtain an address range from SBFunction and SBBlock. This is a second attempt to land the reverted PR #92014.
Recently we have disabled this test for Windows host and Linux target.
Now we faced the same issue #92419 in case of Linux x86_64 host and
Linux Aarch64 target.
The new tests added in #92014 have been flaky on Linaro's
Windows on Arm bot. They appear to be hitting a deadlock trying
to clean up the test process.
This only happens in async mode and I don't see why this test
case needs async mode, so the simple workaround is to stick to
sync mode.
# Motivation
Individual callers of `SBDebugger::SetDestroyCallback()` might think
that they have registered their callback and expect it to be called when
the debugger is destroyed. In reality, only the last caller survives,
and all previous callers are forgotten, which might be a surprise to
them. Worse, if this is called in a race condition, which callback
survives is less predictable, which may case confusing behavior
elsewhere.
# This PR
Allows multiple destroy callbacks to be registered and all called when
the debugger is destroyed.
**EDIT**: Adds two new APIs: `AddDestroyCallback()` and
`ClearDestroyCallback()`. `SetDestroyCallback()` will first clear then
add the given callback. Tests are added for the new APIs.
## Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/debugger/TestDebuggerAPI.py
```
## (out-dated, see comments below) Semantic change to
`SetDestroyCallback()`
~~Currently, the method overwrites the old callback with the new one.
With this PR, it will NOT overwrite. Instead, it will hold on to both.
Both callbacks get called during destroy.~~
~~**Risk**: Although the documentation of `SetDestroyCallback()` (see
[C++](https://lldb.llvm.org/cpp_reference/classlldb_1_1SBDebugger.html#afa1649d9453a376b5c95888b5a0cb4ec)
and
[python](https://lldb.llvm.org/python_api/lldb.SBDebugger.html#lldb.SBDebugger.SetDestroyCallback))
doesn't really specify the behavior, there is a risk: if existing call
sites rely on the "overwrite" behavior, they will be surprised because
now the old callback will get called. But as the above said, the current
behavior of "overwrite" itself might be unintended, so I don't
anticipate users to rely on this behavior. In short, this risk might be
less of a problem if we correct it sooner rather than later (which is
what this PR is trying to do).~~
## (out-dated, see comments below) Implementation
~~The implementation holds a `std::vector<std::pair<callback, baton>>`.
When `SetDestroyCallback()` is called, callbacks and batons are appended
to the `std::vector`. When destroy event happen, the `(callback, baton)`
pairs are invoked FIFO. Finally, the `std::vector` is cleared.~~
# (out-dated, see comments below) Alternatives considered
~~Instead of changing `SetDestroyCallback()`, a new method
`AddDestroyCallback()` can be added, which use the same
`std::vector<std::pair<>>` implementation. Together with
`ClearDestroyCallback()` (see below), they will replace and deprecate
`SetDestroyCallback()`. Meanwhile, in order to be backward compatible,
`SetDestroyCallback()` need to be updated to clear the `std::vector` and
then add the new callback. Pros: The end state is semantically more
correct. Cons: More steps to take; potentially maintaining an
"incorrect" behavior (of "overwrite").~~
~~A new method `ClearDestroyCallback()` can be added. Might be
unnecessary at this point, because workflows which need to set then
clear callbacks may exist but shouldn't be too common at least for now.
Such method can be added later when needed.~~
~~The `std::vector` may bring slight performance drawback if its
implementation doesn't handle small size efficiently. However, even if
that's the case, this path should be very cold (only used during init
and destroy). Such performance drawback should be negligible.~~
~~A different implementation was also considered. Instead of using
`std::vector`, the current `m_destroy_callback` field can be kept
unchanged. When `SetDestroyCallback()` is called, a lambda function can
be stored into `m_destroy_callback`. This lambda function will first
call the old callback, then the new one. This way, `std::vector` is
avoided. However, this implementation is more complex, thus less
readable, with not much perf to gain.~~
---------
Co-authored-by: Roy Shi <royshi@meta.com>
# Motivation
Currently, the user can already get the "transcript" (for "what is the
transcript", see `CommandInterpreter::SaveTranscript`). However, the
only way to obtain the transcript data as a user is to first destroy the
debugger, then read the save directory. Note that destroy-callbacks
cannot be used, because 1\ transcript data is private to the command
interpreter (see `CommandInterpreter.h`), and 2\ the writing of the
transcript is *after* the invocation of destory-callbacks (see
`Debugger::Destroy`).
So basically, there is no way to obtain the transcript:
* during the lifetime of a debugger (including the destroy-callbacks,
which often performs logging tasks, where the transcript can be useful)
* without relying on external storage
In theory, there are other ways for user to obtain transcript data
during a debugger's life cycle:
* Use Python API and intercept commands and results.
* Use CLI and record console input/output.
However, such ways rely on the client's setup and are not supported
natively by LLDB.
# Proposal
Add a new Python API `SBCommandInterpreter::GetTranscript()`.
Goals:
* It can be called at any time during the debugger's life cycle,
including in destroy-callbacks.
* It returns data in-memory.
Structured data:
* To make data processing easier, the return type is `SBStructuredData`.
See comments in code for how the data is organized.
* In the future, `SaveTranscript` can be updated to write different
formats using such data (e.g. JSON). This is probably accompanied by a
new setting (e.g. `interpreter.save-session-format`).
# Alternatives
The return type can also be `std::vector<std::pair<std::string,
SBCommandReturnObject>>`. This will make implementation easier, without
having to translate it to `SBStructuredData`. On the other hand,
`SBStructuredData` can convert to JSON easily, so it's more convenient
for user to process.
# Privacy
Both user commands and output/error in the transcript can contain
privacy data. However, as mentioned, the transcript is already available
to the user. The addition of the new API doesn't increase the level of
risk. In fact, it _lowers_ the risk of privacy data being leaked later
on, by avoiding writing such data to external storage.
Once the user (or their code) gets the transcript, it will be their
responsibility to make sure that any required privacy policies are
guaranteed.
# Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/interpreter/TestCommandInterpreterAPI.py
```
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/session/save/TestSessionSave.py
```
---------
Co-authored-by: Roy Shi <royshi@meta.com>
Co-authored-by: Med Ismail Bennani <ismail@bennani.ma>
Parsing of '::' scopes in TypeQuery was very naive and failed for names
with '::''s in template arguments. Interestingly, one of the functions
it was calling (Type::GetTypeScopeAndBasename) was already doing the
same thing, and getting it (mostly (*)) right. This refactors the
function so that it can return the scope results, fixing the parsing of
names like std::vector<int, std::allocator<int>>::iterator.
Two callers of GetTypeScopeAndBasename are deleted as the functions are
not used (I presume they stopped being used once we started pruning type
search results more eagerly).
(*) This implementation is still not correct when one takes c++
operators into account -- e.g., something like `X<&A::operator<>::T` is
a legitimate type name. We do have an implementation that is able to
handle names like these (CPlusPlusLanguage::MethodName), but using it is
not trivial, because it is hidden in a language plugin and specific to
method name parsing.
---------
Co-authored-by: Michael Buch <michaelbuch12@gmail.com>
