Addresses the post-commit review comments from the original PR
(https://github.com/llvm/llvm-project/pull/217126#pullrequestreview-5162589053).
- `SBFile::OpenFdFromHandle` is now visible on all platforms (just
`#ifndef SWIG`) and returns -1 by default.
- Removed non-stable `PyLong_Check` in favor of `PyErr_Occurred` and
`PyErr_Clear`.
(cherry picked from commit ba1687b72476eb368641b9a9f565a38cf41b7393)
The official LLVM and lldb builds use `LLVM_ENABLE_RPMALLOC`. This
changes the default CRT allocator to rpmalloc for performance reasons.
In doing so, it also changes the CRT linkage from dynamic (`/MD`) to
static (`/MT`).
Since the CRT is the one managing file descriptors on Windows, its
universe is now scoped to a module. For lldb, this means we have three
fd-spaces: the driver (lldb/lldb-dap), liblldb, and Python.
We usually pass files around as file descriptors. This breaks when we do
it between modules.
Fortunately, the C runtime provides us with helpers to translate between
the process scoped HANDLEs and file descriptors. This way, we can
allocate an fd for a handle passed across modules. For Python, there's
the [`msvcrt`](https://docs.python.org/3/library/msvcrt.html) library we
can use to translate.
Fixes#216679.
(cherry picked from commit 1ca6698be7708121f987d0acf896c7dc4cf63a51)
SBFrame::GetValueForVariablePath added new parameters in commit
80fffd527c.
This caused an ABI break because
the ABI is based on the function's mangled name, which changes when
parameters are added. Existing users depend on the original symbol being
present in the library.
changing the ABI from:
```cpp
// _ZN4lldb7SBFrame23GetValueForVariablePathEPKcNS_16DynamicValueTypeE
lldb::SBFrame::GetValueForVariablePath(char const*, lldb::DynamicValueType)
```
to
```cpp
// _ZN4lldb7SBFrame23GetValueForVariablePathEPKcNS_16DynamicValueTypeENS_7DILModeE`
lldb::SBFrame::GetValueForVariablePath(char const*, lldb::DynamicValueType, lldb::DILMode)
```
Normally, this can be fixed by manually overloading the function from:
```cpp
GetValueForVariablePath(const char *, DynamicValueType, lldb::DILMode = lldb::eDILModeFull);
```
to
```cpp
GetValueForVariablePath(const char *, DynamicValueType);
GetValueForVariablePath(const char *, DynamicValueType, lldb::DILMode mode);
```
with the two-argument overload setting the mode in the implementation.
However, this approach does not work for the Python API because enums
are converted to integers. The four C++ overloads:
```cpp
GetValueForVariablePath(const char *);
GetValueForVariablePath(const char *, lldb::DILMode);
GetValueForVariablePath(const char *, DynamicValueType);
GetValueForVariablePath(const char *, DynamicValueType, lldb::DILMode mode);
```
would appear in python as:
```py
GetValueForVariablePath(str)
GetValueForVariablePath(str, int)
GetValueForVariablePath(str, int)
GetValueForVariablePath(str, int, int)
```
As a result, there is no way to distinguish between `lldb::DILMode` and
`lldb::DynamicValueType` when they are passed as the second parameter.
Existing test did not catch this because there is not test covering
`GetValueForVariablePath(const char *, DynamicValueType)`).
Instead I added an new function called
```cpp
GetValueForVariablePathWithMode(const char*, lldb::DILMode)
GetValueForVariablePathWithMode(const char*, lldb::DILMode, DynamicValueType)
```
and a python helper.
```python
class SBFrame:
def var_with_mode(self, var_path: str, mode: int, use_dynamic: int, /):
...
```
mirroring the existing `SBFrame.var` helper function.
(cherry picked from commit 4e1a982f130bcd329a062e6c458c3d8818b6ddf3)
Crashed because std::string is constructed with a nullptr. Wrap with
`llvm::StringRef`.
Add a unittest
(cherry picked from commit 69db8489be2f4e3cb1cf3f874403481d206b545c)
In python the method's signature is
```py
class SBStructuredData:
def GetStringValue(self, dst_len: int) -> str:
```
There is no way from python to get the total data from GetStringValue
regardless of how many times you call GetStringValue.
Update the implementation
```py
def GetStringValue(self, len: int = 0) -> str:
```
Fixes crash when input is None in `SBStructuredData.SetStringValue`
Don't trim the result when called with the `dynamic` property.
(cherry picked from commit 93377d703115b45c3ad8f5f218cb85769d24a1b0)
It crashes because the `default` error string may not be a format string
compared to the `fallback` error string
(cherry picked from commit 29bc0749034c4f2bda0752360ecacd4e89d7439c)
IDEs may offer functionality to set a variable to a specific value.
There are many situations where this isn't actually possible, for
example, if the variable's value is a constant or the result of a
complex DWARF expression. Instead of offering to change a value only to
have it fail with an error, this API lets the IDE query whether setting
a value is generally feasible so it can hide the action where it isn't
applicable.
rdar://142358140
Assisted-by: claude
By having ConstString in this interface, we are forcing callers to
create a ConstString in order to perform a lookup. In many cases,
`FindSectionByName` is a best-effort interface, meaning we're creating
ConstStrings speculatively. In other cases, we were creating a
ConstString at the call-site instead of computing it once and re-using
it.
I audited all the callsites and changed all the callsites that created a
ConstString for the purpose of calling `FindSectionByName`. Other
callsites will take more time to change.
That way if an override only wants to override, say, file and line
breakpoints, it can say that when it registers itself, and we won't do
the work of creating the resolver and checking with it if the breakpoint
wasn't of the requested type.
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Instead of a boolean plus writing errors to a stream.
llvm::Error is either success or an error message, unlike
before where we could (and did sometimes) succeed but
also have an error message.
We can do this for DisableLogChannel too but
I'll do that in a follow up.
This finishes the removal of ConstString from FileSpec's interface. Note
that FileSpec is still _backed_ by ConstStrings, that will be changed in
a follow-up.
A non-goal for this change is to rewrite business logic, so I kept the
type of some variables as ConstString where a refactor would necessary
to change that. I converted printf-style formatting to formatv-style
formatting where required, and otherwise tried to leave callsites alone
to the extent that was possible.
Nothing in the Utility or Host layers uses Diagnostics. Its only callers
are Debugger (the always-on log feeder), SBDebugger, and the
SystemInitializerCommon lifecycle. Those all live in Core or above. The
header depends only on Utility primitives (FileSpec, Log, Error), and
lldbInitialization already links lldbCore, so the move adds no new link
dependency anywhere.
Relocating it to Core lets Diagnostics reach Debugger, Target,
CommandInterpreter, and Host, which simplifies an upcoming change that
collect a richer diagnostics bundle (statistics, command snapshots,
invocation, etc) and allows us to implement that directly in the
Diagnostics class.
In responding to the question here:
https://discourse.llvm.org/t/creating-nested-synthetic-children/90997/3
it became clear that getting the persistent variable/type results from
the expression parser was much harder than it needed to be. I filed a
couple issues for this:
https://github.com/llvm/llvm-project/issues/201952https://github.com/llvm/llvm-project/issues/201949
though after thinking about this a bit more, I decided to go with
dedicated API for getting Persistent Expression variables & types. We
don't allow redefinitions of persistent variables and types within the
same Target, so a "FirstType" API didn't make sense, nor did one
returning a list of types or values.
Plus, this will be easier to discover.
Synthetic frame variables can be included in `SBFrame::GetVariables` by
passing an `SBVariablesOptions` with `SetIncludeSynthetic(true)`.
However, they're not included in the other two overloads.
With the same reason they're included in `frame variable`, they should
be included in these overloads. If the user doesn't want them included
there, they can use the overload taking `SBVariablesOptions`.
This almost enables them to be shown in lldb-dap's variables. They're
currently filtered out if `in_scope_only` is specified. Not sure what
the proper fix is. Maybe they should skip that check.
With #201392, we generate a minimal export list when LLDB_ENABLE_
DYNAMIC_SCRIPTINTERPRETERS is set. We therefore no longer need to force
LLDB_EXPORT_ALL_SYMBOLS.
This PR makes the two options independent again. LLDB_EXPORT_ALL_SYMBOLS
now defaults to OFF and, when set, is honored regardless of whether the
script interpreters are built as static or dynamic libraries. otherwise,
when disabled, liblldb exports the generated subset if dynamic
interpreters are enabled, otherwise only the public lldb namespace.
I'm trying to clean up the FileSpec API. Even if it is currently
implemented as a pair of ConstStrings, I think it would be better if
FileSpec users only created new ConstStrings as necessary.
Expose the existing `Process::IsLiveDebugSession()` through the SB API
as `SBProcess::IsLiveDebugSession()`, letting clients distinguish a live
debuggee from a post-mortem session such as a core file or minidump. It
returns `false` when there is no underlying process, consistent with
other `SBProcess` query methods.
Today we can run `target modules dump separate-debug-info --json` to get
a json blob of all the separate debug info, but it has a few
shortcomings when developing some scripting against it. Namely, the
caller has to know the structure of the JSON per architecture that will
be returned.
I've been working on a Minidump packing utility where we enumerate
symbols and source and put them in a package so we can debug with
symbols portably, and it's been difficult to maintain multiple
architectures due to the above shortcomings. To address this for myself,
I've exposed a simple iterator for the SBModule to get all the
separate-debug-info as list of filespecs with no need for the caller to
have context on what kind of data it is.
I also extened the swig interfaces to make writing my test easier and as
a nice to have.
Terminal width and height were communicated to the debugger separately,
via SetTerminalWidth() and SetTerminalHeight(). Each notified the
IOHandler and the statusline, so on a resize they recomputed their
layout twice: once with one dimension updated and the other still stale.
Add Debugger::SetTerminalDimensions(width, height) (exposed through
SBDebugger) that updates both properties before notifying, and
reimplement the single-axis setters and the driver's resize handler in
terms of it.
Also fix SBDebugger::GetTerminalHeight(), which returned the width.
With LLDB_ENABLE_DYNAMIC_SCRIPTINTERPRETERS, the script interpreter
plugins are built as standalone shared libraries that resolve their
lldb_private/llvm references through liblldb's re-exports. liblldb was
falling back to liblldb-private.exports, which allowlists the
lldb_private/llvm namespaces wholesale.
Generate the export list per build instead. For each plugin, run llvm-nm
on its compiled objects, keep undefined references in namespaces liblldb
owns, subtract the plugin's own definitions, and concatenate the
per-plugin lists with liblldb.exports.
The plugin's SHARED target LINK_LIBS liblldb, which makes the plugin's
compile order depend on liblldb. Wiring the exports file in via
add_llvm_symbol_exports' built-in target dependency would form a cycle
through that order-only edge. Add NO_TARGET_DEPENDENCY to
add_llvm_symbol_exports so we can apply the file via LINK_DEPENDS alone;
that still triggers a relink when the symbol set changes, without
participating in compile-order tracking.
Windows is unchanged: msvc_extract_private_symbols.py was already doing
per-symbol extraction at namespace granularity.
Assisted-by: Claude
Drops ${Python3_LIBRARIES} from the SHARED build of
lldbPluginScriptInterpreterPython and lets undefined Python symbols
through at link time (`-undefined dynamic_lookup` on Darwin,
`--allow-shlib-undefined` on Linux; Windows keeps its existing
delay-load + import lib).
SystemInitializerFull::Initialize resolves the Python runtime loader
via ScriptInterpreterRuntimeLoader::Get(eScriptLanguagePython) and
calls Load() before initializing any plugin, so libpython is mapped
into the process before either entry point that references it: the
static script interpreter's Initialize() (which invokes Python via
the LLDB_PLUGIN_INITIALIZE loop) and the dynamic plugin's dlopen
(whose undefined references resolve against the in-process
libpython). This covers both LLDB_ENABLE_DYNAMIC_SCRIPTINTERPRETERS
=ON and =OFF, and keeps Windows working in static builds where the
delay-load thunks live in liblldb itself. The loader is
once_flag-cached, and errors propagate out via the existing Expected
return.
`import lldb` from a running Python works because libpython is
already mapped into the process. The Python runtime loader probes
for stable-ABI symbols (Py_IsInitialized, Py_InitializeFromConfig)
via dlsym(RTLD_DEFAULT) / GetProcAddress and, finding them, returns
success without dlopen'ing a second libpython on top.
The dynamic plugins are added as test-suite dependencies so
check-lldb-shell and check-lldb-api wait for them; they can't be
build-order deps of liblldb itself because the plugins link against
liblldb (CMake SHARED cycle).
The unit-test static variant lldbStaticScriptInterpreterPython keeps
hard-linking libpython so link-time symbol checking is preserved.
This is a follow-up to 1b4a578a9f.
Replace ScriptedThreadPlan's separate `m_class_name` and `m_args_data`
members with a single `m_scripted_metadata`, and update its constructor,
Thread::QueueThreadPlanForStepScripted and the SB/CLI callers to take a
`const ScriptedMetadata &` directly. Drop the now-redundant `args_sp`
parameter from ScriptedThreadPlanInterface::CreatePluginObject; the
Python override constructs a StructuredDataImpl from the metadata's args
dict for the dispatched call.
ScriptedThreadPlan keeps its own `m_scripted_metadata` member because
the interface's metadata is only set when `DidPush()` calls
`CreatePluginObject`, so the plan needs to hold the metadata between
construction and `DidPush()`.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
While working formatter bytecode, one of the C++ formatters needs
`GetParent`. While adding `GetParent` support in the bytecode, I noticed
the SB API also does not expose `GetParent`. This remedies that.
During review of this PR, it was pointed out that `GetParent` does work
with synthetic value objects. This PR also addresses that shortcoming.
Assisted-by: claude
The notion of a "selected target" is something that belongs at the
command layer rather than the debugger layer. Jim and I were discussing
how this is a foot-gun and I suggested moving it into the
CommandInterpreter. Everyone else that needs to get their hands on the
current target should do so through the execution context. The latter is
trivial to swap out if the "selected one" isn't what you should be
operating on.
Add GetNumArguments() and GetArgumentAtIndex() to SBProcessInfo. Add a
convienence property `arguments` in the python api.
lldb-dap uses this to show more information to the user when picking a
process.
Fixes: #197509
This is a general mechanism that allows a custom breakpoint resolver to
"override" the resolver lldb would have used. It is implemented by
making a scripted breakpoint resolver class, implementing an extra
"overrides_resolver" method. When lldb has decided what resolver it is
going to use for the breakpoint, if there are any override resolvers
registered, it will pass the serialized form of the resolver to the
`overrides_resolver` method of the registered resolvers. If any of the
override resolvers return true from the overrides_resolver method, that
resolver will be used in place of the one lldb would have used.
The original motivation for this feature is to make the support of
"debugging a scripting language from the implementation side". We added
the notion of "facade breakpoints" so that a location given in the
context of the scripting language can be handled by the emulation code.
But when you are using lldb in some IDE, when "stopped" in the scripting
language, the IDE will show the scripting language source. It is natural
in that case for the user to expect the UI gesture to set a file and
line breakpoint should work the same way.
To implement that, the implementor would make their facade breakpoint
resolver also an override resolver that checks to see if the incoming
resolver was a file and line breakpoint for a source file of the
scripting language type. If it is, it returns true, and lldb will use
that resolver instead. Note, this will also work when breakpoints are
set on the command line, so you can transparently support "scripting
language" breakpoints from the command line as well.
Several CommandObject subclasses had DoExecute paths that returned
without ever calling SetStatus on the CommandReturnObject. The status
was silently left at its initial eReturnStatusStarted value, which made
Succeeded() report false for what were really successful commands and
left CommandReturnObject in an undefined state.
Add support for building the script interpreter plugins as shared
libraries, controlled by the `LLDB_ENABLE_DYNAMIC_SCRIPTINTERPRETERS`
CMake variable. This allows the plugins, rather than `liblldb`, to link
against their respective dependencies such as Python and Lua, which has
several benefits:
1. It makes it possible to gracefully handle the Python or Lua
dependencies not being present, instead of crashing.
2. It enables programmatically deciding which Python or Lua library to
load.
3. Combined with (2), this opens up a path towards not statically
linking with Python at all, which is required in order to load the LLDB
module in a different Python interpreter (at least this is a hard
requirement on Darwin due to two-level namespaces and install names).
This PR completes #183791
@jimingham You had suggested to @clayborg that we should call DidFork(),
DidVFork() and DidVForkDone() after a corresponding stop in expression
evaluation. Those functions require parameters that are private to the
specific child StopInfo class. I went with reusing the existing
StopInfo::PerformAction() virtual function, rather than create a new one
just for this purpose.
To fit the style guide:
https://llvm.org/docs/CodingStandards.html#error-and-warning-messages
I found these with:
* Find `(\.AppendErrorWithFormat\(([\s\r\n]+)?"(?:(?:\\.|[^"\\])*))\."`
and replace with `$1"` using Visual Studio Code.
* Putting a call to `validate_diagnostic` in `AppendErrorWithFormat`.
* Manual inspection.
Note that this change *does not* include a call to `validate_diagnostic`
because I do not know what's going to crash on platforms that I haven't
tested on.
A very common pattern in our synthetic child providers was to make the
child ValueObject using ValueObjectConstResult::Create or some form of
the static ValueObject::CreateValueObjectFrom*** methods, and store and
hand that out as the child. Doing that creates a "root" ValueObject
whose lifecycle is not linked to the lifecycle of the ValueObject it is
a child of. And that means it is possible that either the child or the
parent could have gotten destroyed when the other ValueObject gets asked
a question about it.
For the most part this doesn't happen because there are usually enough
other shared pointer references binding the two to keep both sides
alive. But we have gotten a small but steady stream of reports for years
now of crashes where a ValueObject accesses its ClusterManager but that
has already been deleted. I've never been able to find a reproducible
case of this, but one plausible cause is that we are violating the
contract that "all the children of a ValueObject have coterminous
lifespans, enforced by the ClusterManager". So it is unsurprising that
they might sometimes not stay alive as long as they should.
This patch addresses that by providing a way to use these static create
methods but passing in the ClusterManager to be used, and adds or
modifies where extant the CreateChildValueObjectFrom*** methods to pass
in the parent manager.
This patch is not complete. It turns out that routing the ClusterManager
from CreateValueObjectFromExpression all the way to where the expression
result VO is made is quite intrusive, and would have made this already
quite large (though largly mechanical) patch become unweildy. Since I am
mostly concerned about synthetic child providers, and we discourage
using expressions for them, I think that's an acceptable separation.
I also added a test where the ValueObjectSynthetic hands out its
children, checking whether the child being handed out is in fact owned
by the parent handing it out. It is controlled by a setting
(target.check-vo-ownership) which is currently off by default - since I
can't provide a way to do this correctly for expressions yet. But I turn
it on in the testsuite. That's the real testing strategy for this patch,
since its goal is to ensure that all the children we hand out are
managed by their parents. I didn't put an equivalent check in
ValueObjectVariable because that system really doesn't allow a way to do
this incorrectly.
I also fixed some of the tests that were making children using
CreateValueObjectFromExpression to make their children with
CreateValueObjectFromData instead. I could have opted the tests out of
the check, but using the expression parser to do this work is not best
practice, so I didn't want to leave bad examples around. I left one case
that does do it with expressions, however, so we still have a test for
that not-recommended but people do it anyway path.
This patch adds a new flag to the lldb_private::StackFrame API to get variable lists: `include_synthetic_vars`. This allows ScriptedFrame (and other future synthetic frames) to construct 'fake' variables and return them in the VariableList, so that commands like `fr v` and `SBFrame::GetVariables` can show them to the user as requested.
This patch includes all changes necessary to call the API the new way - I tried to use my best judgement on when to include synthetic variables or not and leave comments explaining the decision.
As a consequence of producing synthetic variables, this patch means that ScriptedFrame can produce Variable objects with ValueType that contains a ValueTypeExtendedMask in a high bit. This necessarily complicates some of the switch/case handling in places where we would expect to find such variables, and this patch makes best effort to address all such cases as well. From experience, they tend to show up whenever we're dealing with checking if a Variable is in a specified scope, which means we basically have to check the high bit against some user input saying "yes/no synthetic variables".
This patch handles most of the scaffolding for synthetic variable support that isn't directly tied to functional changes. This patch will be used by one following patch that actually modifies the lldb_private::StackFrame API to allow us to fetch synthetic variables.
There were a couple important/interesting decisions made in this patch that should be noted:
- Any value type may be synthetic, which is why it's a mask applied over the top of another value type.
- When printing frame variables with `fr v`, default to showing synthetic variables.
This new value type mask makes some of the ValueType handling more interesting, but since nothing generates objects with this mask until the next patch, we can land the concept in this patch in some amount of isolation.
BroadcastEventSpec owns the broadcaster class its configured to listen
for. Broadcasters usually advertise their broadcast class name with
StringRefs so there's no need to put them in the string pool.
The only exception here is SBListener. There are 2 methods that take
`const char *` values. However, that's handled when converting them to
StringRefs.
This patch makes the default parameter to ConstString::AsCString
explicit. Currently it defaults to `nullptr`. However, there a bunch of
callsites that do `printf("%s", foo.AsCString())`, which is UB (I have
at least 1 crash on me where this was the root cause).
Alternatively we could change the default to `""` instead of `nullptr`,
but code that did `if (foo.AsCString())` would then change meaning.
Command I ran to generate this patch:
```
rg "\.AsCString\(\)" lldb -l | xargs sed -i '' 's/\.AsCString()/.AsCString(nullptr)/g'
```
There's also a commonly used `Status::AsCString` with a default
parameter. So I added a temporary:
```
const char *AsCString(std::nullptr_t) const = delete;
```
overload to `Status`, and fixed all the compile errors.
In a follow-up PR I intend to change the callsites that do `printf("%s",
foo.AsCString(nullptr)` to pass `""` as the fallback parameter. Another
dangrous pattern is `std::string(error.AsCString(nullptr))`
rdar://171219173
The only difference between them is that OptionalBool's third state
is "unknown" and LazyBool's is "calculate". We don't need to tell
the difference in a single context, so I've made a new eLazyBoolDontKnow
which is an alias of eLazyBoolCalculate.
While here:
* Move the constructor to the public section. Almost all ThreadPlan
classes have public constructors.
* Use `std::make_shared()`. It is modern and more efficient.
After this change, Address::SetSection() had only one use left (in a
unit test) and was removed. Address::ClearSection() had no uses, now
also removed. (It is unlikely that someone needs to change the section
without simultaneously changing the section offset, and for that we have
a constructor.)
Currently these functions take a target pointer. In Cross-language
projects this means it's down to chance what typesystem the resulting
value will be in, since the implementation returns the first scratch
type system in the target, which depends on the order of images and
their implementation language.
By passing in an execution context the selected frame is used to
determine the typesystem, which is usually the expected outcome when
using DIL.
This should be entirely NFC for Clang-only LLDBs, but is a necessity for
LLDBs with additional type system plugins such as the Swift plugin.
Assisted by Claude to patch the call sites.
Ensure that use of the GNU driver does not change the library name on
Windows. We would check the build tools being MSVC rather than targeting
Windows to select the output name.
Host::RunShellCommand takes a std::string *command_output argument and a
bool hide_stderr=false defaulted argument. If the shell command returns
stderr and stdout text, it is intermixed in the same command_output,
unless hide_stderr=true.
In SymbolLocatorDebugSymbols::DownloadObjectAndSymbolFile we call an
external program to find a binary and dSYM by uuid, and the external
program returns a plist (xml) output. In some cases, it printed a
(harmless) warning message to stderr, and then a complete plist output
to stdout. We attempt to parse the combination of these two streams, and
the parse fails - we don't get the output.
This patch removes hide_stderr and instead adds a `std::string
*separated_error_output` argument. If `separated_error_output` is
nullptr, output and error texts are returned combined in the
`command_output` argument. If a std::string object address is provided
for `separated_error_output`, then standard error output is separated
into this string. A caller which wants the old `hide_stderr=true`
behavior should pass a throwaway std::string object to `RunShellCommand`
and ignore it.
rdar://168621579