Push `PolicyStack::Get().PushPublicStateRunningExpression()` at all
three expression evaluation entry points
(`LLVMUserExpression::DoExecute`, `FunctionCaller::ExecuteFunction`,
`IRInterpreter`). This policy sets `can_run_breakpoint_actions=false`,
preventing recursive breakpoint callback execution during expression
eval.
Push `PolicyStack::Get().PushPrivateState()` unconditionally for all
PSTs in `RunPrivateStateThread` (not just overrides), giving every PST
the private view while keeping frame providers and recognizers enabled
for normal stop processing. Override PSTs use
`PushPrivateStateRunningExpression()` which additionally disables
providers and recognizers.
With all PSTs and expression eval sites now covered by the policy,
remove all host thread identity check fallbacks:
- `CurrentThreadPosesAsPrivateStateThread()` in `Process::GetState()`
- `CurrentThreadIsPrivateStateThread()` in `Target::GetAPIMutex()`
- `IsOnThread()` in `PrivateStateThread::GetRunLock()`
- `CurrentThreadPosesAsPrivateStateThread()` in
`SelectMostRelevantFrame()`
- `IsRunningExpression()` in `StopInfoBreakpoint::PerformAction()`
`SelectMostRelevantFrame` now checks `!can_run_frame_recognizers`
instead of `View::Private`, so recognizers run during normal PST stop
processing but are skipped during expression evaluation.
----
The following PRs are related to the Policy feature:
- #195762
- #195771
- #198897
- #195774
- #195775
rdar://176223894
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Added support for new `target.process.stop-on-fork` and
`target.process.stop-on-vfork` settings. GDB already has ability to set
[catchpoints](https://www.sourceware.org/gdb/current/onlinedocs/gdb.html/Set-Catchpoints.html)
on `fork` and `vfork`, so having this feature in LLDB might be useful
(e.g. use it to get pid of new process and attach to the new process
from different console).
Two related correctness/clarity improvements to the `Policy`
infrastructure introduced by 504a112:
`PolicyStack::Guard` now stores the std::thread::id of the thread that
created it. Destruction and move operations call
`llvm::report_fatal_error` when they happen on a different thread, since
the `PolicyStack` is `thread_local`: popping from the wrong thread would
silently corrupt that thread's stack.
`Push`/`Pop` on `PolicyStack` are now private. Callers go through named
factories (`PushPrivateState`, `PushPublicStateRunningExpression`) that
return RAII Guards. The transition factories on Policy
(`CreatePrivateState`, `CreatePublicStateRunningExpression`) inherit
from `PolicyStack::Get().Current()` and apply their named change on top,
so pushed policies preserve existing stack state rather than resetting
unrelated fields. `CreatePublicState` remains the baseline reference
value (returns a default `Policy{}`); the stack returns to public state
by popping the private guards, not by pushing a "public" policy on top.
----
The following PRs are related to the Policy feature:
- #195762
- #195771
- #198897
- #195774
- #195775
rdar://176223894
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This finishes the removal of ConstString from FileSpec's interface. Note
that FileSpec is still _backed_ by ConstStrings, that will be changed in
a follow-up.
A non-goal for this change is to rewrite business logic, so I kept the
type of some variables as ConstString where a refactor would necessary
to change that. I converted printf-style formatting to formatv-style
formatting where required, and otherwise tried to leave callsites alone
to the extent that was possible.
Every time a memory region is queried in the inferior with a
ProcessGDBRemote connection, lldb needs to send a packet requesting the
information. When we are querying multiple addresses within the same
region, this can result in many redundant packets - e.g. to test for
whether an address is in stack memory during a backtrace.
This adds a MemoryRegionInfoCache wrapper around RangeDataVector to save
the memory regions that we query, and supply them to future queries. The
cache is flushed on any process resume in the same way that the memory
cache is; the valid pages may change as a program executes.
This PR started as a combination of this cache plus extensions to gdb
remote serial protocol packets so the debug stub can provide memory
region infos when we have a public stop (e.g. the info about each
thread's stack memory region) - including changes to both debugserver
and ProcessGDBRemote to implement that (
https://github.com/llvm/llvm-project/pull/202509 ). I'm separating out
the two pieces into separate patchsets. The test in this PR retains the
full code in the .c file that I will need to test the expedited memory
region infos still, even though it doesn't do antyhing but launch the
inferior process in the current tests.
rdar://179067896
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Resumbitting https://github.com/llvm/llvm-project/pull/205235 with
disabled parallel module loading to fix buildbot deadlock.
ProcessElfCore was reading the NT_FILE list and using that to help
FindModuleUUID to provide UUID information when loading core files. The
NT_FILE list contains resolved paths only, while the
DynamicLoaderPOSIXDYLD plug-in was using paths found in the r_debug
structure which contains a linked list of all of the shared libraries in
a process. The issue was these paths could be symlinks which would cause
ProcessELFCore::FindModuleUUID(...) to fail because the paths wouldn't
match up. This led to the ProcessELFCore often not being able to provide
UUIDs for shared libraries and cause the incorrect binaries to be loaded
from the current machine even when the shared library UUIDs don't match.
The solution was to add the ability for a ModuleSpec to contain a load
address for the shared library. This allows ProcessELFCore to uniquely
identify a library regardless of the name used in NT_FILE. We can now
correctly supply the UUID from the .gnu-build-id to any binaries which
use symlinks when linking, but have differing resolved paths to the
libraries.
The process virtual function for finding a UUID was changed from:
virtual lldb_private::UUID FindModuleUUID(const llvm::StringRef path);
to:
virtual bool FindModuleUUID(ModuleSpec &spec);
to allow Process::FindModuleUUID to rely on other data in the ModuleSpec
since the path isn't enough.
We will be able to use the ModuleSpec's load address for creating a
module from a ModuleSpec, but that isn't in this PR.
ProcessElfCore was reading the NT_FILE list and using that to help
FindModuleUUID to provide UUID information when loading core files. The
NT_FILE list contains resolved paths only, while the
DynamicLoaderPOSIXDYLD plug-in was using paths found in the r_debug
structure which contains a linked list of all of the shared libraries in
a process. The issue was these paths could be symlinks which would cause
ProcessELFCore::FindModuleUUID(...) to fail because the paths wouldn't
match up. This led to the ProcessELFCore often not being able to provide
UUIDs for shared libraries and cause the incorrect binaries to be loaded
from the current machine even when the shared library UUIDs don't match.
The solution was to add the ability for a ModuleSpec to contain a load
address for the shared library. This allows ProcessELFCore to uniquely
identify a library regardless of the name used in NT_FILE. We can now
correctly supply the UUID from the .gnu-build-id to any binaries which
use symlinks when linking, but have differing resolved paths to the
libraries.
The process virtual function for finding a UUID was changed from:
virtual lldb_private::UUID FindModuleUUID(const llvm::StringRef path);
to:
virtual bool FindModuleUUID(ModuleSpec &spec);
to allow Process::FindModuleUUID to rely on other data in the ModuleSpec
since the path isn't enough.
We will be able to use the ModuleSpec's load address for creating a
module from a ModuleSpec, but that isn't in this PR.
This patch merges `IOHandlerProcessSTDIO` and
`IOHandlerProcessSTDIOWindows` in one file: `ProcessIOHandler.h`.
This is an NFC change which reduces the size of `Process.cpp` and allows
to reuse `IOHandlerProcessSTDIOWindows` in GDBRemote, ahead of
https://github.com/llvm/llvm-project/pull/201638.
There are scenarios (especially in the ObjectiveC metadata reading) in
which multiple strings are read over and over again, but through
different code paths. In order to make that part of the code use
MultiMemRead effectively, the memory cache must be integrated into
ReadRangesFromMemory before we can migrate the string reading to
vectorized version.
The Process base class is generally responsible for fixing pointer
metadata before delegating memory reads to concrete Process
specializations. However, ReadMemoryRanges was a direct path into the
derived classes, which made it so that pointer metadata was never
stripped.
This commit creates a non-virtual ReadMemoryRanges in Process, which
clears pointer metadata, before delegating to the new virtual method
DoReadMemoryRanges. This also allows, in the future, to plug into the
memory cache system.
Move the Policy struct and PolicyStack class from lldb/Target to
lldb/Utility. This is a pure relocation -- no API or behavior changes.
This is needed so that lldbHost (which contains ProcessRunLock) can
depend on Policy without introducing a layering violation, since
lldbHost cannot depend on lldbTarget.
----
The following PRs are related to the Policy feature:
- #195762
- #195771
- #198897
- #195774
- #195775
rdar://176223894
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
A future patch will want to control whether all breakpoints are eager or
not based on whether the process is running. Sinking the logic allows
for a smaller diff.
g_process_unique_id is a function-local static used to hand out unique
IDs to each Process. It was a plain uint32_t, so concurrent
SBTarget::LoadCore / Target::CreateProcess calls raced on the increment.
Make it std::atomic<uint32_t>. Prefix operator++ on std::atomic is
already an atomic fetch_add that returns the new value, so the call
sites are unchanged.
Found by ThreadSanitizer as part of #197792.
This PR adds the ability for lldb to print out how to reproduce the
crash by printing out the arguments used with the process on Linux.
This is implemented by extracting arguments from the field `pr_psargs`
of the struct `ELFLinuxPrPsInfo`, a note entries of an ELF coredump
file.
A limitation of this approach is that the `pr_psargs` char array is only
80 character long, so the command arguments that triggers a core dump
might not be fully captured. To alleviate this, if the arguments
extracted matches the maximal length, we emit `(command may be
truncated)` message when printing the arguments to the user.
Calling `process status -v` also prints out said arguments for users, as
suggested by #156427.
Not implemented for BSD-based OSs.
Assisted with Claude.
Push Policy::PrivateState() at all sites where the private state thread
needs to see the private reality instead of the public illusion:
- StopInfoBreakpoint::ShouldStopSynchronous (sync callbacks, WasHit)
- StopInfoWatchpoint::ShouldStopSynchronous
- RunThreadPlan (original PST during expression evaluation)
- RunPrivateStateThread (override PSTs)
Consult the policy at all view/capability decision points:
- Process::GetState() -- private vs public state
- Target::GetAPIMutex() -- private vs public mutex
- PrivateStateThread::GetRunLock() -- private vs public run lock
- Thread::GetStackFrameList() -- parent vs provider-augmented frames
- StackFrameList::SelectMostRelevantFrame() -- skip frame recognizers
- StopInfoBreakpoint::PerformAction() -- skip breakpoint actions
- StopInfoWatchpoint::PerformAction() -- skip watchpoint actions
The existing host thread identity checks remain as fallbacks until
Policy is pushed for all PSTs unconditionally.
This commit also add Stream::operator<<(bool) to allow streaming boolean
values as "true"/"false".
rdar://176223894
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
See the discussion in https://github.com/llvm/llvm-project/pull/192971
When LLDB makes the decision to eagerly send a breakpoint packet, it
should first ensure the delayed breakpoints are flushed, as they may
interfere with the eager breakpoint that is about to be changed.
Implementation note: we could have included the eager breakpoint in the
batch that is about to be flushed. However, it's important to get
information about the error status of this eager breakpoint, and the
current APIs dont make it easy to distinguish which breakpoint caused an
error.
This PR fixes two issues that contribute to `TestEvents.py` being flaky
in CI:
1. `ProcessEventData::DoOnRemoval` runs the full stop-handling logic
(like `ShouldStop` and `RunStopHooks`) every time an event is consumed
from any listener. When the primary listener consumes an event and then
the shadow listener consumes the same event, the logic runs twice. The
second execution can race with subsequent event processing. Fix this by
incrementing `m_update_state` after the first successful run so
secondary listeners skip the full logic.
2. Target::RunStopHooks updates `m_latest_stop_hook_id` (marking a stop
as "handled") before checking whether any threads have stop reasons. If
the check fails and hooks don't run, the stop ID is already consumed,
preventing hooks from ever running for that stop. Fix this by deferring
the update until we're certain we'll actually run hooks.
@jimingham You had suggested to @clayborg that we should call DidFork(),
DidVFork() and DidVForkDone() after a corresponding stop in expression
evaluation. Those functions require parameters that are private to the
specific child StopInfo class. I went with reusing the existing
StopInfo::PerformAction() virtual function, rather than create a new one
just for this purpose.
It may not be obvious from its name, but `Process::m_image_tokens`
contains loaded image addresses, not "tokens". The tokens are actually
indices into that array.
Add a process setting that keeps named threads running during
single-stepping operations. Some programs install in-process Mach
exception handler threads that must continue running to forward
exceptions. When lldb single-steps and suspends all other threads, these
handler threads stall, potentially causing a deadlock if the stepped
instruction raises an exception.
The new `target.process.always-run-thread-names` setting accepts a list
of thread name strings. In `ThreadList::WillResume`, threads whose names
match an entry in this list are resumed with `eStateRunning` instead of
`eStateSuspended`, ensuring they appear in vCont packets even during
single-thread stepping.
rdar://175038920
Frame providers are an overlay on top of the parent reality (the
unwinder stack). The private state thread (PST) manages the stop of that
parent reality, so the correct view for PST logic IS the parent --
providers should only be applied once the process has settled and
clients query the stopped state.
When a scripted breakpoint's `was_hit` callback calls
`EvaluateExpression` on the PST, `RunThreadPlan` spawns an override PST
(Thread B) and reassigns `m_current_private_state_thread_sp` to it. Two
threads then need to see parent frames:
- Thread B (override PST): processes stop events via
`HandlePrivateEvent` -> `ShouldStop` -> `GetStackFrameList`. If it loads
a provider, the provider's Python code can acquire locks held by Thread
A, causing a deadlock.
- Thread A (original PST): processes events inline via
`FindNextEventInternal` -> `DoOnRemoval` -> `GetStackFrameList`. After
the override is created, `CurrentThreadIsPrivateStateThread` no longer
recognizes Thread A (it checks against
`m_current_private_state_thread_sp`, which now points to Thread B).
The existing re-entrancy guard in `GetStackFrameList` (e1cd558) only
triggers when a provider is already active. In the deadlock scenario,
provider loading is being initiated for the first time, so the guard
does not trigger.
This patch introduces `PrivateStateThreadGuard`, an RAII guard that sets
a `thread_local` flag checked by `GetStackFrameList` to return parent
frames instead of loading providers. The guard is activated in two
places:
- `RunPrivateStateThread`: for override PSTs only (identified via a new
`is_override` flag on `PrivateStateThread`). The original PST does not
set the guard here, so normal stepping still sees provider-augmented
frames.
- `RunThreadPlan`: for the original PST, scoped to the event processing
window when an override has been spawned.
rdar://174679105
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This PR upstreams ReadUnsignedIntegersFromMemory, which uses the
MultiMemRead packet to speed up reading unsigned numbers from memory.
Felipe landed this on Github because we didn't have a use-case for it
upstream, until now, which uses it to batch up reading ObjC runtime
symbols.
This patch makes the default parameter to ConstString::AsCString
explicit. Currently it defaults to `nullptr`. However, there a bunch of
callsites that do `printf("%s", foo.AsCString())`, which is UB (I have
at least 1 crash on me where this was the root cause).
Alternatively we could change the default to `""` instead of `nullptr`,
but code that did `if (foo.AsCString())` would then change meaning.
Command I ran to generate this patch:
```
rg "\.AsCString\(\)" lldb -l | xargs sed -i '' 's/\.AsCString()/.AsCString(nullptr)/g'
```
There's also a commonly used `Status::AsCString` with a default
parameter. So I added a temporary:
```
const char *AsCString(std::nullptr_t) const = delete;
```
overload to `Status`, and fixed all the compile errors.
In a follow-up PR I intend to change the callsites that do `printf("%s",
foo.AsCString(nullptr)` to pass `""` as the fallback parameter. Another
dangrous pattern is `std::string(error.AsCString(nullptr))`
rdar://171219173
The LLVM Coding Standards [1] specify that:
> [T]o match error message styles commonly produced by other tools,
> start the first sentence with a lowercase letter, and finish the last
> sentence without a period, if it would end in one otherwise.
Historically, that hasn't been something we've enforced in LLDB, but in
the past year or so I've started to pay more attention to this in code
reviews. This PR brings more error messages in compliance, further
increasing consistency.
I also adopted `createStringErrorV` where it improved the code as a
drive-by for lines I was already touching.
[1] https://llvm.org/docs/CodingStandards.html#error-and-warning-messages
Assisted-by: Claude Code
The only difference between them is that OptionalBool's third state
is "unknown" and LazyBool's is "calculate". We don't need to tell
the difference in a single context, so I've made a new eLazyBoolDontKnow
which is an alias of eLazyBoolCalculate.
Avoids manual memory management.
Uses `shared_ptr` instead of `unique_ptr` because we store references to
the current thread in a backup variable.
Simplifies the private thread `is_secondary` semantics by providing a
backup storage for the current thread instead of a boolean value with a
contract to manage the backup separately.
lldb had three preprocessor defines for logging,
LLDB_LOG - formatv style argument
LLDB_LOGF - printf style argument
LLDB_LOGV - formatv style argument, only when verbose enabled
If you weren't looking at Log.h and the definition of these three, and
wanted to log something with formatv, it was easy to use LLDB_LOGV by
accident. We just had a situation where an important log statement
wasn't logging and it turned out to be this. This is fragile if you
aren't looking at the header directly, so I'd like to make this more
explicit. My proposal:
LLDB_LOG - formatv style argument
LLDB_LOG_VERBOSE - formatv style argument, only when verbose enabled
LLDB_LOGF - printf style argument
LLDB_LOGF_VERBOSE - printf style argument, only when verbose enabled
The new fouth one is to remove several places where we do `if (log &&
log->GetVerbose()) LLDB_LOGF (...)` in the sources today, and make both
styles consistent.
This PR implements that change, mechanically changing all LLDB_LOGV's to
LLDB_LOG_VERBOSE.
It also updates many of the `if (log && log->GetVerbose()) LLDB_LOGF`'s.
Some uses of this conditional expression do extra calculations in
addition to logging, and so those were left as-is so we're not doing
throwaway work when running without verbose logging.
There were many instances throughout lldb where callers are still doing
`if (log) LLDB_LOG*(...)`, a remnant of when all calls were to the `Log`
object's `Printf()` method, and you had to check if your local Log*
pointer was non-nullptr before calling the method. I removed those,
again keeping ones where work for logging is done in the block of code.
The code changes are all mechanical and uninteresting, but the question
of whether this naming change is widely agreed on is maybe worth
discussing.
This PR replaces the Get*CallbackAtIndex pattern in the PluginManager
with returning a snapshot of callbacks that the caller can iterate over
using a range-based for loop. This is a continuation of #184452 which
added thread safety by using snapshots. However, that introduced a bunch
of unnecessary copies which are largely eliminated again by getting the
snapshot once when gather all the callbacks, rather than doing that on
each iteration when querying a plugin for a given index. It also
eliminates the possibility of the snapshot changing underneath you when
iterating over the plugins.
This change was largely mechanical and I used Claude to do the menial
work of updating the signatures and call sites.
This reverts commit 97572c1860.
This patch seems to cause TestWatchpointCommandPython.py to time out
on the ubuntu buildbots (but nowhere else that I can find so far.) The
timeout is weird too, the TEST FILE is timing out but the individual
tests aren't being shown and there's no other output. Grrr...
Anyway I'll revert this and then see if I can do some guessing about
how this change might cause the test to fail.
When lldb stops to run a breakpoint condition or other callback that has
to happen between the private stop and returning control to the user, it
will run in the state where the public state is still "running". But if
the callback needs to run lldb commands or python code, it needs to see
the correct "stop" state.
We used to handle that by switching the public state to stopped before
running the callbacks. However, that opened a window where we are still
handling the stop event and another thread would be allowed to continue
the target or do other actions that can interfere with that orderly
process.
This patch adds the ability to designate a particular thread as "seeing
the private state" while all other threads see the "public state". Then
when we run a breakpoint callback, no threads but the one that is
actually running the callback will see the state change until the event
has been delivered to the primary state listener.
It also adds a test that while a long-running breakpoint callback runs,
another thread continues to see the state as running.
In #184259, Jim noticed that Debugger::FindTargetWithProcess and
Debugger::FindTargetWithProcessID are rather poorly designed APIs as
tehy allow code running in one Debugger to mess with Targets from
another Debugger. The only use is Process::SetProcessExitStatus which
isn't actually used.
Fix asan error mentioned in [this
comment](https://github.com/llvm/llvm-project/pull/179799#issuecomment-3882710678)
This is breaking our internal builds.
Details: The field needs to be cleaned up or we'll have a leak. We could
also make it a smart pointer if you'd like. But as-is, this is the
quickest way to resolve the issue.
In #168245, I attempted to dump the available settings to Markdown. That
required a full build of LLDB. However, to build the docs, only the swig
wrappers should need to be compiled. The comment was that we should be
able to use the definitions from the TableGen files.
Currently, the property definitions in don't have information about the
path where they will be available. They only contain a `Definition`
which groups properties, so they can be added to
`OptionValueProperties`.
With this PR, I'm adding the path for each property definition. For
example, `symbols.enable-external-lookup` would have `Name =
enable-external-lookup, Path = symbols`. In LLDB itself, we don't need
this path, we only need it for the documentation. To avoid mismatches
between the actual path and the declared one, I added a debug-only check
when a property group is added to a parent
(`OptionValueProperties::AppendProperty`).
The TableGen emitter for the properties now additionally emits
`g_{definition}_properties_def`, which includes both the array of
properties and the expected path. This constant has to be used to
initialize a `OptionValueProperties`.
I couldn't test this for everything (e.g. IntelPT or ProcessKDP), but
the necessary changes are simple: (1) set the `Path` in the TableGen
file, (2) update `initialize` to use `_def`.
This ivar was passed to the thread function for the private state thread
- mostly because the equivalent variable was passed in in the original
version of the code. But it was never used, so I didn't notice that the
ivar equivalent wasn't being initialized. I'm going to keep the ivar
because it will be useful when debugging to easily see whether you are
on the main or secondary private state thread.
So in this change, I set m_is_secondary_thread properly in the
constructor, but remove passing it to the thread function since it
wasn't needed there.
We have a problem when some code on the private state thread needs to
run an expression. The private state thread is the one that fetches
"raw" events from the Process Plugin and decides how to handle them. But
if the private state thread needs to fetch the processed events to drive
running an expression, it can't also be the thread processing the raw
events.
We solve this by swapping in a modal private state thread just to handle
the events from the expression evaluation. That worked until you could
cause the expression evaluation to happen from Python, because then it
wasn't just the fetching of events that matter, but also the state of
the process and the state of the runlocks. The modal private state
thread is really a modal version of the thread and its associated state.
This patch gathers all the relevant control parameters into a structure
which we can swap in and out when needed.
It also adds a test using the new "was_hit" breakpoint resolver
affordance, which, since it acts as an asynchronous breakpoint callback,
gets run on the private state thread, showing that with the change we
can call expressions in the `was_hit` callback without problems.
I noticed that Module::GetMemoryObjectFile populates a Status object
upon error but it's effectively dropped on the floor. Instead, the
clients can report the error as desired.
At the moment, all clients are either (1) consuming the error because
it's only trying to find a module, or (2) log the error and bail out
early. I tried to preserve existing behavior as faithfully as possible.
This commit uses Process::ReadMemoryRanges to create an efficient method
for reading multiple strings at once. This method works like the
single-string version, reading 256 bytes at a time, but instead doing it
for _every_ string requested at the same time.