diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 249140956..bec587f0e 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -228,11 +228,22 @@ - [Debugging LLVM](./backend/debugging.md) - [Backend Agnostic Codegen](./backend/backend-agnostic.md) - [Implicit caller location](./backend/implicit-caller-location.md) +- [Debug Info](./debuginfo/intro.md) + - [Rust Codegen](./debuginfo/rust-codegen.md) + - [LLVM Codegen](./debuginfo/llvm-codegen.md) + - [Debugger Interanls](./debuginfo/debugger-internals.md) + - [LLDB Internals](./debuginfo/lldb-internals.md) + - [GDB Internals](./debuginfo/gdb-internals.md) + - [Debugger Visualizers](./debuginfo/debugger-visualizers.md) + - [LLDB - Python Providers](./debuginfo/lldb-visualizers.md) + - [GDB - Python Providers](./debuginfo/gdb-visualizers.md) + - [CDB - Natvis](./debuginfo/natvis-visualizers.md) + - [Testing](./debuginfo/testing.md) + - [(Lecture Notes) Debugging support in the Rust compiler](./debugging-support-in-rustc.md) - [Libraries and metadata](./backend/libs-and-metadata.md) - [Profile-guided optimization](./profile-guided-optimization.md) - [LLVM source-based code coverage](./llvm-coverage-instrumentation.md) - [Sanitizers support](./sanitizers.md) -- [Debugging support in the Rust compiler](./debugging-support-in-rustc.md) --- diff --git a/src/debuginfo/CodeView.pdf b/src/debuginfo/CodeView.pdf new file mode 100644 index 000000000..f899d2517 Binary files /dev/null and b/src/debuginfo/CodeView.pdf differ diff --git a/src/debuginfo/debugger-internals.md b/src/debuginfo/debugger-internals.md new file mode 100644 index 000000000..114ce8a99 --- /dev/null +++ b/src/debuginfo/debugger-internals.md @@ -0,0 +1,14 @@ +# Debugger Internals + +It is the debugger's job to convert the debug info into an in-memory representation. Both the +interpretation of the debug info and the in-memory representation are arbitrary; anything will do +so long as meaningful information can be reconstructed while the program is running. The pipeline +from raw debug info to usable types can be quite complicated. + +Once the information is in a workable format, the debugger front-end then must provide a way to +interpret and display the data, a way for users to interact with it, and an API for extensibility. + +Debuggers are vast systems and cannot be covered completely here. This section will provide a brief +overview of the subsystems directly relevant to the Rust debugging experience. + +Microsoft's debugging engine is closed source, so it will not be covered here. \ No newline at end of file diff --git a/src/debuginfo/debugger-visualizers.md b/src/debuginfo/debugger-visualizers.md new file mode 100644 index 000000000..07e3861a5 --- /dev/null +++ b/src/debuginfo/debugger-visualizers.md @@ -0,0 +1,62 @@ +# Debugger Visualizers + +These are typically the last step before the debugger displays the information, but the results may +be piped through a debug adapter such as an IDE's debugger API. + +The term "Visualizer" is a bit of a misnomer. The real goal isn't just to prettify the output, but +to provide an interface for the user to interact with that is as useful as possible. In many cases +this means reconstructing the original type as closely as possible to its Rust representation, but +not always. + +The visualizer interface allows generating "synthetic children" - fields that don't exist in the +debug info, but can be derived from invariants about the language and the type itself. A simple +example is allowing one to interact with the elements of a `Vec` instead of just it's `*mut u8` +heap pointer, length, and capacity. + +## Performance + +Before tackling the visualizers themselves, it's important to note that these are part of a +performance-sensitive system. Please excuse the break in formality, but: if I have to spend +significant time debugging, I'm annoyed. If I have to *wait on my debugger*, I'm pissed. + +Every millisecond spent in these visualizers is a millisecond longer for the user to see output. +This can be especially painful for large stackframes that contain many/large container types. +Debugger GUI's such as VSCode will request the whole stack frame at once, and this can result in +delays of tens of seconds (or even minutes) before being able to interact with any variables in the +frame. + +There is a tendancy to balk at the idea of optimizing Python code, but it really can have a +substantial impact. Remember, there is no compiler to help keep the code fast. Even simple +transformations are not done for you. It can be difficult to find Python performance tips through +all the noise of people suggesting you don't bother optimizing Python, so here are some things to +keep in mind that are relevant to these scripts: + +* Everything allocates, even `int` +* Use tuples when possible. `list` is effectively `Vec>`, whereas tuples are equivalent +to `Box<[Any]>`. They have one less layer of indirection, don't carry extra capacity and can't +grow/shrink which can be advantageous in many cases. An additional benefit is that Python caches and +recycles the underlying allocations of all tuples up to size 20. +* Regexes are slow and should be avoided when simple string manipulation will do +* Strings are immutable, thus many string operations implictly copy the contents. +* When concatenating large lists of strings, `"".join(iterable_of_strings)` is typically the fastest +way to do it. +* f-strings are generally the fastest way to do small, simple string transformations such as +surrounding a string with parentheses. +* The act of calling a function is somewhat slow (even if the function is completely empty). If the +code section is very hot, consider inlining the function manually. +* Local variable access is significantly faster than global and built-in function access +* Member/method access via the `.` operator is also slow, consider reassigning deeply nested values +to local variables to avoid this cost (e.g. `h = a.b.c.d.e.f.g.h`). +* Accessing inherited methods and fields is about 2x slower than base-class methods and fields. +Avoid inheritance whenever possible. +* Use [`__slots__`](https://wiki.python.org/moin/UsingSlots) wherever possible. `__slots__` is a way +to indicate to Python that your class's fields won't change and speeds up field access by a +noticable amount. This does require you to name your fields in advance and initialize them in +`__init__`, but it's a small price to pay for the benefits. +* Match statements/if..elif..else are not optimized in any way. The conditions are checked in order, +1 by 1. If possible, use an alternative such as dictionary dispatch or a table of values +* Compute lazily when possible +* List comprehensions are typically faster than loops, generator comprehensions are a bit slower +than list comprehensions, but use less memory. You can think of comprehensions as equivalent to +Rust's `iter.map()`. List comprehensions effectively call `collect::>` at the end, whereas +generator comprehensions do not. \ No newline at end of file diff --git a/src/debuginfo/gdb-internals.md b/src/debuginfo/gdb-internals.md new file mode 100644 index 000000000..95f543c8e --- /dev/null +++ b/src/debuginfo/gdb-internals.md @@ -0,0 +1,4 @@ +# (WIP) GDB Internals + +GDB's Rust support lives at `gdb/rust-lang.h` and `gdb/rust-lang.c`. The expression parsing support +can be found in `gdb/rust-exp.h` and `gdb/rust-parse.c` \ No newline at end of file diff --git a/src/debuginfo/gdb-visualizers.md b/src/debuginfo/gdb-visualizers.md new file mode 100644 index 000000000..4027ef897 --- /dev/null +++ b/src/debuginfo/gdb-visualizers.md @@ -0,0 +1,9 @@ +# (WIP) GDB - Python Providers + +Below are links to relevant parts of the GDB documentation + +* [Overview on writing a pretty printer](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Writing-a-Pretty_002dPrinter.html#Writing-a-Pretty_002dPrinter) +* [Pretty Printer API](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Pretty-Printing-API.html#Pretty-Printing-API) (equivalent to LLDB's `SyntheticProvider`) +* [Value API](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Values-From-Inferior.html#Values-From-Inferior) (equivalent to LLDB's `SBValue`) +* [Type API](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Types-In-Python.html#Types-In-Python) (equivalent to LLDB's `SBType`) +* [Type Printing API](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Type-Printing-API.html#Type-Printing-API) (equivalent to LLDB's `SyntheticProvider.get_type_name`) diff --git a/src/debuginfo/intro.md b/src/debuginfo/intro.md new file mode 100644 index 000000000..0f1e59fa6 --- /dev/null +++ b/src/debuginfo/intro.md @@ -0,0 +1,114 @@ +# Debug Info + +Debug info is a collection of information generated by the compiler that allows debuggers to +correctly interpret the state of a program while it is running. That includes things like mapping +instruction addresses to lines of code in the source file, and type layout information so that +bytes in memory can be read and displayed in a meaningful way. + +Debug info can be a slightly overloaded term, covering all the layers between Rust MIR, and the +end-user seeing the output of their debugger onscreen. In brief, the stack from beginning to end is +as follows: + +1. Rustc inspects the MIR and communicates the relevant source, symbol, and type information to LLVM +2. LLVM translates this information into a target-specific debug info format during compilation +3. A debugger reads and interprets the debug info, mapping source-lines and allowing the debugee's +variables in memory to be located and read with the correct layout +4. Built-in debugger formatting and styling is applied to variables +5. User-defined scripts are run, formatting and styling the variables further +6. The debugger frontend displays the variable to the user, possibly through the means of additional +API layers (e.g. VSCode extension by way of the +[Debug Adapter Protocol](https://microsoft.github.io/debug-adapter-protocol/)) + + +> NOTE: This subsection of the dev guide is perhaps more detailed than necessary. It aims to collect +> a large amount of scattered information into one place and equip the reader with as firm a grasp of +> the entire debug stack as possible. +> +> If you are only interested in working on the visualizer +> scripts, the information in the [debugger-visualizers](./debugger-visualizers.md) and +> [testing](./testing.md) will suffice. If you need to make changes to Rust's debug node generation, +> please see [rust-codegen](./rust-codegen.md). All other sections are supplementary, but can be +> vital to understanding some of the compromises the visualizers or codegen need to make. It can +> also be valuable to know when a problem might be better solved in LLVM or the debugger itself. + +# DWARF + +The is the primary debug info format for `*-gnu` targets. It is typically bundled in with the +binary, but it [can be generated as a separate file](https://gcc.gnu.org/wiki/DebugFission). The +DWARF standard is available [here](https://dwarfstd.org/). + +> NOTE: To inspect DWARF debug info, [gimli](https://crates.io/crates/gimli) can be used +> programatically. If you prefer a GUI, the author recommends [DWEX](https://github.com/sevaa/dwex) + +# PDB/CodeView + +The primary debug info format for `*-msvc` targets. PDB is a proprietary container format created by +Microsoft that, unfortunately, +[has multiple meanings](https://docs.rs/ms-pdb/0.1.10/ms_pdb/taster/enum.Flavor.html). +We are concerned with ordinary PDB files, as Portable PDB is used mainly for .Net applications. PDB +files are separate from the compiled binary and use the `.pdb` extension. + +PDB files contain CodeView objects, equivalent to DWARF's tags. CodeView, the debugger that +consumed CodeView objects, was originally released in 1985. Its original intent was for C debugging, +and was later extended to support Visual C++. There are still minor alterations to the format to +support modern architectures and languages, but many of these changes are undocumented and/or +sparsely used. + +It is important to keep this context in mind when working with CodeView objects. Due to its origins, +the "feature-set" of these objects is very limited, and focused around the core features of C. It +does not have many of the convenience or features of modern DWARF standards. A fair number of +workarounds exist within the debug info stack to compensate for CodeView's shortcomings. + +Due to its proprietary nature, it is very difficult to find information about PDB and CodeView. Many +of the sources were made at vastly different times and contain incomplete or somewhat contradictory +information. As such this page will aim to collect as many sources as possible. + +* [CodeView 1.0 specification](./CodeView.pdf) +* LLVM + * [CodeView Overview](https://llvm.org/docs/SourceLevelDebugging.html#codeview-debug-info-format) + * [PDB Overview and technical details](https://llvm.org/docs/PDB/index.html) +* Microsoft + * [microsoft-pdb](https://github.com/microsoft/microsoft-pdb) - A C/C++ implementation of a PDB + reader. The implementation does not contain the full PDB or CodeView specification, but does + contain enough information for other PDB consumers to be written. At time of writing (Nov 2025), + this repo has been archived for several years. + * [pdb-rs](https://github.com/microsoft/pdb-rs/) - A Rust-based PDB reader and writer based on + other publicly-available information. Does not guarantee stability or spec compliance. Also + contains `pdbtool`, which can dump PDB files (`cargo install pdbtool`) + * [Debug Interface Access SDK](https://learn.microsoft.com/en-us/visualstudio/debugger/debug-interface-access/getting-started-debug-interface-access-sdk). + While it does not document the PDB format directly, details can be gleaned from the interface + itself. + +# Debuggers + +Rust supports 3 major debuggers: GDB, LLDB, and CDB. Each has its own set of requirements, +limitations, and quirks. This unfortunately creates a large surface area to account for. + +> NOTE: CDB is a proprietary debugger created by Microsoft. The underlying engine also powers +>WinDbg, KD, the Microsoft C/C++ extension for VSCode, and part of the Visual Studio Debugger. In +>these docs, it will be referred to as CDB for consistency + +While GDB and LLDB do offer facilities to natively support Rust's value layout, this isn't +completely necessary. Rust currently outputs debug info very similar to that of C++, allowing +debuggers without Rust support to work with a slightly degraded experience. More detail will be +included in later sections, but here is a quick reference for the capabilities of each debugger: + +| Debugger | Debug Info Format | Native Rust support | Expression Style | Visualizer Scripts | +| --- | --- | --- | --- | --- | +| GDB | DWARF | Full | Rust | Python | +| LLDB | DWARF and PDB | Partial | C/C++ | Python | +| CDB | PDB | None | C/C++ | Natvis | + +> IMPORTANT: CDB can be assumed to run only on Windows. No assumptions can be made about the OS +>running GDB or LLDB. + +## Unsupported + +Below, are several unsupported debuggers that are of particular note due to their potential impact +in the future. + +* [Bugstalker](https://github.com/godzie44/BugStalker) is an x86-64 Linux debugger written in Rust, +specifically to debug Rust programs. While promising, it is still in early development. +* [RAD Debugger](https://github.com/EpicGamesExt/raddebugger) is a Windows-only GUI debugger. It has +a custom debug info format that PDB is translated into. The project also includes a linker that can +generate their new debug info format during the linking phase. \ No newline at end of file diff --git a/src/debuginfo/lldb-internals.md b/src/debuginfo/lldb-internals.md new file mode 100644 index 000000000..301dda977 --- /dev/null +++ b/src/debuginfo/lldb-internals.md @@ -0,0 +1,205 @@ +# LLDB Internals + +LLDB's debug info processing relies on a set of extensible interfaces largely defined in +[lldb/src/Plugins][lldb_plugins]. These are meant to allow third-party compiler developers to add +language support that is loaded at run-time by LLDB, but at time of writing (Nov 2025) the public +API has not been settled on, so plugins exist either in LLDB itself or in standalone forks of LLDB. + +[lldb_plugins]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins + +Typically, language support will be written as a pipeline of these plugins: `*ASTParser` -> +`TypeSystem` -> `ExpressionParser`/`Language`. + +Here are some existing implementations of LLDB's plugin API: + +* [Apple's fork with support for Swift](https://github.com/swiftlang/llvm-project) +* [CodeLLDB's former fork with support for Rust](https://archive.softwareheritage.org/browse/origin/directory/?branch=refs/heads/codelldb/16.x&origin_url=https://github.com/vadimcn/llvm-project&path=lldb/source/Plugins/TypeSystem/Rust×tamp=2023-09-11T04:55:10Z) +* [A work in progress reimplementation of Rust support](https://github.com/Walnut356/llvm-project/tree/lldbrust/19.x) + +## Rust Support and TypeSystemClang + +As mentioned in the debug info overview, LLDB has partial Rust support. To further clarify, Rust +uses the plugin-pipeline that was built for C/C++ (though it contains some helpers for Rust enum +types), which relies directly on the `clang` compiler's representation of types. This imposes heavy +restrictions on how much we can change when LLDB's output doesn't match what we want. Some +workarounds can help, but at the end of the day Rust's needs are secondary compared to making sure +C and C++ compilation and debugging work correctly. + +LLDB is receptive to adding a `TypeSystemRust`, but it is a massive undertaking. This section serves +to not only document how we currently interact with [`TypeSystemClang`][ts_clang], but also as light +guidance on implementing a `TypeSystemRust` in the future. + +[ts_clang]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/TypeSystem/Clang + +It is worth noting that a `TypeSystem` directly interacting with the target language's compiler is +the intention, but it is not a requirement. One can create all the necessary supporting types within +their plugin implementation. + +> Note: LLDB's documentation, including comments in the source code, is pretty sparse. Trying to +> understand how language support works by reading `TypeSystemClang`'s implementation is somewhat +> difficult due to the added requirement of understanding the `clang` compiler's internals. It is +> recommended to look at the 2 `TypeSystemRust` implementations listed above, as they are written +> "from scratch" without leveraging a compiler's type representation. They are relatively close to +> the minimum necessary to implement language support. + +## DWARF vs PDB + +LLDB is unique in being able to handle both DWARF and PDB debug information. This does come with +some added complexity. To complicate matters further, PDB support is split between `dia`, which +relies on the `msdia140.dll` library distributed with Visual Studio, and `native`, which is written +from scratch using publicly available information about the PDB format. + +> Note: `dia` was the default up to LLDB version 21. `native` is the new default as of +> LLDB 22's release. There are plans to deprecate and completely remove the `dia`-based plugins. As +> such, only `native` parsing will be discussed below. For progress, please see +> [this discourse thread][dia_discourse] and the relevant [tracking issue][dia_tracking]. +> +> `native` can be toggled via the `plugin.symbol-file.pdb.reader` setting added in LLDB 22 or using +> the environment variable `LLDB_USE_NATIVE_PDB_READER=0/1` + +[dia_discourse]: https://discourse.llvm.org/t/rfc-removing-the-dia-pdb-plugin-from-lldb/87827 +[dia_tracking]: https://github.com/llvm/llvm-project/issues/114906 + +## Debug Node Parsing + +The first step is to process the raw debug nodes into something usable. This primarily occurs in +the [`DWARFASTParser`][dwarf_ast] and [`PdbAstBuilder`][pdb_ast] classes. These classes are fed a +deserialized form of the debug info generated from [`SymbolFileDWARF`][sf_dwarf] and +[`SymbolFileNativePDB`][sf_pdb] respectively. The `SymbolFile` implementers make almost no +transformations to the underlying debug info before passing it to the parsers. For both PDB and +DWARF, the debug info is read using LLVM's debug info handlers. + +[dwarf_ast]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/SymbolFile/DWARF +[pdb_ast]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/SymbolFile/NativePDB +[sf_dwarf]: https://github.com/llvm/llvm-project/blob/main/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h +[sf_pdb]: https://github.com/llvm/llvm-project/blob/main/lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.h + +The parsers translate the nodes into more convenient formats for LLDB's purposes. For `clang`, these +formats are `clang::QualType`, `clang::Decl`, and `clang::DeclContext`, which are the types `clang` +uses internally when compiling C and C++. Again, using the compiler's representation of types is not a +requirement, but the plugin system was built with it as a possibility. + +> Note: The above types will be referred to language-agnostically as `LangType`, `Decl`, and +`DeclContext` when the specific implementation details of `TypeSystemClang` are not relevant. + +`LangType` represents a type. This includes information such as the name of the type, the size and +alignment, its classification (e.g. struct, primitive, pointer), its qualifiers (e.g. +`const`, `volatile`), template arguments, function argument and return types, etc. [Here][rust_type] +is an example of what a `RustType` might look like. + +[rust_type]: https://github.com/Walnut356/llvm-project/blob/13bcfd502452606d69faeea76aec3a06db554af9/lldb/source/Plugins/TypeSystem/Rust/TypeSystemRust.h#L618 + +`Decl` represents any kind of declaration. It could be a type, a variable, a static field of a +struct, the value that a static or const is initialized with, etc. + +`DeclContext` more or less represents a scope. `DeclContext`s typically contain `Decl`s and other +`DeclContexts`, though the relationship isn't that straight forward. For example, a function can be +both a `Decl` (because function signatures are types), **and** a `DeclContext` (because functions +contain variable declarations, nested functions declarations, etc.). + +The translation process can be quite verbose, but is usually straightforward. Much of the work here +is dependant on the exact information needed to fill out `LangType`, `Decl`, and `DeclContext`. + +Once a node is translated, a pointer to it is type-erased (`void*`) and wrapped in `CompilerType`, +`CompilerDecl`, or `CompilerDeclContext`. These wrappers associate the them with the `TypeSystem` +that owns them. Methods on these objects delegates to the `TypeSystem`, which casts the `void*` back +to the appropriate `LangType*`/`Decl*`/`DeclContext*` and operates on the internals. In Rust terms, +the relationship looks something like this: + +```Rust +struct CompilerType { + inner_type: *mut c_void, + type_system: Arc, +} + +impl CompilerType { + pub fn get_byte_size(&self) -> usize { + self.type_system.get_byte_size(self.lang_type) + } + +} + +... + +impl TypeSystem for TypeSystemLang { + pub fn get_byte_size(lang_type: *mut c_void) -> usize { + let lang_type = lang_type as *mut LangType; + + // Operate on the internals of the LangType to + // determine its size + ... + } +} +``` + +## Type Systems + +The [`TypeSystem` interface][ts_interface] has 3 major purposes: + +[ts_interface]: https://github.com/llvm/llvm-project/blob/main/lldb/include/lldb/Symbol/TypeSystem.h#L69 + +1. Act as the "sole authority" of a language's types. This allows the type system to be added to +LLDB's "pool" of type systems. When an executable is loaded, the target language is determined, and +the pool is queried to find a `TypeSystem` that claims it can handle the language. One can also use +the `TypeSystem` to retrieve the backing `SymbolFile`, search for types, and synthesize basic types +that might not exist in the debug info (e.g. primitives, arrays-of-`T`, pointers-to-`T`). +2. Manage the lifetimes of the `LangType`, `Decl`, and `DeclContext` objects +3. Customize the "defaults" of how those types appear and how they can be interacted with. + +The first two functions are pretty straightforward so we will focus on the third. + +Many of the functions in the `TypeSystem` interface will look familiar if you have worked with the +visualizer scripts. These functions underpin `SBType` the `SBValue` functions with matching names. +For example, `TypeSystem::GetFormat` returns the default format for the type if no custom formatter +has been applied to it. + +Of particular note are `GetIndexOfChildWithName` and `GetNumChildren`. The `TypeSystem` versions of +these functions operate on a *type*, not a value like the `SBValue` versions. The values returned +from the `TypeSystem` functions dictate what parts of the struct can be interacted with *at all* by +the rest of LLDB. If a field is ommitted, that field effectively no longer exists to LLDB. + +Additionally, since they do not work with objects, there is no underlying memory to inspect or +interpret. Essentially, this means these functions do not have the same purpose as their equivalent +`SyntheticProvider` functions. There is no way to determine how many elements a `Vec` has or what +address those elements live at. It is also not possible to determine the value of the discriminant +of a sum-type. + +Ideally, the `TypeSystem` should expose types as they appear in the debug info with as few +alterations as possible. LLDB's synthetics and frontend can handle making the type pretty. If some +piece of information is useless, the Rust compiler should be altered to not output that debug info +in the first place. + +## Expression Parsing + +The `TypeSystem` is typically written to have a counterpart that can handle expression parsing. It +requires implementing a few extra functions in the `TypeSystem` interface. The bulk of the +expression parsing code should live in [lldb/source/Plugins/ExpressionParser][expr]. + +[expr]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/ExpressionParser + +There isn't too much of note about the parser. It requires implementing a simple interpreter that +can handle (possibly simplified) Rust syntax. They operate on `lldb::ValueObject`s, which are the +objects that underpin `SBValue`. + +## Language + +The [`Language` plugins][lang_plugin] are the C++ equivalent to the Python visualizer scripts. They +operate on `SBValue` objects for the same purpose: creating synthetic children and pretty-printing. +The [CPlusPlusLanguage's implementations][cpp_lang] for the LibCxx types are great resources to +learn how visualizers should be written. + +[lang_plugin]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/Language +[cpp_lang]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/Language/CPlusPlus + +These plugins can access LLDB's private internals (including the underlying `TypeSystem`), so +synthetic/summary providers written as a `Language` plugin can provide higher quality output than +their python equivalent. + +While debug node parsing, type systems, and expression parsing are all closely tied to eachother, +the `Language` plugin is encapsulated more and thus can be written "standalone" for any language +that an existing type system supports. Due to the lower barrier of entry, a `RustLanguage` plugin +may be a good stepping stone towards full language support in LLDB. + +## Visualizers + +WIP \ No newline at end of file diff --git a/src/debuginfo/lldb-visualizers.md b/src/debuginfo/lldb-visualizers.md new file mode 100644 index 000000000..c7bcc2502 --- /dev/null +++ b/src/debuginfo/lldb-visualizers.md @@ -0,0 +1,662 @@ +# LLDB - Python Providers + +> NOTE: LLDB's C++<->Python FFI expects a version of python designated at the time LLDB was +>compiled. LLDB is careful to correspond this version to the minimum in typical Linux and MacOs +>distributions, but on Windows there is no easy solution. If you recieve an import error regarding +>`_lldb` not existing, a mismatched Python version is likely the cause. +> +> LLDB is considering solutions this issue. For updates, see +>[this discussion][minimal_python_install] and [this github issue][issue_167001] + +[minimal_python_install]: https://discourse.llvm.org/t/a-minimal-python-install-for-lldb/88658 +[issue_167001]: https://github.com/llvm/llvm-project/issues/167001 + +> NOTE: Currently (Nov 2025), LLDB's minimum supported Python version is 3.8 with plans to update it to +>3.9 or 3.10 depending on several outside factors. Scripts should ideally be written with only the +>features available in the minimum supported Python version. Please see [this discussion][mrpv] for +>more info. + +[mrpv]: https://discourse.llvm.org/t/rfc-upgrading-llvm-s-minimum-required-python-version/88605/ + +> NOTE: The path to LLDB's python package can be located via the CLI command `lldb -P` + +LLDB provides 3 mechanisms for customizing output: + +* Formats +* Synthetic providers +* Summary providers + +## Formats + +The official documentation is [here](https://lldb.llvm.org/use/variable.html#type-format). In short, +formats allow one to set the default print format for primitive types (e.g. print `25u8` as decimal +`25`, hex `0x19`, or binary `00011001`). + +Rust will almost always need to override `unsigned char`, `signed char`, `char`, `u8`, and `i8`, to +(unsigned) decimal format. + +## Synthetic Providers + +The official documentation is [here](https://lldb.llvm.org/use/variable.html#synthetic-children), +but some information is vague, outdated, or entirely missing. + +Nearly all interaction the user has with variables will be through LLDB's +[`SBValue` objects][sbvalue] which are used both in the Python API, and internally via LLDB's +plugins and CLI. + +[sbvalue]: https://lldb.llvm.org/python_api/lldb.SBValue.html + +A Synthetic Provider is a Python class, written with a specific interface, that is associated with +one or more Rust types. The Synthetic Provider wraps `SBValue` objects and LLDB will call our +class's functions when inspecting the variable. + +The wrapped value is still an `SBValue`, but when calling e.g. `SBValue.GetChildAtIndex`, it will +internally call `SyntheticProvider.get_child_at_index`. You can check if a value has a synthetic +provider via `SBValue.IsSynthetic()`, and which synthetic it is via `SBValue.GetTypeSynthetic()`. If +you want to interact with the underlying non-synthetic value, you can call +`SBValue.GetNonSyntheticValue()`. + + +The expected interface is as follows: + +```python +class SyntheticProvider: + def __init__(self, valobj: SBValue, _lldb_internal): ... + + # optional + def update(self) -> bool: ... + + # optional + def has_children(self) -> bool: ... + + # optional + def num_children(self, max_children: int) -> int: ... + + def get_child_index(self, name: str) -> int: ... + + def get_child_at_index(self, index: int) -> SBValue: ... + + # optional + def get_type_name(self) -> str: ... + + # optional + def get_value(self) -> SBValue: ... +``` + +Below are explanations of the methods, their quirks, and how they should generally be used. If a +method overrides an `SBValue` method, that method will be listed. + +### `__init__` + +This function is called once per object, and must store the `valobj` in the python class so that it +is accessible elsewhere. Very little else should be done here. + +### (optional) `update` + +This function is called prior to LLDB interacting with a variable, but after `__init__`. LLDB tracks +whether `update` has already been called. If it has been, and if it is not possible for the variable +to have changed (e.g. inspecting the same variable a second time without stepping), it will omit the +call to `update`. + +This function has 2 purposes: + +* Store/update any information that may have changed since the last time `update` was run +* Inform LLDB if there were changes to the children such that it should flush the child cache. + +Typical operations include storing the heap pointer, length, capacity, and element type of a `Vec`, +determining an enum variable's variant, or checking which slots of a `HashMap` are occupied. + +The bool returned from this function is somewhat complicated, see: +[`update` caching](#update-caching) below for more info. When in doubt, return `False`/`None`. +Currently (Nov 2025), none of the visualizers return `True`, but that may change as the debug info +test suite is improved. + +#### `update` caching + +LLDB attempts to cache values when possible, including child values. This cache is effectively the +number of child objects, and the addresses of the underlying debugee memory that the child object +represents. By returning `True`, you indicate to LLDB that the number of children and the addresses +of those children have not changed since the last time `update` was run, meaning it can reuse the +cached children. + +**Returning `True` in the wrong circumstances will result in the debugger outputting incorrect +information**. + +Returning `False` indicates that there have been changes, the cache will be flushed, and the +children will be fetched from scratch. It is the safer option if you are unsure. + +The only relationship that matters is parent-to-child. Grandchildren depend on the `update` function +of their direct parent, not that of the grandparent. + +It is important to view the child cache as pointers-to-memory. For example, if a slice's `data_ptr` +value and `length` have not changed, returning `True` is appropriate. Even if the slice is mutable +and elements of it are overwritten (e.g. `slice[0] = 15`), because the child cache consists of +*pointers*, they will reflect the new data at that memory location. + +Conversely, if `data_ptr` has changed, that means it is pointing to a new location in memory, the +child pointers are invalid, and the cache must be flushed. If the `length` has changed, we need to +flush the cache to reflect the new number of children. If `length` has changed but `data_ptr` has +not, it is possible to store the old children in the `SyntheticProvider` itself (e.g. +`list[SBValue]`) and dole those out rather than generating them from scratch, only creating new +children if they do not already exist in the `SyntheticProvider`'s list. + +For further clarification, see [this discussion](https://discourse.llvm.org/t/when-is-it-safe-to-cache-syntheticprovider-update/88608) + +> NOTE: when testing the caching behavior, do not rely on LLDB's heuristic to persist variables when +> stepping. Instead, store the variable in a python object (e.g. `v = lldb.frame.var("var_name")`), +> step forward, and then inspect the stored variable. + +### (optional) `has_children` + +> Overrides `SBValue.MightHaveChildren` + +This is a shortcut used by LLDB to check if the value has children *at all*, without doing +potentially expensive computations to determine how many children there are (e.g. linked list). +Often, this will be a one-liner of `return True`/`return False` or +`return self.valobj.MightHaveChildren()`. + +### (optional) `num_children` + +> Overrides `SBValue.GetNumChildren` + +Returns the total number of children that LLDB should try to access when printing the type. This +number **does not** need to match to total number of synthetic children. + +The `max_children` argument can be returned if calculating the number of children can be expensive +(e.g. linked list). If this is not a consideration, `max_children` can be omitted from the function +signature. + +Additionally, fields can be intentionally "hidden" from LLDB while still being accessible to the +user. For example, one might want a `vec![1, 2, 3]` to display only its elements, but still have the +`len` and `capacity` values accessible on request. By returning `3` from `num_children`, one can +restrict LLDB to only displaying `[1, 2, 3]`, while users can still directly access `v.len` and +`v.capacity`. See: [Example Provider: Vec\](#example-provider-vect) to see an implementation of +this. + +### `get_child_index` + +> Overrides `SBValue.GetIndexOfChildWithName` +> +> Affects `SBValue.GetChildMemberWithName` + +Given a name, returns the index that the child should be accessed at. It is expected that the return +value of this function is passed directly to `get_child_at_index`. As with `num_children`, the +values returned here *can* be arbitrary, so long as they are properly coordinated with +`get_child_at_index`. + +One special value is `$$dereference$$`. Accounting for this pseudo-field will allow LLDB to use the +`SBValue` returned from `get_child_at_index` as the result of a dereference via LLDB's expression +parser (e.g. `*val` and `val->field`) + +### `get_child_at_index` + +> Overrides `SBValue.GetChildAtIndex` + +Given an index, returns a child `SBValue`. Often these are generated via +`SBValue.CreateValueFromAddress`, but less commonly `SBValue.CreateChildAtOffset`, +`SBValue.CreateValueFromExpression`, and `SBValue.CreateValueFromData`. These functions can be a +little finicky, so you may need to fiddle with them to get the output you want. + +In some cases, `SBValue.Clone` is appropriate. It creates a new child that is an exact copy of an +existing child, but with a new name. This is useful for cases like tuples, which have field names of +the style `__0`, `__1`, ... when we would prefer they were named `0`, `1`, ... + +Small alterations can be made to the resulting child before it is returned. This is useful for +`&str`/`String`, where we would prefer if the children were displayed as +`lldb.eFormatBytesWithASCII` rather than just as a decimal value. + +### (optional) `get_type_name` + +> Overrides `SBValue.GetDisplayTypeName` + +Overrides the displayed name of a type. For a synthetic `SBValue` whose type name is overridden, the +original type name can still be retrieved via `SBValue.GetTypeName()` and +`SBValue.GetType().GetName()` + +This can be helpful in shortening the name of common standard library types (e.g. +`std::collections::hash::map::HashMap` -> `HashMap`), or +in normalizing MSVC type names (e.g. `ref$` -> `&str`). + +The string manipulation can be a little tricky, especially on MSVC where we cannot conveniently +access the generic parameters of the type. + +### (optional) `get_value` + +> Overrides `SBValue.GetValue()`, `SBValue.GetValueAsUnsigned()`, `SBValue.GetValueAsSigned()`, +>`SBValue.GetValueAsAddress()`, + +The `SBValue` returned is expected to be a primitive type or pointer, and is treated as the value of +the variable in expressions. + +> IMPORTANT: The `SBValue` returned **must be stored in the `SyntheticProvider`**. There is +>currently (Nov 2025) a bug where if the `SBValue` is acquired within `get_value` and not stored +>anywhere, Python will segfault when LLDB attempts to access the value. + +## Summary Providers + +Summary providers are python functions of the following form: + +```python +def SummaryProvider(valobj: SBValue, _lldb_internal) -> str: ... +``` + +Where the returned string is passed verbatim to the user. If the returned value isn't a string, it +is naively convered to a string (e.g. `return None` prints `"None"`, not an empty string). + +If the `SBValue` passed in is of a type that has a Synthetic Provider, `valobj.IsSynthetic()` will +return `True`, and the synthetic's corresponding functions will be used. If this is undesirable, the +original value can be retrieved via `valobj.GetNonSyntheticValue()`. This can be helpful in cases +like `String`, where individually calling `GetChildAtIndex` in a loop is much slower than accessing +the heap pointer, reading the whole byte array directly from the debugee's memory, and using +Python's `bytes.decode()`. + +### Instance Summaries + +Regular `SummaryProvider` functions take an opaque `SBValue`. That `SBValue` will reflect the type's +`SyntheticProvider` if one exists, but we cannot access the `SyntheticProvider` instance itself, or +any of its internal implementation details. This is deterimental in cases where we need some of +those internal details to help complete the summary. Currently (Nov 2025), in the synthetic we just +run the non-synthetic value through the synthetic provider +(`synth = SyntheticProvider(valobj.GetNonSyntheticValue(), _dict)`), but this is obviously +suboptimal and there are plans to use the method outlined below. + +Instead, we can leverage the Python module's state to allow for instance summaries. Prior art for +this technique exists in the [old CodeLLDB Rust visualizer scripts](https://github.com/vadimcn/codelldb/blob/cf9574977b80e29c6de2c44d12f1071a53a54caf/formatters/rust.py#L110). + +In short: every Synthetic Provider's `__init__` function stores a unique ID and a weak reference to +`self` in a global dictionary. The Synthetic Provider class also implements a `get_summary` +function. The type's `SummaryProvider` is a function that looks up the unique ID in this dictionary, +then calls a `get_summary` on the instance it retrieves. + +```python +import weakref + +SYNTH_BY_ID = weakref.WeakValueDictionary() + +class SyntheticProvider: + valobj: SBValue + + # slots requires opting-in to __weakref__ + __slots__ = ("valobj", "__weakref__") + + def __init__(valobj: SBValue, _dict): + SYNTH_BY_ID[valobj.GetID()] = self + self.valobj = valobj + + def get_summary(self) -> str: + ... + +def InstanceSummaryProvider(valobj: SBValue, _dict) -> str: + # GetNonSyntheticValue should never fail as InstanceSummaryProvider implies an instance of a + # `SyntheticProvider`. No non-synthetic types should ever have this summary assigned to them + # We use GetNonSyntheticValue because the synthetic vaobj has its own unique ID + return SYNTH_BY_ID[valobj.GetNonSyntheticValue().GetID()].get_summary() +``` + +For example, one might use this for the Enum synthetic provider. The summary would like to access +the variant name, but there isn't a convenient way to reflect this via the type name or child-values +of the synthetic. By implementing an instance summary, we can retrieve the variant name via +`self.variant.GetTypeName()` and some string manipulation. + +# Writing Visualizer Scripts + +> IMPORTANT: Unlike GDB and CDB, LLDB can debug executables with either DWARF or PDB debug info. +>Visualizers must be written to account for both formats whenever possible. See: +>[rust-codegen](./rust-codegen.md#dwarf-vs-pdb) for an overview of the differences + +Scripts are injected into LLDB via the CLI command `command script import .py`. Once +injected, classes and functions can be added to the synthetic/summary pool with `type synthetic add` +and `type summary add` respectively. The summaries and synthetics can be associated with a +"category", which is typically named after the language the providers are intended for. The category +we use will be called `Rust`. + +> TIP: all LLDB commands can be prefixed with `help` (e.g. `help type synthetic add`) for a brief +description, list of arguments, and examples. + +Currently (Nov 2025) we use `command source ...`, which executes a series of CLI commands from the +file [`lldb_commands`](https://github.com/rust-lang/rust/blob/main/src/etc/lldb_commands) to add +providers. This file is somewhat unwieldy, and will soon be supplanted by the Python API equivalent +outlined below. + +## `__lldb_init_module` + +This is an optional function of the form: + +```python +def __lldb_init_module(debugger: SBDebugger, _lldb_internal) -> None: ... +``` + +This function is called at the end of `command script import ...`, but before control returns back +to the CLI. It allows the script to initialize its own state. + +Crucially, it is passed a reference to the debugger itself. This allows us to create the `Rust` +category and add providers to it. It can also allow us to conditionally change which providers we +use depending on what version of LLDB the script detects. This is vital for backwards compatibility +once we begin using recognizer functions, as recognizers were added in lldb 19.0. + +## Visualizer Resolution + +The order that visualizers resolve in is listed [here][formatters_101]. In short: + +[formatters_101]: https://lldb.llvm.org/use/variable.html#finding-formatters-101 + +* If there is an exact match (non-regex name, recognizer function, or type already matched to +provider), use that +* If the object is a pointer/reference, try to use the dereferenced type's formatter +* If the object is a typedef, check the underlying type for a formatter +* If none of the above work, iterate through the regex type matchers + +Within each of those steps, **iteration is done backwards** to allow new commands to "override" old +commands. This is important for cases like `Box` vs `Box`, were we want a specialized +synthetic for the former, but a more generalized synthetic for the latter. + +## Minutiae + +LLDB's API is very powerful, but there are some "gotchas" and unintuitive behavior, some of which +will be outlined below. The python implementation can be viewed at the path returned by the CLI +command `lldb -P` in `lldb\__init__.py`. In addition to the +[examples in the lldb repo][synth_examples], there are also [C++ visualizers][plugin_cpp] that can +be used as a reference (e.g. [LibCxxVector, the equivalent to `Vec`][cxx_vector]). While C++'s +visualizers are written in C++ and have access to LLDB's internals, the API and general practices +are very similar. + +[synth_examples]:https://github.com/llvm/llvm-project/tree/main/lldb/examples/synthetic +[plugin_cpp]: https://github.com/llvm/llvm-project/tree/main/lldb/source/Plugins/Language/CPlusPlus +[cxx_vector]: https://github.com/llvm/llvm-project/blob/main/lldb/source/Plugins/Language/CPlusPlus/LibCxxVector.cpp + +### `SBValue` + +* Pointer/reference `SBValue`s will effectively "auto-deref" in some cases, acting as if the +children of the pointed-to-object are its own children. +* The non-function fields are typically [`property()`][property] fields that point directly to the +function anyway (e.g. `SBValue.type = property(GetType, None)`). Accessing through these shorthands +is a bit slower to access than just calling the function directly, so they should be avoided. Some +of the properties return special objects with special properties (e.g. `SBValue.member` returns an +object that acts like `dict[str, SBValue]` to access children). Internally, many of these special +objects just allocate a new class instance and call the function on the `SBValue` anyway, resulting +in additional performance loss (e.g. `SBValue.member` internally just implements `__getitem__` which +is the one-liner `return self.valobj.GetChildMemberWithName(name)`) +* `SBValue.GetID` returns a unique `int` for each value for the duration of the debug session. +Synthetic `SBValue`'s have a different ID than their underlying `SBValue`. The underlying ID can be +retrieved via `SBValue.GetNonSyntheticValue().GetID()`. +* When manually calculating an address, `SBValue.GetValueAsAddress` should be preferred over +`SBValue.GetValueAsUnsigned` due to [target-specific behavior][get_address] +* Getting a string representation of an `SBValue` can be tricky because `GetSummary` requires a +summary provider and `GetValue` requires the type be representable by a primitive. In almost all +cases where neither of those conditions are met, the type is a user defined struct that can be +passed through `StructSummaryProvider`. + +[property]: https://docs.python.org/3/library/functions.html#property +[get_address]: https://lldb.llvm.org/python_api/lldb.SBValue.html#lldb.SBValue.GetValueAsAddress + +### `SBType` + +* "Aggregate type" means a non-primitive struct/class/union +* "Template" is equivalent to "Generic" +* Types can be looked up by their name via `SBTarget.FindFirstType(type_name)`. `SBTarget` can be +acquired via `SBValue.GetTarget` +* `SBType.template_args` returns `None` instead of an empty list if the type has no generics +* It is sometimes necessary to transform a type into the type you want via functions like +`SBType.GetArrayType` and `SBType.GetPointerType`. These functions cannot fail. They ask the +underlying LLDB `TypeSystem` plugin for the type, bypassing the debug info completely. Even if the +type does not exist in the debug info at all, these functions can create the appropriate type. +* `SBType.GetCanonicalType` is effectively `SBType.GetTypedefedType` + `SBType.GetUnqualifiedType`. +Unlike `SBType.GetTypedefedType`, it will always return a valid `SBType` regardless of whether or +not the original `SBType` is a typedef. +* `SBType.GetStaticFieldWithName` was added in LLDB 18. Unfortunately, backwards compatibility isn't +always possible since the static fields are otherwise completely inaccessible. + + +# Example Provider: `Vec` + +## SyntheticProvider + +We start with the typical prelude, using `__slots__` since we have known fields. In addition to the +object itself, we also need to store the type of the elements because `Vec`'s heap pointer is a +`*mut u8`, not a `*mut T`. Rust is a statically typed language, so the type of `T` will never +change. That means we can store it during initialization. The heap pointer, length, and capacity +*can* change though, and thus are default initialized here. + +```python +import lldb + +class VecSyntheticProvider: + valobj: SBValue + data_ptr: SBValue + len: int + cap: int + element_type: SBType + + __slots__ = ( + "valobj", + "data_ptr", + "len", + "cap", + "element_type", + "__weakref__", + ) + + def __init__(valobj: SBValue, _dict) -> None: + self.valobj = valobj + # invalid type is a better default than `None` + self.element_type = SBType() + + # special handling to account for DWARF/PDB differences + if (arg := valobj.GetType().GetTemplateArgumentType(0)): + self.element_type = arg + else: + arg_name = next(get_template_args(valobj.GetTypeName())) + self.element_type = resolve_msvc_template_arg(arg_name, valobj.GetTarget()) +``` + +For the implementation of `get_template_args` and `resolve_msvc_template_arg`, please see: +[`lldb_providers.py`](https://github.com/rust-lang/rust/blob/main/src/etc/lldb_providers.py#L136). + +Next, the update function. We check if the pointer or length have changed. We can ommit checking the +capacity, as the number of children will remain the same unless `len` changes. If changing the +capacity resulted in a reallocation, `data_ptr`'s address would be different. + +If `data_ptr` and `length` haven't changed, we can take advantage of LLDB's caching and return +early. If they have changed, we store the new values and tell LLDB to flush the cache. + +```python +def update(self): + ptr = self.valobj.GetChildMemberWithName("data_ptr") + len = self.valobj.GetChildMemberWithName("length").GetValueAsUnsigned() + + if ( + self.data_ptr.GetValueAsAddress() == ptr.GetValueAsAddress() + and self.len == len + ): + # Our child address offsets and child count are still valid + # so we can reuse cached children + return True + + self.data_ptr = ptr + self.len = len + + return False +``` + +`has_children` and `num_children` are both straightforward: + +```python +def has_children(self) -> bool: + return True + +def num_children(self) -> int: + return self.len +``` + +When accessing elements, we expect values of the format `[0]`, `[1]`, etc. to mimic indexing. +Additionally, we still want the user to be able to quickly access the length and capacity, as they +can be very useful when debugging. We assign these values `u32::MAX - 1` and `u32::MAX - 2` +respectively, as we can almost surely guarantee that they will not overlap with element values. Note +that we can account for both the full and shorthand `capacity` name. + +```python + def get_child_index(self, name: str) -> int: + index = name.lstrip("[").rstrip("]") + if index.isdigit(): + return int(index) + if name == "len": + return lldb.UINT32_MAX - 1 + if name == "cap" or name == "capacity": + return lldb.UINT32_MAX - 2 + + return -1 +``` + +We now have to properly coordinate `get_child_at_index` so that the elements, length, and capacity +are all accessible. + +```python +def get_child_at_index(self, index: int) -> SBValue: + if index == UINT32_MAX - 1: + return self.valobj.GetChildMemberWithName("len") + if index == UINT32_MAX - 2: + return ( + self.valobj.GetChildMemberWithName("buf") + .GetChildMemberWithName("inner") + .GetChildMemberWithName("cap") + .GetChildAtIndex(0) + .Clone("capacity") + ) + addr = self.data_ptr.GetValueAsAddress() + addr += index * self.element_type.GetByteSize() + return self.valobj.CreateValueFromAddress(f"[{index}]", addr, self.element_type) +``` + +For the type's display name, we can strip the path qualifier. User defined types named +`Vec` will end up fully qualified, so there shouldn't be any ambiguity. We can also remove the +allocator generic, as it's very very rarely useful. We use `get_template_args` instead of +`self.element_type.GetName()` for 3 reasons: + +1. If we fail to resolve the element type for any reason, `self.valobj`'s type name can still let +the user know what the real type of the element is +2. Type names are not subject to the limitations of DWARF and PDB nodes, so the template type in +the name will reflect things like `*const`/`*mut` and `&`/`&mut`. +3. We do not currently (Nov 2025) normalize MSVC type names, but once we do, we will need to work with the +string-names of types anyway. It's also much easier to cache a string-to-string conversion compared +to an `SBType`-to-string conversion. + +```python +def get_type_name(self) -> str: + return f"Vec<{next(get_template_args(self.valobj))}>" +``` + +There isn't an appropriate primitive value with which to represent a `Vec`, so we simply ommit +the `get_value` function. + +## SummaryProvider + +The summary provider is very simple thanks to our synthetic provider. The only real hiccup is that +`GetSummary` only returns a value if the object's type has a `SummaryProvider`. If it doesn't, it +will return an empty string which is not ideal. In a full set of visualizer scripts, we can ensure +that every type that doesn't have a `GetSummary()` or a `GetValue()` is a struct, and then delegate +to a generic `StructSummaryProvider`. For this demonstration, I will gloss over that detail. + +```python +def VecSummaryProvider(valobj: SBValue, _lldb_internal) -> str: + children = [] + for i in range(valobj.GetNumChildren()): + child = valobj.GetChildAtIndex(i) + summary = child.GetSummary() + if summary is None: + summary = child.GetValue() + if summary is None: + summary = "{...}" + + children.append(summary) + + return f"vec![{", ".join(children)}]" +``` + +## Enabling the providers + +Assume this synthetic is imported into `lldb_lookup.py` + +With CLI commands: + +```txt +type synthetic add -l lldb_lookup.synthetic_lookup -x "^(alloc::([a-z_]+::)+)Vec<.+>$" --category Rust +type summary add -F lldb_lookup.summary_lookup -x "^(alloc::([a-z_]+::)+)Vec<.+>$" --category Rust +``` + +With `__lldb_init_module`: + +```python +def __lldb_init_module(debugger: SBDebugger, _dict: LLDBOpaque): + # Ensure the category exists and is enabled + rust_cat = debugger.GetCategory("Rust") + + if not rust_cat.IsValid(): + rust_cat = debugger.CreateCategory("Rust") + + rust_cat.SetEnabled(True) + + # Register Vec providers + vec_regex = r"^(alloc::([a-z_]+::)+)Vec<.+>$" + sb_name = lldb.SBTypeNameSpecifier(vec_regex, is_regex=True) + + sb_synth = lldb.SBTypeSynthetic.CreateWithClassName("lldb_lookup.VecSyntheticProvider") + sb_synth.SetOptions(lldb.eTypeOptionCascade) + + sb_summary = lldb.SBTypeSummary.CreateWithFunctionName("lldb_lookup.VecSummaryProvider") + sb_summary.SetOptions(lldb.eTypeOptionCascade) + + rust_cat.AddTypeSynthetic(sb_name, sb_synth) + rust_cat.AddSummary(sb_name, sb_summary) +``` + +## Output + +Without providers: + +```text +(lldb) v vec_v +(alloc::vec::Vec) vec_v = { + buf = { + inner = { + ptr = { + pointer = (pointer = "\n") + _marker = {} + } + cap = (__0 = 5) + alloc = {} + } + _marker = {} + } + len = 5 +} +(lldb) v vec_v[0] +error: :1:6: subscripted value is not an array or pointer + 1 | vec_v[0] + | ^ +``` + +With providers (`v ` prints the summary and then a list of all children): + +```text +(lldb) v vec_v +(Vec) vec_v = vec![10, 20, 30, 40, 50] { + [0] = 10 + [1] = 20 + [2] = 30 + [3] = 40 + [4] = 50 +} +(lldb) v vec_v[0] +(int) vec_v[0] = 10 +``` + +We can also confirm that the "hidden" length and capacity are still accessible: + +```text +(lldb) v vec_v.len +(unsigned long long) vec_v.len = 5 +(lldb) v vec_v.capacity +(unsigned long long) vec_v.capacity = 5 +(lldb) v vec_v.cap +(unsigned long long) vec_v.cap = 5 +``` \ No newline at end of file diff --git a/src/debuginfo/llvm-codegen.md b/src/debuginfo/llvm-codegen.md new file mode 100644 index 000000000..cc052b6b9 --- /dev/null +++ b/src/debuginfo/llvm-codegen.md @@ -0,0 +1,12 @@ +# (WIP) LLVM Codegen + +When Rust calls an LLVM `DIBuilder` function, LLVM translates the given information to a +["debug record"][dbg_record] that is format-agnostic. These records can be inspected in the LLVM-IR. + +[dbg_record]: https://llvm.org/docs/SourceLevelDebugging.html#debug-records + +It is important to note that tags within the debug records are **always stored as DWARF tags**. If +the target calls for PDB debug info, during codegen the debug records will then be passed through +[a module that translates the DWARF tags to their CodeView counterparts][cv]. + +[cv]:https://github.com/llvm/llvm-project/blob/main/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp \ No newline at end of file diff --git a/src/debuginfo/natvis-visualizers.md b/src/debuginfo/natvis-visualizers.md new file mode 100644 index 000000000..653e7d5d6 --- /dev/null +++ b/src/debuginfo/natvis-visualizers.md @@ -0,0 +1,3 @@ +# (WIP) CDB - Natvis + +Official documentation for Natvis can be found [here](https://learn.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects) and [here](https://code.visualstudio.com/docs/cpp/natvis) diff --git a/src/debuginfo/rust-codegen.md b/src/debuginfo/rust-codegen.md new file mode 100644 index 000000000..6d5dfe08c --- /dev/null +++ b/src/debuginfo/rust-codegen.md @@ -0,0 +1,183 @@ +# Rust Codegen + +The first phase in debug info generation requires Rust to inspect the MIR of the program and +communicate it to LLVM. This is primarily done in [`rustc_codegen_llvm/debuginfo`][llvm_di], though +some type-name processing exists in [`rustc_codegen_ssa/debuginfo`][ssa_di]. Rust communicates to +LLVM via the `DIBuilder` API - a thin wrapper around LLVM's internals that exists in +[rustc_llvm][rustc_llvm]. + +[llvm_di]: https://github.com/rust-lang/rust/tree/main/compiler/rustc_codegen_llvm/src/debuginfo +[ssa_di]: https://github.com/rust-lang/rust/tree/main/compiler/rustc_codegen_ssa/src/debuginfo +[rustc_llvm]: https://github.com/rust-lang/rust/tree/main/compiler/rustc_llvm + +# Type Information + +Type information typically consists of the type name, size, alignment, as well as things like +fields, generic parameters, and storage modifiers if they are relevant. Much of this work happens in +[rustc_codegen_llvm/src/debuginfo/metadata][di_metadata]. + +[di_metadata]: https://github.com/rust-lang/rust/blob/main/compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs + +It is important to keep in mind that the goal is not necessarily "represent types exactly how they +appear in Rust", rather it is to represent them in a way that allows debuggers to most accurately +reconstruct the data during debugging. This distinction is vital to understanding the core work that +occurs on this layer; many changes made here will be for the purpose of working around debugger +limitations when no other option will work. + +## Quirks + +Rust's generated DI nodes "pretend" to be C/C++ for both CDB and LLDB's sake. This can result in +some unintuitive and non-idiomatic debug info. + +### Pointers and Reference + +Wide pointers/references/`Box` are treated as a struct with 2 fields: `data_ptr` and `length`. + +All non-wide pointers, references, and `Box` pointers are output as pointer nodes, and no +distinction is made between `mut` and non-`mut`. Several attempts have been made to rectify this, +but unfortunately there is not a straightforward solution. Using the `reference` DI nodes of the +respective formats has pitfalls. There is a semantic difference between C++ references and Rust +references that is unreconcilable. + +>From [cppreference](https://en.cppreference.com/w/cpp/language/reference.html): +> +>References are not objects; **they do not necessarily occupy storage**, although the compiler may +>allocate storage if it is necessary to implement the desired semantics (e.g. a non-static data +>member of reference type usually increases the size of the class by the amount necessary to store +>a memory address). +> +>Because references are not objects, **there are no arrays of references, no pointers to references, and no references to references** + +The current proposed solution is to simply [typedef the pointer nodes][issue_144394]. + +[issue_144394]: https://github.com/rust-lang/rust/pull/144394 + +Using the `const` qualifier to denote non-`mut` poses potential issues due to LLDB's internal +optimizations. In short, LLDB attempts to cache the child-values of variables (e.g. struct fields, +array elements) when stepping through code. A heuristic is used to determine which values are safely +cache-able, and `const` is part of that heuristic. Research has not been done into how this would +interact with things like Rust's interrior mutability constructs. + +### DWARF vs PDB + +While most of the type information is fairly straight forward, one notable issue is the debug info +format of the target. Each format has different semantics and limitations, as such they require +slightly different debug info in some cases. This is gated by calls to +[`cpp_like_debuginfo`][cpp_like]. + +[cpp_like]: https://github.com/rust-lang/rust/blob/main/compiler/rustc_codegen_ssa/src/debuginfo/type_names.rs#L813 + +### Naming + +Rust attempts to communicate type names as accurately as possible, but debuggers and debug info +formats do not always respect that. + +Due to limitations in MSVC's expression parser, the following name transformations are made for PDB +debug info: + +| Rust name | MSVC name | +| --- | --- | +| `&str`/`&mut str` | `ref$`/`ref_mut$` | +| `&[T]`/`&mut [T]` | `ref$ >`/`ref_mut$ >`* | +| `[T; N]` | `array$` | +| `RustEnum` | `enum2$` | +| `(T1, T2)` | `tuple$`| +| `*const T` | `ptr_const$` | +| `*mut T` | `ptr_mut$` | +| `usize` | `size_t`** | +| `isize` | `ptrdiff_t`** | +| `uN` | `unsigned __intN`** | +| `iN` | `__intN`** | +| `f32` | `float`** | +| `f64` | `double`** | +| `f128` | `fp128`** | + +> \* MSVC's expression parser will treat `>>` as a right shift. It is necessary to separate +>consecutive `>`'s with a space (`> >`) in type names. +> +> ** While these type names are generated as part of the debug info node (which is then wrapped in +>a typedef node with the Rust name), once the LLVM-IR node is converted to a CodeView node, the type +>name information is lost. This is because CodeView has special shorthand nodes for primitive types, +>and those shorthand nodes to not have a "name" field. + +### Generics + +Rust outputs generic *type* information (`T` in `ArrayVec`), but not generic *value* +information (`N` in `ArrayVec`). + +CodeView does not have a leaf node for generics/C++ templates, so all generic information is lost +when generating PDB debug info. There are workarounds that allow the debugger to retrieve the +generic arguments via the type name, but it is fragile solution at best. Efforts are being made to +contact Microsoft to correct this deficiency, and/or to use one of the unused CodeView node types as +a suitable equivalent. + +### Type aliases + +Rust outputs typedef nodes in several cases to help account for debugger limitiations, but it does +not currently output nodes for [type aliases in the source code][type_aliases]. + +[type_aliases]: https://doc.rust-lang.org/reference/items/type-aliases.html + +### Enums + +Enum DI nodes are generated in [rustc_codegen_llvm/src/debuginfo/metadata/enums][di_metadata_enums] + +[di_metadata_enums]: https://github.com/rust-lang/rust/tree/main/compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums + +#### DWARF + +DWARF has a dedicated node for discriminated unions: `DW_TAG_variant`. It is a container that +references `DW_TAG_variant_part` nodes that may or may not contain a discriminant value. The +hierarchy looks as follows: + +```txt +DW_TAG_structure_type (top-level type for the coroutine) + DW_TAG_variant_part (variant part) + DW_AT_discr (reference to discriminant DW_TAG_member) + DW_TAG_member (discriminant member) + DW_TAG_variant (variant 1) + DW_TAG_variant (variant 2) + DW_TAG_variant (variant 3) + DW_TAG_structure_type (type of variant 1) + DW_TAG_structure_type (type of variant 2) + DW_TAG_structure_type (type of variant 3) +``` + +#### PDB +PDB does not have a dedicated node, so it generates the C equivalent of a discriminated union: + +```c +union enum2$ { + enum VariantNames { + First, + Second + }; + struct Variant0 { + struct First { + // fields + }; + static const enum2$::VariantNames NAME; + static const unsigned long DISCR_EXACT; + enum2$::Variant0::First value; + }; + struct Variant1 { + struct Second { + // fields + }; + static enum2$::VariantNames NAME; + static unsigned long DISCR_EXACT; + enum2$::Variant1::Second value; + }; + enum2$::Variant0 variant0; + enum2$::Variant1 variant1; + unsigned long tag; +} +``` + +An important note is that due to limitations in LLDB, the `DISCR_*` value generated is always a +`u64` even if the value is not `#[repr(u64)]`. This is largely a non-issue for LLDB because the +`DISCR_*` value and the `tag` are read into `uint64_t` values regardless of their type. + +# Source Information + +TODO \ No newline at end of file diff --git a/src/debuginfo/testing.md b/src/debuginfo/testing.md new file mode 100644 index 000000000..f58050875 --- /dev/null +++ b/src/debuginfo/testing.md @@ -0,0 +1,8 @@ +# (WIP) Testing + +The debug info test suite is undergoing a substantial rewrite. This section will be filled out as +the rewrite makes progress. + +Please see [this tracking issue][148483] for more information. + +[148483]: https://github.com/rust-lang/rust/issues/148483 \ No newline at end of file