1. Overview
  2. Debugging workflow
  3. Debugging workflow: CI
  4. Basic interface
  5. Visualizing control flow
  6. Call stacks
  7. Explaining dataflow
  8. Multiprocess
  9. Search box
  10. Source files
  11. Condition and print expressions
  12. Toolbox
  13. Alerts
  14. Application logs
  15. Callees
  16. View operators
  17. Notebook
  18. Instruction execution
  19. Javascript
  20. Browser UI integration
  21. Screenshots
  22. Additional views
  23. GDB
  24. System debug info
  25. Compiler issues
  26. The Pernosco vision
  27. Related work


Pernosco is intended to surpass gdb and rr, but the gdb interface is familiar to users and has some features that Pernosco still lacks. Therefore we have integrated gdb support into Pernosco. In fact, in many ways Pernosco-gdb is the best gdb experience ever.

We created a gdbserver backend that uses Pernosco's omniscient database. gdb's continue and reverse-continue commands don't execute code; instead Pernosco finds the next (or previous) moment at which one of the current breakpoints or watchpoints would be triggered, and set the current time to that moment. Thus gdb commands, like Pernosco interactions, are mostly instantaneous:

We synchronize gdb sessions with the rest of the Pernosco interface. In particular whenever the currently focused moment changes, all gdb sessions also jump to the same moment:

Calling user functions to dump data structures etc is difficult to implement in a record-and-replay system, and even more so in an omniscient debugger where debuggee processes do not normally exist. However it is an important feature and Pernosco-gdb supports it:

Implementing breakpoints and watchpoints with database queries means we're no longer limited to CPU hardware breakpoint/watchpoint limitations. Pernosco-gdb supports an unlimited number of hardware data watchpoints, each of unlimited size. Pernosco-gdb breakpoints don't require inserting breakpoint instructions, and therefore are immune to nasty corner-case issues arising from that (e.g. rr issue 2163).

CPU hardware watchpoints observe modifications to specific virtual addresses and don't trigger when the memory contents are changed through a different virtual address (possibly in a different process). In contrast, Pernosco tracks "physical memory" and will trigger watchpoints no matter how the memory values are modified:

The selected stack frame is synchronized between Pernosco and gdb. (when possible — as noted elsewhere, gdb stacks often don't capture all the frames Pernosco knows about):

It is possible to open multiple gdb sessions attached to different processes and they stay synchronized as you debug:

Pernosco automatically imports project-defined gdb scripts such as pretty-printers. Pernosco-gdb sessions are tightly sandboxed so there are no security issues running untrusted gdb scripts.

Gdb index files speed up gdb startup a lot if you build them with the gdb-add-index tool. Pernosco builds gdb indexes automatically as part of its trace processing.