We will demonstrate debugging a use-after-free in gdb with Pernosco.
The first step is to capture a recording of the bug. We were able to capture a recording of the crash on our first attempt, following the instructions given by the bug reporter, and uploaded it to Pernosco. Once that finished processing Pernosco emailed us a link to our debugging session.
When we open the session we notice an unusual "0x0+0" frame on the top of our stack, suggesting that we jumped to a null pointer. According to the source view, this happened while trying to call
gdb_stdout->flush(). We open the notebook view to take some notes and the instructions executed view to see what is going on at the instruction level. In the instructions view it is immediately obvious that we tried to perform an indirect call to an invalid address (null, in this case) and crashed. If we look backwards just a few instructions, we can see that null was loaded into $rax from memory.
An engineer familiar with calling patterns may recognize the code sequence at
flush_streams+17 as a vtable pointer lookup. Starting from an object pointer (
0x561f96bcf470), the pointer is dereferenced once with no offset to get the address of the virtual table (
0x561f96c20060) and then the vtable pointer is itself dereferenced with an offset to find the specific function
flush(). Otherwise, one can observe the value of
gdb_stdout (this is slightly complicated by the fact that
gdb_stdout is a macro that expands to
*current_ui_gdb_stdout_ptr(), and thus no variable by the name
gdb_stdout is present in the program here). Either way, the object at
0x561f96bcf470 appears to be defective in some way.
We can click on the value
0x561f96bcf470 to see where it originated. Pernosco opens the dataflow view and shows us that this value was previously stored in
cli_interp_base::set_logging. And when we click on that item we then see in the source view that this value was indeed stored to
gdb_stdout here. We write another entry in our notebook for this.
The next question to ask is when did the value of the vtable pointer last get assigned? We would expect it to be set up at object creation and remain untouched. To investigate, we return to the site of the crash by clicking on its entry in the notebook, and then we click on the vtable pointer value (
0x561f96c20060) when it is loaded into a register in the instructions view. The dataflow view now shows us this memory was previously written to in a function that certainly does not look like an object constructor. We click on that entry and between the source view and the call stack we see we are in some sort of memmove within std::vector. Certainly not what we were expecting! We can also see from the position of the tentative annotation in the notebook that this is happening after
gdb_stdout was set, while we expected the object to fully constructed before it was stored in
gdb_stdout. It appears this code is corrupting the our object.
Lets investigate where this function got the address where we expected to find our vtable pointer from. We can see in the instructions view that the address is present in
$rdi. We open the registers view and click on
$rdi's value. The
operator new in the dataflow entries list catches our attention. After clicking on it, we can see that the same address we stored earlier in
gdb_stdout is now being returned by
Just to be certain, after jotting our findings down in the notebook, we evaluate
gdb_stdout (which is a preprocessor macro that expands to
*current_ui_gdb_stdout_ptr()) and see that it is still returning the same value at the exact moment this pointer is being handed out as freshly allocated memory.
The final step is to figure out why the memory at
0x561f96bcf470 is considered free. To do that, we search for executions of
operator delete and apply a condition on the
ptr argument to restrict ourselves to matching invocations. We select the most recent
operator delete call. We can see a
std::unique_ptr is being destroyed, and scan up the call stack until we find a frame that belongs to our application (rather than the standard library). That takes us to
current_interp_set_logging. We can see in the notebook view that this also falls after the assignment into
We have laid out all the components of the use-after-free in our notebook.
gdb_stdout->flush()is made which crashes because the vtable pointer is no longer valid.
current_interp_set_logging and its callee
cli_interp_base::set_logging the bug is that
logfile is a
std::unique_ptr but its stored as a raw pointer in
gdb_stdout. Nothing is done to take ownership of
std::unique_ptr::release is never called) and when
logfile goes out of scope it is destroyed, leaving
gdb_stdout as a dangling pointer.