Overview

GNU debugger, GDB can be your best friend to help you to perform the source-level debugging during the application development. In this article, we will not go through the GDB’s basic usages which should be easy to be located with a google search. This article is specifically to show how GDB can help application crash scenarios by a case of segmentation fault error. Some code snippets are also used to guide how to apply GDB’s commands on it. Additionally, Integrated Eclipse IDE of Intel System Studio 2018 will be used to illustrate GDB's usages.

Use GDB’s backtrace command for tracking the callstack when crash happens

You will want to discover the code line where your application fails at when application crash happens. GDB’s backtrace can help you get the callstack, function call history information , right after the accident occurs in most of cases. Considering the following example codes to write data to an uninitialized string to invoke a segmentation fault, typing “backtrace” in the debugger console window instantly reveal where this segmentation fault fails at in code-line level. Check the screenshot below.

//// code snippets - start
static char buffer [256];
static char* string;

static long do_that() {
 gets(string);
 return 0;
};

static long do_this() {
 do_that();
 return 0;
};

int main()
{
   printf ("input a string: ");
   do_this();
   printf ("\n\n the input string: %s\n", string);
}
//// code snippets - end

 Figure 1. GDB’s backtrace example

Sometimes backtrace does not work

backtrace is useful but it is not always able to reveal the debug callstack in details. Considering the following sample code snippets, backtrace cannot show the helpful information here.  

////code snippets - start
struct A_arg {
    long arg1;
    long arg2;
    long count1;
    long count2;
    long count3;
    long count4;
};

struct B_arg {
    long arg1;
    long arg2;
};

static long fun_A(void* argB) {
    struct A_arg* myarg = (struct A_arg*) argB;
    if (!myarg) return -1;
    myarg->count4 += 1;// access the out-of-boundary data of B_arg, corrupt stack

////code snippets - end

Check the figure below, backtrace misinterpret the last function call “0x7ffffffffbca0 in ??()” which is invalid. This is all due to the code snippets accidentally update the wrong memory content which happens to corrupt the callstack.  

Figure 2. GDB’s backtrace cannot interpret callstack

Use Intel System Studio 2018’s function call history view to review clear call stacks

Instead of interpreting the call stack when the segmentation fault occurs, you can to log the function call history during the runtime and Intel System Studio 2018 integrates a Function Call History view to facilitate the debugging experience.

To use this feature, you have to make sure 4th Intel generation core platform or higher are used. The next step is to run a GDB command “record btrace bts” before the crash happens. “record btrace bts” enables Branch Trace Store feature to record the history of function calls. Once the problematic program hits the exception and sends out signals like SIGSEGV for segmentation fault, you will right away see Function Call History windows to print out the functions history the program has traversed. That gives you a full debug information for how this crash happens. Check the Debugger Console window and Function Call History window for details in the figure below.

Figure 3. Use Intel System Studio 2018 to review function call history.

For higher Intel platforms than 4th Intel Core, you can type “record btrace pt” to enable Intel Processor Trace with a compressed log format which is lower overhead and more records entries compared to bts(Branch Trace Store).

Intel System Studio 2018 provides IDE integrate Function Call History to support viewing the records from “record btrace bts” and “record btrace pt”. You can also use “record function-call-history” to view the records in plain text format in debugger console. Now 90-days free and full license of Intel System Studio 2018 is available via http://intel.ly/system-studio. Check register and download on that web page.

More debug usages you might want to know.

There are quite a few cases of segmentation fault caused by out-of-boundary pointers or arrays access.  You can try the compiler options like --check-pointers, -check-pointers-undimensioned to aid GDB to debug these invalid access. Other tools like <valgrind> can be used to detect memory leakages, memory access validation and other memory issues., <strace> allows you to review logs to understand how the program invokes system calls and interacts with system level components to further debug more complicated issues.

See also

GDB - The GNU* Project Debugger for Intel® Architecture - https://software.intel.com/en-us/articles/gdb-ia-embedded

DebuggingProgramCrash - https://wiki.ubuntu.com/DebuggingProgramCrash

Intel C++ Compiler’s -check-pointers option - https://software.intel.com/en-us/cpp-compiler-18.0-developer-guide-and-reference-check-pointers-qcheck-pointers

Intel C++ Compiler’s -check-pointers-undimensioned option -

https://software.intel.com/en-us/cpp-compiler-18.0-developer-guide-and-reference-check-pointers-undimensioned-qcheck-pointers-undimensioned