From GPU Debugging: Challenges and Opportunities presented at the International Workshop on OpenCL (IWOCL) 2017.
GPU debugging support matches the OpenCL™ 2.0 GPU/CPU driver package for Linux* (64-bit) from OpenCL™ Drivers and Runtimes for Intel® Architecture with the notable exception of processors based on the Broadwell architecture.
Basic concepts to keep in mind for GPU debugging
- There are "host" and "target" components. Host = where you interact with the debugger, target = where the application is run
- There are 3 components: gdbserver, the application to be debugged, and the gdb session.
- The gdb session and application can be on the same or different machines
- Breakpoints in the graphics driver can affect screen rendering. You cannot debug on the same system that is rendering your screen.
- However, non-graphical connections (such as with SSH) are unaffected. The "host" can be connected to remotely as well for gdb's text interface.
Abbreviations used:
KMD - Kernel Mode Driver
RT = OpenCL Runtime
DCD – Debug Companion Driver
- Ring-0 driver, provides low-level gfx access
- Run control flow, breakpoints, etc
DSL – Debug Support Library
- Ring-3 debugger driver (shared library)
- Loaded into the gdbserver process
DSL <--> DCD
- Communicate via IOCTLs
How to set up a debugging session
The simplest option is to use ssh for steps 1,2, and 3. However, gdb can be run locally as well. The target steps (1 and 2) should be run remotely because GPU breakpoints can cause rendering hangs.
1. launch gdbserver
/usr/bin/gdbserver-igfx :1234 --attach 123
2. launch the application
export IGFXDBG_OVERRIDE_CLIENT_PID=123 ./gemm
Note: there is an automatic breakpoint at the first kernel launch
3. launch GDB
source /opt/intel/opencl-sdk/gt_debugger_2016.0/bin/debuggervars.sh /opt/intel/opencl-sdk/gt_debugger_2016.0/bin/launch_gdb.sh –tui
In GDB
target remote :1234 continue x/i $pc
GDB should now be able to step through the kernel code.