Contents

ftrace

ftrace (Function Tracer) is a tracing framework for the Linux kernel. Although its original name, Function Tracer, came from ftrace's ability to record information related to various function calls performed while the kernel is running, ftrace's tracing capabilities cover a much broader range of kernel's internal operations.^[2][3]

With its various tracer plugins, ftrace can be targeted at different static tracepoints, such as scheduling events, interrupts, memory-mapped I/O, CPU power state transitions, and operations related to file systems and virtualization. Also, dynamic tracking of kernel function calls is available, optionally restrictable to a subset of functions by using globs, and with the possibility to generate call graphs and provide stack usage reports. At the same time, ftrace can be used to measure various latencies within the Linux kernel, such as for how long interrupts or preemption are disabled.^{[2][4][5]: 3–11, 14, 18}

An ftrace-enabled Linux kernel is built by enabling the CONFIG_FUNCTION_TRACER kernel configuration option. The entire runtime interaction with ftrace is performed through readable and writable virtual files contained in a specifically mounted debugfs file system; as a result, ftrace requires no specialized userspace utilities to operate.^[2][3][6] However, there are additional userspace utilities that provide more advanced features for data recording, analysis and visualization; examples of such utilities are trace-cmd and KernelShark.^{[2][5]: 31–47 [7][8]}

Internally, ftrace relies on the gcc's profiling mechanism to prepend machine instructions to the compiled versions of all source-level kernel functions, which redirect the execution of functions to the ftrace's trampolines and tracer plugins that perform the actual tracing. These "entry point" instructions created by gcc are altered by ftrace when the kernel is booted, and varied later at runtime by ftrace between NOPs and actual jumps to the tracing trampolines, depending on the tracing types and options configured at runtime.^[9][10]

ftrace is developed primarily by Steven Rostedt, and it was merged into the Linux kernel mainline in kernel version 2.6.27, which was released on October 9, 2008.^[11]

• DTrace – a tracing framework for troubleshooting kernel and application problems, originally developed for Solaris
• ktrace – a BSD Unix and Mac OS X utility that traces kernel–program interactions
• ltrace – a Linux debugging utility, displays the calls a userland application makes to shared libraries
• strace – a debugging utility for Linux and some other Unix-like systems, monitors system calls used by a program and all received signals
• SystemTap – a scripting language and utility used for instrumenting Linux systems

• Debugging the kernel using Ftrace – part 1 and part 2, LWN.net, December 2009, by Steven Rostedt
• Secrets of the Ftrace function tracer, LWN.net, January 20, 2010, by Steven Rostedt
• Ftrace: The hidden light switch, LWN.net, August 13, 2014, by Brendan Gregg
• The source of the e1000e corruption bug, LWN.net, October 21, 2008, by Jonathan Corbet