Can't trace userspace processes with SystemTap

What is System Tap ?

It is an infrastructuer coming with a scripting language that allows collecting information on a running linux kernel. It can be used by System Administrators to see for example which process are hitting hardly the disks, or by Kernel/UserSpace developers to instrument their code and gather information about disk access,  network usage... A great tool to fine tune and optimize application.
It is similar to DTrace available for Solaris and BDS Oses.

A good documentation exists for System Tap: http://sourceware.org/systemtap/tutorial/

User Space debugging

For Debian Squeeze System Tap works out of the box for probing kernel code. But when I wanted to trace userspace code I was facing the following error:

# cat test.stp

probe process("./test").function("main").return
{
    printf ("Exit from the main\n");
}

#stap -v test.stp -c ./test

Pass 1: parsed user script and 68 library script(s) using 49964virt/20928res/1768shr kb, in  10usr/20sys/424real ms.
semantic error: process probes not available without kernel CONFIG_UTRACE while resolving probe point process("./test").function("main").return

So it seems like my Debian kernel is lacking of utrace supports. Let's recompile it :). First of all we have to get a vanilla kernel (because with the debian I can't apply utrace patches properly):

#wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.tar.bz2

Then the utrace patches:

#wget http://web.elastic.org/~fche/frob-utrace/2.6.32/tracehook.patch

#wget http://web.elastic.org/~fche/frob-utrace/2.6.32/utrace.patch

#wget http://web.elastic.org/~fche/frob-utrace/2.6.32/utrace-ptrace.patch

The trick here is that tracehook.patch is a gzip archive and not a plain text file:

#mv tracehook.patch tracehook.gz

#gunzip tracehook.gz

#mv tracehook tracehook.patch

Untar linux sources and apply the patches:

#tar -xvf linux-2.6.32.tar.bz2

#cd linux-2.6.32

#patch -p1 < ../tracehook.patch

#patch -p1 < ../utrace.patch

#patch -p1 < ../utrace-ptrace.patch

Compile the kernel:

#cp /boot/config-`uname -r` .config

#make oldconfig

Enable the option CONFIG_UTRACE

#make menuconfig (->General Setup -> Infrastructure for debugging and tracing user processes ) 


#sudo make

#sudo make modules_install install

#mkinitramfs -o /boot/initrd-img.2.6.32 2.6.32

Also modify grub to add an entry pointing to your new kernel and initrd image.

Reboot and that's it !