ptrace on ia64

I decided to write a blog entry about my porting efforts of sydbox to IA64.
Thanks to ahf who gave me access to an IA64 box, i started porting sydbox to IA64.

All ptrace() related stuff of sydbox resides in trace.c.
So that’s the file we’re interested in. For x86 and x86_64 architectures
everything was so easy, just fill in some registry values and it’s all done. For IA64, however, additional hackery is needed.

First problem, you can’t include both sys/ptrace.h and linux/ptrace.h due to redefinition errors. This is easy to solve, I added an autoconf check:

AC_CHECK_TYPES([struct pt_all_user_regs, struct ia64_fpreg],,,[#include <sys/ptrace.h>])

This one exports two defines to config.h namely HAVE_STRUCT_IA64_FPREG and
HAVE_STRUCT_PT_ALL_USER_REGS, using this two defines it’s possible to include both files using some preprocessor hacks:

#include <sys/ptrace.h>
/* We need additional hackery on IA64 to include linux/ptrace.h to avoid
 * redefinition errors.
 */
#if defined(IA64)
#ifdef HAVE_STRUCT_IA64_FPREG
#define ia64_fpreg XXX_ia64_fpreg
#endif // HAVE_STRUCT_IA64_FPREG
#ifdef HAVE_STRUCT_PT_ALL_USER_REGS
#define pt_all_user_regs XXX_pt_all_user_regs
#endif // HAVE_STRUCT_PT_ALL_USER_REGS
#include <linux/ptrace.h>
#undef ia64_fpreg
#undef pt_all_user_regs
#endif // defined(IA64)

and problem solved.

The second problem is figuring out how to get system call number, this is easy too. PT_R15 is the registry we need.

The third problem is getting and setting the return value of system calls. On IA64 the registry PT_R10 has the return value of the system call. One last thing is the errno. The errno should reside in the PT_R8 registry and it should be positive unlike x86 and x86_64 where return value of the system call is the negated errno.

So far so good, the last problem is getting and setting arguments of functions. This is a bit too long to explain, interested readers are encouraged to look at the trace_ia64_peek function in trace.c.

~ by alip on 2009/07/11.