[Mono-list] catching stack overflow signals

Dietmar Maurer dietmar@ximian.com
19 Jun 2002 17:42:42 +0200

Content-Type: text/plain
Content-Transfer-Encoding: 7bit

Hi all,

today I tried to find a solution to catch stack overflow signals
(segmentation fault) in order to throw a StackOverflow exception, but
unfortunately its not so easy.

On stack overflow the system raises a SIGSEGV signal and transfers
control to the handler. But that action needs the stack, so it raises
another SIGSEGV which then terminates the process immediately.

On solution is to use a separate stack for signal handler. We can use
sigaltstack () to setup such extra stack. I have attached a small test
program I found on the net to show how this works - simply compile the
program with:

gcc sigtest.c -o sigtest

The problem with that solution is that i does not work with pthread. you
can test that if you compile the program with:

gcc sigtest.c -o sigtest -lpthread

Pthread tries to determine the current thread using the stack pointer,
so this fails if we change the stack using sigaltstack.

On workaround is to use a stack allocated memory for sigaltstack, for
example we can define a local variable:

char alttack [8192] __attribute__ (( aligned (32) )); 

That way we solve the above problem, but i am quite sure that this will
confuse pthread somehow, because its thread detection always points to
the same thread in signal handler.

So maybe someone knows a better solution?

- Dietmar

Content-Disposition: attachment; filename=sigtest.c
Content-Transfer-Encoding: quoted-printable
Content-Type: text/x-c; name=sigtest.c; charset=ISO-8859-1

 * Test program to test sigaltstack() system call.
 * Does a stack overflow, which normally linux/i386 will not be able=20
 * to deliver. There are other ways of demonstrating but this is
 * the easiest. Of course apply the sigaltstack() patch first or
 * you wont see diddly.

#include <sys/signal.h>
#include <asm/unistd.h>
#include <stdio.h>
#include <errno.h>
#include <sys/time.h>
#include <sys/resource.h>
#include <unistd.h>
#include <sys/syscall.h>

struct sigaction act;

#define GET_STACK( x ) \
	__asm__ __volatile("movl %%esp, %0\n\t" : "=3Dq" (x) : :"%0" )

void handler()
	unsigned long ss_sp;

	printf( "Caught SIGSEGV and handled stack overflow correctly.\n" );
	GET_STACK( ss_sp );
	printf( "signal handler stack pointer =3D %p\n", ss_sp );
void handler1()

void recurse()
	char buf[ 4096 ];

int main()
	long ret;
	struct sigaltstack sa, sa_old;
	unsigned long ss_sp;

	/* Step 1 - setup your alternate sig-stack */
	sa.ss_sp =3D malloc( MINSIGSTKSZ );
	if( !sa.ss_sp )
		printf( "malloc failed\n" );

	sa.ss_size =3D MINSIGSTKSZ;
	sa.ss_flags =3D SS_ONSTACK;

	if( sigaltstack( &sa, &sa_old ) < 0 ) {
		printf( "failed to install alt-stack!\n" );

/* Step 2 - setup a sighandler and specify we want it delivered
	 * on the alternate stack */
	act.sa_handler =3D handler;
	act.sa_flags =3D SA_STACK;

#if 0
	syscall (SYS_sigaction, SIGSEGV, &act, NULL);
	sigaction( SIGSEGV, &act, 0);

/* Step 3 - Generate a stack-overflow with recursion.
     * Without the sigaltstack you will not handle the SIGSEGV
	 * because there will be no more space left on the processes
	 * main stack for the call.
     * With the patch, you will catch it correctly! Wheee!!=20

	GET_STACK( ss_sp );
	printf( "main stack pointer =3D %p\n", ss_sp );