[Monodevelop-devel] Newbie question about soft debugger

[Christoph Müller]

>> Sdb uses segmentation faults to trigger breakpoints, so a native 
>> debugger will detect them as crashes when they're triggered. Is your 
>> app actually crashing when the (VS) debugger isn't attached?
>> Also, breakpoints will actually cause (unhandled) crashes on 64bit 
>> windows prior to win7sp1, I don't know what version of vista, and all
known versions of xp.
>> (http://support.microsoft.com/kb/976038)

>Thx for your quick reply. I am running on win7sp1 64 bit. Yes, my app also
crashes with the 
>VS debugger not attached, with both, my self-built version of mono-2.0.dll
as well as the 
>one shipped with Mono 2.10. Well, it's not quite a crash - it's rather some
application 
>message box saying that there was some severe error and the app is going to
close now.
>Maybe there's some segmentation fault exception handler built into the
native app? 
>How do I get the soft debugger exception handler (don't know if that's the
correct 
>terminology) to handle the breakpoint exception first?
>I'll probably try to recreate the situation with a simple console C++
application to
>see if there's a difference to the application I'm currently using.
>Thanks again

Sorry to annoy you again. I'm still looking for a way to embed mono into a
C++Plugin-API of a commercial product (I don't have the sources) and still
trying to get the monodevelop soft-debugger connected. So I am initializing
mono from some C++ Plugin for that product I have written. I can start the
app from my self-written Monodevelop-Addon and everything runs quite well.
Whenever I set a breakpoint in Monodevelop, the app crashes when hitting the
breakpoint. 

As it seems, the product wraps every call to Plugins into some sort of __try
{} __except block. At least that's the most probable explanation why the
seh_handler set by SetUnhandledExceptionFilter (exceptions-x86.c) is never
reached - the exception (or segmentation fault) generated by a breakpoint is
swallowed by the application and results in an error message from the
application code. 

>From my plugin, I was able to set my own exception block around some code
calling into mono like the following:

MonoObject *CallPluginStart(MonoMethod *methodStart, void *pluginobject)
{
	MonoObject *result = NULL;
	__try
	{
		// plugin.Start();
		result = mono_runtime_invoke(methodStart, pluginobject,
NULL, NULL);
	}
	__except(GetExceptionCode() == EXCEPTION_ACCESS_VIOLATION ?
EXCEPTION_EXECUTE_HANDLER) :
EXCEPTION_CONTINUE_SEARCH)
	{
		volatile int i = 42;
	}
	return result;
}

And I verified that I get into the exception block when a breakpoint is set
and that the invoke call returns without exception when no breakpoint is
hit. 

So now my question is: Is there a way to bypass the application's exception
handling to enable the seh_handler installed by mono to handle the
segmentation fault (and report the breakpoint hit to the to the connected
debugger)?

My first idea was to call the seh_handler directly from the code above, but
I had problems with building my mono-2.0-dll and I wanted to know if that's
a promising solution before carrying on. 

The other idea I had was: Why not install the exception handler using
AddVectoredExceptionHandler instead of (or in addition to)
SetUnhandledExceptionFilter. A handler installed that way can be told to be
the first in line, so it would be called before the application gets
signaled. Also trying this approach would mean that I need to build my own
mono-2.0.dll.

I'd be glad if you could give me some expert opinion what would be the best
way for me to proceed.

Thanks a lot,
Christoph