[Mono-devel-list] RE: [Gc] Segmentation fault with "GC_ENABLE_INCREMENTAL"

[Boehm, Hans]

For the incremental GC to work correctly, every page in the heap needs to either be
protected or marked dirty, since we need to keep track of all pointer writes.
Normally pages are unprotected only in response to a write fault, which can be
caused by either client or mutator.  The collector
explicitly unprotects a page and marks it dirty to optimize certain cases.

This mechanism does not work with system calls that write to the heap,
since OS kernels don't treat those the same as ordinary writes.  Typically the
system call fails rather than invoking the handler.  Most seriously,
there is often no way to recover from a write fault generated during a system call.
One could argue that this is a misdesign in the kernel interface; but it has a long
history and all sorts of code expects it to work the way it does.

Possible ways to deal with this:

1) Provide a "dirty bit" facility in the kernel that does the right thing.
This would also eliminate the signal handling overhead.
No doubt impractical, especially in the short term, except on the one or two
platforms for which it currently exists.

2) Change Mono so that it doesn't write to the heap in a system call, e.g. by 
having the system call write to the stack first and then copying.  Expensive.
Probably fine for system calls that write small amounts of data.

3) Explicitly unprotect memory before issuing a system call that writes to
the heap.  See the GC_begin_syscall(), GC_end_syscall(), GC_unprotect_range()
calls.  This doesn't really work very well for multithreaded code, since
you need to hold the GC lock during the system call, which is very bad if the
call is e.g. a blocking read(). 

4) Make sure that all system call writes go to heap blocks allocated with
GC_malloc_atomic().  Since system calls rarely write pointers, this can
usually be arranged.  Enable incremental collection only if
GC_incremental_protection_needs() & (GC_PROTECTS_PTRFREE_HEAP | GC_PROTECTS_STATIC_DATA | GC_PROTECTS_STACK)
is zero.  This should be the case on X86 with a standard collector configuration.
On other architectures you may need to adjust HBLKSIZE in the collector to match the
page size to make this work.  

I would try for a combination of (2) and (4).

Hans

> -----Original Message-----
> From: Okehee Goh [mailto:ogoh at asu.edu]
> Sent: Friday, July 18, 2003 1:31 PM
> To: Boehm, Hans; 'gc at linux.hpl.hp.com'; 
> mono-devel-list at lists.ximian.com
> Subject: RE: [Gc] Segmentation fault with "GC_ENABLE_INCREMENTAL"
> 
> 
> 
> 
>  Hello Dr.Boehm,
>  Thank you so much for your reply.
>  I am looking at libgc that comes with Mono-0.25. I'd like to 
> understand it
> firmly.
> 
>  The memory protection during an incremental GC in BoehmGC6.1 
> seems to be
> enabled such that
>    (1) SIGSEV is associated to trigger GC_write_fault_handler at
> GC_dirty_init(), and GC_write_fault_handler() records pages 
> that violate a
> write protection,
>    (2) a heap is protected as just read-only during incremental GC at
> GC_read_dirty() which is called at stopped_mark(),
>    (3) dirty pages are re-visited again at mark_some()...
> ( os_dep.c is very complicating to support various platforms. 
> I followed
> functions for MPROTECT_VDB for my system, i386 and Linux)
> 
>  If so, some system calls from Mono, that attempt to write on 
> a protected
> heap can cause SIGSEV even though they are nothing to do with 
> application's
> memory writing that must be caught for Write Barrier.
>  But, why aren't they trapped by GC_write_fault_handler()?
>  And also could you give any advice about what to check or 
> what to do to
> support the current implementation for an incremental GC with Mono?
> 
>  This is another question.
>  At the end of GC_write_fault_handler(), it releases the 
> protection of the
> page that violates a write protection after recording the 
> page as a dirty
> page. But, I couldn't find where the protection for other pages  are
> released before starting an application(mutator).
> 
>  Thank you so much in advance.
> 
>  Regards,
> 
>  Okehee
> 
> 
> -----Original Message-----
> From: Boehm, Hans [mailto:hans_boehm at hp.com]
> Sent: Wednesday, July 16, 2003 3:45 PM
> To: 'Okehee Goh'; 'gc at linux.hpl.hp.com'; ximian-mono-list at ximian.com
> Subject: RE: [Gc] Segmentation fault with "GC_ENABLE_INCREMENTAL"
> 
> 
> Unless Mono specifically supports this by being careful about 
> where system
> calls can write, this is unlikely to work.
> 
> It will almost certainly fail if any system calls (e.g. 
> "read") write to
> heap sections allocated with GC_malloc, though sometimes 
> sections allocated
> with GC_malloc_atomic are OK.  There are vague plans to hide 
> all of this and
> make this directly usable in gcj, but I don't think it has 
> happened there
> yet, either.
> 
> I assume you're doing this because you saw unacceptable pause 
> times without
> incremental collection?  Other things that might help a bit are:
> 
> 1) Making sure that pointer-free objects (e.g. bitmaps, 
> arrays of numbers)
> are known by the collector to be pointer-free.  I don't know 
> whether Mono
> already does this, or whether you are allocating any other 
> objects to which
> this might apply.
> 
> 2) If you are running on a dual processor (or possibly 
> "hyperthreaded" P4),
> turn on the parallel GC.
> 
> Hans
> 
> > -----Original Message-----
> > From: Okehee Goh [mailto:ogoh at asu.edu]
> > Sent: Wednesday, July 16, 2003 2:11 PM
> > To: 'gc at linux.hpl.hp.com'; ximian-mono-list at ximian.com
> > Subject: [Gc] Segmentation fault with "GC_ENABLE_INCREMENTAL"
> >
> >
> >
> >
> >  Hello,
> >  I run Mono-0.25 with "GC_ENABLE_INCREMENTAL" to enable an
> > incremental GC.
> >  But it just gave a segmentation fault.
> >  One of documents regarding Boehm's GC says that the
> > incremental GC can
> > cause "unintended system call failure."
> >  (Refer to the following..)
> >  Is there anybody who didn't encounter this problem? My
> > system is Linux
> > kernel 2.4 and i686.
> >
> >   ---
> >  GC_ENABLE_INCREMENTAL - Turn on incremental collection at
> > startup. Note
> > that,
> > depending on platform and collector configuration, this
> > may involve write protecting pieces of the heap to
> > track modifications. These pieces may include pointerfree
> > objects or not. Although this is intended to be
> > transparent, it may cause unintended system call failures.
> > Use with caution.
> >
> >
> >  Any suggestion will be appreciated.
> >
> >  Regards,
> >
> >  Okehee
> >
> > _______________________________________________
> > Gc mailing list
> > Gc at linux.hpl.hp.com
> > http://linux.hpl.hp.com/cgi-bin/mailman/listinfo/gc
> >
>