[Mono-dev] making mono builds reproducible (xamarin bz #26842)

[Jeroen Frijters]

Thanks, that was very helpful. I've implemented something similar. By setting the UniverseOptions.DeterministicOutput flag, IKVM.Reflection will now do the same as Roslyn. This is currently not compatible with PDB file generation (because the PDB file generates another random GUID in the debug directory), but for Mono that is not an issue.

Marek, if mcs sets this flag it won't need to do anything else and given that Roslyn behaves the same, it is unlikely to cause problems (although for large files there is a small perf impact of hashing the output file).

Regards,
Jeroen

> -----Original Message-----
> From: Alexander Köplinger [mailto:alex.koeplinger at outlook.com]
> Sent: Tuesday, February 17, 2015 13:03
> To: Jeroen Frijters; Daniel Kahn Gillmor; mono development list;
> marek.safar marek.safar at gmail.com
> Cc: Henrik Feldt; Jo Shields; Debian Reproducible Builds
> Subject: RE: [Mono-dev] making mono builds reproducible (xamarin bz
> #26842)
> 
> FYI, roslyn made the switch to being deterministic by default in April
> last year:
> https://github.com/dotnet/roslyn/commit/04462c44e30dfa91267581abdb029f31
> 02796486
> 
> Quoting from the commit:
> "(1) The timestamp in the header is replaced with 0 (which is
> specifically allowed by the spec)
>  (2) The module version ID Guid (Mvid) is computed by hashing the
> contents of the generated assembly (with zero
>       where the Mvid will go for the purposes of computing the hash)
> 
> The name of the "private implementation details" class no longer
> includes the Mvid."
> 
> -- Alex
> 
> 
> > From: jeroen at sumatra.nl
> > To: dkg at fifthhorseman.net; mono-devel-list at lists.ximian.com;
> > marek.safar at gmail.com
> > Date: Tue, 17 Feb 2015 11:12:14 +0000
> > CC: henrik at logibit.se; directhex at apebox.org;
> > reproducible-builds at lists.alioth.debian.org
> > Subject: Re: [Mono-dev] making mono builds reproducible (xamarin bz
> > #26842)
> >
> > Hello all,
> >
> > I'm not a big fan of using an environment variable for this, but I
> won't oppose it. In the mean time, I've added a new public API
> (ModuleBuilder.__PEHeaderTimeDateStamp) to specify the time stamp. There
> is already a public API to specify the module version GUID.
> >
> > As Miguel said, hard coding the GUID violates the spec and can lead to
> weird problems. I think the proper solution is to base the GUID on a
> SHA1 of the contents of the assembly. However, this is non-trivial, so
> it will take some time to develop a complete solution. This will be an
> opt-in feature, so it will also need to be supported by mcs or an
> environment variable.
> >
> > Marek, what do you think about adding an mcs switch to facilitate
> reproducible builds?
> >
> > Regards,
> > Jeroen
> >
> > > -----Original Message-----
> > > From: Daniel Kahn Gillmor [mailto:dkg at fifthhorseman.net]
> > > Sent: Monday, February 16, 2015 23:20
> > > To: mono development list
> > > Cc: Jeroen Frijters; Jo Shields; Debian Reproducible Builds; Henrik
> > > Feldt
> > > Subject: making mono builds reproducible (xamarin bz #26842)
> > >
> > > Hi Mono folks--
> > >
> > > some good discussion has come up on the xamarin bugtracker about
> > > being able to make builds using the mono toolchain reproducible:
> > >
> > > https://bugzilla.xamarin.com/show_bug.cgi?id=26842
> > >
> > > Jo Shields offered a one-liner fix to PEWriter.cs to allow the use
> > > of an environment variable to fix the timestamp epoch, but made it
> > > clear that more feedback is needed, so i'm raising it here:
> > >
> > > from:
> > >
> > > public DWORD TimeDateStamp = (uint)(DateTime.UtcNow - new
> > > DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalSeconds;
> > >
> > > to:
> > >
> > > public DWORD TimeDateStamp =
> > > Environment.GetEnvironmentVariable("IKVM_WRITER_TIMESTAMP_EPOCH") !=
> > > null ?
> > >
> > > uint.Parse(Environment.GetEnvironmentVariable("IKVM_WRITER_TIMESTAMP
> > > _EPO
> > > CH")) :
> > > (uint)(DateTime.UtcNow - new
> > > DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalSeconds;
> > >
> > > (i'm not sure whether the integer number of seconds is the best form
> > > for the environment variable, or whether it would be better to parse
> > > a standard date string -- from debian's perspective, we can deal
> > > with either, of course)
> > >
> > > And this still doesn't solve the secondary issue of the assembly
> > > GUID, as Jo noted.
> > >
> > > Any thoughts about how to best enable binary-reproducible builds
> > > from the mono toolchain if the invoker requests them?
> > >
> > > Regards,
> > >
> > > --dkg
> > _______________________________________________
> > Mono-devel-list mailing list
> > Mono-devel-list at lists.ximian.com
> > http://lists.ximian.com/mailman/listinfo/mono-devel-list