[Mono-dev] Repeat builds of core assemblies

[Rolf Bjarne Kvinge]

Hi,

On Tue, Apr 22, 2014 at 9:53 PM, Miguel de Icaza <miguel at xamarin.com> wrote:

> Hey guys,
>
> I was looking at making the MSBuild system work, and during the process I
> have encountered a few problems that we have in our existing build system
> that are problematic.
>
> The problem is that System, System.XML and System.Configuration are each
> defined in terms of the other assemblies.   So we gradually bring up each
> one of those assemblies up by first compiling a stub System, which we use
> to build System.XML and System.Configuration.   Then we rebuild System,
> this time referencing System.XML and System.Configuration so we can take a
> dependency on them, and so on.
>
> To build a complete System.dll for a particular profile (net_2_0, net_4_0,
> etc) takes three steps:
>
>    - Core Build
>    - Secondary Build:
>       - Core Build +
>       - Defines: XML_DEP + SECURITY_DEP
>       - Refs:
>          - -r:PrebuiltSystem=../lib/Previous/System.dll
>          - -r:System.Xml.dll
>          - -r:MonoSecurity=Mono.Security.dll
>       - Final Build:
>       - Secondary Build +
>       - defines: -d:CONFIGURATION_DEP
>       - Refs:
>          - System.Configuration.dll
>
> Another option is to take advantage of the fact that ilasm does not
require referenced assemblies to exist.

The idea would be do to the following:

Once:
* Use any existing mscorlib.dll, System.dll and System.XML.dll to create
mscorlib.il, System.il and System.XML.il (possibly by using mono-cil-strip
so that only the metadata remains). We commit these files to the repository.

During the build:
* Compile the *.il files to a set of reference assemblies (note that since
ilasm does not require referenced assemblies to exist, we can compile
mscorlib.il to a mscorlib.dll that references System.dll before System.dll
exists).
* Use those reference assemblies to compile the final mscorlib.dll,
System.dll and System.XML.dll

Pros:
* No circular build magic required.
* The build should be faster, each assembly is compiled at most twice (once
the .il and once the final assembly). With the current setup we compile
some assemblies three times.

Cons:
* It'll probably be painful to update/create new .il files when new
profiles comes out from Microsoft.

Rolf



> The above is what is required to bring up System.
>
> Our implementation has one major problem: it overwrites the intermediate
> files.  So the core build output is overwritten by the secondary build, and
> the secondary build is overwritten by the final build.
>
> It seems that historically, instead of introducing temporary directories
> for each stage, instead we hacked our way out of it.   We introduced a
> LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the
> case where Windows was actively telling us we were doing something wrong
> (we were overwriting a file that we were actively referencing!)
>
> The above is also likely going to prevent reliable parallel builds, or
> probably means that we introduced some gross hack to make the above work in
> parallel.
>
> I am going to try to fix this, but the Makefile goop is pretty dense, and
> I might fail.   I just figured I should share my findings in case
> civilization comes to an end and a future archeologist tries to figure out
> why this was not working.
>
> These are the defines that we use to bring up System for each profile:
>
> basic Profile:
>
> basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0
>
> basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0
> -d:XML_DEP
>
>
> Build Profile:
>
> build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0
>
> build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0  -d:XML_DEP
>
>
> Net 2.0 profile:
>
> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0
>
> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0  -d:XML_DEP
> -d:SECURITY_DEP
>
> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0  -d:XML_DEP
> -d:SECURITY_DEP -d:CONFIGURATION_DEP
>
>
> Net 4.0 profile:
>
> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0
>
> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP
>
> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP  -d:CONFIGURATION_DEP
>
>
> Net 4.5 profile:
>
> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
> -d:CONFIGURATION_2_0
>
> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP
>
> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
> -d:CONFIGURATION_2_0  -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP
>
>
> Miguel
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
>


-- 
Explore Xamarin University <http://xamarin.com/university>--unlimited, live,
online, mobile training around the clock.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20140423/f621e65c/attachment.html>