[Mono-dev] Repeat builds of core assemblies

[Miguel de Icaza]

Hello guys,

Just a follow up to my previous posting on this.

I have managed to untangle this mess, and now I have a clean build that
does not involve overwriting assemblies.

In addition to untangling this, I added dependencies on all the assemblies
involved in this circular dependency mess so if you type "make" in any of
System, System.Xml, System.Security, Mono.Security or System.Configuration,
all the dependencies will be properly built.

During the fixing, I noticed that our System.Xml build must have broke a
few eons ago, because there was code in place to perform a 2-stage
System.Xml build as well (without and with System.Configuration support),
but nobody noticed that this had happened.   While I fixed this, it raises
the obvious point that nobody really cares (or likes) System.Configuration.

While doing this review, I found a few other places that also have these
ugly loops, so I am going to be fixing those as well.

Miguel


On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza <miguel at xamarin.com> wrote:

> Hey guys,
>
> I was looking at making the MSBuild system work, and during the process I
> have encountered a few problems that we have in our existing build system
> that are problematic.
>
> The problem is that System, System.XML and System.Configuration are each
> defined in terms of the other assemblies.   So we gradually bring up each
> one of those assemblies up by first compiling a stub System, which we use
> to build System.XML and System.Configuration.   Then we rebuild System,
> this time referencing System.XML and System.Configuration so we can take a
> dependency on them, and so on.
>
> To build a complete System.dll for a particular profile (net_2_0, net_4_0,
> etc) takes three steps:
>
>    - Core Build
>    - Secondary Build:
>       - Core Build +
>       - Defines: XML_DEP + SECURITY_DEP
>       - Refs:
>          - -r:PrebuiltSystem=../lib/Previous/System.dll
>          - -r:System.Xml.dll
>          - -r:MonoSecurity=Mono.Security.dll
>       - Final Build:
>       - Secondary Build +
>       - defines: -d:CONFIGURATION_DEP
>       - Refs:
>          - System.Configuration.dll
>
> The above is what is required to bring up System.
>
> Our implementation has one major problem: it overwrites the intermediate
> files.  So the core build output is overwritten by the secondary build, and
> the secondary build is overwritten by the final build.
>
> It seems that historically, instead of introducing temporary directories
> for each stage, instead we hacked our way out of it.   We introduced a
> LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the
> case where Windows was actively telling us we were doing something wrong
> (we were overwriting a file that we were actively referencing!)
>
> The above is also likely going to prevent reliable parallel builds, or
> probably means that we introduced some gross hack to make the above work in
> parallel.
>
> I am going to try to fix this, but the Makefile goop is pretty dense, and
> I might fail.   I just figured I should share my findings in case
> civilization comes to an end and a future archeologist tries to figure out
> why this was not working.
>
> These are the defines that we use to bring up System for each profile:
>
> basic Profile:
>
> basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0
>
> basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0
> -d:XML_DEP
>
>
> Build Profile:
>
> build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0
>
> build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0  -d:XML_DEP
>
>
> Net 2.0 profile:
>
> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0
>
> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0  -d:XML_DEP
> -d:SECURITY_DEP
>
> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0  -d:XML_DEP
> -d:SECURITY_DEP -d:CONFIGURATION_DEP
>
>
> Net 4.0 profile:
>
> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0
>
> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP
>
> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP  -d:CONFIGURATION_DEP
>
>
> Net 4.5 profile:
>
> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
> -d:CONFIGURATION_2_0
>
> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP
>
> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
> -d:CONFIGURATION_2_0  -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP
>
>
> Miguel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20140502/9c8ffae9/attachment.html>