[Mono-dev] Repeat builds of core assemblies

[Miguel de Icaza]

Hey guys,

Another update: I am almost done with the work, only one cycle left to
resolve and I will be able to land the changes.

All the changes are on a branch on github.

MIguel


On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza <miguel at xamarin.com> wrote:

> Hello guys,
>
> Just a follow up to my previous posting on this.
>
> I have managed to untangle this mess, and now I have a clean build that
> does not involve overwriting assemblies.
>
> In addition to untangling this, I added dependencies on all the assemblies
> involved in this circular dependency mess so if you type "make" in any of
> System, System.Xml, System.Security, Mono.Security or System.Configuration,
> all the dependencies will be properly built.
>
> During the fixing, I noticed that our System.Xml build must have broke a
> few eons ago, because there was code in place to perform a 2-stage
> System.Xml build as well (without and with System.Configuration support),
> but nobody noticed that this had happened.   While I fixed this, it raises
> the obvious point that nobody really cares (or likes) System.Configuration.
>
> While doing this review, I found a few other places that also have these
> ugly loops, so I am going to be fixing those as well.
>
> Miguel
>
>
> On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza <miguel at xamarin.com>wrote:
>
>> Hey guys,
>>
>> I was looking at making the MSBuild system work, and during the process I
>> have encountered a few problems that we have in our existing build system
>> that are problematic.
>>
>> The problem is that System, System.XML and System.Configuration are each
>> defined in terms of the other assemblies.   So we gradually bring up each
>> one of those assemblies up by first compiling a stub System, which we use
>> to build System.XML and System.Configuration.   Then we rebuild System,
>> this time referencing System.XML and System.Configuration so we can take a
>> dependency on them, and so on.
>>
>> To build a complete System.dll for a particular profile (net_2_0,
>> net_4_0, etc) takes three steps:
>>
>>    - Core Build
>>    - Secondary Build:
>>       - Core Build +
>>       - Defines: XML_DEP + SECURITY_DEP
>>       - Refs:
>>          - -r:PrebuiltSystem=../lib/Previous/System.dll
>>          - -r:System.Xml.dll
>>          - -r:MonoSecurity=Mono.Security.dll
>>       - Final Build:
>>       - Secondary Build +
>>       - defines: -d:CONFIGURATION_DEP
>>       - Refs:
>>          - System.Configuration.dll
>>
>> The above is what is required to bring up System.
>>
>> Our implementation has one major problem: it overwrites the intermediate
>> files.  So the core build output is overwritten by the secondary build, and
>> the secondary build is overwritten by the final build.
>>
>> It seems that historically, instead of introducing temporary directories
>> for each stage, instead we hacked our way out of it.   We introduced a
>> LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the
>> case where Windows was actively telling us we were doing something wrong
>> (we were overwriting a file that we were actively referencing!)
>>
>> The above is also likely going to prevent reliable parallel builds, or
>> probably means that we introduced some gross hack to make the above work in
>> parallel.
>>
>> I am going to try to fix this, but the Makefile goop is pretty dense, and
>> I might fail.   I just figured I should share my findings in case
>> civilization comes to an end and a future archeologist tries to figure out
>> why this was not working.
>>
>> These are the defines that we use to bring up System for each profile:
>>
>> basic Profile:
>>
>> basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0
>>
>> basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0
>> -d:XML_DEP
>>
>>
>> Build Profile:
>>
>> build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:CONFIGURATION_2_0
>>
>> build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:CONFIGURATION_2_0  -d:XML_DEP
>>
>>
>> Net 2.0 profile:
>>
>> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0
>>
>> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0  -d:XML_DEP
>> -d:SECURITY_DEP
>>
>> net_2_0: -d:NET_1_1 -d:NET_2_0  -d:CONFIGURATION_2_0  -d:XML_DEP
>> -d:SECURITY_DEP -d:CONFIGURATION_DEP
>>
>>
>> Net 4.0 profile:
>>
>> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:CONFIGURATION_2_0
>>
>> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP
>>
>> net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP  -d:CONFIGURATION_DEP
>>
>>
>> Net 4.5 profile:
>>
>> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:NET_4_5 -d:CONFIGURATION_2_0
>>
>> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP  -d:SECURITY_DEP
>>
>> net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0
>> -d:NET_4_5 -d:CONFIGURATION_2_0  -d:XML_DEP -d:SECURITY_DEP
>> -d:CONFIGURATION_DEP
>>
>>
>> Miguel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20140512/e03f71f1/attachment.html>