[Mono-dev] How to improve Dataset Performance
Jonathan Chambers
joncham at gmail.com
Wed Dec 23 10:01:48 EST 2009
Hello,
Here is some summarized profiler output in case anyone is interested in
where the time is going. This is the above sample for 1000 rows instead of
10000, and with --profiler=default:time.
Thanks,
Jonathan
On Wed, Dec 9, 2009 at 9:20 AM, Oskar Berggren <oskar.berggren at gmail.com>wrote:
> 2009/12/9 APS <dev.malst at apsystems.it>:
> > For sure I can pass a test case, is better that I file a bug report? I
> think
> > that Mono team already knows where datasets are slower than MS.Net and
> maybe
> > can suggest me some "best practice".
> > Btw with the code provided below with MS.Net on a standard developer
> machine
> > and Mono on a bi-processor testing machine (with 2GB of ram):
> >
> > 1000 rows
> > MS.Net: 130ms
> > Mono 2.4.3: 1.2s
> > Mono 2.0.1: 3.5s
> >
> > 10.000 rows
> > MS.Net: less than a second
> > Mono 2.4.3: 114s
> > Mono 2.0.1: 361s
>
>
> So when the data size increase by a factor of 10, on ms.net the time
> taken also seem to increase by a factor of 10, but on mono by a factor
> of 100. Seems like an an O(N) versus an O(N²) algorithm.
>
> The code below contains a loop and you claim the Select part is taking
> the most time. If that is true, it seems to indicate that ms.net
> Select() for this case is O(1) but mono's O(N), with N being the
> number of rows in the table.
>
> /Oskar
>
>
> > I have tables with more than 50.000 rows...I don't know how much it takes
> to
> > perform the elaboration. RowFilter speed up only if applied on small
> > rowsets.
> > The problem is in the .Select method, removing those lines the
> elaboration
> > is quicker in Mono. Testing machines are not with the same resources but
> the
> > difference is too much to think to an hardware difference.
> > Here is the code...
> >
> > DataSet ds1 = new DataSet();
> > ds1.Tables.Add();
> > ds1.Tables[0].Columns.Add("COL1", typeof(string));
> > ds1.Tables[0].Columns.Add("COL2", typeof(int));
> > ds1.Tables[0].Columns.Add("COL3", typeof(bool));
> > for (int i = 0; i < 10000; i++) {
> > ds1.Tables[0].Rows.Add(new object[] { "test", i, true });
> > }
> > DataSet ds2 = new DataSet();
> > ds2.Tables.Add();
> > ds2.Tables[0].Columns.Add("COL1", typeof(string));
> > ds2.Tables[0].Columns.Add("COL2", typeof(int));
> > ds2.Tables[0].Columns.Add("COL3", typeof(bool));
> > for (int i = 0; i < 10000; i++) {
> > ds2.Tables[0].Rows.Add(new object[] { "test", i, true });
> > }
> > DataSet ds3 = new DataSet();
> > ds3.Tables.Add();
> > ds3.Tables[0].Columns.Add("COL1", typeof(string));
> > ds3.Tables[0].Columns.Add("COL2", typeof(int));
> > ds3.Tables[0].Columns.Add("COL3", typeof(bool));
> > for (int i = 0; i < 10000; i++) {
> > ds3.Tables[0].Rows.Add(new object[] { "test", i, true });
> > }
> > int idx=0;
> > foreach (DataRow dr in ds1.Tables[0].Rows) {
> > DataRow[] dr2 = ds2.Tables[0].Select("COL2=" + idx);
> > DataRow[] dr3 = ds3.Tables[0].Select("COL2=" + idx);
> >
> > idx++;
> > }
> >
> >
> >
> >
> > At 21.59 02/12/2009, Nagappan Alagappan wrote:
> >
> > Hello,
> >
> > Do you have a test case to reproduce this issue, which you could share
> with
> > the Mono team ?
> >
> > Thanks
> > Nagappan
> >
> > On Wed, Dec 2, 2009 at 4:05 AM, APS <dev.malst at apsystems.it > wrote:
> > Hi,
> >
> > I'm testing some old elaboration that cycles on a large datatable and
> > for every rows makes several selections on separated datatables.
> > This elaboration is really slow on mono and fast on MS.NET, let say
> > 400secs vs 4 secs.
> > It seems that last rows are slower that first rows.
> > I made some test on datasets scroll and I saw that mono performance
> > is better than MS.NET so the delay is due to inner selections.
> > I changed datatable.select in defaultview.sort and I gained 200secs,
> > still too slow.
> > Now I've identified some datatable.compute as the main delay source.
> > I'll try to change them using Linq but the question is: there's some
> > known performance problem in dataset heavy usage, there's some best
> > pratice that you can suggest to speed up elaborations?
> > Thanks in advance.
> >
> >
> > _______________________________________________
> > Mono-devel-list mailing list
> > Mono-devel-list at lists.ximian.com
> > http://lists.ximian.com/mailman/listinfo/mono-devel-list
> >
> >
> >
> >
> > --
> > Linux Desktop (GUI Application) Testing Project -
> > http://ldtp.freedesktop.org
> > http://nagappanal.blogspot.com
> >
> > --
> > Il messaggio e' stato analizzato alla ricerca di virus o
> > contenuti pericolosi da MailScanner, ed e'
> > risultato non infetto.
> >
> > _______________________________________________
> > Mono-devel-list mailing list
> > Mono-devel-list at lists.ximian.com
> > http://lists.ximian.com/mailman/listinfo/mono-devel-list
> >
> >
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ximian.com/pipermail/mono-devel-list/attachments/20091223/ec21beaf/attachment.html
-------------- next part --------------
Total time spent compiling 0 methods (sec): 0
Time(ms) Count P/call(ms) Method name
########################
102953.904 1 102953.904 .<Module>::runtime_invoke_void_object(object,intptr,intptr,intptr)
Callers (with count) that contribute at least for 1%:
########################
102953.902 1 102953.902 .Foo::Main(string[])
Callers (with count) that contribute at least for 1%:
1 100 % .<Module>::runtime_invoke_void_object(object,intptr,intptr,intptr)
########################
102256.739 2000 51.128 System.Data.DataTable::Select(string)
Callers (with count) that contribute at least for 1%:
2000 100 % .Foo::Main(string[])
########################
102254.160 2000 51.127 System.Data.DataTable::Select(string,string,DataViewRowState)
Callers (with count) that contribute at least for 1%:
2000 100 % System.Data.DataTable::Select(string)
########################
101706.476 2000 50.853 System.Data.DataTable::GetIndex(DataColumn[],ListSortDirection[],DataViewRowState,IExpression,bool,bool)
Callers (with count) that contribute at least for 1%:
2000 100 % System.Data.DataTable::Select(string,string,DataViewRowState)
########################
101668.556 2000 50.834 System.Data.Common.Index::.ctor(Key)
Callers (with count) that contribute at least for 1%:
2000 100 % System.Data.DataTable::GetIndex(DataColumn[],ListSortDirection[],DataViewRowState,IExpression,bool,bool)
########################
101666.608 2000 50.833 System.Data.Common.Index::Reset()
Callers (with count) that contribute at least for 1%:
2000 100 % System.Data.Common.Index::.ctor(Key)
########################
101664.276 2000 50.832 System.Data.Common.Index::RebuildIndex()
Callers (with count) that contribute at least for 1%:
2000 100 % System.Data.Common.Index::Reset()
########################
93049.697 2000000 0.047 System.Data.Common.Key::GetRecord(DataRow)
Callers (with count) that contribute at least for 1%:
2000000 100 % System.Data.Common.Index::RebuildIndex()
########################
88072.318 2000000 0.044 System.Data.Common.Key::CanContain(int)
Callers (with count) that contribute at least for 1%:
2000000 100 % System.Data.Common.Key::GetRecord(DataRow)
########################
86382.649 2000000 0.043 Mono.Data.SqlExpressions.Comparison::EvalBoolean(DataRow)
Callers (with count) that contribute at least for 1%:
2000000 100 % System.Data.Common.Key::CanContain(int)
########################
49904.807 2000000 0.025 Mono.Data.SqlExpressions.ColumnReference::Eval(DataRow)
Callers (with count) that contribute at least for 1%:
2000000 100 % Mono.Data.SqlExpressions.Comparison::EvalBoolean(DataRow)
########################
30554.465 2009000 0.015 System.Data.DataRow::get_Item(DataColumn)
Callers (with count) that contribute at least for 1%:
2000000 99 % Mono.Data.SqlExpressions.ColumnReference::Eval(DataRow)
########################
30162.481 2000000 0.015 Mono.Data.SqlExpressions.Comparison::Compare(IComparable,IComparable,bool)
Callers (with count) that contribute at least for 1%:
2000000 100 % Mono.Data.SqlExpressions.Comparison::EvalBoolean(DataRow)
########################
28866.007 2009000 0.014 System.Data.DataRow::get_Item(DataColumn,DataRowVersion)
Callers (with count) that contribute at least for 1%:
2009000 100 % System.Data.DataRow::get_Item(DataColumn)
########################
26899.479 2009000 0.013 System.Data.DataRow::get_Item(int,DataRowVersion)
Callers (with count) that contribute at least for 1%:
2009000 100 % System.Data.DataRow::get_Item(DataColumn,DataRowVersion)
########################
26536.273 2000000 0.013 System.Convert::ChangeType(object,Type)
Callers (with count) that contribute at least for 1%:
2000000 100 % Mono.Data.SqlExpressions.Comparison::Compare(IComparable,IComparable,bool)
########################
12895.972 2004014 0.006 System.Globalization.CultureInfo::get_CurrentCulture()
Callers (with count) that contribute at least for 1%:
2000000 99 % System.Convert::ChangeType(object,Type)
########################
11264.164 2009000 0.006 System.Data.DataColumn::get_Item(int)
Callers (with count) that contribute at least for 1%:
2009000 100 % System.Data.DataRow::get_Item(int,DataRowVersion)
########################
9607.594 2009000 0.005 System.Data.Common.DataContainer::get_Item(int)
Callers (with count) that contribute at least for 1%:
2009000 100 % System.Data.DataColumn::get_Item(int)
########################
7355.540 2000000 0.004 Mono.Data.SqlExpressions.ColumnReference::Unify(object)
Callers (with count) that contribute at least for 1%:
2000000 100 % Mono.Data.SqlExpressions.ColumnReference::Eval(DataRow)
########################
6577.252 2000000 0.003 Mono.Data.SqlExpressions.ColumnReference::GetReferencedRow(DataRow)
Callers (with count) that contribute at least for 1%:
2000000 100 % Mono.Data.SqlExpressions.ColumnReference::Eval(DataRow)
########################
6037.793 2000000 0.003 System.Convert::ToType(object,Type,IFormatProvider,bool)
Callers (with count) that contribute at least for 1%:
2000000 100 % System.Convert::ChangeType(object,Type)
########################
5753.030 2004015 0.003 System.Threading.Thread::get_CurrentCulture()
Callers (with count) that contribute at least for 1%:
2004014 99 % System.Globalization.CultureInfo::get_CurrentCulture()
########################
5499.963 4000000 0.001 Mono.Data.SqlExpressions.ColumnReference::GetColumn(DataRow)
Callers (with count) that contribute at least for 1%:
2000000 50 % Mono.Data.SqlExpressions.ColumnReference::GetReferencedRow(DataRow)
2000000 50 % Mono.Data.SqlExpressions.ColumnReference::Eval(DataRow)
More information about the Mono-devel-list
mailing list