Pet, November 07, 2007 - 9:49 am UTC
Table is already having data which are aggregated by month.
It has 7 million records for a month , and for an entire year , it has 7*12 = 96 million records.
We need the output of 7 million rows with each monthly buckets ( 12 buckets ) as columns, filled with corresponding month data.
For ex.
code_combination_id, set_of_books_id,currency_code,period_year,jan_begin_balance_dr,jan_begin_balance_cr,jan_period_net_cr,jan_period_net_dr,feb_begin_balance... repeat for 12 months
1201212,1001,USD,2007,100.00, 200.00,300.00,400.00,.....
1201213,1001,USD,2007,200.00,204.00,400.00,500.00,....
November 07, 2007 - 6:03 pm UTC
Pet - you do see how contradictory this might appear to the rest of us???
Table is already having data which are aggregated by month.
It has 7 million records for a month , and for an entire year , it has 7*12 =
96 million records.
If the tale is already aggregated by month - well, ummm, hrrmmmm - where do these 7 million records per month come from??? or is your report 7 million lines LONG????
if so -
a) I question the usefulness of this thing - in fact I'll go as far as to call it a TPS report squared. (If you do not understand that reference, google office space tps report)
b) I don't see how you'd expect it to be really fast - but - it should not be horribly slow either - a nice big full scan and sort aggregate, given proper pga settings, this should be doable in a reasonable amount of time.
Don't do it (the MV) then ...
A writer, November 07, 2007 - 1:07 pm UTC
Tom was suggesting MVing the aggregation and then pivot the smaller result ... which makes sense.
If that's not applicable to you then it means there is no aggregation required ... because "Table is already having data which are aggregated by month".
If that's the case then to "make this faster" ... don't do it all.
How is your pivoted data being consumed? If it is by OLTP-type queries then pivot as part of those queries. You may want to define a View pivoting the data and ensuring the predicates from your queries gets pushed into the View; that is, your queries first filter data and then pivot the considerable smaller result.
Batch Process
Pet, November 07, 2007 - 1:21 pm UTC
Just to clarify.
It's a batch process to pivot the data and push it to a datamart. The process runs for 45 minutes to pivot 96 million records to 7 million records and insert into a staging table. So having a view may not help me.
We don't have a MV right now, I mentioned MV because I tried that option.
Any suggestion to improve the performance?
November 07, 2007 - 6:16 pm UTC
trace it and see where you spend the time. are you spending your time in temp, maybe larger pga settings. are you spending your time full scanning? maybe looking at making IO more efficient.
find where you spend your time and then... there you go.
maybe pivot no loger THE factor ...
A writer, November 07, 2007 - 4:14 pm UTC
I mentioned MV because I tried that option
You mentioned a fast refresh MV .. it seemed you wanted the pivoted data available in real-time.
But if it is a batch process, maybe the savings are to be found on the write rather than read side. Insert /*+ append */, if applicable to your circumstances?
There just aren't enough details about your process and its context to suggest anything precise really. Maybe others could ...
Cheers