Large DB Block Buffers
Krish, January 21, 2002 - 12:43 pm UTC
The question of large db block buffers is indeed intriguing.
I would like to mention here a case where I am stuck at:
I have a database with 45000 db blocks (8k each).
The 2 tables have 5M rows each.
SELECT /*+ RULE */ count(*)
from kentity e, kbenefit_entity be
WHERE last_name like :l and first_name like :f
and be.entity_id = e.entity_id ;
SELECT STATEMENT Optimizer=HINT: RULE
SORT (AGGREGATE)
NESTED LOOPS
INDEX (RANGE SCAN) OF KENTITY_FULLNAME_IDX (NON-UNIQUE)
INDEX (RANGE SCAN) OF KBENENT_ENTID_ROLEID_BENID_IDX (NON-UNIQUE)
The entity table returns 1100 rows (<20 index blocks) for the criteria and the query (in almost single user mode) as a whole returns 1200 (<20 index blocks) rows. But it takes 40 seconds to return the result every time (except when cached). Each index segment has <10 extents and data (if relevant) is "colocated". And we are using one of the fastest disk technologies, per our experts.
Session_event shows no waits.
The delay is inexplicable and I wonder if it has to do anything with the large db block buffers value.
Could you pl throw light on this issue?
Thanks
KU
January 21, 2002 - 12:52 pm UTC
tkprof it. see what that says. Since it runs "fast" when cached -- i would have to say that the large buffer cache is not having a negative impact but rather that it must be scanning a TON of entries in the index (doesn't matter how many rows it RETURNS -- it matters how many rows it had to INSPECT for the like).
I don't know what the indexes are one, but say its on last_name, first_name and you supply '%x%', it'll read EVERY index entry and if it has an x in it -- it'll then do the index probe into the other table. That could be a TON of singleton (slow) IO's.
More, Please
Harrison Picot, January 21, 2002 - 10:03 pm UTC
Hi Tom:
"Plump but comfortable" sounds good, but I didn't understand the part before that:
You said: " reorg could actually slow down an OLTP system, all of those freshly rebuilt, compact indexes must now get broken down again, split the blocks -- get back to where they were -- plump but comfortable."
Why must the indexes get broken down? ("in the book" is an ok answer, if true).
January 21, 2002 - 10:31 pm UTC
An index is a data structure (a pretty complex one). Many times -- depending on what you do, they end up with "extra space" in them. If you rebuild them (as you did) they get smaller -- but lo and behold over time -- they grow again. This "extra space" is really needed -- sometimes its used, sometimes not used.
When you rebuild (compact) the index, all of this space goes away. The deleted entries (caused by deletes or updates) are removed. Well, we NEED those slots -- we are going to use them tommorow! The next day we won't but the day after we need them again.
Well, we'll have to split blocks - move data around -- grow the structure to get this "extra space" back again. In a couple of days or weeks -- the index will be exactly the same size it was before -- with all of that extra space. Getting that space back added overhead.
So, any index on a column that is updated frequently could cause this to happen. As you update it -- you'll leave a "deleted" entry where the old value was and use or make space for an entry. So, say you
update t set x = 'M' where id = 55; (x was 'C' before the update)
well, that leaves an empty slot on the block where that 'C' was before the update and gets space where the 'M' was. In a freshly compacted index -- we might have had to split the block where the update ended up (the index block the M goes on).
Now, the next guy that does:
update t set x = 'C' where id = 66;
will be able to use the space we left behind from the first 'C' that is not there anymore.
The problem after a compact is there are no more "deleted" entries, no place to go and we'll have to split the index until there is.
Many indexes have a steady state they'll get into over time, regardless of how often you "reorg" them and you pay the price of getting into this steady state over and over.
Large buffer cache
terry, January 22, 2002 - 1:39 am UTC
Hi,
Thank you for helping on this issue. I would like to say that we have lost some indexes after reorg, but we have made a list to compare and we built them straight the way.
Firstly, please confirm whether it is not harm on larger buffer cache as long as we are not 'Page in and page out' on unix memory.
Regarding to the index, you said that 'the indexes get broken down...then, go back to the original steady state'.
This does confuse me...as many oralce books show the necessary of index rebuild.
If it is not necessary to rebuild index in most case, what kind of situation we can gain a signicant benefit from rebuilding index? Or, if not necessary in any case, I am wondering why Oracle provide such function and keep improve its functionality from, say, oracle 7 to oracle 8i online index rebuild.
Thank you in advance.
January 22, 2002 - 7:11 am UTC
A large buffer cache is in the eye of the beholder. Yes, an excessively large one can cause issues -- but you were happy with the prior performance....
Many oracle books are not 100% accurate or only tell half of the story. Me, I've never actually rebuilt an index due to "fragmentation". Get Jonathan Lewis's book -- practical Oracle8i, building efficient databases. Lots of good common sense in there.
We provide that which customers demand. The rebuild online is GREAT for moving indexes from tablespace to tablespace, changing storage parameters, or after a mass load (eg: set the indexes UNUSABLE, load, REBUILD the indexes -- that way you NEVER EVER lose them ;)
So you see -- rebuild has many other alternate uses.
I would ask you -- now that you have spent UNTOLD hours, incurred serious performance issues, tore your hair out fixing the problem -- was it worth it?
Did you get measurable performance increases?
Was it an efficient use of your time?
Think about those two questions before you do your next reorg.
One more thing to say
terry, January 22, 2002 - 2:33 am UTC
Hi Tom,
Regarding to your exmaple above "
As you update it -- you'll leave a "deleted" entry where the old value was
and use or make space for an entry. So, say you
update t set x = 'M' where id = 55; (x was 'C' before the update)
well, that leaves an empty slot on the block where that 'C' was before the
update and gets space where the 'M' was. In a freshly compacted index -- we
might have had to split the block where the update ended up (the index block the
M goes on).
Now, the next guy that does:
update t set x = 'C' where id = 66;
will be able to use the space we left behind from the first 'C' that is not
there anymore.
The problem after a compact is there are no more "deleted" entries, no place to
go and we'll have to split the index until there is.
"
I agree with you but it seems that you are based on the assumption that the deleted data will update back again in the future. I think that it is not necessary to reuild index in this circumstance.
However, it is not the case, at least in my site, the deleted index entry will not come back. As time goes, more and more delelted slots come out and index is less and less balancing. At this case, the unbalance index will require more i/o, so whether we need to rebuild index or not? Please explain for me. Thank you so much.
January 22, 2002 - 7:15 am UTC
Well, it depends on how you delete the data. Do your indexes grow infinitely large over time? If not -- we must be doing some space reuse... (the indexes tend to stay pretty balanced).
If the indexes grow infinitely larger over time -- they are a candidate for rebuilds. If not, -- make sure you MEASURE response time BEFORE and AFTER a rebuild to see if you GAINED anything. Write a program that randomly selectes via the index a couple of thousand/million times, use sql_trace and tkprof. rebuild the index and do it again. compare. If all you got back was a little space (which will get allocated AGAIN very soon) don't rebuild. If the performance goes up measurably, consider rebuilding at some point in the future.
Yes, Sir ! I will study "index" further
Terry, January 22, 2002 - 10:27 am UTC
Hi Tom,
Thank you so much. You do give me a lot of clues...I will pay more attention on it, since "index" is extremely important part of the RDBMS. Great!
Performance gains (revisited)
Harrison Picot, January 24, 2002 - 2:44 am UTC
Hi Tom:
When I saw your comments on the lack of performance gains from rebuilding indexes, I had the feeling that in the dim past some smart Oracle people had differed, and finally found the paper, one by Cary Millsap in 1992. Actually I remembered his saying the performance gains that come from raw disks are no greater (and no more permanent) than those from moving a database, which "reduces row chaining to its theoretical minimum and also nicely repacks and balances every index." I remembered the indexes, but what I should have remembered was the next sentence: "The same five- to twenty-percent performance improvement can be gained by moving a badly chained table from raw disks to a Unix file system". The present point being that sometimes we pick one thing that has changed and associate better performance to to it, but have not held other things constant. Your plan for testing the indexes makes a lot of sense (and my going back and reading what I think I remember doesn't hurt either). I am always impressed by how throughly you test and document things, and how little of that some of the rest of us do. Maybe that is something to work on.
thanks for the help
Harrison
Buffer cache size is too large?
A reader, August 09, 2002 - 6:31 pm UTC
I agree with you table re-org is utter waste of time but what if I run a select statement against an index which has too many fragements ? Don't the "selects" take more time?
August 10, 2002 - 9:24 am UTC
define "fragments"
If you mean "over time, due to updates and deletes, my index is 50% 'whitespace' and I think I should rebuild it" -- you might be very wrong (or you might be right).
Indexes, like people, have a certain weight they like to be at. No matter how many times I go on a diet, lose weight -- I end up back at the same weight (well, maybe a little heavier even). Same with an index.
This quote:
A reorg could actually slow down an OLTP system, all of those freshly rebuilt,
compact indexes must now get broken down again, split the blocks -- get back to
where they were -- plump but comfortable.
from above typifies what I mean. We put the index on a diet (rebuilt it) but it goes right back. Why? because your updates and deletes CONTINUE on this data and the index moves all around. An index like this might just want to be 50% whitespace forever and no amount of rebuilding will change that. You can put it on a diet (rebuild) as often as you like -- it'll just consume resources getting "fat" again.
A reader, May 19, 2003 - 8:48 am UTC
Hi Tom,
We are using siebel crm application.Siebel DBA visited out site last week and give one suggestion about buffer cache that surprise me.
He(DBA) came here to tune our bad sql statement generated by siebel.
We have 150 Concurrent users and our block buffer size is 22000 and out block size is 8k.
Siebel DBA recommanded to dubble block buffer(ie 44000).
According to him we need to allocate 5 to 10 MB buffer cache per user.
I never heard buffer cache allocation like this.
He also suggested to make init.ora parameter pre_page_sga to TRUE
Please advide.
Thanks,
May 19, 2003 - 10:37 am UTC
"Siebel DBA" is what is known as an oxymoron.
Ask them why they propose such things. You do not size a buffer cache by numbers of users, that is silly.
Pre_Page_Sga is equally as doubtful.
In order to tune a system, you must diagnose what people are waiting on. do you have high IO waits? Is it likely that you are aging blocks out frequently causing to you read and re-read the same blocks? if not, increasing the buffer cache will just consume more ram (could make you slower) and pre-paging will just make you take longer to startup.