Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Senthil.

Asked: August 30, 2001 - 9:08 am UTC

Last updated: March 17, 2005 - 2:03 pm UTC

Version: 8.1.5

Viewed 10K+ times! This question is

You Asked

Tom,

Is there anyway that we can calculate/predict space for index. i know there are few methods to estimate space for table growth. Like that, do we have any method or atleast workaround to estimate space for all indexes created for the table.

thanks,
senthil.

and Tom said...

Not with any accuracy. Here is why.

Not only do you need to know how big your index entries are (hard/impossible with varchars and numbers as they are varying length fields) but you need to know how your index is modified.

Lets say you only insert and update in this index. Well, your index could end up being many times larger then it would be if you rebuilt it. The reason is that if you update a value -- the position it was in is "still there" in the index structure. We can reuse it -- but it depends on how you use the data. For example:


ops$tkyte@ORA817DEV.US.ORACLE.COM> create table t ( x varchar2(1) );
Table created.

ops$tkyte@ORA817DEV.US.ORACLE.COM> create index t_idx on t(x);
Index created.

ops$tkyte@ORA817DEV.US.ORACLE.COM> insert into t values ( 'a' );
ops$tkyte@ORA817DEV.US.ORACLE.COM> insert into t values ( 'b' );
ops$tkyte@ORA817DEV.US.ORACLE.COM> insert into t values ( 'd' );
ops$tkyte@ORA817DEV.US.ORACLE.COM> commit;
Commit complete.

ops$tkyte@ORA817DEV.US.ORACLE.COM> update t set x = 'e' where x = 'b';
1 row updated.

ops$tkyte@ORA817DEV.US.ORACLE.COM> commit;
Commit complete.

ops$tkyte@ORA817DEV.US.ORACLE.COM> analyze index t_idx validate structure;
Index analyzed.

ops$tkyte@ORA817DEV.US.ORACLE.COM> select lf_rows, del_lf_rows from index_stats;

LF_ROWS DEL_LF_ROWS
---------- -----------
4 1

that shows that there are 4 entries in the index -- but one of them is "deleted". There is a hole between 'a' and 'd' in the index. If we put in some data that fits there (note: it does not have to be the SAME data -- just has to fall between 'a' and 'd'):

ops$tkyte@ORA817DEV.US.ORACLE.COM> insert into t values ( 'c' );
1 row created.

ops$tkyte@ORA817DEV.US.ORACLE.COM> analyze index t_idx validate structure;
Index analyzed.

ops$tkyte@ORA817DEV.US.ORACLE.COM> select lf_rows, del_lf_rows from index_stats;

LF_ROWS DEL_LF_ROWS
---------- -----------
4 0

we'll reuse that.

So, estimating on paper how an index will grow can be extremely tricky. You through some deletes in there with the updates -- the fact that an index block will be reclaimed when ALL index entries on it are marked as deleted (space is reused) but won't be if one entry remains, the fact that it is a complex data structure and estimation before the fact is hard.


You can estimate the initial size of the index (search for sizing on my site) using the same technique I use for sizing a table (load some representative data, index it and multiply).

Estimating growth is something you can do after you either really use the index or simulate how you will use the index (benchmark).

Rating

  (5 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, August 30, 2001 - 9:46 am UTC

That was a nice explanation...thanks.

necessitates index rebuilds

Mark, August 30, 2001 - 1:06 pm UTC

so i am surmising that for an increasing sequential primary key, deleted index space will never be reused, and will always be growing...thus necessitating an index rebuild at appropriate times...
BTW: I enjoy your well run site and the competent, documented knowledge that you disseminate.

Tom Kyte
August 30, 2001 - 4:06 pm UTC

No, not at all. Only in the worst of degenerate cases.

If you insert 1,2,3,4,5,6........ <big number>

and then delete 1,2,3,4,5,6.......<some number>

we'll reclaim all of the space. If you delete every OTHER number -- we'll end up with lots of holes that will not be reused (by the nature of the sequence).

I find in general however that most applications tend to "archive" old data and would tend to delete all of the old data. In this case, the index space would be reused.

It is only the application that leaves some of the old data behind forever that might necessitate a rebuild over a long period of time.

mark, August 30, 2001 - 4:19 pm UTC

thanks tom, i think i have the hole picture...

Tom Kyte
August 30, 2001 - 7:28 pm UTC

No pun intended with "hole" in that comment was there ;)

Size of index

A reader, January 07, 2005 - 7:33 pm UTC

I have a table with 100 columns most of them NOT NULL 1-byte indicator/flag columns. PK is 12 bytes. Contains 50million rows. Size of the table is 900MB.

But the size of an index on the table containing 6 frequently used columns (2 varchar, 3 date and 3 numbers) is 1.1GB!

This is just after the index is created.

Why would the size of the index be more than the size of the table?

If I have 2-3 of these indexes, the indexes would use 3-4 times more space than the table! Seems like a waste?

Thanks

Tom Kyte
January 08, 2005 - 7:14 pm UTC

a waste


hmmm -- lets see:

select * from t where pk = :x;

on a 

a) non indexed, 900meg table versus
b) an indexed, 900meg table


which might burn up the disks -- verus getting the answer in a nanosecond??


you see, you have to define "waste".  indexes are there to generally "make things go faster", not to be "smaller than the table"


The index has the rowid in it (the table, it does not).  rowids are sizeable ...

the table -- don't know how you loaded it or anything, but perhaps it was loaded "packed" (pctfree=0) maybe even compressed.  

But the index was created "using the defaults" and has 10% free.


but actually, i don't know whats up because I created a 50,000,000 row table:

ops$tkyte@ORA9IR2> declare
  2          l_stmt long := 'create table t ( pk char(12), d1 date default sysdate, d2 date default sysdate, d3 date default sysdate, n1 number, n2 number, n3 number';
  3  begin
  4          for i in 1 .. 100
  5          loop
  6                  l_stmt := l_stmt || ', c' || i || ' char(1) default ''x'' not null ';
  7          end loop;
  8          l_stmt := l_stmt || ' )';
  9          execute immediate l_stmt;
 10  end;
 11  /
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> insert /*+ APPEND */ into t (pk,n1,n2,n3)
  2  select rownum, rownum,rownum,rownum from big_table.big_table where rownum <= 50000000;
 
50000000 rows created.

ops$tkyte@ORA9IR2>  select segment_name, segment_type, bytes/1024/1024 meg, bytes/1024/1024/1024 gig
  2  from user_segments;
 
SEGMENT_NAME                   SEGMENT_TYPE              MEG        GIG
------------------------------ ------------------ ---------- ----------
T                              TABLE              14491.0156 14.1513824



ops$tkyte@ORA9IR2> create index t_idx on t(pk,d1,d2,d3,n1,n2,n3);
Index created.

ops$tkyte@ORA9IR2>  select segment_name, segment_type, bytes/1024/1024 meg, bytes/1024/1024/1024 gig
  2  from user_segments;


SEGMENT_NAME                   SEGMENT_TYPE              MEG        GIG
------------------------------ ------------------ ---------- ----------
T                              TABLE              14491.0156 14.1513824
T_IDX                          INDEX                    3621 3.53613281
 


ops$tkyte@ORA9IR2> exec show_space( 'T' );
Free Blocks.............................               0
Total Blocks............................       1,854,848
Total Bytes.............................  15,194,914,816
Total MBytes............................          14,491
Unused Blocks...........................           4,317
Unused Bytes............................      35,364,864
Last Used Ext FileId....................               8
Last Used Ext BlockId...................       1,847,177
Last Used Block.........................           3,875
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec show_space( 'T_IDX', p_type => 'INDEX' );
Free Blocks.............................               0
Total Blocks............................         463,488
Total Bytes.............................   3,796,893,696
Total MBytes............................           3,621
Unused Blocks...........................             122
Unused Bytes............................         999,424
Last Used Ext FileId....................               8
Last Used Ext BlockId...................       2,310,153
Last Used Block.........................           8,070
 
PL/SQL procedure successfully completed.



<b>so, i guess we are at the point where "things just don't add up".  I would expect 50 million rows to take more than 900 MB, especially given your description since:


 
ops$tkyte@ORA9IR2> select 900*1024*1024/50000000 from dual;
 
900*1024*1024/50000000
----------------------
             18.874368


900 meg -- mulitplied out to bytes and divided by 50 M, works out to about 19 bytes max per row in a perfect perfect world....



And if you got 6 columns into that index as described, in 1.1 gig -- well -- what can I say besides "well done"


So -- can you re-check your numbers?</b>



 

Where I can find the show_space procedure ?

Philippe LYON, March 17, 2005 - 1:11 pm UTC

In your example, you uses a procedure show_space.
I don't find this procedure in my database (9.2.0.5.0).

Is it a specific procedure that you have write or a standard procedure that should be installed

Tom Kyte
March 17, 2005 - 2:03 pm UTC