Skip to Main Content

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Igor.

Asked: May 24, 2000 - 1:03 pm UTC

Last updated: April 13, 2016 - 2:31 pm UTC

Version: Oracle 8.1.5

Viewed 10K+ times! This question is

You Asked

My question is - how to reuse space in Index Organized Table (IOT) after deleting substantial amount of rows.
When I deal with regular table, after deleting a lot of rows I rebuild the indexes in order not to leave gaps in the index blocks, because we know that this empty space in the indexes will not be reused (unlike, when it is reused in table blocks after PCTUSED threshold met in the block).
So, what could be done with IOT in order to prevent constant growth, even after a lot of rows being deleted?

TIA

and Tom said...

The answer to this is pretty neat actually -- Oracle8i release 8.1 allows you to do 2 new things that make this interesting.

o an online index rebuild
o 'move' a table

Since an IOT is just really an index.... We can actually rebuild the index by moving the table and do this in an online fashion (eg: people are modifying this data while we are rebuilding...)

Here is an example:

ops$tkyte@dev8i> create table demo_iot
2 ( object_id int primary key,
3 oname varchar2(30),
4 owner varchar2(30),
5 status varchar2(30) )
6 organization index;

Table created.

ops$tkyte@dev8i> insert into demo_iot
2 select object_id, object_name, owner, status
3 from all_objects
4 /

22525 rows created.

ops$tkyte@dev8i> column index_name new_value iname
ops$tkyte@dev8i> select index_name
2 from user_indexes
3 where table_name = 'DEMO_IOT'
4 /

INDEX_NAME
------------------------------
SYS_IOT_TOP_87241

ops$tkyte@dev8i> analyze index &iname validate structure;
old 1: analyze index &iname validate structure
new 1: analyze index SYS_IOT_TOP_87241 validate structure
Index analyzed.

ops$tkyte@dev8i> select height, blocks, del_lf_rows from index_stats;

HEIGHT BLOCKS DEL_LF_ROWS
---------- ---------- -----------
2 192 0


Now we will just delete half of the rows in the table more or less. Trying for "every other row"

ops$tkyte@dev8i> delete from demo_iot where mod(object_id,2) = 1;
11270 rows deleted.

ops$tkyte@dev8i> analyze index &iname validate structure;
old 1: analyze index &iname validate structure
new 1: analyze index SYS_IOT_TOP_87241 validate structure

Index analyzed.

ops$tkyte@dev8i> select height, blocks, del_lf_rows from index_stats;

HEIGHT BLOCKS DEL_LF_ROWS
---------- ---------- -----------
2 192 11270

So, we have lots of deleted rows in the index (none of the blocks got totally empty). How can we "clean" that up...



ops$tkyte@dev8i> alter table demo_iot move online;
Index altered.


ops$tkyte@dev8i> analyze index &iname validate structure;
old 1: analyze index &iname validate structure
new 1: analyze index SYS_IOT_TOP_87241 validate structure

Index analyzed.

ops$tkyte@dev8i> select height, blocks, del_lf_rows from index_stats;

HEIGHT BLOCKS DEL_LF_ROWS
---------- ---------- -----------
2 128 0

There we go -- its all cleaned up now. As a test, you should have other sessions open when we do the alter table move -- just to see that the table is online and available for all DML




Rating

  (105 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, August 23, 2001 - 1:23 pm UTC

you are talking two method
one is
an online index rebuild

i tried this one but it didn't work.

alter index SYS_IOT_TOP_87241 rebuild
*
ERROR at line 1:
ORA-28650: Primary index on an IOT cannot be rebuilt


Tom Kyte
August 23, 2001 - 3:12 pm UTC

See
</code> http://docs.oracle.com/docs/cd/A81042_01/DOC/server.816/a76956/tables.htm#1913 <code>

It is done via:

ALTER TABLE docindex MOVE ONLINE;



Kimberly, May 10, 2002 - 10:14 am UTC

I have been searching for this answer for 2 days. I should have come here first. This is exactly what I was looking for.

Deleting rows from IOTs with foreign keys

Brigitte, May 16, 2002 - 8:54 am UTC

Hallo Tom,

your "answer and reply" is very useful to learn many details!

I had changed some tables to IOT some month before.
Now I had problems with deleting more than one rows in the IOTs. The IOTs have foreign keys with on delete cascade to one table (TestDok).
I don't know whether the problem is new or not because normally rows don't be deleted in TestDok, they were signed. 

SQL> r
  1  delete from TESTDok
  2* where objektid = '38060'
delete from TESTDok
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [25012], [8], [10], [], [], [], [], []


I saw in the trc-file the problem occurs when oracle must delete the rows from the child table.


SQL> select * from TESTDokH where objektid = '38060' or superobjektid = '38060';

OBJEKTID                         SUPEROBJEKTID
-------------------------------- --------------------------------
27654                            38060
28830                            38060

2 rows selected.

SQL> delete from TESTDokH where superobjektid = '38060';
delete from TESTDokH where superobjektid = '38060'
                    *
ERROR at line 1:
ORA-00600: internal error code, arguments: [25012], [8], [10], [], [], [], [], []

The tables are now :

/* TESTDok */
CREATE TABLE TESTDok
(
    OBJEKTID        VARCHAR2(32) 
            CONSTRAINT PK_TESTDok PRIMARY KEY
            using index tablespace user_i,
    MANDANT              VARCHAR2(30) NOT NULL,
    NOTESID         VARCHAR2(32) 
            CONSTRAINT UK_TESTDok UNIQUE 
            using index tablespace user_i,
    DELETED             VARCHAR2(1) DEFAULT '0',
. . .
    AID_MZ         VARCHAR2(32),
    AID_EF         VARCHAR2(32),
CONSTRAINT FK_TESTDok1 FOREIGN KEY (AID_MZ)
        REFERENCES TESTAR (FE_ID)
        ON DELETE SET NULL,
CONSTRAINT FK_TESTDok2 FOREIGN KEY (AID_EF)
        REFERENCES TESTAR (FE_ID)
        ON DELETE SET NULL
) tablespace user_tg pctfree 8 pctused 86
/

create bitmap index BM_TESTDok_mandant ON TESTDok (mandant) 
tablespace user_i
/

/* TESTDokH IOT*/
CREATE TABLE TESTDokH
(
    OBJEKTID        VARCHAR2(32),
    SUPEROBJEKTID    VARCHAR2(32),
  CONSTRAINT IOTPK_TESTDokHiera PRIMARY KEY (OBJEKTID, SUPEROBJEKTID),
  CONSTRAINT FKTESTDokHiera FOREIGN KEY (OBJEKTID)
        REFERENCES TESTDOK (OBJEKTID)
        ON DELETE CASCADE,  
  CONSTRAINT FKTESTDokHiera2 FOREIGN KEY (SUPEROBJEKTID)
        REFERENCES TESTDOK (OBJEKTID)
        ON DELETE CASCADE
)ORGANIZATION INDEX PCTTHRESHOLD 5
tablespace user_tg pctfree 10
/

What is wrong?

Thanks
 
 

Tom Kyte
May 16, 2002 - 1:44 pm UTC

as with all internal errors, you should contact support. Many times, a patch exists for this problem.



IOT pk modification.

Alvin, July 15, 2003 - 12:57 am UTC

If i have say an Index organized table 'T' who has columns a,b,c,d and declared a,b to be the primary key.

What i really needed columns a,b,c to be the primary key. Is there an alter table command that allows me to alter IOT and add another column in the composite primary key ?

Many thanks.



Tom Kyte
July 15, 2003 - 9:18 am UTC

in Oracle9i, you can use an online rebuild.

In 8i, you'll have to use a Create table as select, drop old, rename new.

prefix...

Alvin, July 15, 2003 - 1:05 am UTC

If i have say an Index organized table 'T' who has columns a,b,c,d

this time i've a,b,c as primary key in that order also.

And i found out that majority of my queries has a,c in the predicate will the composite key of a,b,c be useful ? say compared to composite key a,c ?

Tom Kyte
July 15, 2003 - 9:20 am UTC

arg, a primary key is a primary key -- that is what decides what is the key.

if the majority of your queries access the table via a,c -- then the primary key should be on a,c,b

follow up question..

Alvin, July 15, 2003 - 9:49 pm UTC

Half of my queries are a,c and the other half is a,b

Will making a,b,c as composite primary key work for both ? a,b has a little bit more queries than a,c.

I tried in our developmental server the effects of using an IOT as compared to the heap organized table and the difference via tkprof was enough to convince me to implement it on our production server.

It's a 24x7 system and downtime should be kept to a minimum.

How will i freeze the said table ? is there a 'alter table <tablename> read only command can't seem to find it on 8i. Or will the CTAS freeze the source table ?

I was planning to do the following on our off-peak times.

1. rename the orgininal table to <temp>- so no further changes would be introduced
2. create table <original name> as select from <temp>

I did check for other object dependencies and found none. The table's pk are not enforced as fk on other tables and the said table doesn't contain any columns defined as fk's.

Tom Kyte
July 15, 2003 - 10:10 pm UTC

in one session

lock table t in exclusive mode;

in another, copy it. 

then in the fist, drop it and rename the other.


in 8i, if you have queries of the type a,b and a,c, you'll probably want to index

a,b,c
c


the reason only "c" is because all indexes on an IOT have the primary keys in them already.  consider:

ops$tkyte@ORA817DEV> create table t ( a int, b int, c int, d char(20), primary key (a,b,c) ) organization index;

Table created.

ops$tkyte@ORA817DEV> insert into t
  2  select rownum, -rownum, rownum*2, 'x'
  3  from all_objects where rownum <= 5000;

5000 rows created.

ops$tkyte@ORA817DEV>
ops$tkyte@ORA817DEV> create index t_idx on t(c);

Index created.

ops$tkyte@ORA817DEV> analyze table t compute statistics for table for all indexes for all indexed columns;

Table analyzed.

ops$tkyte@ORA817DEV>
ops$tkyte@ORA817DEV> set autotrace traceonly explain
ops$tkyte@ORA817DEV> select a,c from t where c = 44 and a = 22;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=1 Bytes=6)
   1    0   INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=1 Card=1 Bytes=6)



ops$tkyte@ORA817DEV> set autotrace off


See how it was able to use the index in its entirety to answer the question?  it did not need to goto the table to get A, it was already there. 

Index rebuild required?

Tony, July 16, 2003 - 1:28 am UTC

why should we rebuild indexes?. You're against index rebuild. Refer your answer

</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:6601312252730, <code>

Your comment please.

Tom Kyte
July 16, 2003 - 9:21 am UTC

interesting - one might think you would have just posted this on that question...

anyway

I'm not AGAINST rebuilding ever.

I'm against the scheduled rebuilding of indexes under the premise that "well, everyone knows you should".

I'm against doing things that have not been shown to

a) make the system better
b) not cause adverse effects


bitmap indexes -- may need to be rebuilt after some amount of DML

my text index on asktom -- i rebuild that from time to time after lots of DML (it in fact is very similar to a bitmap index)

b*tree indexes, hardly ever (hint read about COALESCE, most of the benefit, fraction of the work)


IOT OVERFLOW

michael, July 16, 2003 - 11:40 am UTC

Tom,

Thank you for the great services you provide.

I created IOT as such for use as a drop down list

create table category
(name VARCHAR2(50)
CONSTRAINT category$name$nn NOT NULL
,display_yn VARCHAR2(1)
CONSTRAINT category$display$nn NOT NULL
,CONSTRAINT category$pk
PRIMARY KEY (name)
)
organization index tablespace indx
including display_yn overflow;

alter table category add constraint category$display$ck01
check(display_yn in ('Y','N'));

create unique index category$name$x on category(UPPER(name))
tablespace indx;

My questions are:

1. Basically, there are two indexes for the table on the same column. Is this efficient and how can I get around with using only one index?

NEREUS@mdl1> select index_name, index_type from dba_indexes
2 where owner='NEREUS' and index_name like 'CATEGORY%';

INDEX_NAME INDEX_TYPE
------------------------------ ---------------------------
CATEGORY$NAME$X FUNCTION-BASED NORMAL
CATEGORY$PK IOT - TOP

2. How do I name the OVERFLOW table?

NEREUS@mdl1> select table_name, iot_name from dba_tables
2 where owner='NEREUS'
3 and table_name='CATEGORY'
4 or IOT_NAME='CATEGORY';

TABLE_NAME IOT_NAME
------------------------------ -----------------------------
CATEGORY
SYS_IOT_OVER_27251 CATEGORY

Any recommendation you can make for performance design as well would be great.


Tom Kyte
July 16, 2003 - 1:22 pm UTC

1) well, there is only ONE index that I see -- the fbi you created. the other "index" is the table really.

2) you do not need, want, nor desire the overflow in this case. In fact, I would say if you need an overflow, chances are you might not really want an IOT (there are of course exceptions to that).


If you always search by upper, you might consider

create table category
(upper_name varchar2(50) constraint cat$pk primary key,
name VARCHAR2(50) CONSTRAINT category$name$nn NOT NULL,
display_yn VARCHAR2(1) CONSTRAINT category$display$nn NOT NULL
)
organization index;


and just inserting the upper name, name and display_yn

that way you have

o just the IOT
o no ambiguities -- eg right now, you could have 'hello' and 'Hello' in your table as a "primary key" value




How oracle locates a row in IOT?

Samuel Bernard, August 23, 2003 - 1:21 pm UTC

Tom, does oracle use PK value to locate the row fast in IOT? it does not store rowid as conventional btree index does. Could you explain the underlying mechanism? Thanks.

Tom Kyte
August 23, 2003 - 6:22 pm UTC

the "rowid" is a universal rowid -- not a real rowid rowid.

as such, it actually incorporates the primary key value.  Consider:

ops$tkyte@ORA920> create table t ( x varchar2(50) primary key ) organization index;

Table created.

ops$tkyte@ORA920>
ops$tkyte@ORA920> insert into t values ( 'xxx' );

1 row created.

ops$tkyte@ORA920> insert into t values ( rpad('x',50,'x') );

1 row created.

ops$tkyte@ORA920> select x, rowid from t;

X                                                  ROWID
-------------------------------------------------- -------------------------------------------------------------------------------
xxx                                                *BAGAAjoDeHh4/g
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx *BAGAAjoyeHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHj+




see how big a rowid can be... 

Great Info

Samuel Bernard, August 23, 2003 - 9:20 pm UTC

Tom, if I create an index on a column other than the primary key columns in IOT, what does the index just created on the column store? Does it store what you have shown in the demo? I don't know much about using IOT so I would appreciate your time in explaining this concept. Thanks much.

Tom Kyte
August 23, 2003 - 9:35 pm UTC

an index on an IOT always includes the primary key.

index key contains .....

David, August 23, 2003 - 11:06 pm UTC

Tom, does it mean that the index entry for the indexed column (secondary index) in IOT will have key value, primary key value, rowid (logical?)

thanks.

Tom Kyte
August 24, 2003 - 7:36 am UTC

yes, note how in the following example, only the index T_IDX is accessed -- never the table.  even tho the index is on Y and Y alone

ops$tkyte@ORA920> create table t ( x int primary key, y int ) organization index;

Table created.

ops$tkyte@ORA920>
ops$tkyte@ORA920> create index t_idx on t(y);

Index created.

ops$tkyte@ORA920>
ops$tkyte@ORA920> exec dbms_stats.set_table_stats( user, 'T', numrows => 100000 );

PL/SQL procedure successfully completed.

ops$tkyte@ORA920>
ops$tkyte@ORA920> set autotrace traceonly explain
ops$tkyte@ORA920>
ops$tkyte@ORA920> select x from t where y = 1;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1000 Bytes=26000)
   1    0   INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=2 Card=1000 Bytes=26000)



ops$tkyte@ORA920> select y from t where y = 1;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1000 Bytes=13000)
   1    0   INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=2 Card=1000 Bytes=13000)



ops$tkyte@ORA920> select rowid from t where y = 1;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1000 Bytes=30000)
   1    0   INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=2 Card=1000 Bytes=30000)



ops$tkyte@ORA920>
ops$tkyte@ORA920> set autotrace off
ops$tkyte@ORA920> 

what does set do?

Reader, August 24, 2003 - 6:35 pm UTC

ops$tkyte@ORA920> exec dbms_stats.set_table_stats( user, 'T', numrows => 100000
);

Tom, what does set_table_stats do? Thanks.


Tom Kyte
August 24, 2003 - 9:06 pm UTC

sets the stats. i told the optimizer "please trust me, there are 100,000 rows in this table"

set stats

Reader, August 24, 2003 - 9:35 pm UTC

Is it used to study the execution plan for certain number of rows even though the table is empty? Thanks.

Tom Kyte
August 25, 2003 - 6:10 am UTC

it can be.

it can be used when you know the stats already (instead of analyzing to save time)


Size for IOT

A reader, October 16, 2003 - 11:29 am UTC

Hi Tom,

I am considering converting some "big and lean" tables of more than 10000 rows to IOT and I found it helps the performance a lot.

In your opinion, do you think it makes sense to convert tables of less than 10000 rows to IOT? Will it give me much benefit, compared to using heap table with indexes?


Thanks.

Tom Kyte
October 16, 2003 - 5:01 pm UTC

yes, definitely. you could end up doing 1/3 the logical IO on a lookup. If you do that lookup lots -- 100 rows, 10,000 rows -- doesn't really matter.

COALESCE vs MOVE ONLINE

Jan, October 30, 2003 - 4:59 am UTC

1)When to use ALTER iot_table COALESCE and when ALTER iot_table MOVE ONLINE? Is there some circumstance when using of COALESCE is reasonable?

2) If there is an index on iot_table and we do MOVE ONLINE do we need to rebuild also that index?


Thank you, Jan

Tom Kyte
October 30, 2003 - 7:17 am UTC

1) use coalesce to "compact" the iot.

use move to MOVE it. total rebuild.  2x the space.

2) indexes on iots are based on primary keys -- the move doesn't change the key, so no.


ops$tkyte@ORA920LAP> create table heap( x int primary key, y int );

Table created.

ops$tkyte@ORA920LAP> create index heap_idx on heap(y);

Index created.

ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> create table iot( x int primary key, y int ) organization index;

Table created.

ops$tkyte@ORA920LAP> create index iot_idx on iot(y);

Index created.

ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> insert into heap select rownum, rownum from all_objects;

30694 rows created.

ops$tkyte@ORA920LAP> insert into iot  select rownum, rownum from all_objects;

30694 rows created.

ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> analyze table heap compute statistics for table;

Table analyzed.

ops$tkyte@ORA920LAP> analyze table iot  compute statistics for table;

Table analyzed.

ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> set autotrace on explain
ops$tkyte@ORA920LAP> select * from heap where y = 55;

         X          Y
---------- ----------
        55         55


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=307 Bytes=3684)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'HEAP' (Cost=2 Card=307 Bytes=3684)
   2    1     INDEX (RANGE SCAN) OF 'HEAP_IDX' (NON-UNIQUE) (Cost=1 Card=123)



ops$tkyte@ORA920LAP> select * from iot  where y = 55;

         X          Y
---------- ----------
        55         55


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=307 Bytes=4298)
   1    0   INDEX (RANGE SCAN) OF 'IOT_IDX' (NON-UNIQUE) (Cost=1 Card=307 Bytes=4298)



ops$tkyte@ORA920LAP> set autotrace off
ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> alter table heap move;

Table altered.

ops$tkyte@ORA920LAP> alter table iot  move online;

Table altered.

ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> set autotrace on explain
ops$tkyte@ORA920LAP> select * from heap where y = 55;
select * from heap where y = 55
*
ERROR at line 1:
ORA-01502: index 'OPS$TKYTE.HEAP_IDX' or partition of such index is in unusable state


ops$tkyte@ORA920LAP> select * from iot  where y = 55;

         X          Y
---------- ----------
        55         55


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=307 Bytes=4298)
   1    0   INDEX (RANGE SCAN) OF 'IOT_IDX' (NON-UNIQUE) (Cost=1 Card=307 Bytes=4298)



ops$tkyte@ORA920LAP> set autotrace off


the heap table needs to have indexes rebuilt (rowids changed) the iot does not 

Index with Large Block size

Jagjeet Singh, December 26, 2003 - 6:22 am UTC

Hi,

SQL> Select Tablespace_name,block_size from Dba_Tablespaces
  2  /

TABLESPACE_NAME                BLOCK_SIZE
------------------------------ ----------
....
TEST                                 2048
SIXTEEN_K                           16384

6 rows selected.

SQL> Alter user j default tablespace test;

SQL> create table a(  a  varchar2(700) primary key )
     organization index;

Table created.

SQL> alter table a modify (a   varchar2(4000));
alter table a modify (a   varchar2(4000))
                      *
ERROR at line 1:
ORA-01404: ALTER COLUMN will make an index too large  <<--

SQL> Select index_name,table_name,tablespace_name from user_indexes;

INDEX_NAME         TABLE_NAME  TABLESPACE_NAME
------------------ ----------- ----------------
SYS_IOT_TOP_25958  A           TEST     <<--- 2k Tablespace

SQL> Alter table a move tablespace sixteen_k;

Table altered.

SQL> select index_name,table_name,tablespace_name from user_indexes ;

INDEX_NAME        TABLE_NAME TABLESPACE_NAME
----------------- ---------- ---------------
SYS_IOT_TOP_25958 A          SIXTEEN_K

SQL> alter table a modify (a   varchar2(4000));

Table altered.

SQL> 
SQL> alter table a move tablespace test ;   <<--- Moving back in 2k

Table altered.

SQL> select index_name,table_name,tablespace_name from user_indexes ;

INDEX_NAME        TABLE_NAME TABLESPACE_NAME
----------------- ---------- ---------------
SYS_IOT_TOP_25958 A          TEST   ??????

how can an index can have a 4000+ bytes within 2k block size
for a single entry.


 

Tom Kyte
December 26, 2003 - 10:58 am UTC

looks like a bug, they forgot to check. if the table had rows that exceeded the max size, you would have gotten:

alter table t move tablespace ts_2k
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [17023], [0x5CC5E94C], [], [], [], [], [], []

if the alter move succeeds, you will get:

insert into t values ( rpad('*',4000,'*') )
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [kdisle:nrows], [1], [], [], [], [], [], []

upon trying to insert the too large row.


There is no data corruption or anything with this -- just a missed check before moving. I've file a bug against this issue for you with this testcase:

create tablespace ts_2k datafile size 1m blocksize 2k;
create tablespace ts_16k datafile size 1m blocksize 16k;
create table t ( x varchar2(500) primary key )
organization index tablespace ts_2k;
alter table t modify x varchar2(4000);
alter table t move tablespace ts_16k;
alter table t modify x varchar2(4000);
insert into t values ( rpad('*',4000,'*') );
alter table t move tablespace ts_2k;
delete from t;
alter table t move tablespace ts_2k;
insert into t values ( rpad('*',4000,'*') );




How can you transfer normal table to IOT

Jelena, March 23, 2004 - 7:09 am UTC

IOT seems to be nice way to organize data in my case. We have 6 million rows in a table, which is always accessed via one index, only 5 columns and 3 are already in index, so IOT seems to be perfect solution. But i have two concerns, how can i actually transfer this table to IOT?
if i create new table and do: insert into ... select * ... on 6 million rows it will eventually crash (after eating up all rollback segments) so is there any fancy way to change the organization. Or if not, if i export table, and drop and create with the same name, but different organization, will import work?
Is there any reason (intensive insert/update...) or any other reason why i should _not_ use IOT?
And one more question, can you partition IOT? does it make sense at all?
Thanks!
Jelena

Tom Kyte
March 23, 2004 - 7:56 am UTC

it will not crash, if you size your RBS for the work you need to do (i work with 10 million or more wide rows on my laptop frequently for heavens sake. My laptop!)

You can partition an IOT, yes.
Yes, it can make sense (but 6million rows is pretty small still)

If you use

"CREATE TABLE IOT ( columns..., primary key(...) ) organization index
as
select...."

You'll find you use just about, oh, 1 rollback block :) you are worrying about something that isn't going to happen (and in archive log mode, you could wip a nologging in there after consulting your DBA to schedule a backup of this new data asap, that would remove any redo log issues you might think you have)

big_table@ORA9IR2> desc test
Name Null? Type
----------------------------- -------- --------------------
ID NUMBER
OBJECT_ID NOT NULL NUMBER
CREATED NOT NULL DATE
OWNER NOT NULL VARCHAR2(30)
OBJECT_NAME NOT NULL VARCHAR2(30)



big_table@ORA9IR2> select count(*) from test;

COUNT(*)
----------
6000000


big_table@ORA9IR2> create table iot
2 ( id, object_id, created, owner, object_name, primary key(id,object_id,created))
3 organization index
4 as
5 select * from test;

Table created.

Elapsed: 00:04:21.35


and it used one block of UNDO!

Partitioning of an IOT

Jelena, March 25, 2004 - 3:47 am UTC

Hi Tom,
thanks for your answer, it's really helpfull.
I was worried that i have to do
"insert into ... select..."
on 6 mil rows, which would mean for sure some downtime for our production (since we are running 7x24 system, its not so easy to organize) "Create table ... as select" looks perfect!
Besides that, you said that it makes sense to partition IOT (not in my case but in general). My question is: since it's index organized table, and if you partition it, does oracle create "index organization" in each partition (ie the effect of local index) or it's still organized as global index, and in that case what's the use of partitioning?
Thanks,
Jelena

Tom Kyte
March 25, 2004 - 9:13 am UTC

yes, it is many IOT's if you partition it.

range partitioned, IOT, compressed table

cosmin ioan, March 25, 2004 - 2:41 pm UTC

hi Tom,
on a 9iR2, I have a thin (3 column) table of about 1 billion rows and growing fast. I'm considering a range partitioned, compressed, IOT scenario.

Is this asking too much? :-)
I've already build the partitioned IOT table and it looks like the performance (and not only) is greatly improved, from many aspects which I won't dwell on.

I'll do some tests on the feasibility/speed etc of the COMPRESSED scenario (on top of the above) but my main question is IF (if presuming this yields better results) this is a STABLE long term, "production" solution, ie, if there are no known bugs, "gotchas" or otherwise issues (for this scenario) that would be considered as major showstoppers or of major future potential risk.

Tom Kyte
March 25, 2004 - 9:21 pm UTC

3 skinny columns = perfect for IOT

1,000,000,000 rows = sounds like a partition candidate

if leading column(s) are repetitive -- compression = probable good idea....


I'm not aware of any show stopping gotchas, no.

...just to be sure

cosmin ioan, March 26, 2004 - 4:26 pm UTC

hi Tom,
thanks much for the feedback but I want to make certain I've explained myself clearly, as this is crital for our environment:

what I want it an INDEX ORGANIZED, RANGE PARTITIONED AND COMPRESSED table, all in one, like below (I'll find the compressed syntax somewhere) [and thanks for the tip on the 'first repetitive column']

It sounded for a second, if I understood right, that you were advocating either IOT or a partitioned table but I was thinking of wanting both AND the compression and furthermore was concerned about long term chances of corruption, etc. We have plenty memory, bazillion of RAM so the compression time would not be of concern, I believe (but still need to run a benchmark). I'm more concerned about corruption and the like, PLUS if I issue a TRUNCATE (which incidentally I have yet to get the syntax on the truncation of a partition) if I have to rebuild the ENTIRE table or just that IOT'ed partition :-)

Basically, has the above scenario been approached by any major companies with success and little administrative headaches.
:-)


create table myparttable
(
field1 number(10),
field2 number(10),
field3 number(10),
constraint table_pk primary key (field1,field2,field3)
organization index nologging
partition by range(field1)
(partition p7 values less than (8) tablespace yadayada.....

This alone will **vastly** improve our processing however, for the kicker, are there new schema/schematic/storage features in 10g (other than compiled PL/SQL which we try to use as little as we can-- mostly pure sql or analytical funcs for speed purposes) that would be worth considering to make this even more of a speed deamon as a "dream of a point in time" where we'd be swamped processing tens if not hundreds of billions of operations.

Tom Kyte
March 26, 2004 - 6:51 pm UTC

nope, I was saying -- in a building on previous statement sort of fashion -- that an IOT, that was partitioned and perhaps compressed would be a very good idea.

I am aware of large partitioned IOTs (not sure on the compression, that is sort of an after thought in many cases) and they are running successfully.

partitioned IOT; compression vs cost

cosmin, March 29, 2004 - 9:17 pm UTC

hi Tom,
thx a bunch for all the info.
I've compressed the range partitioned IOT'ed table (with the most repeatable value columns first) and yeah, it compresses to about 1/2 the size which is really cool. Now, one thing that I don't understand is why the actual cost (of various simple operations; after 'analysis' of partitions) is about a thousand-fold higher than on the non-compressed counterpart table.

Do I need to be looking into histograms and the like or is there something else, more specific compression-related that I'm overlooking?

I'll play in the next few months w/ 10g, just for kicks but thought I'd inquire about this on 9iR2 as this is the release we're deploying on next.

Tom Kyte
March 30, 2004 - 7:29 am UTC

do you have an example of what you mean?  I show full scan and index scan costs that are lower for the compressed, partitioned IOT -- can you help me make this example mimick what you are seeing?

ops$tkyte@ORA9IR2> create table t1
  2  ( object_type, owner, object_name, object_id,
  3    constraint t1_pk primary key(object_type,owner,object_name))
  4  partition by range (object_type)
  5  ( partition values less than ( 'S' ),
  6    partition values less than ( MAXVALUE )
  7  )
  8  as
  9  select object_type, owner, object_name, max(object_id)
 10    from all_objects
 11   group by object_type, owner, object_name
 12  /
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> create table t2
  2  ( object_type, owner, object_name, object_id,
  3    constraint t2_pk primary key(object_type,owner,object_name))
  4  organization index compress 2
  5  partition by range (object_type)
  6  ( partition values less than ( 'S' ),
  7    partition values less than ( MAXVALUE )
  8  )
  9  as
 10  select * from t1
 11  /
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> analyze table t1 compute statistics for table for all indexes;
 
Table analyzed.
 
ops$tkyte@ORA9IR2> analyze table t2 compute statistics for table for all indexes;
 
Table analyzed.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select * from t1;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=23 Card=30543 Bytes=1374435)
   1    0   PARTITION RANGE (ALL)
   2    1     TABLE ACCESS (FULL) OF 'T1' (Cost=23 Card=30543 Bytes=1374435)
 
 
 
ops$tkyte@ORA9IR2> select * from t2;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=16 Card=30543 Bytes=977376)
   1    0   PARTITION RANGE (ALL)
   2    1     INDEX (FAST FULL SCAN) OF 'T2_PK' (UNIQUE) (Cost=16 Card=30543 Bytes=977376)
 
 
 
ops$tkyte@ORA9IR2> select object_id from t1 where owner = 'X' and object_type = 'Y' and object_name='Z';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=44)
   1    0   TABLE ACCESS (BY GLOBAL INDEX ROWID) OF 'T1' (Cost=2 Card=1 Bytes=44)
   2    1     INDEX (UNIQUE SCAN) OF 'T1_PK' (UNIQUE) (Cost=1 Card=16890)
 
 
 
ops$tkyte@ORA9IR2> select object_id from t2 where owner = 'X' and object_type = 'Y' and object_name='Z';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=1 Bytes=32)
   1    0   INDEX (UNIQUE SCAN) OF 'T2_PK' (UNIQUE) (Cost=1 Card=1 Bytes=32)
 
 
 
ops$tkyte@ORA9IR2> set autotrace off
 

Deciding on IOT

Dhrubo, March 30, 2004 - 9:35 am UTC

Hi Tom,
I was reading your book Expert one on One Oracle.I read about the four main conditions when i can possibly use IOT.Say i have an Address table where i store Address information both for my customers as well as sales force - the structure of the table is like this :
AddressId NUMBER,
CustomerId NUMBER,
SaleForceId NUMBER,
Street1 VARCHAR2(100),
Street2 VARCHAR2(100),
Street3 VARCHAR2(100),
AddressType CHAR(1),
City NUMBER,
Zip VARCHAR2(10),
State NUMBER,
Country NUMBER

Now am i supposed to use IOT in this case.Or traditional heap organized table with AddressId as my primary key and a bitmap index on AddressType which can have possibly three value and my Address table can have millions of rows so the cardinality is < 0.1.What is your suggestion?

Tom Kyte
March 30, 2004 - 10:31 am UTC

I cannot really answer that.

do you care that addressid = 55 is right next to addressid=56? an IOT would give you that....


that looks a bit fat for an IOT. Probably a conventional heap table .


As for addresstype, why would you even need to index it????

partitioned IOT; compression vs cost; followup

cosmin, March 30, 2004 - 6:08 pm UTC

hi Tom,
I was able to replicate your test/results almost identically; I rebuilt my table and indeed the COST went down substantially, well below the cost of the uncompressed table (when I analyzed the index as well, in addition to the table, I think that was the issue) HOWEVER, the actual elapsed time of a simple select or a count(*) would not finish in minutes (vs. milliseconds). I guess toying w/ TKPROF might come in handy. Also, the funny thing is that the space taken by a simple compressed partition vs the identical non-compressed partition slightly increased, even though I started the table definition w/ the most frequent value columns first.

That being said, I don't think we'll go to compression this release as we're sometimes doing intensive write IO and I don't think it's feasible, besides the fact that we're doing the upgrade v. soon and we're now at 'crunch time' and a lot more research is required on my part to get make this a strong case for our management.

I'll toy w/ compression more in the future, perhaps also on 10g however for now, Partitioned IOT should give us substantial performance increase;

I will come back w/ more feedback/questions, in the next few weeks I hope, after I put out all the current fires and I document myself a bit more :-)


thx much for all your help & looking forward for your book on 10g :-)
Cosmin

IOT and secondary indexes

VKOUL, July 01, 2004 - 3:08 pm UTC

Excerpt
"Great Info  August 23, 2003 
Reviewer:  Samuel Bernard  from Salt Lake City 

Tom, if I create an index on a column other than the primary key columns in IOT, 
what does the index just created on the column store? Does it store what you 
have shown in the demo? I don't know much about using IOT so I would appreciate 
your time in explaining this concept. Thanks much. 


Followup:  
an index on an IOT always includes the primary key. 
 

Bookmark Review | Bottom | Top 
index key contains .....  August 23, 2003 
Reviewer:  David  from Austin, TX 

Tom, does it mean that the index entry for the indexed column (secondary index) 
in IOT will have key value, primary key value, rowid (logical?)

thanks. 


Followup:  
yes,"

Pls. review the following :

SQL> create table iot_table(x number, y number, constraint pk_iot_table primary key (x,y));

Table created.

SQL> create index idx_iot_table on iot_table(x, y);
create index idx_iot_table on iot_table(x, y)
                                        *
ERROR at line 1:
ORA-01408: such column list already indexed


SQL> create index idx_iot_table on iot_table(y);

Index created.

SQL> drop index idx_iot_table;

Index dropped.

SQL> create unique index idx_iot_table on iot_table(y);

Index created.

SQL> disc
Disconnected from Oracle8i Enterprise Edition Release 8.1.7.3.0 - 64bit Production
With the Partitioning option
JServer Release 8.1.7.3.0 - 64bit Production

Q1 : If secondary index on an IOT table always contain primary columns, then creating secondary index in first and (second or third) case shouldn't have given the same error, which was raised in first case, may be I got it all wrong.

Q2 : Since IOT tables are indexes in reality (I assume), What difference on structural level they have in comparision with indexes on columns in case all columns of my IOT table are primary key, such as

Table IOT(x number, y number), PK : x,y
Table HEAP(x number, y number), PK : x,y

What structural level difference do IOT table and Primary key index of HEAP have ?

Thanks 

Tom Kyte
July 01, 2004 - 7:56 pm UTC

q1) you had an index on x,y -- the primary key.

You are allowed then to add indexes on (y) or on (x).

having one on (x) would be "not so smart", having one on (y) could be very smart -- lets you go via X=.... or Y=..... values.

the "rowid" for the IOT is the primary key value, that is all.

q2) there are minor (non-relevant to you and me, only interesting to people that want to dump blocks) differences. for all intents and purposes consider them "the same" -- except the IOT would not have the overhead of the heap table that a regular index would.

basics please

pooja, July 02, 2004 - 4:45 am UTC

hi tom,
i'm not aware of this index organized tables and also the concepts of optimization.and i have to attend the interview.can u please kindly let me know those concepts
please tom send me some useful links



Tom Kyte
July 02, 2004 - 9:51 am UTC

see my home page -- I've written two books on Oracle, see the server concepts (free on otn.oracle.com). READ the concepts guide from cover to cover.


Should IDs in the predicate by in sorted order?

Peter Tran, July 02, 2004 - 9:56 am UTC

Hi Tom,

1) If I have a very large IOT table with the following PK (id number, booking_date date).
2) I'm querying for a list of IDs.
3) The list of IDs can be huge - several thousands.
4) IDs are random (1, 4000288, 1200, 2, 1924562, etc.)

My query looks like SELECT * FROM LARGE_IOT WHERE ID IN (?, ?, ?, ..., ?);

Would it help if I sort the IDs first? Or would it not matter at all.

Thanks,
-Peter


Tom Kyte
July 02, 2004 - 10:53 am UTC

it would not matter at all.

Jane, July 02, 2004 - 11:19 am UTC

Tom
Why Oracle has introduced the Index Organised Tables? What is it's advantage over the normal table.

Thanks

IOT tables Vs single table clusters

VKOUL, July 06, 2004 - 8:27 pm UTC

Thanks

I have one question relating IOT & Clustered Table.

Q1. Since IOT tables group data based on primary key, and single table cluster too organizes data on cluster key, given a choice what are the pros & cons of using one over the other ?



Tom Kyte
July 06, 2004 - 8:46 pm UTC

I compared and contrasted them in both of my books in some detail.

In most *general* cases, IOT's "win" but...

o secondary indexes, not a concern in clusters, they have real rowids.
o storing data from more than one table is something an IOT cannot do.


you have to size clusters which can be hard, but secondary indexes don't exist (which is good) and you can "pre-join" data (which is good)....



When is not a good option to use

Manish Ahuja, July 07, 2004 - 12:03 am UTC

Hey Tom

I am new to IOT's. Only read about them never used them. Looking at the advantages it offers do you think we should always use IOT's & not opt for regular tables (with indexes).

What I mean to ask is when are IOT's not to be used. In which scenarios will IOT's not be a good candidate to choose over regular tables (with indexes)

Rgrds

Tom Kyte
July 07, 2004 - 7:58 am UTC

no, if I did, I would say "stop using heaps"

I'll never say that.


what you need to do is understand why you might use them (to force position of data in a table, to make certain rows that sort together, be together), when you might use them.

I've written extensively about them in both Expert one on one Oracle and Effective Oracle by design. In fact, I covered IOTs, b*tree clusters, hash clusters, compression, etc -- you might be interested in that.

to VKOUL

Jan, July 07, 2004 - 3:19 am UTC

IOT - data ARE ordered by PK - good for index range scan (e.g. ... WHERE pk BETWEEN 1 AND 100) - but inserts/updates are probably more expensive (due to ordering - but I did not tested it)

Single Cluster - data ARE NOT ordered - good enough for equality conditions (e.g. ... WHERE cluster_key = 50), worse for index range scan (data are not ordered by cluster key), but probably faster inserts/updates (again - I did not tested it)



IOTs VS CLUSTER TABLE

VKOUL, July 07, 2004 - 11:47 am UTC

Thanks Tom & Jan

INDEX ORGANIZED TABLES

chandra, July 12, 2004 - 4:16 pm UTC

Tom,

I have created two tables one is index organized table
and the other normal heap table

sql>create table IOT(x int primary key)
2 organization index
3 /

sql>create table NON_IOT(x int primary key)
2 /

When I query user_table I do not see the tablespace name for the table IOT.

sql> select table_name, tablespace_name from user_tables
2 where table_name in ('IOT', 'NON_IOT')
3 /

TABLE_NAME TABLESPACE_NAME
------------------- ------------------------------
IOT
NON_IOT USERS

Why does it not display the tablespace_name from Index organized tables?.






Tom Kyte
July 12, 2004 - 11:35 pm UTC

there is no table, just an index -- that is why.

look in user_indexes.

Intersection tables

A reader, August 28, 2004 - 11:37 pm UTC

My database has several "intersection tables" i.e. resolver tables for many-to-many relationships. These tables have the (surrogate) PKs for upto 3-4 tables as their composite PK. So, this intersection table has a 3-column composite PK plus an additional 3-4 columns like timestamp, updated by, note, etc.

Since they dont contain any real, usable information, they are *always* used by joining with the 3 real tables by each of the PK columns.

1. Would using IOTs for these intersection tables make sense?

2. Since they are always joined, would that negate the benefits of using IOTs, i.e. would clusters be more suitable here?

3. I read that "skinny" tables benefit from being IOT so that no overflow segment, etc are needed.

How skinny is skinny? Upto what row length?

Thanks



Tom Kyte
August 29, 2004 - 11:41 am UTC

pk's for 3-4 tables? really? interesting, association objects in my experience are for 2 tables. I don't know as if I've seen a 3/4 way one.


These would make sense for IOTs in general. Absolutely.

I don't see why "joining" would negate their benefits? A cluster would make sense if all of the objects shared the same key (like DEPTNO with dept and emp). Here you are saying explicitly that these objects do not share a common key, they are random associations created by linking primary key1 to primary key2.


skinny is something less than about 10-40% of the block size. If the IOT row is too "wide", the index structure will split frequently (that has performance implications) and if the row needs an overflow, then accessing the overflow data is just like "index range scan -> table access by index rowid" - that is, much of the benefit of the IOT is overtaken, the amount of work performed to retrieve becomes the same.


Your "note" field opens an area of concern -- and could be a reasonable overflow column as I doubt it is accessed very often at all (so the retrieval price isn't an issue). It sounds "wide", perhaps you could model it as an out of line CLOB even...


Thanks

A reader, August 29, 2004 - 10:30 pm UTC

<quote>
pk's for 3-4 tables? really? interesting, association objects in my experience are for 2 tables. I don't know as if I've seen a 3/4 way one.
<quote>

Well, it is a association object for a many-many-many relationship! :) Or think of it as a "fact" table. The dimensions are A, B and C. The association/fact table is a_b_c (with composite pk as (a_pk, b_pk, c_pk)

<quote>
Your "note" field opens an area of concern -- and could be a reasonable overflow column as I doubt it is accessed very often at all (so the retrieval price isn't an issue). It sounds "wide", perhaps you could model it as an out of line CLOB even
<quote>

You hit the nail on the head. This "note" field is just defined in the model, but most apps dont use it, it is NULL for >95% of the rows.

1. How can I define it as a out of line CLOB?

2. What are the similarities & differences between using overflow segments and using this out-of-line clob technique?

Thanks

Tom Kyte
August 30, 2004 - 8:22 am UTC

if I think of it as a fact table, I'm not liking the idea of a IOT anymore.  

1) using the lob storage clause

ops$tkyte@ORA9IR2> CREATE TABLE t
  2  (X NUMBER,
  3   Y CLOB)
  4  LOB (Y) STORE AS
  5  (TABLESPACE USERS
  6   DISABLE STORAGE IN ROW
  7   CHUNK 8192
  8   PCTVERSION 10
  9   NOCACHE )
 10  /
 
Table created.
 
2) they are similar in that any dereference of the lob locator will mandate a read elsewhere in another segment, like reading an overflow would - but the lob allows for arbitrarily long notes, not just 4000 bytes. 

Fact table?

A reader, August 30, 2004 - 8:39 am UTC

"if I think of it as a fact table, I'm not liking the idea of a IOT anymore"

Why?

All my statements above are still the same, so just because it is a fact table (2+ tables) and not an association table (2 tables), why does it make you not like the idea of an IOT?

As per conventional DW theory, fact tables are always
joined with the dimension tables (since they have non-intelligent surrogate keys), so IOTs would make perfect sense, since the index structure itself contains all the data, no table access by rowid needed!

Thanks

Tom Kyte
August 30, 2004 - 9:02 am UTC

because you come at a fact table via any direction.

an IOT is best come at from "one" direction (the key direction)


the DIMENSIONS make great sense for IOTs

the FACTS not as much.

Dimensions for IOTs

A reader, August 30, 2004 - 9:56 am UTC

"the DIMENSIONS make great sense for IOTs"

What?! Typical DIMENSION tables are very wide, contain lots of descriptive attributes and would lead to heavy use of IOT overflow segment! IMHO dimension tables are a bad choice for IOT

"an IOT is best come at from "one" direction (the key direction"

And in the case of dimensions (most likely having surrogate keys), it would never be come from the key direction! So I am surprised that you suggest using IOTs for dimension tables!


Tom Kyte
August 30, 2004 - 10:34 am UTC

typical dimension tables in my experience are not very wide?

date -> week
-> month -> qty -> fy

for example.

and dimensions are frequently accessed via their keys. ?

but at the end of the day -- it all comes down to "how and what you are doing". Your experience is different than mine. That is why we need to not have ROT (rules of thumb) but use cases.


In general, i think of IOT's for lookups.
Dimensions are lookups to me.




Dimensions

A reader, August 30, 2004 - 5:07 pm UTC

You are right, thats why it is so frustrating to discuss data warehousing-type things with others. Every person has a different thing in mind when they say DW/DSS, etc.

You are talking about the quinessential "time dimension", which is very "skinny".

I am talking about the "real" dimensions in the DW which are the CUSTOMER, INVOICE_DETAIL, etc. CUSTOMER dimension in a CRM-type data warehouse is extremely wide, more than 100 columns and lots of VARCHAR2(2000) columns!

Thanks

Tom Kyte
August 30, 2004 - 5:39 pm UTC

"real" is in the eye of the beholder, that is why i like to use examples with most of my discussions -- case studies. because ROT just doesn't compute at the end of the day.

google:

</code> http://www.google.com/search?hl=en&lr=&ie=UTF-8&safe=off&q=define%3A+quintessential&btnG=Search <code>

love the definition it came up with ;)

That is why lots of times when sitting in an audience listening to people talk -- I have a very quizzical look on my fact -- due to generalizations being thrown out that I can come up with 5 cases where "it just doesn't apply".

Dimension

A reader, August 30, 2004 - 6:54 pm UTC

OK maybe "quintessential" was not the right choice of words, but the fact remains that TIME dimension is not a real busines-oriented dimension. For a day-grain time dimension, populate heck even 100 years worth, its only 36500 rows and 99% of the companies out there would have the exact same table. Its nothing but a darn calendar, for crying out loud.

The other "real" (for lack of a better term) dimensions have lots of real-world headaches. handling slowly changing dimensions, type1 or type 2 or a hybrid, handling their loading, handling history, etc, etc.

Anyway, I dont want to sidetrack this thread on IOTs into a discussion about data warehousing techniques, but I think still think that most real-world dimensions out there in mid-to-large sized DWs are very wide and NOT suitable for a IOT.

Thanks

Tom Kyte
August 30, 2004 - 7:31 pm UTC

well, it is "real" enough that we invented an entire "create dimension" thing in order to make dimensions like that contain yet more meta-data that can be used by the optimizer with query rewrite and MV's (eg: it is common enough that we added tons of stuff to the server *just* in support of them)

many many dimensions are "hierarchies" like that. that 99% of the companies out there would have the same thing doesn't "obviate that"

we'll have to agree to "IOT's are a data structure that, once you understand what they are, how they work, what they do well -- may well be used in your OLTP, DW, DSS, reporting, mixed workload, etc etc etc etc systems in a variety of places"

A reader, August 30, 2004 - 7:56 pm UTC

"we'll have to agree to "IOT's are a data structure that, once you understand what they are, how they work, what they do well -- may well be used in your OLTP, DW, DSS, reporting, mixed workload, etc etc etc etc systems in a variety of places"

I agree completely.

Few more questions, if you dont mind

I have some table(s) that I think are suitable to be converted to IOTs.

1. How do I go about methodically verifying and quantifying the benefit of doing this? Run all the queries that use the table as a regular table and a IOT, compare the LIOs, elapsed time, etc? Do you have any guidelines?

2. The existing tables have row-level triggers, secondary unique indexes, sps, views on them, etc. Do IOTs support all this or are there any restrictions I have to consider?

3. How can I seamlessly migrate the existing table to IOT while preserving all the things above? Can I use the dbms_redef package for this? How?

Thanks a lot

Tom Kyte
August 30, 2004 - 8:09 pm UTC

1) i view IOT's as a "clustering" thing or a "i've got a table i access by primary key mostly" thing.

to me, it is mostly about "locale of data" (the stock example leaps immediately to mind) or an association table (for many to many's) or lookups...

2) secondary indexes on IOT's can be problematic. they include rowid hints that go stale frequently if the table is modified lots (rows move in an index, rows do not "move" in a table really - they might migrate but that is another story).

to dereference a row in an IOT via a secondary index, we use the rowid hint - see if the data is there, if not, use the primary key which is also stored in there to range scan the structure to refind the row. so a table access by index rowid into an IOT could be an IO to find the row isn't there and then two or three or four (depends on index height) to find the row.

in m:m tables, this isn't a problem since:

create table t ( id1 int, id2 int, primary key(id1,id2) ) organization index;
create index on t(id2);

lets me go back and forth either way via index only access (the primary key index, the table, goes id1->id2. the secondary index has the primary key AND is sorted by id2 so goes id2->id1 without accessing the table)

else, in 9ir2 -- an IOT has all of the abilities of a table... partitioning, et.al.

3) yes, definitely. search for dbms_redefintition on this site for examples of using that package.

Facing Problem with IOT

kiran kumar, August 31, 2004 - 12:33 am UTC

Hi Tom,

I am trying to create a IOT and rebuild the indexes. I am getting error like ....

SQL> ed
Wrote file afiedt.buf

  1  create table demo_iot
  2      ( object_id int primary key,
  3        oname     varchar2(30),
  4        owner     varchar2(30),
  5        status    varchar2(30) )
  6*     organization index
SQL> /

Table created.

SQL> insert into demo_iot select object_id, object_name, owner, status
  2  from all_objects;

30233 rows created.

SQL> column index_name new_value iname
SQL> ed
Wrote file afiedt.buf

  1  select index_name
  2        from user_indexes
  3*      where table_name = 'DEMO_IOT'
SQL> /

INDEX_NAME
------------------------------
SYS_IOT_TOP_35158

SQL> alter index SYS_IOT_TOP_35158 validate structure;
alter index SYS_IOT_TOP_35158 validate structure
                              *
ERROR at line 1:
ORA-02243: invalid ALTER INDEX or ALTER MATERIALIZED VIEW option

..... I am using oracle 9.2.0.4.0 (test database) and having dba privs.

 

Tom Kyte
August 31, 2004 - 8:47 am UTC

<b>that is not a "rebuild" and demo_iot is a "table"</b>

ops$tkyte@ORA9IR2> create table t ( x int primary key, y int ) organization index;
 
Table created.
 
ops$tkyte@ORA9IR2> insert into t select rownum, user_id from all_users;
 
48 rows created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> analyze table t validate structure;
 
Table analyzed.

 
ops$tkyte@ORA9IR2> select name, lf_rows from index_stats;
 
NAME                              LF_ROWS
------------------------------ ----------
SYS_IOT_TOP_136082                     48


<b>
and to "rebuild", you would:
</b>

ops$tkyte@ORA9IR2> alter table t move online;
 
Table altered.

<b>or</b>

ops$tkyte@ORA9IR2> alter table t move;
 
Table altered.


 

OK

James, August 31, 2004 - 5:56 am UTC

Hello Tom,
Is it possible to transform a IOT to Heap Organized Table?
Do you have any way to that?
Bye!


Tom Kyte
August 31, 2004 - 9:54 am UTC

you need to rebuild it

dbms_redefinition can be used to do this "online"

else, you are just rebuilding

create table new_table as select
drop old_table
rename new_table

What if I dont give INCLUDING clause?

A reader, December 15, 2004 - 1:29 am UTC

While creating IOT what if I dont give INCLUDING clause but specify PCTTHRESHOLD ?

Tom Kyte
December 15, 2004 - 1:33 pm UTC

from expert one on one Oracle:

<quote>
The OVERFLOW clause allows you to setup another segment where the row data for the IOT can overflow onto when it gets too large. Notice that an OVERFLOW reintroduces the PCTUSED clause to an IOT. PCTFREE and PCTUSED have the same meanings for an OVERFLOW segment as they did for a heap table. The conditions for using an overflow segment can be specified in one of two ways:

o PCTTHRESHOLD – When the amount of data in the row exceeds that percentage of the block, the trailing columns of that row will be stored in the overflow. So, if PCTTHRESHOLD was 10% and your block size was 8k, any row that was greater then about 800 bytes in length would have part of it stored elsewhere – off of the index block.

o INCLUDING – All of the columns in the row up to and including the one specified in the INCLUDING clause are stored on the index block, the remaining columns are stored in the overflow.

....

The gray boxes are the index entries, part of a larger index structure (in the section on indexing, you'll see a larger picture of what an index looks like). The index structure is a tree – and the leaf blocks, where our data is, are in effect a doubly-linked list to make it easier to traverse the nodes in order once you have found where you want to start at in the index. The white box represents an OVERFLOW segment. This is where data that exceeds our PCTTHRESHOLD setting will be stored. Oracle will work backwards from the last column up to but not including the last column of the primary key to find out what columns need to be stored in the overflow segment. In this example, the number column X and the date column Y will always fit in the index block. The last column, Z, is of varying length. When it is less than about 190 bytes or so (10% of a 2k block is about 200 bytes, add in 7 bytes for the date and 3 to 5 for the number) – it will be stored on the index block. When it exceeds 190 bytes, Oracle will store the data for Z in the overflow segment and set up a pointer to it.

The other option is to use the INCLUDING clause. Here you are stating explicitly what columns you want stored on the index block and which should be stored in the overflow.
......

Which is better then – PCTTHRESHOLD, INCLUDING, some combination of both ? It depends on your needs. If you have an application that always, or almost always, uses the first four columns of a table, and rarely accesses the last 5 columns, this sounds like an application for INCLUDING. You would include up to the fourth column and let the other five be stored out of line. At runtime, if you need them, they will be retrieved in much the same way as a migrated or chained row would be. Oracle will read the 'head' of the row, find the pointer to the rest of the row, and then read that. If on the other hand, you cannot say that you almost always access these columns and hardly ever access those columns, you would be giving some consideration to PCTTHRESHOLD. Setting the PCTTHRESHOLD is easy once you determine the number of rows you would like to store per index block on average. Suppose you wanted 20 rows per index block. Well, that means each row should be 1/20th (5%) then. Your PCTTHRESHOLD would be 5 – each chunk of the row that stays on the index leaf block should consume no more then 5% of the block.
<quote>

pctthreshold and including are complimentary.

Is secondary index needed for other id in associative table?

Scot, January 05, 2005 - 10:47 am UTC

A couple of posts above, you made the comments:

--
create table t ( id1 int, id2 int, primary key(id1,id2) ) organization index;
create index on t(id2);

lets me go back and forth either way via index only access (the primary key
index, the table, goes id1->id2. the secondary index has the primary key AND is
sorted by id2 so goes id2->id1 without accessing the table)
--

I was testing (in 10gr1) IOT's, and found that even without creating the secondary index on id2, that queries looking for id2 still used the primary key index of the IOT. The difference was that queries looking for id1 did an index range scan (very few lio's) and those looking for id2 used an index fast full (lots more lio's).

But regardless of whether I had id1 = :x or id2 = :x in the predicate, both queries used the iot pk index.

Then, when I created a secondary index on just id2, and re-ran, the optimizer chose the same path and the secondary index was not used. This of course could be due to the size and type of data I was testing with on my slow pc.


Tom Kyte
January 05, 2005 - 11:09 am UTC

ops$tkyte@ORA10G> create table t (id1, id2, primary key(id1,id2) ) organization index
  2  as
  3  select object_id, -object_id from all_objects;
 
Table created.
 
ops$tkyte@ORA10G>
ops$tkyte@ORA10G> create index t_idx on t(id2);
 
Index created.
 
ops$tkyte@ORA10G>
ops$tkyte@ORA10G> exec dbms_stats.gather_table_stats( user, 'T', cascade=>true );
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10G>
ops$tkyte@ORA10G> set autotrace traceonly explain
ops$tkyte@ORA10G> select * from t where id1 = 5;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=ALL_ROWS (Cost=2 Card=1 Bytes=10)
   1    0   INDEX (RANGE SCAN) OF 'SYS_IOT_TOP_73989' (INDEX (UNIQUE)) (Cost=2 Card=1 Bytes=10)
 
 
 
ops$tkyte@ORA10G> select * from t where id2 = -5;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=ALL_ROWS (Cost=1 Card=1 Bytes=10)
   1    0   INDEX (RANGE SCAN) OF 'T_IDX' (INDEX) (Cost=1 Card=1 Bytes=10)
 
 
 
ops$tkyte@ORA10G> set autotrace off
ops$tkyte@ORA10G> select * from v$version;
 
BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.1.0.3.0 - Prod
PL/SQL Release 10.1.0.3.0 - Production
CORE    10.1.0.3.0      Production
TNS for Linux: Version 10.1.0.3.0 - Production
NLSRTL Version 10.1.0.3.0 - Production
 
 

Hmm, I must be doing something wrong?

Scot, January 05, 2005 - 11:58 am UTC

SQL> 
SQL> create table iop1
  2  (
  3           p1pk         number
  4           ,data         char(100)
  5           ,constraint iop1 primary key (p1pk)
  6  )
  7  ;

Table created.

SQL> 
SQL> insert into iop1 select rownum, 'x' from all_objects where rownum <= 400;

400 rows created.

SQL> 
SQL> create table iop2
  2  (
  3           p2pk         number
  4           ,data         char(100)
  5           ,constraint iop2 primary key (p2pk)
  6  )
  7  ;

Table created.

SQL> 
SQL> insert into iop2 select rownum, 'x' from all_objects where rownum <= 400;

400 rows created.

SQL> 
SQL> create table ioa
  2  (
  3           p1pk         number
  4           ,p2pk         number
  5           ,data         char(20)
  6           ,constraint ioapk primary key (p1pk,p2pk)
  7           ,constraint ioafk1 foreign key (p1pk) references iop1 (p1pk)
  8           ,constraint ioafk2 foreign key (p2pk) references iop2 (p2pk)
  9  )
 10  organization index
 11  ;

Table created.

SQL> 
SQL> create index ioa_idx on ioa(p2pk);

Index created.

SQL> 
SQL> -- intentional cartesian product
SQL> insert /*+ append */ into ioa select p1pk, p2pk, 'x' from iop1, iop2;

160000 rows created.

SQL> 
SQL> commit;

Commit complete.

SQL> 
SQL> exec dbms_stats.gather_table_stats(user,'iop1',cascade=>true);

PL/SQL procedure successfully completed.

SQL> exec dbms_stats.gather_table_stats(user,'iop2',cascade=>true);

PL/SQL procedure successfully completed.

SQL> exec dbms_stats.gather_table_stats(user,'ioa',cascade=>true);

PL/SQL procedure successfully completed.

SQL> 
SQL> 
SQL> explain plan for select * from ioa where p1pk = 1;

Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------
Plan hash value: 121472429

--------------------------------------------------------------------------
| Id  | Operation        | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT |       |   401 | 11228 |     4   (0)| 00:00:01 |
|*  1 |  INDEX RANGE SCAN| IOAPK |   401 | 11228 |     4   (0)| 00:00:01 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("P1PK"=1)

13 rows selected.

SQL> 
SQL> set autotrace traceonly statistics;
SQL> --alter session set sql_trace=true;
SQL> select /* ioap1pk */ * from ioa where p1pk = 1;

400 rows selected.


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
         31  consistent gets
          0  physical reads
          0  redo size
       8459  bytes sent via SQL*Net to client
        794  bytes received via SQL*Net from client
         28  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        400  rows processed

SQL> --alter session set sql_trace=false;
SQL> set autotrace off;
SQL> 
SQL> 
SQL> explain plan for select * from ioa where p2pk = 1;

Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------
Plan hash value: 3782169291

------------------------------------------------------------------------------
| Id  | Operation            | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT     |       |   401 | 11228 |   157   (0)| 00:00:02 |
|*  1 |  INDEX FAST FULL SCAN| IOAPK |   401 | 11228 |   157   (0)| 00:00:02 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("P2PK"=1)

13 rows selected.

SQL> 
SQL> set autotrace traceonly statistics;
SQL> --alter session set sql_trace=true;
SQL> select /* ioap2pk */ * from ioa where p2pk = 1;

400 rows selected.


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
        783  consistent gets
          0  physical reads
          0  redo size
       8459  bytes sent via SQL*Net to client
        794  bytes received via SQL*Net from client
         28  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        400  rows processed

SQL> --alter session set sql_trace=false;
SQL> set autotrace off;
SQL> 
SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Prod
PL/SQL Release 10.1.0.2.0 - Production
CORE    10.1.0.2.0    Production
TNS for 32-bit Windows: Version 10.1.0.2.0 - Production
NLSRTL Version 10.1.0.2.0 - Production

SQL> 
SQL> spool off;
 

Tom Kyte
January 05, 2005 - 12:38 pm UTC

No, it is a significantly different thing here.

the question the optimizer was asking itself was "should I fast full scan the index to find the <b>400</b> rows, or should I do an index range scan with <b>400 MORE index range scans</b>"


It found it cheaper to fast full scan the index rather than do 401 index range scans (1 range scan to find p2pk = 1, 400 more to pick up the rows again to get "data")

apples and oranges here.


lower the number of rows returned by p2pk = 1

ops$tkyte@ORA10G> create table ioa
  2  (
  3           p1pk         number
  4           ,p2pk         number
  5           ,data         char(20)
  6           ,constraint ioapk primary key (p1pk,p2pk)
  7  )
  8  organization index
  9  ;
 
Table created.
 
ops$tkyte@ORA10G> create index ioa_idx on ioa(p2pk);
 
Index created.
 
ops$tkyte@ORA10G> insert /*+ append */ into ioa select p1pk, rownum, 'x' from iop1, iop2;
 
160000 rows created.


and it'll:

ops$tkyte@ORA10G> select /* ioap2pk */ * from ioa where p2pk = 1;
 
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=ALL_ROWS (Cost=3 Card=1 Bytes=29)
   1    0   INDEX (UNIQUE SCAN) OF 'IOAPK' (INDEX (UNIQUE)) (Cost=3 Card=1 Bytes=29)
   2    1     INDEX (RANGE SCAN) OF 'IOA_IDX' (INDEX) (Cost=1 Card=1)
 


quite naturally 

Although I can get an index skip scan

Scot, January 05, 2005 - 12:27 pm UTC

If I increase the size of the data field from char(20) to char(400) in ioa, I get an index skip scan, but still it is of ioapk (and much slower than range scan):

SQL> 
SQL> set echo on
SQL> 
SQL> -- asktom_iot_2.sql
SQL> -- io = indexed organized; p1 and p2 are entities; ioa is their association
SQL> 
SQL> drop table ioa;

Table dropped.

SQL> drop table iop1;

Table dropped.

SQL> drop table iop2;

Table dropped.

SQL> 
SQL> create table iop1
  2  (
  3           p1pk         number
  4           ,data         char(100)
  5           ,constraint iop1 primary key (p1pk)
  6  )
  7  ;

Table created.

SQL> 
SQL> insert into iop1 select rownum, 'x' from all_objects where rownum <= 400;

400 rows created.

SQL> 
SQL> create table iop2
  2  (
  3           p2pk         number
  4           ,data         char(100)
  5           ,constraint iop2 primary key (p2pk)
  6  )
  7  ;

Table created.

SQL> 
SQL> insert into iop2 select rownum, 'x' from all_objects where rownum <= 400;

400 rows created.

SQL> 
SQL> create table ioa
  2  (
  3           p1pk         number
  4           ,p2pk         number
  5           ,data         char(400)
  6           ,constraint ioapk primary key (p1pk,p2pk)
  7           ,constraint ioafk1 foreign key (p1pk) references iop1 (p1pk)
  8           ,constraint ioafk2 foreign key (p2pk) references iop2 (p2pk)
  9  )
 10  organization index
 11  ;

Table created.

SQL> 
SQL> create index ioa_idx on ioa(p2pk);

Index created.

SQL> 
SQL> -- intentional cartesian product
SQL> insert /*+ append */ into ioa select p1pk, p2pk, 'x' from iop1, iop2;

160000 rows created.

SQL> 
SQL> commit;

Commit complete.

SQL> 
SQL> exec dbms_stats.gather_table_stats(user,'iop1',cascade=>true);

PL/SQL procedure successfully completed.

SQL> exec dbms_stats.gather_table_stats(user,'iop2',cascade=>true);

PL/SQL procedure successfully completed.

SQL> exec dbms_stats.gather_table_stats(user,'ioa',cascade=>true);

PL/SQL procedure successfully completed.

SQL> 
SQL> 
SQL> explain plan for select * from ioa where p1pk = 1;

Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------
Plan hash value: 121472429

--------------------------------------------------------------------------
| Id  | Operation        | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT |       |   400 |   159K|    24   (0)| 00:00:01 |
|*  1 |  INDEX RANGE SCAN| IOAPK |   400 |   159K|    24   (0)| 00:00:01 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("P1PK"=1)

13 rows selected.

SQL> 
SQL> set autotrace traceonly statistics;
SQL> --alter session set sql_trace=true;
SQL> select /* ioap1pk */ * from ioa where p1pk = 1;

400 rows selected.


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
         49  consistent gets
         23  physical reads
          0  redo size
       8842  bytes sent via SQL*Net to client
        794  bytes received via SQL*Net from client
         28  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        400  rows processed

SQL> --alter session set sql_trace=false;
SQL> set autotrace off;
SQL> 
SQL> 
SQL> explain plan for select * from ioa where p2pk = 1;

Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------
Plan hash value: 2253818249

--------------------------------------------------------------------------
| Id  | Operation        | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT |       |   400 |   159K|   415   (1)| 00:00:05 |
|*  1 |  INDEX SKIP SCAN | IOAPK |   400 |   159K|   415   (1)| 00:00:05 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("P2PK"=1)
       filter("P2PK"=1)

14 rows selected.

SQL> 
SQL> set autotrace traceonly statistics;
SQL> --alter session set sql_trace=true;
SQL> select /* ioap2pk */ * from ioa where p2pk = 1;

400 rows selected.


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
        519  consistent gets
        191  physical reads
          0  redo size
       8842  bytes sent via SQL*Net to client
        794  bytes received via SQL*Net from client
         28  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        400  rows processed

SQL> --alter session set sql_trace=false;
SQL> set autotrace off;
SQL> 
SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Prod
PL/SQL Release 10.1.0.2.0 - Production
CORE    10.1.0.2.0    Production
TNS for 32-bit Windows: Version 10.1.0.2.0 - Production
NLSRTL Version 10.1.0.2.0 - Production

SQL> 
SQL> spool off;
 

Tom Kyte
January 05, 2005 - 12:52 pm UTC

about how big is your buffer cache? looks like it could be tiny -- and we are being forced to do tons more PIO than you would in real life.

to "fix" that you would have system statistics for your site, but this looks to be a test box, single user - and it is hard to get a representative load on a machine like that.

(and did you compare it to the OTHER plan, use an index hint and see if it actually is significantly faster the other way about)

Yep

Scot, January 05, 2005 - 1:51 pm UTC

Yep this is a test db on my pc. I'm not sure what my db_cache_size is, but I set sga_target (and sga_max_size) to 128M, so it has got to be small. I initially had them at 64M but kept running into ORA shared memory errors.

Thanks for your feedback on my two questions, it is very helpful. I'm off to do more reading and testing, including forcing use of the index via hint to compare.

need an IOT ?

Nilanjan Ray, January 28, 2005 - 4:14 am UTC

Dear Tom,

having read this useful thread I would like to have your opinion on the following

I have the following table structures

create table SSP_CMCL_BREAK_PATTERN (
cmcl_break_pattern_key number (10) not null,
channel_key number (4) not null,
schd_episode_key number (10),
cmcl_break_dt date not null,
start_tm varchar2 (6) not null,
duration number (5,2) not null,
break_type varchar2 (2) not null,
schd_type varchar2 (4) not null,
next_schd_episode_key number (10),
day_type varchar2 (1) not null,
modify_by varchar2 (10) not null,
modify_dt date not null,
prog_break_key number (10) not null,
constraint ssp_cmcl_break_pattern_pk
primary key ( cmcl_break_pattern_key ) ) ;

create table SSP_COMMERCIAL_BREAK (
cmcl_break_key number (10) not null,
cmcl_break_pattern_key number (10) not null,
cmcl_break_narr varchar2 (60),
territory_key number (4) not null,
airtime_status varchar2 (1) not null,
avail_airtime_flg varchar2 (1) default 'y' not null,
unmatched_airtime_flg varchar2 (1) default 'n' not null,
txm_flg varchar2 (1) default 'n',
spl_flg varchar2 (1) default 'n',
modify_dt date not null,
modify_by varchar2 (10),
cert_no number (3),
national_flg varchar2 (1) default 'n' not null,
sequenced varchar2 (1) default 'n',
constraint ssp_commercial_break_uk1
unique (cmcl_break_pattern_key, territory_key),
constraint ssp_commercial_break_pk
primary key ( cmcl_break_key ) ) ;

alter table ssp_commercial_break add constraint scb_fk2
foreign key (cmcl_break_pattern_key)
references stowner.ssp_cmcl_break_pattern (cmcl_break_pattern_key) ;

TABLE_NAME NUM_ROWS
------------------------------ ----------
SSP_CMCL_BREAK_PATTERN 368345
SSP_COMMERCIAL_BREAK 418770

Tables SSP_CMCL_BREAK_PATTERN and SSP_COMMERCIAL_BREAK are accessed
1. through cmcl_break_key and/or cmcl_break_pattern_key
2. often through queries like this
select ....
from SSP_CMCL_BREAK_PATTERN a,SSP_COMMERCIAL_BREAK b
where a.cmcl_break_pattern_key = b.cmcl_break_pattern_key
and a.cmcl_break_dt = <>
OR
select ....
from SSP_CMCL_BREAK_PATTERN a,SSP_COMMERCIAL_BREAK b
where a.cmcl_break_pattern_key = b.cmcl_break_pattern_key
and a.cmcl_break_dt between <> and <>
OR
select ....
from SSP_CMCL_BREAK_PATTERN a,SSP_COMMERCIAL_BREAK b
where a.cmcl_break_pattern_key = b.cmcl_break_pattern_key
and b.cmcl_break_key = <>

These two tables are heavily accessed from front-end applications and background jobs as well.
My question is :

1. Will having SSP_CMCL_BREAK_PATTERN (and/or SSP_COMMERCIAL_BREAK) as IOT improve on the performance?
2. I needs few secondary indexes on SSP_CMCL_BREAK_PATTERN and SSP_COMMERCIAL_BREAK. what could be the perils if I use IOT?

Thanks in advance for your help and guidence.

Regards





Tom Kyte
January 28, 2005 - 7:32 am UTC

simulate it, benchmark it, test it..........

It could remove the need for an IO when accessing via the key, but the stuff with cmcl_break_dt isn't going to benefit much, might even be damaging over time as you would need a secondary index on the date column -- and the rowid hint contained therein might go "stale" over time as you modify the data and so a "index range scan" + "table access by rowid" type of lookup becomes "index range scan" + "table access by rowid" + "another index range scan by primary key value" to find the row.

IOT & surrogate key

Nilanjan Ray, January 30, 2005 - 11:33 pm UTC

Thanks Tom,

I too had the same feeling about secondary indexes on IOT.
I always wondered whether a master-detail type relationship where both table uses surrogate keys as PK will ever benfit much with IOT as in my case above ? Here you would always needs a unique key on the detail table for the FK constraint column.

You view would be much appreciated.

Regards

Tom Kyte
January 31, 2005 - 8:07 am UTC

but the fkey of a child table could always be (primary key of parent+some unique number) if you are using surrogates anyway.

Then the iot will be organized around the fkey (good).

the surrogate can be anything you want here. and it need not be a single column.



stale secondary index

Nilanjan Ray, February 07, 2005 - 3:52 am UTC

Tom,

Is there any way to determine when a secondary index on IOT goes stale? Probably then an index-rebuild would be called for..(index-rebuild!!! ;-) probably this would be one good reason).

Regards

Tom Kyte
February 07, 2005 - 4:54 am UTC

statistcs tells you the pct of guesses that are "stale"

quets

amitabh deo, February 20, 2005 - 5:39 am UTC

2. How do you find out from the RMAN catalog if a particular archive log has been backed-up?

The database should be in ARCHIVE mode if you are going to use RMAN as backup method.


3. How can you tell how much space is left on a given file system and how much space each of the file system's subdirectories take-up?

4. Define the SGA and:

&#8226; How you would configure SGA for a mid-sized OLTP environment?
&#8226; What is involved in tuning the SGA?


5. What is the cache hit ratio, what impact does it have on performance of an Oracle database and what is involved in tuning it?

6. Other than making use of the statspack utility, what would you check when you are monitoring or running a health check on an Oracle 8i or 9i database?

7. How do you tell what your machine name is and what is its IP address?

8. How would you go about verifying the network name that the local_listener is currently using?

9. You have 4 instances running on the same UNIX box. How can you determine which shared memory and semaphores are associated with which instance?

10. What view(s) do you use to associate a user's SQLPLUS session with his o/s process?

11. What is the recommended interval at which to run statspack snapshots, and why?

12. What spfile/init.ora file parameter exists to force the CBO to make the execution path of a given statement use an index, even if the index scan may appear to be calculated as more costly?

13. Assuming today is Monday, how would you use the DBMS_JOB package to schedule the execution of a given procedure owned by SCOTT to start Wednesday at 9AM and to run subsequently every other day at 2AM.

14. How would you edit your CRONTAB to schedule the running of /test/test.sh to run every other day at 2PM?

15. What do the 9i dbms_standard.sql_txt() and
dbms_standard.sql_text() procedures do?

16. In which dictionary table or view would you look to determine at which time a snapshot or MVIEW last successfully refreshed?

17. How would you best determine why your MVIEW couldn't FAST REFRESH?

18. How does propagation differ between Advanced Replication and Snapshot Replication (read-only)?

19. Which dictionary view(s) would you first look at to
understand or get a high-level idea of a given Advanced Replication environment?

20. How would you begin to troubleshoot an ORA-03113 error?

21. Which dictionary tables and/or views would you look at to diagnose a locking issue?

22. An automatic job running via DBMS_JOB has failed. Knowing only that "it's failed", how do you approach troubleshooting this issue?

23. How would you extract DDL of a table without using a GUI tool?

24. You're getting high "busy buffer waits" - how can you find what's causing it?

25. What query tells you how much space a tablespace named "test" is taking up, and how much space is remaining?

26. Database is hung. Old and new user connections alike hang on impact. What do you do? Your SYS SQLPLUS session IS able to connect.

27. Database crashes. Corruption is found scattered among the file system neither of your doing nor of Oracle's. What database recovery options are available? Database is in archive log mode.

28. Illustrate how to determine the amount of physical CPUs a Unix Box possesses (LINUX and/or Solaris).

29. How do you increase the OS limitation for open files (LINUX and/or Solaris)?

30. Provide an example of a shell script which logs into SQLPLUS as SYS, determines the current date, changes the date format to include minutes & seconds, issues a drop table command, displays the date again, and finally exits.


Tom Kyte
February 20, 2005 - 9:55 am UTC

wow, a quiz. about seemingly random topic. on a question about IOT's.




Online rebuild of Partitioned IOT.

Sai, February 21, 2005 - 4:59 am UTC

Tom,

Is there any workaround for the following error, other than a drop and re-create of the table.

01:56:25 SQL> show rel
release 801070400

01:56:26 SQL> create table t (a number primary key , b number) organization index
01:56:26   2  partition by range(a)
01:56:26   3  (partition t1 values less than (10),
01:56:26   4  partition t2 values less than (40),
01:56:26   5  partition t3 values less than (80),
01:56:26   6  partition t4 values less than (maxvalue))
01:56:26   7  /

Table created.

01:56:26 SQL> alter table t move online initrans 10 storage (buffer_pool keep);
alter table t move online initrans 10 storage (freelists 23 freelist groups 17 buffer_pool keep)
            *
ERROR at line 1:
ORA-14511: cannot perform operation on a partitioned object

Thanks,
 Sai. 

Tom Kyte
February 21, 2005 - 10:42 am UTC

you have to rebuild to get freelist groups but the others can be altered in

ops$tkyte@ORA9IR2> create table t (a number primary key , b number) organization index
  2  partition by range(a)
  3  (partition t1 values less than (10),
  4  partition t2 values less than (40),
  5  partition t3 values less than (80),
  6  partition t4 values less than (maxvalue))
  7  /
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> alter table t modify partition t1 initrans 10 storage (freelists 23 buffer_pool keep);
 
Table altered.


so change the storage and then "move" 

Why the Quiz?

Kevin, February 21, 2005 - 6:44 am UTC

Is someone trying to get you to do their exam for them? :-)

Physical reads on IOT while inserting

Tri, May 31, 2005 - 3:19 pm UTC

I have a big IOT that experiences a lot of physical reads during massive inserts:

CREATE TABLE t (
        member_id INTEGER NOT NULL,
        newsletter_id INTEGER NOT NULL,
        CONSTRAINT t_pk PRIMARY KEY (member_id, newsletter_id) )
ORGANIZATION INDEX;

This table is large -- about 83M rows at the moment. Index structure:

SQL> analyze index T_PK validate structure;

Index analyzed.

SQL> select height, blocks, del_lf_rows from index_stats;

    HEIGHT     BLOCKS DEL_LF_ROWS
---------- ---------- -----------
         4     331792           0

The table experiences massive single-row inserts each and every day. About 1 Million of the following insert can happen within a day by multiple concurrent sessions (about 8-10 concurrent sessions) :

insert into t (member_id, newsletter_id) values (:member_id, :newsletter_id);

Freelists is currently at 20. Freelist groups at 1.

Doing a TKPROF for the above insert statement shows:

********************************************************************************
INSERT INTO T ( MEMBER_ID,NEWSLETTER_ID)
VALUES (:member_id, :newsletter_id)


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.07          2         69          0           0
Execute      1      0.00       0.01          1          2          6           1
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      0.00       0.08          3         71          6           1

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 85
********************************************************************************

My questions:

1) Why consistent read for an insert? (based on what I see in tkprof above). I would think it's mostly concurrent read at best?

2) Our 15-min statspack reports often show that the most dominating wait event as "db file sequential read" and it's contributed mainly by the insert statement above. Any suggestion you may have to reduce that? We are thinking of adding another column to the leading edge of the primary key, say, "org_id" (and enable compression) and thus the table would look something like:

CREATE TABLE t (
        org_id INTEGER NOT NULL,
        member_id INTEGER NOT NULL,
        newsletter_id INTEGER NOT NULL,
        CONSTRAINT t_pk PRIMARY KEY (org_id, member_id, newsletter_id) )
ORGANIZATION INDEX compress 2;

We know that the massive inserts happen for member_id's within one or two specific org_id's at a time. Perhaps this new design helps us reduce the physical reads. 

3) Building on #2 above, perhaps we can add partitioning to the org_id column. Since we are using 8.1.7.4, we can only do range partitioning (not hash) on an IOT. But we can work around that. We assume that would help us even more with reducing physical reads?

Thank you very much for any insights. 

Tom Kyte
June 01, 2005 - 7:55 am UTC

1) because all DML is processed via consistent reads with current mode gets. To reduce the amount of current mode gets we have to do since they are the expensive gets (how many people can get a current mode read image of a block at the same time....)

2) but it blows your data model, member_id,newsletter_id is no longer unique.

but "dominant" does not imply "huge problem", just implies "biggest". Is this a "big problem".


Yes, it would seem logical that by placing all of the newly inserted rows into the same set of blocks it would reduce the number of blocks modified by the insert and hence reduce the number of blocks that need be present in the cache to do the insert.

Are you prepared to modify every query to include org_id in the predicate?

3) doubtful, putting org_id on the leading edge would accomplish that -- I'd assume that the first 3 levels of the index were very well cached typically -- partitioning MIGHT decrease the height of the index.

Are you really prepared to modify every query to include org_id in the predicate?




And why massive single row inserts?? Why no BULK operations?

alter table IOT move consume 10GB PGA

jianhui, September 23, 2005 - 2:22 pm UTC

Hi Tom,
I have an IOT around 100MB, plus indexes total < 300MB. When I run 'ALTER TABLE ... MOVE', the process eventually gets ORA-03113, and 0RA-04030 error found in user dump file. I constantly check v$process for PGA usage of this session, it keeps growing and reaches 10GB before the process fails.

Why such a small table requires 10GB+ PGA memory, does it look like a bug?

Sincerely

Tom Kyte
September 23, 2005 - 8:50 pm UTC

Need example, I have a 200mb IOT:

ops$tkyte@ORA10G> exec show_space( 'SYS_IOT_TOP_80546', user, 'INDEX');
Unformatted Blocks .....................               0
FS1 Blocks (0-25)  .....................               0
FS2 Blocks (25-50) .....................               1
FS3 Blocks (50-75) .....................               0
FS4 Blocks (75-100).....................               0
Full Blocks        .....................          24,212
Total Blocks............................          24,576
Total Bytes.............................     201,326,592
Total MBytes............................             192
Unused Blocks...........................             164
Unused Bytes............................       1,343,488
Last Used Ext FileId....................               4
Last Used Ext BlockId...................          66,441
Last Used Block.........................             860
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10G> @mystat "session pga memory max"
ops$tkyte@ORA10G> set echo off
 
NAME                                                    VALUE
-------------------------------------------------- ----------
session pga memory max                                1874508
 
ops$tkyte@ORA10G> alter table iot move;
 
Table altered.
 
ops$tkyte@ORA10G> @mystat2
ops$tkyte@ORA10G> set echo off
 
NAME                                                        V
-------------------------------------------------- ----------
DIFF
------------------
session pga memory max                              118790732
     116,916,224
 

<b>and did not need 10gb of pga?</b>
 

Quantifying IOT benefits

Duke Ganote, September 26, 2005 - 10:14 am UTC

An August 24, 2004 reader asked: "How do I go about methodically verifying and quantifying the benefit of doing this? Run all the queries that use the table as a regular table and a IOT, compare the LIOs, elapsed time, etc? Do you have any guidelines?"

I had exactly the same question as I read your Expert 1-on-1 chapter/section on IOTs last night. The usually recommended technique of counting LIOs doesn't seem like it would work (e.g. </code> http://asktom.oracle.com/pls/ask/f?p=4950:8:16781446805926749277 <code>) because -- as I recall -- the book says that LIOs won't change. Am I correct in guessing there is no simple means of quantifying the benefits of IOTs without falling back on raw timings on a single user system ??

Tom Kyte
September 26, 2005 - 11:00 am UTC

not sure what you mean? you can measure LIO??? that is part of why you would test with an IOT - to cluster your data together and reduce significantly the LIO's if you can???


Not sure what you mean?

IOT LIO PIO, hoist the grog!

Duke Ganote, September 27, 2005 - 8:57 am UTC

Hmm, that's what I get for reading late into the night. Now I have the book before me. In 1-on-1 (p. 215) you wrote "Having all of the addresses.... physically located near each other will reduce the amount of I/O... The logical I/O would be the same, the physical I/O could be significantly less." I didn't find any LIO/PIO measurements in the IOT section, so when I read the answer you gave to the August 24, 2004 reader...it seemed to confirm my (groggy) impression.

I subsequently found </code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:627128171401 <code>and ran some confirmations that LIO and PIO are measureably different for IOT and heap tables. Thank you!

What table type to use?

Mobra, December 07, 2005 - 10:53 am UTC

Hi Tom,

I went to your seminar in Norway the other day, and I really enjoyed it.

In the seminar, you talked a bit about different table types: heap, iot, and hash cluster.

Let's say I have a table DISTANCE that contains distances (in, say, miles) from A to B (between two cities).

create table distance
(from varchar2(20),
to varchar2(20),
distance number);

insert into distance (from, to, distance) values ('OSLO', 'ROTTERDAM', 1000);

etc.

The table will be used frequently to look up the distance between two given cities. The table will initially be populated with a lot of combinations (around 4 million combinations), but there will be more inserts as the users add their own distances.

1) What would be the best physical table type to use? I'm guessing an IOT would be better than a hash cluster, since the table will keep growing, and we don't know the max size?

2) What about compression? There will be a lot of repeating data here, so I guess there will be a huge potential for space saving by compression. Could you give an example of the syntax to enable compression (for an IOT, or other table type depending on your answer to 1, above).

Thanks and keep up the good work!

Tom Kyte
December 08, 2005 - 1:09 am UTC

I would likely want to set up a rule that says "city1 < city2" and use an IOT.

index key compression (probably compress 1 to remove the OSLO that would repeat over and over and over) could be useful.


create table dist ( city1 varchar2(), city2 varchar2(), distance number not null,
primary key(city1,city2), check (city1 < city2) ) organization index compress 1;

the reason for city1<city2 is so you can search:

select distance rom dist
where city1 = least(:bv1,:bv2) and city2 = greatest(:bv1,bv2);

and not have to check "both ways"

Can't move IOT..

A reader, January 03, 2006 - 7:52 am UTC

Hi Tom. I am experiencing some problems trying to move a IOT. DB version is 8.1.7. Please, see below:

SQL> select segment_type, count( * )
  2    from dba_segments 
  3   where tablespace_name = 'PVD' 
  4   group by segment_type;

SEGMENT_TYPE         COUNT(*)
------------------ ----------
INDEX                       2
SPACE HEADER                1

SQL> select 'alter index ' || owner || '.' || index_name || ' rebuild tablespace pvd_tmp;'
  2    from dba_indexes
  3   where tablespace_name = 'PVD';

'ALTERINDEX'||OWNER||'.'||INDEX_NAME||'REBUILDTABLESPACEPVD_TMP;'
----------------------------------------------------------------------------------------------------
alter index PV.SYS_IOT_TOP_136545 rebuild tablespace pvd_tmp;
alter index PV.SYS_IOT_TOP_136550 rebuild tablespace pvd_tmp;

SQL> alter index PV.SYS_IOT_TOP_136545 rebuild tablespace pvd_tmp;
alter index PV.SYS_IOT_TOP_136545 rebuild tablespace pvd_tmp
*
ERROR at line 1:
ORA-28650: Primary index on an IOT cannot be rebuilt

As per your suggestion, I tried:


SQL> alter table PV.SYS_IOT_TOP_136545 move online tablespace pvd_tmp;
alter table PV.SYS_IOT_TOP_136545 move online tablespace pvd_tmp
               *
ERROR at line 1:
ORA-00942: table or view does not exist

But received error messages.. could you please tell me what else should I try or is this the normal behaivour ? What I am trying to achieve, as you may notice, is to move this table to another tablespace.

Thank you 

Tom Kyte
January 03, 2006 - 8:24 am UTC

use the name of your IOT. As demonstrated above....

Thanks!

A reader, January 03, 2006 - 10:35 am UTC

Tom, Thanks.

And sorry to bother you again, but do you know how can I 'move' this SPACE HEADER object ? I have never heard of it before neither I can't seem to find any info about it on the dba views.

Thanks again!

Tom Kyte
January 03, 2006 - 10:52 am UTC

SPACE HEADER segments are the extent bitmaps, you'll see them after you "migrate to local". You cannot touch them.

A reader, May 15, 2006 - 10:31 pm UTC

Is there a way to MOVE/COALESCE a partitioned IOT( am using a range partitioned one) in 9i?

Thanks






Tom Kyte
May 16, 2006 - 6:58 am UTC

ops$tkyte@ORA10GR2> CREATE TABLE t
  2  (
  3    dt  date primary key,
  4    x   int,
  5    y   varchar2(25)
  6  )
  7  organization index
  8  PARTITION BY RANGE (dt)
  9  (
 10    PARTITION part1 VALUES LESS THAN (to_date('13-mar-2003','dd-mon-yyyy')) ,
 11    PARTITION part2 VALUES LESS THAN (to_date('14-mar-2003','dd-mon-yyyy')) ,
 12    PARTITION junk VALUES LESS THAN (MAXVALUE)
 13  )
 14  /

Table created.

ops$tkyte@ORA10GR2> insert into t select to_date( '01-mar-2003' ) + rownum, rownum, rownum from all_users;

34 rows created.

ops$tkyte@ORA10GR2> select count(*) from t partition(part1);

  COUNT(*)
----------
        11

ops$tkyte@ORA10GR2>
ops$tkyte@ORA10GR2> alter table t move partition part1;

Table altered.

 

Statistics on an IOT

Rennie, May 16, 2006 - 10:20 am UTC

Tom,

Does it make sense to gather statistics on an IOT? Wouldn't gathering statistics on the primary key suffice?

IOT always gets accessed through its primary key, right? I cannot build/force an access plan without the PK ... and hence the question.

Thanks,
Rennie

Tom Kyte
May 16, 2006 - 10:56 am UTC

Yes, you want statistics on IOT's (index organized tables)

IOT's can have secondary indexes.
IOT's have a "size"
IOT's can range scan, index full scan or fast full scan - the optimizer wants information regarding the structure in order to determine which one to do.

Since you are always using the CBO (cost based optimmizer) with IOTs - it is really pretty crucial to have statistics in place.

A reader, May 16, 2006 - 12:50 pm UTC

The statement from the beginning of the thread:

"because we know that this empty space in the indexes will not be reused (unlike, when it is reused in table blocks after PCTUSED threshold met in the block)."

Is that statement strictly true? A key has a specific place in an index and if its the same key being inserted again, wont the space be reused?

Thanks

Tom Kyte
May 16, 2006 - 1:44 pm UTC

it is not true - deleted space would be reused in an index structure by any key that happened to want to be located on that block.

Cardinality of a partitioned IOT

A reader, May 21, 2006 - 9:00 pm UTC

Why is the cardinality off for the partitioned IOT?

SQL> CREATE TABLE tt
  2  (
  3     a int,
  4     b   int,
  5     c int,
  6      d   varchar2(25),
  7      e int,
  8      constraint pk_tt primary key(a,b,c,d,e)
  9    )
 10    organization index compress 3
 11    PARTITION BY RANGE (b)
  (
 12   13      PARTITION part1 VALUES LESS THAN (2) ,
 14      PARTITION part2 VALUES LESS THAN (3) ,
 15      PARTITION junk VALUES LESS THAN (MAXVALUE)
 16    )
 17  /

Table created.

Elapsed: 00:00:00.13

SQL> insert into tt select 1,1,1,to_char(trunc(dbms_random.value(1,999))),trunc(dbms_random.value(1,999))
from all_users;
  2
35 rows created.

Elapsed: 00:00:00.43
SQL> insert into tt select 2,2,2,to_char(trunc(dbms_random.value(1,999))),trunc(dbms_random.value(1,999))
from all_users;  2

35 rows created.

Elapsed: 00:00:00.43

SQL> commit;

Commit complete.

Elapsed: 00:00:00.00
SQL> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.57
SQL> set autot traceonly exp stat;
SQL> select * from tt where a=1;

35 rows selected.

Elapsed: 00:00:00.39

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=35 Bytes=560)
   1    0   PARTITION RANGE (ALL)
   2    1     INDEX (RANGE SCAN) OF 'PK_TT' (UNIQUE) (Cost=1 Card=35 B
          ytes=560)





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          6  consistent gets
          0  physical reads
          0  redo size
       1828  bytes sent via SQL*Net to client
        673  bytes received via SQL*Net from client
          4  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         35  rows processed
SQL> insert into tt select 1,1,1,to_char(trunc(dbms_random.value(1,999))),trunc(dbms_random.value(1,999))
from all_users;  2
/
35 rows created.


Elapsed: 00:00:00.43

SQL> /

35 rows created.

Elapsed: 00:00:00.40

SQL> /

35 rows created.

Elapsed: 00:00:00.40

SQL> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.58
SQL> select * from tt where a=1;

140 rows selected.

Elapsed: 00:00:00.39

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=88 Bytes=1408
          )

   1    0   PARTITION RANGE (ALL)
   2    1     INDEX (RANGE SCAN) OF 'PK_TT' (UNIQUE) (Cost=1 Card=88 B
          ytes=1408)





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         13  consistent gets
          0  physical reads
          0  redo size
       4985  bytes sent via SQL*Net to client
        750  bytes received via SQL*Net from client
         11  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        140  rows processed


After another set of insert, commit and stats:

SQL> select * From tt where a=1;

210 rows selected.

Elapsed: 00:00:00.39

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=123 Bytes=196
          8)

   1    0   PARTITION RANGE (ALL)
   2    1     INDEX (RANGE SCAN) OF 'PK_TT' (UNIQUE) (Cost=1 Card=123
          Bytes=1968)





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         17  consistent gets
          0  physical reads
          0  redo size
       6966  bytes sent via SQL*Net to client
        794  bytes received via SQL*Net from client
         15  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        210  rows processed
 

Tom Kyte
May 22, 2006 - 7:37 am UTC

You must be using 9i or before.  The default method opt there didn't pick up histograms.  That this is an IOT (or even a partitioned table), with other columns and a primary is a red herring, not relevant to the problem at all.

Consider in 9i:



ops$tkyte@ORA9IR2> CREATE TABLE tt
  2  (
  3     a int
  4  )
  5  /
 
Table created.
 
ops$tkyte@ORA9IR2> insert into tt
  2  select 1
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA9IR2> insert into tt
  2  select 2
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA9IR2> commit;
 
Commit complete.
 
ops$tkyte@ORA9IR2> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select * from tt where a=1;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=35 Bytes=105)
   1    0   TABLE ACCESS (FULL) OF 'TT' (Cost=2 Card=35 Bytes=105)
 
 
 
ops$tkyte@ORA9IR2> set autotrace off
ops$tkyte@ORA9IR2> select count(*), count(distinct a), count(*)/count(distinct a) guess,
  2         count(case when a=1 then 1 end) count_a
  3    from tt;
 
  COUNT(*) COUNT(DISTINCTA)      GUESS    COUNT_A
---------- ---------------- ---------- ----------
        70                2         35         35
 
<b>that is what the optimizer had to work with given your dbms_stats command - it knows

a) 70 rows
b) two values of A
c) therefore about 35 rows will be returned.... (the guess)</b>

ops$tkyte@ORA9IR2> insert into tt
  2  select 1
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA9IR2> /
 
35 rows created.
 
ops$tkyte@ORA9IR2> /
 
35 rows created.
 
ops$tkyte@ORA9IR2> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select * from tt where a=1;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=88 Bytes=264)
   1    0   TABLE ACCESS (FULL) OF 'TT' (Cost=2 Card=88 Bytes=264)
 
 
 
ops$tkyte@ORA9IR2> set autotrace off
ops$tkyte@ORA9IR2> select count(*), count(distinct a), count(*)/count(distinct a) guess,
  2         count(case when a=1 then 1 end) count_a
  3    from tt;
 
  COUNT(*) COUNT(DISTINCTA)      GUESS    COUNT_A
---------- ---------------- ---------- ----------
       175                2       87.5        140

<b>Now, it knows:

A) 175 rows
B) still two distinct values for A
C) therefore ABOUT 88 rows will be returned. 

it did not know "140" because the default method_opt is "size 1" - no histograms</b>
 
ops$tkyte@ORA9IR2> insert into tt
  2  select 1
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA9IR2> /
 
35 rows created.
 
ops$tkyte@ORA9IR2> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select * From tt where a=1;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=123 Bytes=369)
   1    0   TABLE ACCESS (FULL) OF 'TT' (Cost=2 Card=123 Bytes=369)
 
 
 
ops$tkyte@ORA9IR2> set autotrace off
ops$tkyte@ORA9IR2> select count(*), count(distinct a), count(*)/count(distinct a) guess,
  2         count(case when a=1 then 1 end) count_a
  3    from tt;
 
  COUNT(*) COUNT(DISTINCTA)      GUESS    COUNT_A
---------- ---------------- ---------- ----------
       245                2      122.5        210

<b>you get the picture by now....

Now, in 10g, you would see something totally different.  This is due to what dbms_stats defaults to - size auto.

See
https://www.oracle.com/technetwork/issue-archive/2006/06-may/o36asktom-099107.html
"Why Does My Plan Change?"
for what is going on under the covers here - but basically by runnin the first query with "where a=1", you told the database "I query by A" and the database would on subsequent gathers - collect histograms on A if it wants to (and it did).  So, in 10g, the first run gets the right cardinality ABSOLUTELY BY ACCIDENT - the rest are right "on purpose" because of the histograms</b>


ops$tkyte@ORA10GR2> CREATE TABLE tt
  2  (
  3     a int
  4  )
  5  /
 
Table created.
 
ops$tkyte@ORA10GR2> insert into tt
  2  select 1
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA10GR2> insert into tt
  2  select 2
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA10GR2> commit;
 
Commit complete.
 
ops$tkyte@ORA10GR2> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10GR2> set autotrace traceonly explain
ops$tkyte@ORA10GR2> select * from tt where a=1;

<b>that query will be remembered by the database - and that it was executed will affect FUTURE gather stats..</b>
 
Execution Plan
----------------------------------------------------------
Plan hash value: 264906180
 
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |    35 |   105 |     3   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| TT   |    35 |   105 |     3   (0)| 00:00:01 |
--------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   1 - filter("A"=1)
 
ops$tkyte@ORA10GR2> set autotrace off
ops$tkyte@ORA10GR2> select count(*), count(distinct a), count(*)/count(distinct a) guess,
  2         count(case when a=1 then 1 end) count_a
  3    from tt;
 
  COUNT(*) COUNT(DISTINCTA)      GUESS    COUNT_A
---------- ---------------- ---------- ----------
        70                2         35         35
 
ops$tkyte@ORA10GR2> insert into tt
  2  select 1
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA10GR2> /
 
35 rows created.
 
ops$tkyte@ORA10GR2> /
 
35 rows created.
 
ops$tkyte@ORA10GR2> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10GR2> set autotrace traceonly explain
ops$tkyte@ORA10GR2> select * from tt where a=1;
 
Execution Plan
----------------------------------------------------------
Plan hash value: 264906180
 
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |   140 |   420 |     3   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| TT   |   140 |   420 |     3   (0)| 00:00:01 |
--------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   1 - filter("A"=1)
 
ops$tkyte@ORA10GR2> set autotrace off
ops$tkyte@ORA10GR2> select count(*), count(distinct a), count(*)/count(distinct a) guess,
  2         count(case when a=1 then 1 end) count_a
  3    from tt;
 
  COUNT(*) COUNT(DISTINCTA)      GUESS    COUNT_A
---------- ---------------- ---------- ----------
       175                2       87.5        140

<b>We get the "right" cardinality becaue of the histograms</b>
 
ops$tkyte@ORA10GR2> insert into tt
  2  select 1
  3    from (select 1 from dual connect by level <= 35);
 
35 rows created.
 
ops$tkyte@ORA10GR2> /
 
35 rows created.
 
ops$tkyte@ORA10GR2> exec dbms_stats.gather_table_stats( user, 'TT',cascade=>true);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10GR2> set autotrace traceonly explain
ops$tkyte@ORA10GR2> select * From tt where a=1;
 
Execution Plan
----------------------------------------------------------
Plan hash value: 264906180
 
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |   210 |   630 |     3   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| TT   |   210 |   630 |     3   (0)| 00:00:01 |
--------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   1 - filter("A"=1)
 
ops$tkyte@ORA10GR2> set autotrace off
ops$tkyte@ORA10GR2> select count(*), count(distinct a), count(*)/count(distinct a) guess,
  2         count(case when a=1 then 1 end) count_a
  3    from tt;
 
  COUNT(*) COUNT(DISTINCTA)      GUESS    COUNT_A
---------- ---------------- ---------- ----------
       245                2      122.5        210

 

A reader, May 22, 2006 - 12:42 pm UTC

Thanks

partition tables with many columns composite PK

A reader, June 06, 2006 - 1:08 pm UTC

Hi

I am reviewing an application design, there are many partition tables. The primary key of these partitioned tables are made up many many columns.

For example a table with 7 columns (one of them is BLOB), the PK is composed of 6 columns (except the BLOB field). Is that a good candidate to make it IOT?
There is another table with 12 columns and PK again is composed of 8 columns, not sure if this is another candidate?

These tables are used to store historical data. HP-UX running Oracle 9.2.0.7

RIMMA

Tom Kyte
June 06, 2006 - 1:59 pm UTC

likely answer is "yes" for that one, since the only non-key column is the blob (and they are small when stored out of line)

IOT and loading process

A reader, June 07, 2006 - 6:45 am UTC

Hi

Since IOT are B-Tree structured therefore during a load process I guess it will be slower than a heap table without index correct?

Tom Kyte
June 07, 2006 - 7:14 am UTC

probably slower than a table with an index even.

IOT's, clusters, partitioning - anything that forces "data to go somewhere" is likely to be slower to load.

They are mostly about enhancing data retrieval

Sizing of a partitioned IOT

Abu Ahmadh, June 26, 2006 - 11:02 am UTC

Tom,

How to calculate the space used by a partitioned IOT (not the entire partition, just 1 partition only)? Should the size of overflow segment be considered?



Tom Kyte
June 26, 2006 - 11:06 am UTC

I only know one way to size.

load it, measure it, multiply it up if needed. It'll be "close" but not 100% accurate. Nothing will (except for hindsight)

</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:406873025890 <code>

and yes, you will of course want to measure the amount of space needed for the overflow segment.

Analyze partitioned-IOT?

Abu Ahmadh, July 04, 2006 - 12:43 pm UTC

Tom,

A few questions.

1) What is the best approach to analyze a 1 billion rows partitioned-IOT in 10g?

2) Is there a way to measure the used space for a given partition without analyzing the partitioned-IOT?

3) Is it recommended to have 10 columns as part of PK for an IOT table?

Thanks

Tom Kyte
July 07, 2006 - 9:00 pm UTC

1) there are two answers for every question

a) why
b) depends

I think b) most applies here. How big is "1 billion rows"? Do you always query precisely one partition (and does the optimizer KNOW that at parse time?)

It might be... Never analyze it, use dbms_stats to SET THE STATS (because you know them well enought)

It might be... A 3% sample should be sufficient...

It might be... Well, that is small enought, just compute them.

2) *_segments will show you the allocated space for any segment (partitions are segments). If you want, you can also use dbms_space (search for show_space for a script I use)

3) we do not recommend using 1 column or 10 columns. It is not really a recommendation thing - it is more of a "does it really make sense in the grand scheme of things" thing.

So, does it?

To IOT or not to IOT

Guy Lambregts, July 28, 2006 - 9:46 am UTC

Dear Tom,

Since I have your book "Effective Oracle by design" and since I used to read the documentation I am aware of the advantages and concept of IOT's.
As a DBA I am involved in the design of very large OLTP databases, I can decide whether a table will be heap either index organized.
In our databases I notice a lot of IOT candidates because of

1. limited row length
2. table access always or almost always by the primary key
3. No updates of the primary key (off course not) almost never updates of the other columns

However

1. insert frequency & concurrency is rather high, let' s say 1 insert per [ 5 sec - 20 sec ]
2. delete frequency will be lower but number of deleted rows will be important ( will be done in batch )
2. since our Java Middle Tier team didn' t want composite primary keys, my candidate IOT's have a sequence based primary key.
3. number of rows in the range [ 5.000.000 - 50.000.000 ]

I think in many cases I can still go for IOT since, and this is my - possible bad - opinion

1. I can move online the IOT with the "alter table iot_name move online" or rebuild with "alter table iot_name coalesce" clause which does not invalidate secundary indexes. ( if the table is not partitioned )
either with the "alter table iot_name modify partition partition_name coalesce" ( if IOT is partitioned )
I think there will be the need to rebuild the IOT now and then since the sequence based primary key "trigger" a "right hand" IOT and as such delete entries at the left site won' t be reused.
What is your opinion please ?

2. Am I right when I think the hash partitioning can partially reduce the hot spot at the right site of the IOT, since there will be n hot blocks ( where n is the number of partitions )
What is your opinion please ?

3. After hours of sql tuning in test databases I believe in hash partitiong and the partition wise join because of equi partitioning of the join columns
(I am aware of the difference between hash and range partitioning with regard to partition wise join)
I have the join between the IOT and heap organized table which are BOTH hash partitioned on the join column.
What is your opinion please ?

4. With IOT' s I always save disk space.
I guess you will always agree with that

5. In your book "Effective Oracle by design" at page 404 I read : "... there are issues with very wide tables in an IOT structure ..."
Let' s say we keep 20.000.000 rows with an avg row length of 100 up to max 200 bytes. Is that a too big for an IOT ? I think no
What is your opinion please ?

6a. Is an insert frequency of let' s say 1 per 10 seconds too high for an IOT ? I think no
6b. Is an insert frequency of let' s say 1 per second too high for an IOT ? I think possible yes
What is your opinion please ?

The motivation behind my question is the lack of practical experience with IOT whereas my limited theoretical knowledge say, ... go for it.

As always thank you in advance and

Respectfull Greetings

Guy Lambregts

Tom Kyte
July 28, 2006 - 8:10 pm UTC

1) if you have a sweeper index (sweeps from left to right - always deleting the old stuff and adding new) AND you have to find the oldest record (select min(id) from iot) - you might find that query start taking more IO's over time.

If you always "where id = :x" and don't really range scan looking for the oldest, you won't have to reorganize since you won't ever read over any of the dead left hand blocks.

2) yes, that is true.

3) don't know what you wanted me to comment on there.

4) I would say "probably", not always.

5) that is not too big for an IOT.

6a) no
6b) no


OK

Kumar, December 19, 2006 - 4:35 am UTC

Hello Tom,
Any v$ views or x$ tables exist specific to
Index Organized Tables?

Tom Kyte
December 19, 2006 - 8:17 am UTC

specific? not really - what are you looking for precisely?

IOT with UNIQUE not PRIMARY KEY

Nathan, April 13, 2007 - 11:45 am UTC

I have an extremely large partitioned table (150 Million rows, lowest detail level in a Scorecard engine) that has a unique key & index, but does not qualify for a Primary Key (some of the key fields are NULL).

All queries against this table currently hit the unique index - none go against the table. As such, the table (and associated 20 Gig of space) is useless.

Since the index is currently the only usefull structure, is there a way that I can eleminate the underlying table? Is there some way to create an IOT without an associated Primary Key index?
Tom Kyte
April 13, 2007 - 2:12 pm UTC

no, there is not.

maybe you would be a candidate for a hash cluster (lose the index), however - you would not be able to partition that.

Thanks Tom

Nathan, April 13, 2007 - 2:40 pm UTC

Thanks for your reply!

Unfortunatly a hash cluster has a limited nubmer of keys and doesn't do well with range scans (which I have many).

I guess I will have to convert the NULL values to something else so that I can use IOTs, and modify all of the retrieval code as necessary.

I just can't stand the thought of that much wasted space...
Tom Kyte
April 13, 2007 - 7:15 pm UTC

a hash cluster has a pre-defined size, yes.

but it does "range scans" just dandy- don't know what you mean there - you can have conventional indexes on a hash cluster.



IOT Vs Hash Clusters

prasanna, July 18, 2007 - 4:22 pm UTC

Hi Tom,

This site rocks!
Some of the tables in our application have the following pattern of columns (just mentioning the relevant ones):

Table test:

col_a
col_b
date1
date2
flag

This is right now a heap table with PK being (col_a,col_b) and unique index on (col_a,date1,date2,flag)
Most of the queries surrounding these tables are accessed via col_a,date1,date2,flag with a range scan...
Like:
SELECT col_a,col_b FROM test
WHERE col_a = (join with another table)
AND (V_Calendar_Dt BETWEEN date1 AND date2)
AND Flag = 'Y'

I dont have a problem with single table access on these tables...
But, I find an enormous performance hit in cases where I have to join these tables to other heap tables just like shown above...

Volume wise, these tables are small with the max record table containing about 100000 records ...
But, when they are joined to other bigger heap tables(which run into millions of records), then it really is a grind to get data back quickly...

I was also considering to convert them to Hash clusters , but, based on the forum discussions here, looks like hash cluster is not a good option since I will
seldom be querying based on the PK...

But, are these kind of tables good candidates for
IOT instead of heap...since, I need to range scan them as well...
based on "AND (V_Calendar_Dt BETWEEN date1 AND date2)" in the query...

Any pros/cons that can be listed would be much help...

Thanks...
Tom Kyte
July 19, 2007 - 10:37 am UTC

your query doesn't make sense to me.

"where col_a = (join with another table)" ???

IOT

prasanna, July 19, 2007 - 10:40 pm UTC

Bad query example...
My apologies...
Here it is:
"test" is as explained before...
assume test1 to have col_a to be the PK...

SELECT a.col_a,a.col_b,a.some_other_columns
FROM test a,test1 b
WHERE a.col_a = b.col_a
AND (:Calendar_Dt BETWEEN a.date1 AND b.date2)
AND a.Flag = 'Y'

I did try this out by creating the "test" as IOT and then executed the above query (please note that test1 is still a heap here)...
But, considering all other things such as data returned and cpu time to be equal, I found IOT version query to perform more LIOs than when I tried with both tables being heaps...!

What am I missing here...

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=82 Bytes=4059
0)

1 0 INDEX (UNIQUE SCAN) OF 'SYS_IOT_TOP_59831' (UNIQUE) (Cost=
1 Card=82 Bytes=40590)

2 1 INDEX (RANGE SCAN) OF 'XAK1CFD_IOT' (UNIQUE) (Cost=1 Car
d=33)

Here , XAK1CFD_IOT is the secondary unique index that I created on col_a,date1,date2,flag.(This definitely needs to be enforced in our application)
SYS_IOT_TOP_59831 is the PK on col_a,col_b...

Now, I do know that when it comes to secondary indexes, IOTs are a dicey...But, i have to have that secondary unique index ...no helping that...
Any pros/cons that can be listed as to where exactly I might see a gain in using IOT/index clustered tables/hash clusters would be much help...



Tom Kyte
July 20, 2007 - 8:14 am UTC

well, basically, there is no good way to "organize" this data to optimize that query.

your predicate (the thing that'll reduce data) is not particularly "location friendly"

You have a predicate on a.flag = 'Y', period.

You have a join on a.col_a to b.col_a (no filters)

you have a predicate that can be applied after the join - well, given the database will really process:

where a.col_a = b.col_a
  and a.date1 <= :c
  and b.date2 >= :c
  and a.flag = 'Y'



so, there is the opportunity for an index on a.flag,a.date1 to perhaps be useful to local rows in A to join to B - and an index on B(col_a,date2) might be useful .

So, if A and B were organized by those "keys" - a(flag,date1), b(col_a,date2), it MIGHT be useful, but doubtful - the predicate that crosses the table

IOT

prasanna, July 20, 2007 - 12:02 pm UTC

Hi Tom,

I am really sorry for the typo in my query from my end that caused a mis-interpretion from you regarding the relationship between the 2 tables : test,test1

"test" as before has columns:
col_a
col_b
date1
date2
flag

col_a and col_b being PK...
col_a,date1,date2,flag have a unique index on them...

"test1" just has one main column(in addition to other columns that are usual attributes) col_a to be the PK...
Its more like a parent child relation with test1.col_a being referenced as a foreign key in test...

But, the correct query is:

SELECT a.col_a,a.col_b,a.some_other_columns
FROM test a,test1 b
WHERE a.col_a = b.col_a
AND (:Calendar_Dt BETWEEN a.date1 AND a.date2)
AND a.Flag = 'Y'

Please note the change here: both date1 and date2 belong to test...

Now, can you please answer my earlier question if this is a possible candidate for IOT?
Can test1 be a heap if the above is possible or should it be some thing like a hash cluster where predominantly things get accessed by PK(test1.col_a)?

But, LIO wise, I could not see much of a difference in performance between test being a heap or IOT...
Our main performance hits so far have stemmed from the fact when tables like test,test1 are joined with other huge volume intensive tables that run in millions...

Left alone , just a simple join among test and test1 are fine ...but, we seldom use them like that...
Tom Kyte
July 20, 2007 - 5:12 pm UTC

i already did answer. probably NOT.

Can IOT take more space than heap table

mohnish, November 06, 2007 - 8:13 pm UTC

Hi ,
I am new to Oracle and was wondering can IOT take up more space than heap table. 
I have a situation. 

SQL> select count(*) from dm_message_route_s;

  COUNT(*)
----------
    353401

SQL> select num_rows,blocks,empty_blocks,avg_row_len,avg_space,num_freelist_bloc
ks from user_tables where table_name='DM_MESSAGE_ROUTE_S';

  NUM_ROWS     BLOCKS EMPTY_BLOCKS AVG_ROW_LEN  AVG_SPACE NUM_FREELIST_BLOCKS
---------- ---------- ------------ ----------- ---------- -------------------
    353401       2346           86          62         68                   0


SQL> select sum(blocks)  from (select extent_id,bytes,blocks from user_extents w
here segment_name='DM_MESSAGE_ROUTE_S' and segment_type='TABLE');

SUM(BLOCKS)
-----------
       2432

Space used by table data only = 2346 * 8192 - 2346*68 = 19072980 = 18Mb.

SQL> create table dm_message_route_iot(r_object_id,addr_id,message_object_id, route_bit_flags,route_type,i_is_replica,i_vstamp, constraint pk_route_iot primary key(r_object_id)) organization index overflow tablespace dm_oraperf_docbase as (
select * from dm_message_route_s);

SQL> select bytes,blocks,extents from user_segments where segment_name='PK_ROUTE
_IOT'
  2  ;

     BYTES     BLOCKS    EXTENTS
---------- ---------- ----------
  28311552       3456         42

Space allocated to the index segment not necessarily the space on disk = 27Mb.

SQL> select num_rows,blocks,empty_blocks,avg_row_len,avg_space,num_freelist_bloc
ks from user_tables where table_name='DM_MESSAGE_ROUTE_IOT';

  NUM_ROWS     BLOCKS EMPTY_BLOCKS AVG_ROW_LEN  AVG_SPACE NUM_FREELIST_BLOCKS
---------- ---------- ------------ ----------- ---------- -------------------
    353401                                  64
Am I doing this right. I know I am calculating the IOT size from index, is this the right way. 


Tom Kyte
November 07, 2007 - 5:08 pm UTC

an iot could take more room than a heap, a heap could take more room than an iot, in short "x can take more room then y" - plug in anything you want for X and Y, it'll hold true.


you don't tell us how the tablespaces involve here allocate space - for all we know, the index allocated an extent that is almost entirely empty - and the used space is less than the table.

But, why didn't you add in the size of the primary key index on the table? The IOT = HEAP+INDEX-ON-PRIMARY-KEY


Size of IOT

Mohnish, November 13, 2007 - 10:57 pm UTC

Hi,
You said that IOT= HEAP+INDEX ON PRIMARY KEY.
and asked me why I didnt add the size of the primary key.
Doesnt the query select bytes,blocks,extents from user_segments where segment_name='PK_ROUTE
_IOT'

Do exactly that where PK_ROUTE_IOT was the primary key on the table.
Is the value that I get 27Mb right value.
Can you give me an example so that I can learn to calculate the size properly?

thanks
Mohnish
Tom Kyte
November 19, 2007 - 4:35 pm UTC

that got the IOT.

I'm saying you need to compare

sum( heap table + index used to support primary key on heap table)

versus

sizeof( IOT )

ops$tkyte%ORA10GR2> create table t_heap (
  2  OWNER, OBJECT_NAME, SUBOBJECT_NAME, OBJECT_ID, DATA_OBJECT_ID, OBJECT_TYPE, CREATED,
  3  LAST_DDL_TIME, TIMESTAMP, STATUS, TEMPORARY, GENERATED, SECONDARY, constraint t_pk primary key(object_id) )
  4  as
  5  select * from all_objects;

Table created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> create table t_iot (
  2  OWNER, OBJECT_NAME, SUBOBJECT_NAME, OBJECT_ID, DATA_OBJECT_ID, OBJECT_TYPE, CREATED,
  3  LAST_DDL_TIME, TIMESTAMP, STATUS, TEMPORARY, GENERATED, SECONDARY, primary key(object_id) )
  4  organization index
  5  as
  6  select * from t_heap;

Table created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> select segment_name, segment_type, blocks from user_segments ;

SEGMENT_NAME                   SEGMENT_TYPE           BLOCKS
------------------------------ ------------------ ----------
SYS_IOT_TOP_56193              INDEX                     768
T_PK                           INDEX                     120
T_HEAP                         TABLE                     768



see, it takes the HEAP table PLUS the index on the primary key of the heap table

to be the same as the IOT all by itself.

why IOT does not need a phyiscal(normal) rowid, but urowid?

jian huang zheng, March 07, 2009 - 4:15 am UTC

Hi Tom

I am wondering why a IOT table does not need a normal rowid, but has a urowid?
Is it because rowid only exists for a heap table?

Thanks!
Tom Kyte
March 07, 2009 - 12:46 pm UTC

In a heap table, when you insert a row it goes into a specific block, in a certain file, in a certain slot on that block. Rarely, an update to that row might cause it to migrate - rarely. So normally a row in a heap table has a very very very stable address, it rarely if ever moves. So, in a heap table, we use a rowid - and if the row does migrate - we leave a forwarding address in the original location (the rows rowid does not change) and move the row. If we ever move the row again - we just change the forwarding address again. It is very rare for a row to migrate (in general) so the overhead of maintaining the forwarding address is almost non-existent.


In a index however, rows move all of the time, every time a leaf block fills up - it splits and splits and splits and splits. (index) rows move all of the time - constantly (in general). The maintenance of the forwarding rowid would be huge AND almost every row (in general) would move at some point so the storage overhead would be pretty large too (very large)


rowids exist for tables other than simple heaps - hash clustered and b*tree clustered tables have rowids too.

A reader, March 07, 2009 - 12:53 pm UTC

In our current production environment, we have 10 IOT tables for which 100% stats are gathered every day which takes a couple of hrs. What is the best way to maintain stats for IOTs ?
Tom Kyte
March 07, 2009 - 1:01 pm UTC

why do you gather computer statistics every day?

... What is the best way
to maintain stats for IOTs ? ...

if there were a single answer for that, there would be only one way to do it and we would have hard coded it - why would we have implemented other inferior ways? The answer to "what is the best way" is ALWAYS (100% ALWAYS) - it depends.

A reader, March 23, 2009 - 11:09 am UTC

I also have a requirement to consider about the best method of gathering statistics on IOT tables. From your latest reply above, you said that "it depends". What does this depend on ? What do we have to consider and how do we determine the best method to do stats on IOT tables. Need some help / guidance.

Thanks.
Tom Kyte
March 26, 2009 - 12:52 pm UTC

you didn't answer my question - why do you compute statistics everyday?

what was the thought process behind that, you can apply that same thought process to your IOT, an IOT is a segment - just like a table or index.

Does this IOT change much?

Does the fact that this IOT is modified frequently mean anything? (eg: what if you update it 1,000,000 times a day - but nothing relevant changes, the row counts are the same, the data distributions are the same)

Do you run queries that required/desire/benefit from histograms? Maybe you need to gather them, probably not.

Think about what statistics are. Think about how the statistics on your segments would change from day to day. If you have some tables whereby "they really don't change" (eg: a lookup table, a table that is modified but the relevant facts about it don't really change, ....) - then you probably do not need to gather against that segment very often *if ever again*. On the other hand, if you have tables that are very dynamic - they will need a faster paced gathering scheme - unlikely to be "compute every day", but you would know that if you know how the data changes....


If you are totally "not sure", use the automatic job in 10g, it'll compute or sample only the segments it feels it needs to.

Unique Constraint on IOT

Parthiban Nagarajan, April 27, 2009 - 12:19 am UTC

Hi Tom

I read somewhere that "we cannot create unique constraint in IOTs". { URLs : http://www.dbasupport.com/oracle/ora8/iot.shtml ; http://books.google.co.in/books?id=xxx0KAwY_ZMC&pg=PA184&lpg=PA184&dq=unique+constraint+iot+oracle&source=bl&ots=4DmjBEq-4z&sig=mVrIRgfKGuueEU5lu6dd861NPeA&hl=en&ei=oS31SfjlENCAkQXi2_juCg&sa=X&oi=book_result&ct=result&resnum=5 }But I found it to be false in 9i R2 database. I googled for these things but not able to come to any conclusion. Could you verify the statement ? Is it true for lower versioned databases ?

Thanks for your Follow-Up :)

Transform range partitioned table to Index Organized table

A reader, April 27, 2009 - 2:25 pm UTC

Dear Tom,

1. Is it possible to transform range partitioned table to Index Oraganised table?

Thanks very much


Tom Kyte
April 27, 2009 - 2:45 pm UTC

only by moving every bit and byte, the object has to be entire reorganized.

If you need to do this online, it could look like this - else just a create table as select would do it


ops$tkyte%ORA10GR2> CREATE TABLE t1
  2  (
  3    dt  date,
  4    x   int,
  5    y   varchar2(30)
  6  )
  7  PARTITION BY RANGE (dt)
  8  (
  9    PARTITION part1 VALUES LESS THAN (to_date('01-jan-2008','dd-mon-yyyy')) ,
 10    PARTITION part2 VALUES LESS THAN (to_date('01-jan-2009','dd-mon-yyyy'))
 11  )
 12  ;

Table created.

ops$tkyte%ORA10GR2> insert into t1 select to_date( '12-mar-2003','dd-mon-yyyy')+mod(rownum,3), user_id, username from all_users;

42 rows created.

ops$tkyte%ORA10GR2> alter table t1 add constraint t1_pk primary key(x);

Table altered.

ops$tkyte%ORA10GR2> select count(*) from t1;

  COUNT(*)
----------
        42

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> create table t2 ( dt date, x int primary key, y varchar2(30) ) organization index;

Table created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> exec dbms_redefinition.start_redef_table( user, 'T1', 'T2' )

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> REM other things might need be done here, like getting constraints/grants/etc in place
ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> exec dbms_redefinition.finish_redef_table( user, 'T1', 'T2' );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> drop table t2;

Table dropped.

ops$tkyte%ORA10GR2> select count(*) from t1;

  COUNT(*)
----------
        42

ops$tkyte%ORA10GR2> select dbms_metadata.get_ddl( 'TABLE', 'T1' ) from dual;

DBMS_METADATA.GET_DDL('TABLE','T1')
-------------------------------------------------------------------------------

  CREATE TABLE "OPS$TKYTE"."T1"
   (    "DT" DATE,
        "X" NUMBER(*,0),
        "Y" VARCHAR2(30),
         PRIMARY KEY ("X") ENABLE
   ) ORGANIZATION INDEX NOCOMPRESS PCTFREE 10 INITRANS 2 MAXTRANS 255 LOGGING
  STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT)
  TABLESPACE "USERS"
 PCTTHRESHOLD 50


Secondary Indexes on IOT.

Rajeshwaran, Jeyabal, December 10, 2010 - 3:24 am UTC

rajesh@11GR2> create table t(
  2     owner,
  3     object_name,
  4     object_type,
  5     object_id,
  6     created,
  7     constraint t_pk primary key(object_id)
  8  )organization index
  9  nologging
 10  as
 11  select owner,object_name,object_type,object_id,created
 12  from all_objects;

Table created.

Elapsed: 00:00:14.32
rajesh@11GR2> begin
  2     dbms_stats.gather_table_stats(
  3     ownname=>user,
  4     tabname=>'T',
  5     cascade=>true,
  6     method_opt=>'for all columns size 254',
  7     estimate_percent=>100);
  8  end;
  9  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:03.86
rajesh@11GR2>
rajesh@11GR2> set autotrace traceonly explain statistics;
rajesh@11GR2>
rajesh@11GR2> select * from t
  2  where created between to_date('01/07/2010','dd/mm/yyyy') and sysdate;

862 rows selected.

Elapsed: 00:00:00.01

Execution Plan
----------------------------------------------------------
Plan hash value: 320690585

------------------------------------------------------------------------------
| Id  | Operation             | Name | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT      |      |   987 | 51324 |   163   (1)| 00:00:02 |
|*  1 |  FILTER               |      |       |       |            |          |
|*  2 |   INDEX FAST FULL SCAN| T_PK |   987 | 51324 |   163   (1)| 00:00:02 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss')<=SYSDATE@!)
   2 - filter("CREATED">=TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss') AND "CREATED"<=SYSDATE@!)


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        604  consistent gets
          0  physical reads
          0  redo size
      20471  bytes sent via SQL*Net to client
        470  bytes received via SQL*Net from client
          7  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        862  rows processed

rajesh@11GR2> create index t_ind on t(created) nologging;

Index created.

Elapsed: 00:00:00.92
rajesh@11GR2> select * from t
  2  where created between to_date('01/07/2010','dd/mm/yyyy') and sysdate;

862 rows selected.

Elapsed: 00:00:00.01

Execution Plan
----------------------------------------------------------
Plan hash value: 170811138

----------------------------------------------------------------------------
| Id  | Operation          | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |       |   987 | 51324 |    14   (0)| 00:00:01 |
|*  1 |  FILTER            |       |       |       |            |          |
|*  2 |   INDEX UNIQUE SCAN| T_PK  |   987 | 51324 |    14   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN| T_IND |   987 |       |     5   (0)| 00:00:01 |
----------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss')<=SYSDATE@!)
   2 - access("CREATED">=TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss') AND "CREATED"<=SYSDATE@!)
   3 - access("CREATED">=TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss') AND "CREATED"<=SYSDATE@!)


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        873  consistent gets
          0  physical reads
          0  redo size
      20471  bytes sent via SQL*Net to client
        470  bytes received via SQL*Net from client
          7  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        862  rows processed

rajesh@11GR2>
rajesh@11GR2> drop index t_ind;

Index dropped.

Elapsed: 00:00:00.93
rajesh@11GR2> create index t_ind on t(object_id,created) nologging;

Index created.

Elapsed: 00:00:00.18
rajesh@11GR2>
rajesh@11GR2> set autotrace traceonly explain statistics;
rajesh@11GR2>
rajesh@11GR2> select * from t
  2  where created between to_date('01/07/2010','dd/mm/yyyy') and sysdate;

862 rows selected.

Elapsed: 00:00:00.12

Execution Plan
----------------------------------------------------------
Plan hash value: 320690585

------------------------------------------------------------------------------
| Id  | Operation             | Name | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT      |      |   987 | 51324 |   163   (1)| 00:00:02 |
|*  1 |  FILTER               |      |       |       |            |          |
|*  2 |   INDEX FAST FULL SCAN| T_PK |   987 | 51324 |   163   (1)| 00:00:02 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss')<=SYSDATE@!)
   2 - filter("CREATED">=TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss') AND "CREATED"<=SYSDATE@!)


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
        604  consistent gets
          0  physical reads
          0  redo size
      20471  bytes sent via SQL*Net to client
        470  bytes received via SQL*Net from client
          7  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        862  rows processed

rajesh@11GR2>


Tom:

<quote>
Secondary indexes provide fast and efficient access to index-organized tables using columns that are neither the primary key nor a prefix of the primary key.
</quote>

http://download.oracle.com/docs/cd/E11882_01/server.112/e16508/indexiot.htm#sthref408
I dont find the usefullness of Secondry Indexes on IOT. Can you tell me when this will be really usefull?
Tom Kyte
December 10, 2010 - 10:13 am UTC

you've really got to stop posting huge examples (with no text until the very end) - especially when the examples are not necessary.



guess what would happen in your particular example if the table wasn't an IOT? You are basically saying "I don't see the point in indexes" - period (think about it, that is truly what you are saying)"

Documentation probably also says something " indexes provide fast and efficient access to tables". Is that ALWAYS true? Of course not, nothing is ever "always true" - except for this statment :)

Never say never
Never say always
I Always say




How about trying

select * from iot where created = to_date( '10-dec-2010', 'dd-mon-yyyy');

then you'll see a clear example of where a secondary index would be useful.


secondary index on IOT.

Rajeshwaran, Jeyabal, December 10, 2010 - 11:14 am UTC

Thanks Tom its really helpful to understand.

rajesh@11GR2> select * from t
  2  where created = to_date('01/07/2010','dd/mm/yyyy')
  3  /

no rows selected

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
Plan hash value: 2448192542

---------------------------------------------------------------------------
| Id  | Operation         | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |       |    44 |  2288 |     1   (0)| 00:00:01 |
|*  1 |  INDEX UNIQUE SCAN| T_PK  |    44 |  2288 |     1   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN| T_IND |    44 |       |     1   (0)| 00:00:01 |
---------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("CREATED"=TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss'))
   2 - access("CREATED"=TO_DATE(' 2010-07-01 00:00:00', 'syyyy-mm-dd
              hh24:mi:ss'))


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
          2  consistent gets
          0  physical reads
          0  redo size
        539  bytes sent via SQL*Net to client
        404  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          0  rows processed

Tom Kyte
December 10, 2010 - 11:25 am UTC

don't forget - this does not apply to secondary indexes exclusively - this is about indexes, all kinds, in all cases.

secondary index on IOT

Rajeshwaran, Jeyabal, December 10, 2010 - 12:07 pm UTC

Tom:

this does not apply to secondary indexes exclusively.

what i have created above (T_IND) is acutally a secondary indexes . Index on IOT. I couldn't get you clearly.
Tom Kyte
December 10, 2010 - 12:57 pm UTC

I trying to tell you - there is nothing special about secondary indexes, what you demonstrated applies to *every* index under the planet.

Sometimes Indexes are good.
Sometimes full scans are better.
Sometimes ....

The word sometimes should be silently placed in front of everything you read.


I'm just trying to say that what you learned applies to *every type of index*

Bitmap Index on IOT.

Rajeshwaran, Jeyabal, December 10, 2010 - 12:30 pm UTC

Tom:

This really confuses a lot.

http://download.oracle.com/docs/cd/E11882_01/server.112/e16508/indexiot.htm#sthref408

rajesh@11GR2> create bitmap index t_bitmap on t(deptno);
create bitmap index t_bitmap on t(deptno)
                                *
ERROR at line 1:
ORA-28669: bitmap index can not be created on an IOT with no mapping table


Elapsed: 00:00:00.12
rajesh@11GR2>


Don't know where this mapping table needs to be specified. it that documented in SQL Reference manuals?

http://download.oracle.com/docs/cd/E11882_01/server.112/e17118/statements_5012.htm#i2129648
Tom Kyte
December 10, 2010 - 1:02 pm UTC



right there on the same page you linked to:

http://docs.oracle.com/docs/cd/E11882_01/server.112/e17118/statements_5012.htm#SQLRF53982

is a link to:

"CREATE TABLE for information on mapping tables"


use ctl-f to search for mapping and you would have found it straight away

Bitmap Index on IOT.

Rajeshwaran, Jeyabal, December 11, 2010 - 2:47 am UTC

<quote>
Oracle Database creates the mapping table or mapping table partition in the same tablespace as its parent index-organized table or partition. You cannot query, perform DML operations on, or modify the storage characteristics of the mapping table or its partitions.
</quote>

http://download.oracle.com/docs/cd/E11882_01/server.112/e17118/statements_7002.htm#i2146256

rajesh@11GR2> SELECT table_name,iot_type, uo.object_id
  2  FROM user_tables ut, user_objects uo
  3  where iot_type like 'IOT%'
  4  and uo.object_name = ut.table_name
  5  /

TABLE_NAME                     IOT_TYPE      OBJECT_ID
------------------------------ ------------ ----------
SYS_IOT_MAP_76674              IOT_MAPPING       76675
T                              IOT               76674

Elapsed: 00:00:00.42
rajesh@11GR2>
rajesh@11GR2>
rajesh@11GR2> alter table sys_iot_map_76674 nologging;
alter table sys_iot_map_76674 nologging
*
ERROR at line 1:
ORA-28668: cannot reference mapping table of an index-organized table


Elapsed: 00:00:00.37
rajesh@11GR2>
rajesh@11GR2> delete from sys_iot_map_76674;
delete from sys_iot_map_76674
            *
ERROR at line 1:
ORA-28668: cannot reference mapping table of an index-organized table


Elapsed: 00:00:00.00
rajesh@11GR2>
rajesh@11GR2> select count(*) from sys_iot_map_76674;

  COUNT(*)
----------
     71790

Elapsed: 00:00:00.00
rajesh@11GR2>


Tom:

Its mentioned in doc's that We cannot query the Mapping table. But see above, i am able to query that.
Tom Kyte
December 11, 2010 - 2:29 pm UTC

ok, so then there is either

a) a documentation bug
b) a database bug

please utilize support for that one. Not a show stopping event - most likely a documentation bug.

Move Index to Another Tablespace

Jim Cox, February 25, 2011 - 10:10 am UTC

Hi Tom

since i tried the conventional move and get this error:
alter index IX_ANCESTORS_1 rebuild tablespace LIVELINK_INDEX ONLINE
*
ERROR at line 1:
ORA-28650: Primary index on an IOT cannot be rebuilt

just to make sure I understand these IOT indexes, i want to move the index to my index tablespace so would i just say:
ALTER TABLE IX_ANCESTORS_1 MOVE TABLESPACE LIVELINK_INDEX;

or is there more to it ?

Thanks
Jim
Tom Kyte
February 25, 2011 - 1:28 pm UTC

there is conceptually no index on an IOT - there is just the table, and the table just happens to be an index but that is secondary.

So, to rebuild or move an IOT, you use alter table. There is no index to deal with really.

Now I Am Confused

Jim Cox, February 25, 2011 - 2:05 pm UTC

Hi Tom

thanks for the follow-up, but now I am lost on this concept

I have this table and these two indexes:
OWNER TABLE_NAME TABLESPACE_NAME IOT_TYPE STATUS
-------------- ------------------------------ ------------------------------------ ------------ --------
LLPROD DTREEANCESTORS IOT VALID


OWNER TABLESPACE_NAME TABLE_NAME INDEX_NAME STATUS INDEX_TYPE
-------------- ------------------------------------ ------------------------------ ------------------------------ -------- ----
LLPROD LIVELINK_DATA DTREEANCESTORS IX_ANCESTORS_1 VALID IOT - TOP
LLPROD LIVELINK_INDEX DTREEANCESTORS IX_ANCESTORS_2 VALID NORMAL



i moved IX_ANCESTORS_2 successfully into Livelink_index from Livelink_Data

so how can i move IX_ANCESTORS_1 into Livelink_Index ?

or because it is an IOT, should i just leave it where it is ?

Thanks
Jim


Tom Kyte
February 28, 2011 - 7:50 am UTC

move the TABLE.

ops$tkyte%ORA11GR2> create table t ( x int constraint t_pk primary key, y int ) organization index;

Table created.

ops$tkyte%ORA11GR2> create index t_idx on t(y);

Index created.

ops$tkyte%ORA11GR2> insert into t values ( 1, 1 );

1 row created.

ops$tkyte%ORA11GR2> commit;

Commit complete.

ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> select index_name, tablespace_name, index_type from user_indexes;

INDEX_NAME                     TABLESPACE_NAME                INDEX_TYPE
------------------------------ ------------------------------ ---------------------------
T_PK                           USERS                          IOT - TOP
T_IDX                          USERS                          NORMAL

ops$tkyte%ORA11GR2> alter table t move tablespace example;

Table altered.

ops$tkyte%ORA11GR2> alter index t_idx rebuild tablespace example;

Index altered.

ops$tkyte%ORA11GR2> select index_name, tablespace_name, index_type from user_indexes;

INDEX_NAME                     TABLESPACE_NAME                INDEX_TYPE
------------------------------ ------------------------------ ---------------------------
T_PK                           EXAMPLE                        IOT - TOP
T_IDX                          EXAMPLE                        NORMAL







excellent work on IOT

A reader, February 25, 2011 - 3:57 pm UTC

excellent work tom!

index rebuild

sam, February 25, 2011 - 9:41 pm UTC

Tom:

Does the above only apply to IOT or I can monitor del_lf_rows column to determine if any indexes need to be rebuilt

select height, blocks, del_lf_rows from index_stats;



Tom Kyte
February 28, 2011 - 8:41 am UTC

How would you use del_lf_rows to do anything? It is a highly unreliable number.

ops$tkyte%ORA11GR2> create sequence s;

Sequence created.

ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> create table t
  2  as
  3  select s.nextval id, stage.*
  4    from stage;

Table created.

ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> alter table t add constraint t_pk primary key(id);

Table altered.

ops$tkyte%ORA11GR2> analyze index t_pk validate structure;

Index analyzed.

ops$tkyte%ORA11GR2> select del_lf_rows, lf_rows, lf_blks, lf_rows/lf_blks from index_stats;

DEL_LF_ROWS    LF_ROWS    LF_BLKS LF_ROWS/LF_BLKS
----------- ---------- ---------- ---------------
          0      72704        151      481.483444


so, we have about 481 leaf rows/block in our index. Now, let's do a purge:

ops$tkyte%ORA11GR2> delete from t where mod(id,482) > 5;

71799 rows deleted.

ops$tkyte%ORA11GR2> alter system flush buffer_cache;

System altered.

ops$tkyte%ORA11GR2> commit;

Commit complete.

ops$tkyte%ORA11GR2> analyze index t_pk validate structure;

Index analyzed.

ops$tkyte%ORA11GR2> select del_lf_rows, lf_rows, lf_blks, lf_rows/lf_blks from index_stats;

DEL_LF_ROWS    LF_ROWS    LF_BLKS LF_ROWS/LF_BLKS
----------- ---------- ---------- ---------------
          0        905        151      5.99337748


I did a flush on that just to get blocks written to disk - that simulates what would happen during a large purge (and it might not even be necessary for this 'demo' depending on your SGA size, the flush might happen naturally even with something this small).

Now, where are those del_lf_rows? That is an index that needs to be reorganized since we'll never reuse that space (we never insert a sequence of 100 again, we'll never reuse space on the left hand side of the index since we deleted MOST but not ALL of the entries). We carefully left about 5 leaf rows/block - and we'll never insert values "in between" those values again.


ops$tkyte%ORA11GR2> insert into t
  2  select s.nextval id, stage.*
  3    from stage;

72704 rows created.

ops$tkyte%ORA11GR2> analyze index t_pk validate structure;

Index analyzed.

ops$tkyte%ORA11GR2> select del_lf_rows, lf_rows, lf_blks, lf_rows/lf_blks from index_stats;

DEL_LF_ROWS    LF_ROWS    LF_BLKS LF_ROWS/LF_BLKS
----------- ---------- ---------- ---------------
          0      73609        287      256.477352


so as we continue to load up that table - the values will be increasing and we'll never be able to use the left hand side again. while this index looks better utilized - you know it is not using space very efficiently. If we purge again:

ops$tkyte%ORA11GR2> delete from t where mod(id,482) > 5;

71798 rows deleted.

ops$tkyte%ORA11GR2> alter system flush buffer_cache;

System altered.

ops$tkyte%ORA11GR2> commit;

Commit complete.

ops$tkyte%ORA11GR2> analyze index t_pk validate structure;

Index analyzed.

ops$tkyte%ORA11GR2> select del_lf_rows, lf_rows, lf_blks, lf_rows/lf_blks from index_stats;

DEL_LF_ROWS    LF_ROWS    LF_BLKS LF_ROWS/LF_BLKS
----------- ---------- ---------- ---------------
          0       1811        287      6.31010453


we are back where we were again... We need to reorganize this one:

ops$tkyte%ORA11GR2> alter index t_pk coalesce;

Index altered.

ops$tkyte%ORA11GR2> analyze index t_pk validate structure;

Index analyzed.

ops$tkyte%ORA11GR2> select del_lf_rows, lf_rows, lf_blks, lf_rows/lf_blks from index_stats;

DEL_LF_ROWS    LF_ROWS    LF_BLKS LF_ROWS/LF_BLKS
----------- ---------- ---------- ---------------
          0       1811          4          452.75



This is the "sweeper" index.

Now I Am Confused -Followup

Jim Cox, March 02, 2011 - 11:01 am UTC

Thanks Very Much Tom

I will try this out

Jim

Now I Am Confused -Followup

Jim Cox, March 02, 2011 - 2:05 pm UTC

Hi Tom

I have mimicked your example with my own tablespace names
Can you tell me why there is no entry for the tablespace of the table when i look at user_tables or dba_tables for that matter

Thanks
Jim


SQL> create table t ( x int constraint t_pk primary key, y int ) organization index;

Table created.

SQL> create index t_idx on t(y);

Index created.

SQL> insert into t values ( 1, 1 );

1 row created.

SQL> commit;

Commit complete.

SQL> select index_name, tablespace_name, index_type from user_indexes;

INDEX_NAME                     TABLESPACE_NAME                INDEX_TYPE
------------------------------ ------------------------------ ------------
T_PK                           TEST_DATA                      IOT - TOP
T_IDX                          TEST_DATA                      NORMAL

2 rows selected.

SQL> alter table t move tablespace test_index;

Table altered.

SQL> alter index t_idx rebuild tablespace test_index;

Index altered.

SQL> select index_name, tablespace_name, index_type from user_indexes;

INDEX_NAME                     TABLESPACE_NAME                INDEX_TYPE
------------------------------ ------------------------------ ------------
T_PK                           TEST_INDEX                     IOT - TOP
T_IDX                          TEST_INDEX                     NORMAL

2 rows selected.


SQL> select table_name, tablespace_name, iot_name, iot_type from user_tables;

TABLE_NAME                     TABLESPACE_NAME   IOT_NAME    IOT_TYPE
------------------------------ ----------------- ----------- ------------
T                                                            IOT

1 row selected.

SQL>

Tom Kyte
March 02, 2011 - 2:08 pm UTC

Because they decided to be inconsistent ;)

Logically there is no index, there is just the table. That is why you use ALTER TABLE to move it, not ALTER INDEX.

Physically there is no table, just an index. That is why you see the tablespace in user_indexes and not in user_tables.

Possible to Import Conventional Table Into Converted IOT

Jim Cox, March 22, 2011 - 11:56 am UTC

Hi Tom

I have a question about importing a conventional table's data into a converted IOT.
First, is that possible and second, if not, what would be the best way to import the data
from the conventional table export into the now converted IOT ?

Reason I ask is that when the Vendor ran their script to convert the table to IOT, 
it lost all the data in the conventional table and thus the new IOT was created empty
(approx 82.75 million rows of data=> 1.2 GB)
All I have is a full export before the change was made


The tables have the same two defined columns in them


IOT

SQL> desc dtreeancestors
 Name                                      Null?    Type
 ----------------------------------------- -------- --------------
 DATAID                                    NOT NULL NUMBER(10)
 ANCESTORID                                NOT NULL NUMBER(10)

TABLE_NAME      TABLESPACE_NAME IOT_TYPE
--------------- --------------- ----------
DTREEANCESTORS                  IOT


TABLESPACE_NAME      TABLE_NAME      INDEX_NAME       STATUS   INDEX_TYPE
-------------------- --------------- ---------------- -------- ------------
LIVELINK_DATA        DTREEANCESTORS  IX_ANCESTORS_1   VALID    IOT - TOP
LIVELINK_INDEX       DTREEANCESTORS  IX_ANCESTORS_2   VALID    NORMAL



Conventional 

SQL> desc dtreeancestors
 Name                                      Null?    Type
 ----------------------------------------- -------- --------------
 DATAID                                    NOT NULL NUMBER(10)
 ANCESTORID                                NOT NULL NUMBER(10)

TABLE_NAME      TABLESPACE_NAME IOT_TYPE
--------------- --------------- ----------
DTREEANCESTORS  LIVELINK_DATA  

      
TABLESPACE_NAME      TABLE_NAME      INDEX_NAME       STATUS   INDEX_TYPE
-------------------- --------------- ---------------- -------- ------------
LIVELINK_INDEX       DTREEANCESTORS  IX_ANCESTORS_2   VALID    NORMAL
LIVELINK_INDEX       DTREEANCESTORS  IX_ANCESTORS_3   VALID    NORMAL



Thanks

Tom Kyte
March 22, 2011 - 12:04 pm UTC

it should import in fine as long as it obeys the primary key.

ops$tkyte%ORA11GR2> create table t ( x int primary key, y date );

Table created.

ops$tkyte%ORA11GR2> insert into t select user_id, created from all_users;

41 rows created.

ops$tkyte%ORA11GR2> select count(*) from t;

  COUNT(*)
----------
        41

ops$tkyte%ORA11GR2> commit;

Commit complete.

ops$tkyte%ORA11GR2> !exp userid=/ tables=t

Export: Release 11.2.0.2.0 - Production on Tue Mar 22 13:02:35 2011

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.


Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses WE8MSWIN1252 character set (possible charset conversion)

About to export specified tables via Conventional Path ...
. . exporting table                              T         41 rows exported
Export terminated successfully without warnings.

ops$tkyte%ORA11GR2> drop table t;

Table dropped.

ops$tkyte%ORA11GR2> create table t ( x int primary key, y date ) ORGANIZATION INDEX;

Table created.

ops$tkyte%ORA11GR2> !imp userid=/ tables=t ignore=y

Import: Release 11.2.0.2.0 - Production on Tue Mar 22 13:02:36 2011

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.


Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

Export file created by EXPORT:V11.02.00 via conventional path
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses WE8MSWIN1252 character set (possible charset conversion)
. importing OPS$TKYTE's objects into OPS$TKYTE
. importing OPS$TKYTE's objects into OPS$TKYTE
. . importing table                            "T"         41 rows imported
IMP-00017: following statement failed with ORACLE error 28667:
 "ALTER TABLE "T" ADD  PRIMARY KEY ("X") USING INDEX PCTFREE 10 INITRANS 2 MA"
 "XTRANS 255 STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 FREELISTS 1 FREE"
 "LIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "USERS" LOGGING ENABLE"
IMP-00003: ORACLE error 28667 encountered
ORA-28667: USING INDEX option not allowed for the primary key of an IOT
Import terminated successfully with warnings.

ops$tkyte%ORA11GR2> select count(*) from t;

  COUNT(*)
----------
        41



you can use constraints=n if you don't want the primary key error, but then you'd have to create the others yourself.

Possible to Import Conventional Table Into Converted IOT- Thanks

Jim Cox, March 22, 2011 - 12:24 pm UTC

Thanks Very Much Tom for your time

I will give it a shot

Jim

Rebuilding (Moving) Index-Organized Tables

Jim Cox, April 21, 2011 - 2:02 pm UTC

Hi Tom

in the Administrators Guide for 11g it says on page 20-61:
Moving (Rebuilding) Index-Organized Tables

Because index-organized tables are primarily stored in a B-tree index, you can encounter fragmentation as a consequence of incremental updates. However, you can
use the ALTER TABLE...MOVE statement to rebuild the index and reduce this fragmentation.

My question is how can I determine if there is fragmentation on the table and at what point would i consider doing the rebuild (move) ?

Thanks
Tom Kyte
April 25, 2011 - 8:32 am UTC

this will only be true if the index is on a monotonically increasing/decreasing value and you delete a LOT but not all of the older values.

search this site for the word:

sweeper

to read about that type of index. If the index is on a value that arrives randomly (like last name for example) then your index will probably be about 40-50% "empty" over time and if you rebuild it, it'll just spend a lot of time getting back to 40-50% empty again.

Rebuilding (Moving) Index-Organized Tables-Thanks

Jim Cox, May 03, 2011 - 2:54 pm UTC

Thanks Tom

appreciate it

sonu, June 23, 2011 - 1:22 pm UTC

Hi Tom,

I was reading your book - Expert Oracle database architecture.

"wherever a table stored in a heap is unorganized, data in an IOT is stored and sorted by primary key.

does it mean in IOT table - it will keep some room for same data comes in future.. i mean in a example you created heap_address table emono, addr_type is the key.. lets say when we insert 2 employee address then first record fit in first block, then it will keep some few blocks and then store second employee data so in future if we get first employee other address that can fit in the earlier free space. that way data will be always together for one employee and retrieval will be faster. is this correct understanding.. i am missing something to understand.. can you pl. help me what exactly happy inside with iot? thanks
Tom Kyte
June 23, 2011 - 2:27 pm UTC

No, it is managed just like an index would be managed. If you refer to the chapter on indexes - the b*tree index part specifically - you'll get an overview of that.


Let's say the primary key of a table was "last_name" (bad key, I know, but easy to visualize).

In that IOT - there would be a block full of the last names that begin with K. Let's say that index leaf block was 100% full. Furthermore, you decided to add my name to the IOT. Now, I have to go on the block next to all of the other K's - but there isn't any room - so we'll take that one leaf block and split it into two leaf blocks - each about 50% empty. On the block with the other K's near KYTE - my record will get inserted.

partitioned index organized table

a reader, August 16, 2011 - 3:14 pm UTC

In a previous post, you said a partitioned IOT would be many IOTs. Then if the partition key is not the leading column, the performance would be worse than not partitioned, isn't it?

For example, I have a IOT on (id, dt), partitioned on date column dt, then query select * from t where id = :1 and dt between :2 and :3;

would have to access multiple IOTs (partitions), while non-partitioned table just have one IOT.

Is partitioning a IOT just for the benefit of administration at cost of slower query?
Tom Kyte
August 16, 2011 - 5:45 pm UTC

you cannot have what your "for example" states.

ops$tkyte%ORA11GR2> CREATE TABLE t
  2  (
  3    x   int primary key,
  4    dt  date,
  5    y   varchar2(30)
  6  )
  7  organization index
  8  PARTITION BY RANGE (dt)
  9  (
 10    PARTITION part1 VALUES LESS THAN (to_date('13-mar-2003','dd-mon-yyyy')) ,
 11    PARTITION part2 VALUES LESS THAN (to_date('14-mar-2003','dd-mon-yyyy')) ,
 12    PARTITION junk VALUES LESS THAN (MAXVALUE)
 13  )
 14  
ops$tkyte%ORA11GR2> /
PARTITION BY RANGE (dt)
                    *
ERROR at line 8:
ORA-25199: partitioning key of a index-organized table must be a subset of the
primary key




so, your basic assumption is incorrect and cannot happen.


You might partition and IOT for

o higher availability
o ease of administration
o improved query performance

- that is, the same reasons you would partition anything.


For improved query performance - if I have an IOT partitioned on some column like DT:

ops$tkyte%ORA11GR2> CREATE TABLE t
  2  (
  3    x   int,
  4    dt  date,
  5    y   varchar2(30),
  6    primary key(dt,x)
  7  )
  8  organization index
  9  PARTITION BY RANGE (dt)
 10  (
 11    PARTITION part1 VALUES LESS THAN (to_date('13-mar-2003','dd-mon-yyyy')) ,
 12    PARTITION part2 VALUES LESS THAN (to_date('14-mar-2003','dd-mon-yyyy')) ,
 13    PARTITION junk VALUES LESS THAN (MAXVALUE)
 14  )
 15  
ops$tkyte%ORA11GR2> /

Table created.


and I inserted data without a time component (so all of the DT values in a partition would be identical for a daily partitioned table) and varied the X values - a query of the form:

select * from t where dt = ? and ....;

could use partition elimination and do an index fast full scan against that single partition instead of using a slower index range scan type of access.

partition IOT

a reader, August 17, 2011 - 9:05 am UTC

Sorry Tom, I wasn't clear, the PK of IOT is on (id, dt), and range partitioned on dt. The access pattern is

select * from t where id =:1 and dt between :2 and :3;
Tom Kyte
August 17, 2011 - 4:30 pm UTC

so, give a full example like I always do - explain the problem concisely with a complete example.

I find many times that as I build the complete example - I answer my own questions...



we would be able to partition eliminate the above query - we would have a pstart/pstop and would be scanning a range of partitions. In that case, it would be less efficient to have the partitions.

But that would be 100% true of a regular table as well, when partitioning ANYTHING, ANY OBJECT - you have to think about how you'll be accessing the data and partition appropriately.

Initial Extent of IOT Index

Venkat, March 20, 2012 - 1:35 pm UTC

Hi Tom,

Thank you very much for your time.

We have an IOT table(for ex., test1) and index on primary key(for ex., test1_idx). Initial extent size of test1_idx is 2.6GB. Currently this table doesn't have any rows. Not from last 1month atleast. Even though it will increase, I don't want the initial extent to be 2.6GB. I would like to have the default settings(1MB or so).
Since we couldn't drop and recreate the primary key index on the IOT, how to change the initial extent size of this index? Please provide some inputs.

Database version: 11.1.0.7


Tom Kyte
March 20, 2012 - 3:25 pm UTC

if you have an IOT, there would be no index on a primary key - the index IS the table.

Not sure what you mean?


Just rebuild the IOT - that can be done online and you can specify whatever storage characteristics you would like.


Initial Extent of IOT

Venkat, March 20, 2012 - 4:18 pm UTC

Hi Tom,

Sorry for not being clear.

From dba_indexes I can see there is an index TEST2_IDX of type "IOT_TOP" on the table TEST2.

I already rebuilt the table using alter table move. But I am mainly concerned about the index INITIAL_EXTENT which has 2.6GB size. I want to make this set to default(1MB).

Through dbms_metadata.get_ddl, I pulled the scripts of index and table(whatever is showing from dba_indexes). Please check it here.

create table test2
( COL1 VARCHAR2(50 CHAR) NOT NULL ENABLE,
COL2 CHAR(1 CHAR) NOT NULL ENABLE,
COL3 NUMBER NOT NULL ENABLE,
COL4 NUMBER NOT NULL ENABLE,
.....
COL12 VARCHAR2(40 CHAR) NOT NULL ENABLE,
CONSTRAINT CONST_NAME PRIMARY KEY(COL1,COL2,COL3,COL4,COL5,COL6) ENABLE
) ORGANIZATION INDEX;

CREATE UNIQUE INDEX TEST2_IDX ON TEST2(COL1,COL2,COL3,COL4,COL5,COL6);

Thanks for all your inputs.
Tom Kyte
March 20, 2012 - 7:37 pm UTC

this is not possible:




ops$tkyte%ORA11GR2> create table t
  2  ( x int,
  3    y int,
  4    z int,
  5    a varchar2(200),
  6    constraint t_pk primary key(x,y,z)
  7  )
  8  organization index;

Table created.

ops$tkyte%ORA11GR2> create unique index t_idx on t(x,y,z);
create unique index t_idx on t(x,y,z)
                               *
ERROR at line 1:
ORA-01408: such column list already indexed



an IOT is an index, there would not be a TEST2_IDX on those columns. Please give a full example from start to finish.

INITIAL Extent of IT

Venkat, March 20, 2012 - 10:46 pm UTC

Sorry Tom.

My editing mistake. I should have changed TEST2_IDX to CONST_NAME in previous question.

Infact it's primary key index.

SQL> create table t (x int,y int,z int,a varchar2(200),constraint t_pk primary key(x,y,z)) organization index;

Table created.

SQL> select index_name,index_type,table_type from dba_indexes where table_name like 'T';

INDEX_NAME                     INDEX_TYPE                  TABLE_TYPE
------------------------------ --------------------------- -----------
T_PK                           IOT - TOP                   TABLE

In the above example, (Let's say) T_PK has 3G INITIAL extent(not sure how it got at the first place). How can we make it to default 1M?

Sorry once again.

Thanks for all your help.

Tom Kyte
March 21, 2012 - 9:45 am UTC

ops$tkyte%ORA11GR2> create table t ( x int, y int, z int, a varchar2(30),
  2                   constraint t_pk primary key(x,y,z) )
  3  organization index;

Table created.

ops$tkyte%ORA11GR2> insert into t select rownum, rownum, rownum, username from all_users;

49 rows created.

ops$tkyte%ORA11GR2> commit;

Commit complete.

ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> select segment_name, initial_extent from user_segments;

SEGMENT_NAME                   INITIAL_EXTENT
------------------------------ --------------
T_PK                                    65536

ops$tkyte%ORA11GR2> alter table t move storage ( initial 1m );

Table altered.

ops$tkyte%ORA11GR2> select segment_name, initial_extent from user_segments;

SEGMENT_NAME                   INITIAL_EXTENT
------------------------------ --------------
T_PK                                  1048576


Stats on large IOT tables

madopal, April 13, 2016 - 2:22 pm UTC

Hi - We recently upgraded our prod database from 11.2.0.3 to 11.2.0.4. Before the upgrade, gather stats on 2 large IOT tables with 250+ mil rows took around 5-8 mins. But after the upgrade, stats on one of the tables is taking around 18 hrs and the other is taking around 5 hrs. There are a few other IOT tables (with rows ranging from a few hundreds to 50+ mil rows) and they seem to be doing fine. Just wanted to check in and see if there is something specific that would cause this. We ran fixed objects stats and dictionary stats after the upgrade
Chris Saxon
April 13, 2016 - 2:31 pm UTC

I'm not aware of something that would cause this. Try tracing the stats gathering session including waits to see what's going on:

exec dbms_monitor.session_trace_enable(<stats sid>, <stats serial#>, waits => true);

If you need help with this, post a new question with the results of your findings.

Thanks for the followup

madopal, April 13, 2016 - 2:39 pm UTC

will do