Skip to Main Content
  • Questions
  • Partitioning the table by range on a column and no index/key on this ?

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, sush.

Asked: February 15, 2003 - 3:21 pm UTC

Last updated: November 21, 2004 - 9:34 am UTC

Version: 8.1.7

Viewed 1000+ times

You Asked

Hi Tom ,

I want to partition by range on column partid (for each thread) on a table say "xyz" as I'll be loading(inserting) this table with multiple threads(say 4).The reason to partition this table is for restartability as it gives me flexibility to truncate and reload if any of the threads fail.

My questions:

1)Is it necessary to have index/key on this column partid.
I'll not be referring to this column partid in any of my queries.
2)Is it ok to analyze one partition while loading into another paritition?
3)Are local indexes on the table better or global indexes on the partitioned table.

I always admire your answers.

and Tom said...

1) no, not necessary

2) yes, but -- you'll WANT global statistics across all partitions after all is said and done as well. You want both partition level and TABLE level stats. Pretty important to have both if you are not sure 100% of the queries will always come down to a single partition (which it doesn't sound like they will)

3) if one were inheritly better then the other -- we would not have IMPLEMENTED the other kind.

What you need to do -- this is important -- is understand the ins and outs of the physics of partitions.

For example, you are using partitions to have a "restartable load". Ok, fair enough -- that is your stated goal.

Now, suppose you use local indexes on all -- well, when you query: "select * from t where indexed_column = :x" -- unless indexed_column IS part of your partition key (doesn't sound like it) you will have easily QUADRUPULED your IO on your system as we have 4 index range scans to do.

So, suppose you use a global index on indexed_column -- now, the index IO is down to normal -- but if you truncate a partition (as stated) the entire local index will go "invalid" and have to be rebuilt.


So -- you will want to do further investigation, benchmarking, testing, and understanding of the physics of partitioning -- before going further.


Partitions are a tool -- they can do many things, they can slow you down to a crawl when used wrong, they can have no impact whatsoever in other cases and from time to time can make you go faster. Understand what and how they do it -- and you'll come along way to understanding how and where to apply them.

Sorry this is sort of "preachy" -- it is just that I see people "slamming" partitioning because they deploy it and it works the way it was designed, not the way they imagined ;) Just want to make sure you don't get to that point.

Rating

  (16 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

sush, February 15, 2003 - 7:56 pm UTC

Thanks for the response.It's very useful.My earlier question perhaps was not clear so I'm rewriting the questions.

question 2 : Can we do analyze of a partition while other partition is being loaded(I mean simultaneously).I thought one of it will fail.

question 3 : Pls correct me if i'm wrong,In a datawarehouse we should go with local indexes rather than with global indexes.

Tom Kyte
February 16, 2003 - 10:31 am UTC

q2) the partitions are independent of eachother.

You should be able to do that. You can analyze whilst others are even writing to the same segment in fact.


q3) maybe yes -- maybe no -- maybe maybe. All of the above, none of the below. Sometimes, never, always.

They are all valid responses as I said -- understand the "physics" behind partitioning and you'll understand what I mean. Understand the motiviation behind using partitions (your motivation) and implement accordingly. Just remember to consider all aspects of the system

o query
o administration
o load
o anything else you do

and make sure that what you do to optimize one does not hurt the others!



sush, February 16, 2003 - 3:28 pm UTC

One more quick Q.I've one column in this partitioned table "xyz" say code_type(has values say a,b,c,d,e,f,g,h)As I'm loading this table with multiple threads , I'm thinking to create a bitmap index on this column.Is it ok ?Thanks in advance.


Tom Kyte
February 16, 2003 - 3:35 pm UTC

After you load this table, go ahead and create the bitmap

It would be a totally KILLER to load into the table with a bitmapped index on it

what if that bitmap index is local on the partitions.

Ajeet, February 16, 2003 - 7:41 pm UTC

Tom --does the load timing will be impacted even the bitmap index is local to partitions.?If yes -- then should we set those indexes unusable before the load and then load .?

Tom Kyte
February 17, 2003 - 9:48 am UTC

depends on how you load -- I'm guessing you are writing your own loader (just have that impression). If so -- disable those bitmaps and rebuild at the end.

Very informative, as usual

Colin Davies, February 17, 2003 - 2:50 pm UTC

I think that this confirms what I have always used as guideline: if an index on a range-partitioned table INCLUDES the partitioning key, then make it a local index, otherwise make it a global index. Is that correct?

Tom Kyte
February 17, 2003 - 3:25 pm UTC

well.... no. not really


table partitioned by x
local index on y


select * from t where x = :x and y = :y;

for example -- local index is OK -- partition elmination kicks in (in this case the partition key is implicitly in the index as well -- since the index is equi-partitioned)


or -- I'm in a DW, doing parallel queries anyway -- parallel index range scans....


It all depends -- there are no "rules". Thats why I'm into the physics behind partitioning. Understand where and how the structures are -- and what your popular queries will have to do in order to use them -- and make the decisions based on that.


You might go for a locally partitioned index that you have to range scan N times per query because the cost of maintaining a global index in your system is too high.

You might go for a globally partitioned index that you have to range scan 1 time per query because the cost of range scanning N indexes is much higher to you then the maintanence of the global index.

Very informative

Colin Davies, February 17, 2003 - 4:06 pm UTC

I agree about understanding the specifics in each case. Here is a test I used to verify conditions in a real database:

The DSSMART.PRODUCT_EVENTS table is range partitioned on START_DT. All of the indexes are currently LOCAL indexes.The index PRODUCT_EVENTS_IDX11 is on column ORD_TYPE_CD.

Some examples:

SELECT COUNT(*) FROM DSSMART.PRODUCT_EVENTS WHERE START_DT > SYSDATE - 2;

Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop

SELECT STATEMENT Hint=CHOOSE 1 600
SORT AGGREGATE 1 8
PARTITION RANGE ITERATOR KEY 44
PARTITION HASH ALL 1 4
INDEX RANGE SCAN PRODUCT_EVENTS_IDX1 1 M 12 M 600 KEY 176

This uses partition elimination as expected since START_DT is the partitioning key and is also part of the primary key index.


Let's add another predicate:

SELECT COUNT(*) FROM DSSMART.PRODUCT_EVENTS WHERE START_DT > SYSDATE - 2 AND ORD_TYPE_CD = 1;

Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop

SELECT STATEMENT Hint=CHOOSE 1 600
SORT AGGREGATE 1 12
PARTITION RANGE ITERATOR KEY 44
PARTITION HASH ALL 1 4
INDEX RANGE SCAN PRODUCT_EVENTS_IDX1 124 K 1 M 600 KEY 176

So far, so good - including START_DT still gives us partition elimination.


Now let's lose the partitioning key from the query:

SELECT COUNT(*) FROM DSSMART.PRODUCT_EVENTS WHERE ORD_TYPE_CD = 1;

Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop

SELECT STATEMENT Hint=CHOOSE 1 1251
SORT AGGREGATE 1 4
PARTITION RANGE ALL 1 44
PARTITION HASH ALL 1 4
INDEX FAST FULL SCAN PRODUCT_EVENTS_IDX11 2 M 9 M 1251 1 176

We now have PARTITION RANGE ALL, which means that even though it can use index PRODUCT_EVENTS_IDX11, it will still have to do an index scan on every partition of this local index.


Single column bitmap on partition key

Peter Scott, April 01, 2003 - 4:10 am UTC

For our sins my group has "inherited" a DWH with loads of range partitioned aggregate tables (8.1.7) Most data is parititioned by week, i.e. for week level summaries each partition has a single date value.

We have found that the designers have created bitmap indexes on each of the dimension columns of the fact summary INCLUDING the time column - surely this is not needed as there will only be 1 value referenced in each index partition. Unless any of you know otherwise ;-}

My thinking is that the data dictionary knows which partition contains which week, and the bitmap index is not giving us any value even for date predicate queries.

Tom Kyte
April 01, 2003 - 7:34 am UTC

did they do it for star transformations?

</code> http://docs.oracle.com/docs/cd/A87860_01/doc/server.817/a76994/schemas.htm#11292 <code>



Answer to Tom

Peter Scott, April 01, 2003 - 8:20 am UTC

Sadly, no
Star transforms are not enabled on this DB.

That is one area we are going to fix after a scheduled upgrade to 9.2

Are you saying that STAR transforms also require a bitmap index on the partition keys

Tom Kyte
April 01, 2003 - 8:36 am UTC

on the foreign keys to the dimension tables.

Dave, April 01, 2003 - 9:56 am UTC

Star transforms were a little buggy in early 8.1.7 releases, but 8.1.7.4 was good. you might like to give them a go prior the the 9i upgrade -- they gave us a great performance boost.

Aside from the star tansformation issue, Oracle doesn't know for sure that there is a single value per partition (aside from column histogram information), only that any values fall within a particular range. So if you wanted to place a predicate on that column that stated "where week_date ='01-jan-2003'" it would still be beneficial to have that bitmap index there so that it could be bitmap-AND'ed with other predicated colum bitmaps

Thanks Dave (& Tom, of course)

Peter Scott, April 01, 2003 - 10:47 am UTC

That was most helpful. The DB was devloped in 8.1.4 i.e before STAR Transforms worked well.

We will certainly give this a try on our test system - just have to find some time to put in the dimensions !

vj, April 23, 2003 - 11:04 am UTC

referring to this partition by range..can i partition with more than one column...

say col1,col2 range x,y
col1, col2 range z,k
col1, col2 range f,h



Tom Kyte
April 23, 2003 - 7:21 pm UTC

yes, it is all documented for you actually. the documentation would describe this...


see also

</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:1200046826714 <code>



concatenated indexes on foreign keys for star query

anonymous, May 27, 2003 - 9:28 pm UTC

for the star transform the docs say:

There is no need to create many concatenated indexes where the different column orders match different patterns of constrained dimensions in different queries.

so:

does this imply that for star queries without bitmap indexes, that we must have many different concatenated indexes to match the different patterns of constrained dimensions in the different queries?



Alvin, September 16, 2003 - 12:24 am UTC

I've a partitioned table and i've created local partitions on some columns and have indexed the partitioning key (TIME).

I've several similar queries that would benefit immensely if they'd use the index on the partitioning key.

1. How would i tell oracle that this is the way to go ?? My queries are dynamically created and i cannot readily change the front end program to make use of the hints.

2. Would it be better to drop the index ??

3. how would i force oracle to full scan the partition instead ? I think block selectivity or block distribution of my data are evenly distributed hence the benefit of full partition scans.

SELECT /*+INDEX (rtbetelcalldetailrecords,CDR_TIME) */ count(*)
FROM rtbetelephony.rtbetelcalldetailrecords
WHERE groupid = 2741
and (TIME BETWEEN 1062219600487 AND 1062305999487)
5 /

COUNT(*)
--------------
59679


1 row selected.

Elapsed: 00:03:209.62

Execution Plan
----------------------------------------------------------
0
SELECT STATEMENT Optimizer=CHOOSE (Cost=68612 Card=1 Bytes=13)


1 0
SORT (AGGREGATE)


2 1
TABLE ACCESS (BY LOCAL INDEX ROWID) OF 'RTBETELCALLDETAILRECORDS' (Cost=6861
2 Card=1636 Bytes=21268)


3 2
INDEX (RANGE SCAN) OF 'CDR_TIME' (NON-UNIQUE) (Cost=268 Card=1636)








Statistics
----------------------------------------------------------
8 recursive calls
0 db block gets
126521 consistent gets
10888 physical reads
0 redo size
205 bytes sent via SQL*Net to client
316 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processed


1 SELECT count(*)
2 FROM rtbetelephony.rtbetelcalldetailrecords
3 WHERE groupid = 2741
4* and (TIME BETWEEN 1062219600487 AND 1062305999487)

COUNT(*)
--------------
59679


1 row selected.

Elapsed: 00:20:1251.90

Execution Plan
----------------------------------------------------------
0
SELECT STATEMENT Optimizer=CHOOSE (Cost=33806 Card=1 Bytes=13)


1 0
SORT (AGGREGATE)


2 1
TABLE ACCESS (BY LOCAL INDEX ROWID) OF 'RTBETELCALLDETAILRECORDS' (Cost=3380
6 Card=1636 Bytes=21268)


3 2
INDEX (RANGE SCAN) OF 'CDR_PART_GRPID' (NON-UNIQUE) (Cost=248 Card=1636)








Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
389986 consistent gets
214864 physical reads
0 redo size
205 bytes sent via SQL*Net to client
316 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processed



Tom Kyte
September 16, 2003 - 8:06 am UTC

tell you what -- when I'm taking questions, this looks like a new one. You see -- one would need many more details in order to provide a meaningful answer.

you say you have other queries that would benefit from using this index. I'd need to see the plan we are generating and the plan you force to use the index to see that this is so (and not just a guess) and we can work from there.

I'd need to understand the table/indexing strategy better.

I'd need to see the explain plan as show by $ORACLE_HOME/rdbms/admin/utlxpls.sql so I can see the partition information as well as the CBO stats used...

all more then I want in a followup here.


I can say the above queries would benefit IMMENSELY if you added groupid to this index. If this is a "popular" type of query -- groupid should be in that index.

Should I index the partition key

A reader, May 28, 2004 - 3:21 pm UTC

I have a table partitioned by day. One partition for every day. Is there any reason ( I can't think of one) why I would want to bitmap index the date field?

Tom Kyte
May 28, 2004 - 3:47 pm UTC

yes there *could* be. but rather then hypothesize, I'd wait until someday when a problem presents itself and I say "hey, a bitmap index on that column would be neat"

there are two cases -- the date field might be truncated to a DAY (eg: constant in a partition). Then, no, I probably don't see it.

or the date field might be truncated to an HOUR within a DAY -- then there are 24 values.... maybe I would (how many hits did we get between 1pm and 2pm...)






Is it possible to Create composite index only for a few partitions of a partitioned table ?

Anto, November 20, 2004 - 10:17 pm UTC

Hi Tom,

I have a range partitioned table, partitioned by a date field, which holds data from year 2000 onwards. My question is - Is it possible to create a btree or bitmap composite index(even though bitmap composite indexes are not that common) only for a few of the table partitions - say for partitions which contains the data for year 2003 and 2004. Since each of the partition contains many millions of rows, a btree composite index on the full table will be really big and most of the old data will be very rarely used.Most of the queries access only the current year and previous year's data. Currently we have local, single column bitmap indexes in place for this table. Is it possible to create a composite index(either local or global, btree or bitmap) only for 1 or 2 partitions of the table ? Ours is a data warehouse which uses range partitioned tables and the queries on these tables use bitmap index/star transformation

Tom Kyte
November 21, 2004 - 9:04 am UTC

Anto, November 21, 2004 - 9:32 am UTC

Thanks, Tom.

I had in fact seen this article in Oracle magazine and on the net,some time back, but was searching for 'condtional index' instead of 'selective index' and thus unable to find that, of late.

Thanks again

A reader, November 21, 2004 - 9:34 am UTC

was searching for 'conditional index' not 'condtional index'

More to Explore

VLDB

If you are new to partitioning, check out Connor McDonald's introduction series here.

VLDB

Documentation set on VLDB and Partitioning.