sush, February 15, 2003 - 7:56 pm UTC
Thanks for the response.It's very useful.My earlier question perhaps was not clear so I'm rewriting the questions.
question 2 : Can we do analyze of a partition while other partition is being loaded(I mean simultaneously).I thought one of it will fail.
question 3 : Pls correct me if i'm wrong,In a datawarehouse we should go with local indexes rather than with global indexes.
February 16, 2003 - 10:31 am UTC
q2) the partitions are independent of eachother.
You should be able to do that. You can analyze whilst others are even writing to the same segment in fact.
q3) maybe yes -- maybe no -- maybe maybe. All of the above, none of the below. Sometimes, never, always.
They are all valid responses as I said -- understand the "physics" behind partitioning and you'll understand what I mean. Understand the motiviation behind using partitions (your motivation) and implement accordingly. Just remember to consider all aspects of the system
o query
o administration
o load
o anything else you do
and make sure that what you do to optimize one does not hurt the others!
sush, February 16, 2003 - 3:28 pm UTC
One more quick Q.I've one column in this partitioned table "xyz" say code_type(has values say a,b,c,d,e,f,g,h)As I'm loading this table with multiple threads , I'm thinking to create a bitmap index on this column.Is it ok ?Thanks in advance.
February 16, 2003 - 3:35 pm UTC
After you load this table, go ahead and create the bitmap
It would be a totally KILLER to load into the table with a bitmapped index on it
what if that bitmap index is local on the partitions.
Ajeet, February 16, 2003 - 7:41 pm UTC
Tom --does the load timing will be impacted even the bitmap index is local to partitions.?If yes -- then should we set those indexes unusable before the load and then load .?
February 17, 2003 - 9:48 am UTC
depends on how you load -- I'm guessing you are writing your own loader (just have that impression). If so -- disable those bitmaps and rebuild at the end.
Very informative, as usual
Colin Davies, February 17, 2003 - 2:50 pm UTC
I think that this confirms what I have always used as guideline: if an index on a range-partitioned table INCLUDES the partitioning key, then make it a local index, otherwise make it a global index. Is that correct?
February 17, 2003 - 3:25 pm UTC
well.... no. not really
table partitioned by x
local index on y
select * from t where x = :x and y = :y;
for example -- local index is OK -- partition elmination kicks in (in this case the partition key is implicitly in the index as well -- since the index is equi-partitioned)
or -- I'm in a DW, doing parallel queries anyway -- parallel index range scans....
It all depends -- there are no "rules". Thats why I'm into the physics behind partitioning. Understand where and how the structures are -- and what your popular queries will have to do in order to use them -- and make the decisions based on that.
You might go for a locally partitioned index that you have to range scan N times per query because the cost of maintaining a global index in your system is too high.
You might go for a globally partitioned index that you have to range scan 1 time per query because the cost of range scanning N indexes is much higher to you then the maintanence of the global index.
Very informative
Colin Davies, February 17, 2003 - 4:06 pm UTC
I agree about understanding the specifics in each case. Here is a test I used to verify conditions in a real database:
The DSSMART.PRODUCT_EVENTS table is range partitioned on START_DT. All of the indexes are currently LOCAL indexes.The index PRODUCT_EVENTS_IDX11 is on column ORD_TYPE_CD.
Some examples:
SELECT COUNT(*) FROM DSSMART.PRODUCT_EVENTS WHERE START_DT > SYSDATE - 2;
Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop
SELECT STATEMENT Hint=CHOOSE 1 600
SORT AGGREGATE 1 8
PARTITION RANGE ITERATOR KEY 44
PARTITION HASH ALL 1 4
INDEX RANGE SCAN PRODUCT_EVENTS_IDX1 1 M 12 M 600 KEY 176
This uses partition elimination as expected since START_DT is the partitioning key and is also part of the primary key index.
Let's add another predicate:
SELECT COUNT(*) FROM DSSMART.PRODUCT_EVENTS WHERE START_DT > SYSDATE - 2 AND ORD_TYPE_CD = 1;
Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop
SELECT STATEMENT Hint=CHOOSE 1 600
SORT AGGREGATE 1 12
PARTITION RANGE ITERATOR KEY 44
PARTITION HASH ALL 1 4
INDEX RANGE SCAN PRODUCT_EVENTS_IDX1 124 K 1 M 600 KEY 176
So far, so good - including START_DT still gives us partition elimination.
Now let's lose the partitioning key from the query:
SELECT COUNT(*) FROM DSSMART.PRODUCT_EVENTS WHERE ORD_TYPE_CD = 1;
Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop
SELECT STATEMENT Hint=CHOOSE 1 1251
SORT AGGREGATE 1 4
PARTITION RANGE ALL 1 44
PARTITION HASH ALL 1 4
INDEX FAST FULL SCAN PRODUCT_EVENTS_IDX11 2 M 9 M 1251 1 176
We now have PARTITION RANGE ALL, which means that even though it can use index PRODUCT_EVENTS_IDX11, it will still have to do an index scan on every partition of this local index.
Single column bitmap on partition key
Peter Scott, April 01, 2003 - 4:10 am UTC
For our sins my group has "inherited" a DWH with loads of range partitioned aggregate tables (8.1.7) Most data is parititioned by week, i.e. for week level summaries each partition has a single date value.
We have found that the designers have created bitmap indexes on each of the dimension columns of the fact summary INCLUDING the time column - surely this is not needed as there will only be 1 value referenced in each index partition. Unless any of you know otherwise ;-}
My thinking is that the data dictionary knows which partition contains which week, and the bitmap index is not giving us any value even for date predicate queries.
April 01, 2003 - 7:34 am UTC
Answer to Tom
Peter Scott, April 01, 2003 - 8:20 am UTC
Sadly, no
Star transforms are not enabled on this DB.
That is one area we are going to fix after a scheduled upgrade to 9.2
Are you saying that STAR transforms also require a bitmap index on the partition keys
April 01, 2003 - 8:36 am UTC
on the foreign keys to the dimension tables.
Dave, April 01, 2003 - 9:56 am UTC
Star transforms were a little buggy in early 8.1.7 releases, but 8.1.7.4 was good. you might like to give them a go prior the the 9i upgrade -- they gave us a great performance boost.
Aside from the star tansformation issue, Oracle doesn't know for sure that there is a single value per partition (aside from column histogram information), only that any values fall within a particular range. So if you wanted to place a predicate on that column that stated "where week_date ='01-jan-2003'" it would still be beneficial to have that bitmap index there so that it could be bitmap-AND'ed with other predicated colum bitmaps
Thanks Dave (& Tom, of course)
Peter Scott, April 01, 2003 - 10:47 am UTC
That was most helpful. The DB was devloped in 8.1.4 i.e before STAR Transforms worked well.
We will certainly give this a try on our test system - just have to find some time to put in the dimensions !
vj, April 23, 2003 - 11:04 am UTC
referring to this partition by range..can i partition with more than one column...
say col1,col2 range x,y
col1, col2 range z,k
col1, col2 range f,h
concatenated indexes on foreign keys for star query
anonymous, May 27, 2003 - 9:28 pm UTC
for the star transform the docs say:
There is no need to create many concatenated indexes where the different column orders match different patterns of constrained dimensions in different queries.
so:
does this imply that for star queries without bitmap indexes, that we must have many different concatenated indexes to match the different patterns of constrained dimensions in the different queries?
Alvin, September 16, 2003 - 12:24 am UTC
I've a partitioned table and i've created local partitions on some columns and have indexed the partitioning key (TIME).
I've several similar queries that would benefit immensely if they'd use the index on the partitioning key.
1. How would i tell oracle that this is the way to go ?? My queries are dynamically created and i cannot readily change the front end program to make use of the hints.
2. Would it be better to drop the index ??
3. how would i force oracle to full scan the partition instead ? I think block selectivity or block distribution of my data are evenly distributed hence the benefit of full partition scans.
SELECT /*+INDEX (rtbetelcalldetailrecords,CDR_TIME) */ count(*)
FROM rtbetelephony.rtbetelcalldetailrecords
WHERE groupid = 2741
and (TIME BETWEEN 1062219600487 AND 1062305999487)
5 /
COUNT(*)
--------------
59679
1 row selected.
Elapsed: 00:03:209.62
Execution Plan
----------------------------------------------------------
0
SELECT STATEMENT Optimizer=CHOOSE (Cost=68612 Card=1 Bytes=13)
1 0
SORT (AGGREGATE)
2 1
TABLE ACCESS (BY LOCAL INDEX ROWID) OF 'RTBETELCALLDETAILRECORDS' (Cost=6861
2 Card=1636 Bytes=21268)
3 2
INDEX (RANGE SCAN) OF 'CDR_TIME' (NON-UNIQUE) (Cost=268 Card=1636)
Statistics
----------------------------------------------------------
8 recursive calls
0 db block gets
126521 consistent gets
10888 physical reads
0 redo size
205 bytes sent via SQL*Net to client
316 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processed
1 SELECT count(*)
2 FROM rtbetelephony.rtbetelcalldetailrecords
3 WHERE groupid = 2741
4* and (TIME BETWEEN 1062219600487 AND 1062305999487)
COUNT(*)
--------------
59679
1 row selected.
Elapsed: 00:20:1251.90
Execution Plan
----------------------------------------------------------
0
SELECT STATEMENT Optimizer=CHOOSE (Cost=33806 Card=1 Bytes=13)
1 0
SORT (AGGREGATE)
2 1
TABLE ACCESS (BY LOCAL INDEX ROWID) OF 'RTBETELCALLDETAILRECORDS' (Cost=3380
6 Card=1636 Bytes=21268)
3 2
INDEX (RANGE SCAN) OF 'CDR_PART_GRPID' (NON-UNIQUE) (Cost=248 Card=1636)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
389986 consistent gets
214864 physical reads
0 redo size
205 bytes sent via SQL*Net to client
316 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processed
September 16, 2003 - 8:06 am UTC
tell you what -- when I'm taking questions, this looks like a new one. You see -- one would need many more details in order to provide a meaningful answer.
you say you have other queries that would benefit from using this index. I'd need to see the plan we are generating and the plan you force to use the index to see that this is so (and not just a guess) and we can work from there.
I'd need to understand the table/indexing strategy better.
I'd need to see the explain plan as show by $ORACLE_HOME/rdbms/admin/utlxpls.sql so I can see the partition information as well as the CBO stats used...
all more then I want in a followup here.
I can say the above queries would benefit IMMENSELY if you added groupid to this index. If this is a "popular" type of query -- groupid should be in that index.
Should I index the partition key
A reader, May 28, 2004 - 3:21 pm UTC
I have a table partitioned by day. One partition for every day. Is there any reason ( I can't think of one) why I would want to bitmap index the date field?
May 28, 2004 - 3:47 pm UTC
yes there *could* be. but rather then hypothesize, I'd wait until someday when a problem presents itself and I say "hey, a bitmap index on that column would be neat"
there are two cases -- the date field might be truncated to a DAY (eg: constant in a partition). Then, no, I probably don't see it.
or the date field might be truncated to an HOUR within a DAY -- then there are 24 values.... maybe I would (how many hits did we get between 1pm and 2pm...)
Is it possible to Create composite index only for a few partitions of a partitioned table ?
Anto, November 20, 2004 - 10:17 pm UTC
Hi Tom,
I have a range partitioned table, partitioned by a date field, which holds data from year 2000 onwards. My question is - Is it possible to create a btree or bitmap composite index(even though bitmap composite indexes are not that common) only for a few of the table partitions - say for partitions which contains the data for year 2003 and 2004. Since each of the partition contains many millions of rows, a btree composite index on the full table will be really big and most of the old data will be very rarely used.Most of the queries access only the current year and previous year's data. Currently we have local, single column bitmap indexes in place for this table. Is it possible to create a composite index(either local or global, btree or bitmap) only for 1 or 2 partitions of the table ? Ours is a data warehouse which uses range partitioned tables and the queries on these tables use bitmap index/star transformation
November 21, 2004 - 9:04 am UTC
Anto, November 21, 2004 - 9:32 am UTC
Thanks, Tom.
I had in fact seen this article in Oracle magazine and on the net,some time back, but was searching for 'condtional index' instead of 'selective index' and thus unable to find that, of late.
Thanks again
A reader, November 21, 2004 - 9:34 am UTC
was searching for 'conditional index' not 'condtional index'