OK
Mirza Amer Baig, October 15, 2001 - 5:51 am UTC
That's okay. I think the last solution is the only option available. Thanks for your support. You are my best resource.
Thanks
Amer
Roland Wallner, May 15, 2002 - 3:08 pm UTC
This was what I've been looking for... I think? This looks like multicolumn partition key, but you said composite partition? If it's a mulicolumn, wouldn't any insert for the month of OCT go into the first partition, since it's less the the bound?
If it's a composite, how do you get each country code to hash to the proper location?
Thanks
May 16, 2002 - 12:50 pm UTC
It is a multi-column partition key.
I said to use a composit KEY in a range partition.
The row would go into the partition is was specified to go into. I cannot say if OCT would go into the first partition since we never really said where each month goes.
My response was to NOT USE hashing at all -- but rather a "multi-column partition key" that was range partitioned by DATE and CITY_CODE
Roland Wallner, May 16, 2002 - 8:50 pm UTC
Thanks Tom! We've been trying to implement a multicolumn key partition and been having some trouble.
Right now we have partitions based on Month, so each months transactions would go into a separate partition. The partitions use the Date as the key. With one client type this hasn't been problem.
Now we are introducing another client type, so we want to put another key along with the date... client_id.
Our example...
Create Table P_main_Test
(
Trans_ID number Not Null,
Client_ID number(4) Not Null,
P_Date date Not Null,
f1 varchar2(10),
f2 varchar2(10),
f3 varchar2(10)
)
storage(initial 1m next 1m) tablespace users
partition by range(P_Date,client_ID)
(
partition G_may99 values less than (to_date('01-JUN-1999','DD-MON-YYYY'),4999)
TABLESPACE users,
partition C_may99 values less than (to_date('01-JUN-1999','DD-MON-YYYY'),9999)
TABLESPACE users,
partition G_jun99 values less than (to_date('01-JUL-1999','DD-MON-YYYY'),4999)
TABLESPACE users,
partition C_jun99 values less than (to_date('01-JUL-1999','DD-MON-YYYY'),9999)
TABLESPACE users,
partition G_jul99 values less than (to_date('01-AUG-1999','DD-MON-YYYY'),4999)
TABLESPACE users,
partition C_jul99 values less than (to_date('01-AUG-1999','DD-MON-YYYY'),9999)
TABLESPACE users
);
Sample Insert....
insert into P_Main_Test
(Trans_id, P_date, client_id,f1,f2,f3)
values
(1, '02-MAY-1999',4000,'aaa','bbb','ccc');
REM
insert into P_Main_Test
(Trans_id, P_date, client_id,f1,f2,f3)
values
(1, '02-MAY-1999',6000,'aaa','bbb','ccc');
insert into P_Main_Test
(Trans_id, P_date, client_id,f1,f2,f3)
values
(1, '01-JUN-1999',4000,'aaa','bbb','ccc');
REM
insert into P_Main_Test
(Trans_id, P_date, client_id,f1,f2,f3)
values
(1, '01-JUN-1999',6000,'aaa','bbb','ccc');
sample selects...
select * from P_Main_Test partition(G_MAY99)
2 /
TRANS_ID CLIENT_ID P_DATE F1 F2 F3
---------- ---------- --------- ---------- ---------- ----------
1 4000 02-MAY-99 aaa bbb ccc
1 6000 02-MAY-99 aaa bbb ccc
1 4000 01-JUN-99 aaa bbb ccc
SQL> Prompt C - May 99
C - May 99
SQL> select * from P_Main_Test partition(C_MAY99)
2 /
TRANS_ID CLIENT_ID P_DATE F1 F2 F3
---------- ---------- --------- ---------- ---------- ----------
1 6000 01-JUN-99 aaa bbb ccc
I understand why, based on the Vector discussions, the both client id's are in G_MAY99 and why 1 June is in G_MAY99... I'm just trying to figure out a way to make this work...
Thanks again
May 17, 2002 - 8:11 am UTC
I don't think you can do it with your case (so many values for client_id).
In their case, they had 5 city ids. They could set up a range like:
partition c107_0102 values less than (107,to_date('01-feb-2002','DD-MON-YYYY')),
partition c107_0202 values less than (107,to_date('01-mar-2002','DD-MON-YYYY')),
partition c107_0302 values less than (107,to_date('01-apr-2002','DD-MON-YYYY')),
partition c107_0402 values less than (107,to_date('01-may-2002','DD-MON-YYYY')),
...
partition c107_1202 values less than (107,to_date('01-jan-2003','DD-MON-YYYY')),
partition c108_0102 values less than (108,to_date('01-feb-2002','DD-MON-YYYY')),
partition c108_0202 values less than (108,to_date('01-mar-2002','DD-MON-YYYY')),
partition c108_0302 values less than (108,to_date('01-apr-2002','DD-MON-YYYY')),
partition c108_0402 values less than (108,to_date('01-may-2002','DD-MON-YYYY')),
...
partition c108_1202 values less than (108,to_date('01-jan-2003','DD-MON-YYYY')),
......
since they have 5 discrete values -- doing this is feasible.
You have 10,000 values, not feasible. (see they can have 12 partitions for city_id = 107, 12 for city_id = 108 and so on. You would have to have N partitions for cust_id = 1, N partitions for cust_id = 2 and so on.
In your case, recommand range partitioning by P_DATE and hash subpartitioning by client_id (client_id has a big set of values -- 1..10000 -- hence it'll hash evenly into two buckets very nicely. You'll achieve what you in the end.
Similiar issue
Rob, September 09, 2002 - 1:46 pm UTC
Tom:
I have a similiar issue to the second follow-up in this thread.
I have a table that is partitioned by week (52 partitions). Many of the queries against this table have a where clause whereby they select a group of stores but don't have a reference to week in the where clause. Each query then has to scan every partition to get the results. I was thinking of subpartitioning the table on the store column (several thousand unique values in the store column) with 8 or 16 subpartitions. My hope is that for a query with a range of store values in the where clause it will only have to scan a limited number of subpartitions for each week partition, thereby cutting down on the IO. For other reasons I don't want to have an index on the store column.
What is the best way to determine if I am getting partition elimination with subpartitions. I have looked at @$ORACLE_HOME/rdbms/admin/utlxpls.sql, but I am not clear on the meaning. Here is an example:
Plan Table
--------------------------------------------------------------------------------
| Operation | Name | Rows | Bytes| Cost | Pstart| Pstop |
--------------------------------------------------------------------------------
| SELECT STATEMENT | | 31 | 3K| 773308 | | |
| PARTITION RANGE INLIST | | | | |KEY(I) |KEY(I) |
| PARTITION HASH ALL | | | | | 1 | 8 |
| TABLE ACCESS FULL |TDS_QCF | 31 | 3K| 773308 |KEY(I) |KEY(I) |
--------------------------------------------------------------------------------
Does partition range inlist KEY(I) KEY(I) mean that it will read a limited number of partitions, and then within those partitions, the partition hash all Pstart and Pstop indicate the subpartitions for each partition that will be accessed since the same value of store will hash to the same subpartition within each partition, for example store_id 404 would hash to subpartition 3 within each partition.
Rob
September 09, 2002 - 7:40 pm UTC
Well, with a comment like "My hope is that for a query with a range of store values in the where clause it will only have to scan a limited number of subpartitions for each week partition", you can stop. HASH partitions cannot do "range" eliminations.
So, if you do "where store between A and B" -- no joy.
If you do "where store = A or store = B" -- you will have joy.
The above query plan shows partition elimination kicking in. I would want to see the query to diagnose the plan normally
Some addl. info
Rob, September 09, 2002 - 9:52 pm UTC
Tom:
I have been doing some more thinking about this composite partitioned table. It seems like the docs recommend hash partitioning on something close to a unique key. While I have about 3000 unique values for the store_id column, the table contains 1.2 billion rows.
I am trying to decide how many subpartitions to specify. My thinking is that the more subpartitions there are the less IO there will be in query that has a where clause like
where store_id in (23,56,980) since it may be that all 3 of these values hash to a different subpartition, each subpartition will be smaller if I choose to create many subpartitions. On the other hand, is there a lot of overhead to having several thousand subpartitions. I now find that there may be 120 weeks of data therefore 120 partitions and if I have 32 subpartitions that would be a total of 3840 subpartitions.
What do you think of the package dbms_pclxutil for rebuilding partitioned indexes in parallel. Have you used it?
September 10, 2002 - 7:26 am UTC
And with only 3000 values, it could well be that some of those 32 subpartitions could be empty -- depends on how well your stores hash (should probably be OK)
4000 partitions would be doable. It'll take a couple of minutes for the create table to go...
That package works just fine. It basically creates the index unusable and then lets you bulid the index subpartitions a bit at a time using dbms_jobs in the background. You could achieve the same yourself manually if you choose (just fire off a couple of "alter index rebuilds online" for the index partitions simultaneously)
A reader, January 03, 2003 - 8:14 am UTC
Hi Tom
When I execute package dbms_pclxutil I get an Oracle-error:
PROCEDURE dbms_pclxutil.build_part_index
Argument Name Type In/Out Default?
------------------------------ ----------------------- ------ --------
JOBS_PER_BATCH NUMBER IN DEFAULT
PROCS_PER_JOB NUMBER IN DEFAULT
TAB_NAME VARCHAR2 IN DEFAULT
IDX_NAME VARCHAR2 IN DEFAULT
FORCE_OPT BOOLEAN IN DEFAULT
exec dbms_pclxutil.build_part_index(1,4,'PART_TABLE','PART_INDEX')
ORA-06502: PL/SQL: numeric or value error: character to number conversion error
ORA-06512: at "SYS.DBMS_PCLXUTIL", line 50
ORA-06512: at line 1
Because package body is wrapped and no help on Metalink is present I cannot see the problem on line 50. Please advice. Thanks.
January 03, 2003 - 8:20 am UTC
can you give me the entire schema necessary to reproduce this? versions, OS, etc appreciated as well.
Marc, January 03, 2003 - 8:28 am UTC
Thanks for extreme fast feedback. We have Oracle Version 8.1.7.4.0 - 64bit Production on Sun Solaris 2.8 with 4 CPU's and 16 GB RAM. The table part_table has 4 partitions and two indexes. I also have tried DBMS_PCLXUTIL for other partitioned tables with the same problem.
Kind Regards.
January 03, 2003 - 9:41 am UTC
give me the create table and everything please.
Marc, January 03, 2003 - 10:29 am UTC
Here is my test script:
CREATE TABLE PART_TABLE
(DAT_PER DATE NOT NULL
,CIFNR VARCHAR2(12) NOT NULL
,WERTEART VARCHAR2(5) NOT NULL
,WERT NUMBER(15,2) NOT NULL
)
TABLESPACE USERS
PARTITION BY RANGE (DAT_PER)
(PARTITION PART_TABLE_2002_M01 VALUES LESS THAN (TO_DATE('01.02.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M02 VALUES LESS THAN (TO_DATE('01.03.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M03 VALUES LESS THAN (TO_DATE('01.04.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M04 VALUES LESS THAN (TO_DATE('01.05.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M05 VALUES LESS THAN (TO_DATE('01.06.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M06 VALUES LESS THAN (TO_DATE('01.07.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M07 VALUES LESS THAN (TO_DATE('01.08.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M08 VALUES LESS THAN (TO_DATE('01.09.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M09 VALUES LESS THAN (TO_DATE('01.10.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M10 VALUES LESS THAN (TO_DATE('01.11.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M11 VALUES LESS THAN (TO_DATE('01.12.2002','DD.MM.YYYY'))
,PARTITION PART_TABLE_2002_M12 VALUES LESS THAN (TO_DATE('01.01.2003','DD.MM.YYYY'))
);
CREATE INDEX PART_INDEX ON PART_TABLE
(WERTEART)
TABLESPACE USERS
LOCAL
UNUSABLE;
execute dbms_pclxutil.build_part_index(1,4,'PART_TABLE','PART_INDEX')
Thanks for your investigations.
January 03, 2003 - 11:20 am UTC
just for grins -- what are your NLS/character set settings?
Marc, January 03, 2003 - 12:00 pm UTC
SQL> select * from NLS_SESSION_PARAMETERS order by 1;
PARAMETER VALUE
------------------------------ ----------------------------------------
NLS_CALENDAR GREGORIAN
NLS_COMP BINARY
NLS_CURRENCY $
NLS_DATE_FORMAT DD-MON-YYYY
NLS_DATE_LANGUAGE AMERICAN
NLS_DUAL_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_LANGUAGE AMERICAN
NLS_NUMERIC_CHARACTERS .,
NLS_SORT BINARY
NLS_TERRITORY AMERICA
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZH:TZM
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZH:TZM
SQL> select * from NLS_DATABASE_PARAMETERS order by 1;
PARAMETER VALUE
------------------------------ ----------------------------------------
NLS_CALENDAR GREGORIAN
NLS_CHARACTERSET WE8ISO8859P1
NLS_COMP BINARY
NLS_CURRENCY $
NLS_DATE_FORMAT DD-MON-YYYY
NLS_DATE_LANGUAGE AMERICAN
NLS_DUAL_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_LANGUAGE AMERICAN
NLS_NCHAR_CHARACTERSET WE8ISO8859P1
NLS_NUMERIC_CHARACTERS .,
NLS_RDBMS_VERSION 8.1.7.4.0
NLS_SORT BINARY
NLS_TERRITORY AMERICA
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZH:TZM
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZH:TZM
January 03, 2003 - 12:50 pm UTC
sorry -- tried everything I could to reproduce, I cannot. This looks like a "new problem" -- I'll have to refer you to support -- they can help set some events to see what is happening.
A reader, January 04, 2003 - 6:05 am UTC
Thank you for feedback. I'll contact Oracle Support.
subpartitions
umesh, February 05, 2003 - 4:00 am UTC
why is my insert inserting 3 rows here please explain
1 CREATE TABLE MYEMP
2 (EMPNO NUMBER,JOB VARCHAR2(20), DEPTNO NUMBER)
3 PARTITION BY RANGE (DEPTNO)
4 SUBPARTITION BY HASH (JOB) SUBPARTITIONS 4
5 ( PARTITION P1 VALUES LESS THAN (11),
6 PARTITION P2 VALUES LESS THAN (21),
7 PARTITION P3 VALUES LESS THAN (31),
8* PARTITION P4 VALUES LESS THAN (MAXVALUE))
scott> /
Table created.
scott> INSERT INTO MYEMP
2 SELECT &EMPNO,'&ENAME',&DEPTNO FROM DUAL;
Enter value for empno: 1
Enter value for ename: MANAGER
Enter value for deptno: 10
old 2: SELECT &EMPNO,'&ENAME',&DEPTNO FROM DUAL
new 2: SELECT 1,'MANAGER',10 FROM DUAL
3 rows created.
scott> /
Enter value for empno: 2
Enter value for ename: CLARK
Enter value for deptno: 20
old 2: SELECT &EMPNO,'&ENAME',&DEPTNO FROM DUAL
new 2: SELECT 2,'CLARK',20 FROM DUAL
3 rows created.
scott> SELECT * FROM MYEMP;
EMPNO JOB DEPTNO
---------- -------------------- ----------
1 MANAGER 10
1 MANAGER 10
1 MANAGER 10
2 CLARK 20
2 CLARK 20
2 CLARK 20
6 rows selected.
scott>
scott> select trigger_name from user_triggers
2 where table_name = upper('myemp');
no rows selected
February 05, 2003 - 8:10 am UTC
you have 3 rows in dual. dual is magical, dual is a SYS owned object. You have allowed someone to manually modify a SYS owned table which is not allowable.
ops$tkyte@ORA817DEV> create table t ( x int );
Table created.
ops$tkyte@ORA817DEV> insert into dual values ( 'x' );
1 row created.
ops$tkyte@ORA817DEV> insert into dual values ( 'x' );
1 row created.
ops$tkyte@ORA817DEV> select * from dual;
D
-
x
ops$tkyte@ORA817DEV> select count(*) from dual;
COUNT(*)
----------
3
ops$tkyte@ORA817DEV> insert into t select rownum from dual;
3 rows created.
ops$tkyte@ORA817DEV> select * from t;
X
----------
1
2
3
ops$tkyte@ORA817DEV> rollback;
Rollback complete.
correct your DUAL table by:
delete from dual where rownum <= 2;
commit;
How many values for subpartition
Scott Wesley, June 26, 2003 - 11:48 pm UTC
G'day Tom.
You say has subpartitions work best with a LARGE amount of subpartitions.
Please define large.
A few Metalink notes I've read discussing composite partitioning (eg: Note 125314.1) use 8 subpartitions, and recommend using a number to the power of 2. (2,4,8,16 etc).
We are proposing partitioning a sales table by financial year, and subpartitioning by state (9 possible values in Australia), however your comments suggest that these subpartitions may not be used.
Your comments would be appreciated.
Thanks.
June 27, 2003 - 8:42 am UTC
sorry, it was not clear.
It is not that you want/need LARGE numbers of subpartitions but rather that:
....
The hash function (as do all hash functions) work best with a LARGE number of
values. Using hash subpartitions with a partition key that has only 5 values is
not very useful ....
You want a column with a large number of distinct values (a unique column works best) to be the column you hash on. If you hash into 8 subpartitions using a key with 12 distinct values, you may well find that many of your subpartitions are *empty* since nothing will hash there.
The hash key should have a large domain of values so as to achieve a good distribution across the partititions
Range + hash with common prefix on hashed string
Mike, January 22, 2004 - 3:22 pm UTC
Based on above discussion, we have data with a great profile for V9.2 range partitioning by date + hash subpartitioning on a unique string identifier. We have hundreds of unique string ids so it would seem the range+hash partitioning would distribute the data nicely into, say, 200 or so subpartitions for each month.
Note: our string identifiers are unique but all have a common suffix. Example:
SPQID
----------
usabakerj
usamastersj
canporterp
canllensl
Please don't scold me for IDs with embedded country code. Rather, I'm interested to know if you think the hash distribution of strings like this will distribute somewhat evenly of if the hash algorithm will steer all the values into just a few sub-partitions (e.g. usa% and can%)?
I had a crazy thought to reverse the key:
partition by range ( spq_date )
subpartition by hash( reverse( spqid ) )
but adding the reverse() function leads to parse errors during table creation.
Thanks for any comments you may have,
-Mike
January 22, 2004 - 8:26 pm UTC
ops$tkyte@ORA10G> CREATE TABLE t
2 (
3 data varchar2(80)
4 )
5 PARTITION BY hash (data) partitions 4
6 /
Table created.
ops$tkyte@ORA10G>
ops$tkyte@ORA10G> insert into t
2 select 'usa' || object_name from all_objects;
47079 rows created.
ops$tkyte@ORA10G>
ops$tkyte@ORA10G> declare
2 l_cnt number;
3 begin
4 for x in ( select partition_name
5 from user_tab_partitions
6 where table_name = 'T' )
7 loop
8 execute immediate
9 'select count(*)
10 from t partition(' || x.partition_name || ')'
11 into l_cnt;
12 dbms_output.put_line
13 ( 'partition ' || x.partition_name || ' ' || l_cnt );
14 end loop;
15 end;
16 /
partition SYS_P185 11856
partition SYS_P186 12062
partition SYS_P187 11664
partition SYS_P188 11497
PL/SQL procedure successfully completed.
ops$tkyte@ORA10G>
it'll spread it out
Thank you!
Mike, January 24, 2004 - 11:36 pm UTC
Great example and test. Thank you very much and I will charge ahead with partitioning our data.
A reader, June 17, 2004 - 6:03 pm UTC
I have created a range partition table based on a composite key. Earlier the same table was range partitioned on one attribute.
After recreating the table with a composite key, the number of partitions went up to 9000. Earlier it was 900.
I am seeing the normal DML commands like Insert into the partitioned table is taking almost 2 minutes to insert 20K records and select from the table for the same 20k records is taking almost 1 minute.
On a non-partitioned table for the same 20k records, the insert and select are under a second.
Why such a drastic change in the DML statements on a composite key range partitioned table?
Thanks,
June 17, 2004 - 6:43 pm UTC
tkprof says? show me a tkprof for both.
Partioning qn
Rich, June 17, 2004 - 7:01 pm UTC
Tom:
I have a table which will store monthly usage data. We have to keep this data for several years. This data will be used for trending reports. I am thinking of creating the table with 12 partition - each partion per month. But when I have data for couple of years I don't want all Jan data in one partition, Feb data in 2nd partition etc etc. I would like to have month and year as the partition key. How do I create a table with month and year as the composite partition key.
June 17, 2004 - 7:09 pm UTC
just like you would have for month? using ranges -- why do you think it would be different?
LIST can be used as subpartitions in 9.2
Adrian, June 18, 2004 - 5:31 am UTC
Tom,
Regarding your statement:-
>> Even in 9i when LIST partitions are added -- they will not help you achieve your specific goal in this case as they cannot be used for subpartitions...
It might help future readers of this thread to know that this is certainly possible in 9.2 ( I guess you must have been talking about 9.0 given the age of the thread ).
920> CREATE TABLE sub_by_list
2 ( x INT
3 , y DATE
4 , z VARCHAR2(1)
5 )
6 PARTITION BY RANGE ( y )
7 SUBPARTITION BY LIST ( z )
8 SUBPARTITION TEMPLATE
9 (
10 SUBPARTITION sp1 VALUES ( 'A', 'B', 'C' )
11 , SUBPARTITION sp2 VALUES ( 'D', 'E', 'F' )
12 , SUBPARTITION df VALUES ( DEFAULT )
13 )
14 (
15 PARTITION p1 VALUES LESS THAN ( DATE '2003-01-01' )
16 , PARTITION p2 VALUES LESS THAN ( DATE '2004-01-01' )
17 , PARTITION mv VALUES LESS THAN ( MAXVALUE )
18 );
Table created.
920>
920> INSERT INTO sub_by_list
2 SELECT object_id
3 , created
4 , SUBSTR( object_name, 1, 1 )
5 FROM user_objects;
706 rows created.
920>
920> ANALYZE TABLE sub_by_list COMPUTE STATISTICS;
Table analyzed.
920>
920> SELECT table_name
2 , partition_name
3 , subpartition_name
4 , num_rows
5 FROM user_tab_subpartitions;
TABLE_NAME PARTITION_NAME SUBPARTITION_NAME NUM_ROWS
------------ --------------- -------------------- ----------
SUB_BY_LIST MV MV_SP1 94
SUB_BY_LIST MV MV_SP2 17
SUB_BY_LIST MV MV_DF 169
SUB_BY_LIST P1 P1_SP1 0
SUB_BY_LIST P1 P1_SP2 0
SUB_BY_LIST P1 P1_DF 0
SUB_BY_LIST P2 P2_SP1 156
SUB_BY_LIST P2 P2_SP2 73
SUB_BY_LIST P2 P2_DF 197
9 rows selected.
Regards
Adrian
June 18, 2004 - 10:45 am UTC
yes, thanks, things change :)
A reader, June 18, 2004 - 1:48 pm UTC
I have generated the tkprof for both the insert statements and pasting it below.
The first insert statement inserts into a range partitioned table. The number of partitions is 8000.
the second insert statement is a simple statement on a non-partitioned table.
********************************************************************************
insert into rpt_fact_retail_blend_0617(bipad,issue_code,yr_term_code)
select bipad,issue_code,yr_term_code from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.03 0.10 0 0 0 0
Execute 1 5.08 225.26 13228 17635 54762 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 5.11 225.36 13228 17635 54762 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Rows Row Source Operation
------- ---------------------------------------------------
20229 TABLE ACCESS FULL RPT_BIPAD_ISSUE_0617 (cr=120 r=159 w=0 time=843776 us)
************************************************************
insert into a select * from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.01 0 3 0 0
Execute 1 0.15 0.93 124 270 745 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.15 0.94 124 273 745 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Rows Row Source Operation
------- ---------------------------------------------------
20229 TABLE ACCESS FULL RPT_BIPAD_ISSUE_0617 (cr=120 r=117 w=0 time=589824 us)
************************************************************
Thanks,
June 18, 2004 - 1:58 pm UTC
umm, we are comparing apples and toaster ovens here?
compare partitioned to partitioned as you said you had please.
and if you could, use a 10046 level 12 trace and show your waits.
A reader, June 18, 2004 - 2:17 pm UTC
Alright, the tkprof report for a insert into a partitioned table. This partitoned table has 900 partitions.
********************************************************************************
insert into rpt_fact_retail_blend3(bipad,issue_code,yr_term_code)
select bipad,issue_code,yr_term_code from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.02 0 0 0 0
Execute 1 1.33 20.58 4402 8309 12839 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 1.35 20.60 4402 8309 12839 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
********************************************************************************
If I see the CPU time, elapsed time, it is lowest on a non-partioned table, better in a partitoned table that has 900 partitions and probably worse in a partitioned table that has 8000 partitions.
I am not sure how to explain this to my manager.
Help me out.
Thanks,
June 18, 2004 - 4:48 pm UTC
well, it is mostly a matter of "math"
lets say you get 50 rows/block (pretty low here, with such a small row)
a non partitioned table -- dirty about 400 blocks (and not think a bit about where they go)
a 900 partition table -- with random distribution, each partition gets about 22 rows. 900 dirtied blocks
a 9000 partition table -- with random distribution, each partition gets about 2/3 rows. 9,000 dirtied blocks.
Now, lets talk about indexes that might exist!
And the cpu used to figure out which of the 9,000 partitions you go into.
I set up a
o regular nonpartitioned table T0
o a 900 partition table T1 (where x = 1 goes into part1, x=2 into part2 and so on)
o a 9000 partition table T2 (same x= logic)
Now consider:
insert into t0 select mod(rownum,1), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.49 0.54 116 514 1798 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.49 0.54 116 514 1798 50000
********************************************************************************
insert into t1 select mod(rownum,1), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.44 0.48 3 138 1723 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.44 0.48 3 138 1723 50000
********************************************************************************
insert into t2 select mod(rownum,1), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.05 0 0 0 0
Execute 1 0.53 0.52 3 317 1752 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.54 0.57 3 317 1752 50000
see, they all did about the same amount of work -- about the same amount of cpu. All data went into a single partition here -- when all things are constant, all things are "constant"
but....
insert into t0 select mod(rownum,900), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.50 0.55 120 538 1981 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.50 0.55 120 538 1981 50000
********************************************************************************
insert into t1 select mod(rownum,900), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 10.90 25.62 130523 51036 167155 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 10.90 25.62 130523 51036 167155 50000
********************************************************************************
insert into t2 select mod(rownum,9000), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.01 0 0 0 0
Execute 1 36.93 241.99 191710 59138 313037 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 36.94 242.00 191710 59138 313037 50000
when you slosh the data all over the place, data arriving way out of order (i inserted 1, 2, 3, 4, ... 9000, 1, 2, 3, .... 9000, and so on) you get the worst case. "insert into part1", insert into part2, and so on. Did *the most work* possible in order to insert this data.
It is interesting to note what happens when we remove the scattered arrival of data:
insert into t0 select mod(rownum,900), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.00 0 0 0 0
Execute 1 0.52 0.55 113 530 1955 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.53 0.56 113 530 1955 50000
********************************************************************************
insert into t1 select * from t0 order by x
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 1 0 0
Execute 1 0.54 3.16 2995 1073 3137 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.54 3.16 2995 1074 3137 50000
here, the 900 partition table does about 2x the work . Just by making the data get inserted into the partition "in order" in this bulk load (eg: order by partition key). Now, there was still extra work since there were so many partitions and so little data. going further:
insert into t0 select mod(rownum,9000), rownum, rownum
from big_table.big_table no_index
where rownum <= 50000
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.50 0.62 113 537 1941 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.50 0.62 113 537 1941 50000
********************************************************************************
insert into t2 select * from t0 order by x
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.02 0.01 0 1 0 0
Execute 1 7.79 96.48 27881 18173 189207 50000
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 7.81 96.50 27881 18174 189207 50000
again, a large reduction, but an expected "difference". It is like you are inserting into 9,000 tables, with 9,000 freelists, with 9,000 segment headers to check out and so on
So, it'll be purely a function of how many partitions you actually touch here. And whether you do them haphazzardly or "orderly".
Almost every "physical structure" will impose (necessarily) an insert overhead -- for the goals of better retrieval/admin (we retrieve many times, insert once)
o IOTS
o hash cluster
o b*tree cluster
o partitioned tables
all of these physical origanization things add work to the insert process (you are dictating "data must go somewhere") as opposed to the standard, boring HEAP table.
A reader, June 18, 2004 - 4:18 pm UTC
I have generated tkprof by setting events 10046 level 12.
The first insert statement is on a non-partitioned table.
the second one is on a range partitioned table that has 8000 partitions.
The third insert statement is on a range partitioned table that has 900 partitioned tables
********************************************************************************
insert into a select * from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.01 0 0 0 0
Execute 1 0.15 0.54 123 386 749 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.16 0.56 123 386 749 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Rows Row Source Operation
------- ---------------------------------------------------
20229 TABLE ACCESS FULL RPT_BIPAD_ISSUE_0617 (cr=120 r=118 w=0 time=370688 us)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 6 0.01 0.07
db file scattered read 8 0.05 0.32
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 2.56 2.56
********************************************************************************
insert into rpt_fact_retail_blend_0617(bipad,issue_code,yr_term_code)
select bipad,issue_code,yr_term_code from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.06 0.06 0 0 0 0
Execute 1 8.11 205.77 14130 19579 55737 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 8.18 205.84 14130 19579 55737 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Rows Row Source Operation
------- ---------------------------------------------------
20229 TABLE ACCESS FULL RPT_BIPAD_ISSUE_0617 (cr=120 r=223 w=0 time=772096 us)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 13911 0.59 197.41
db file scattered read 19 0.12 0.63
latch free 4 0.01 0.01
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 20.62 20.62
********************************************************************************
insert into rpt_fact_retail_blend3(bipad,issue_code,yr_term_code)
select bipad,issue_code,yr_term_code from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 1.65 15.05 1583 8314 12848 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 1.65 15.06 1583 8314 12848 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file scattered read 8 0.07 0.44
db file sequential read 1466 0.15 12.95
SQL*Net message to client 1 0.00 0.00
********************************************************************************
Thanks
June 18, 2004 - 4:48 pm UTC
see above
A reader, June 18, 2004 - 5:09 pm UTC
Thanks for the great example. I tried to insert into a table that has 8000 partitions ordering by the range partitioned column. It seems even worse.
I have the tkprof report for that.
insert into rpt_fact_retail_blend_0617(bipad,issue_code,yr_term_code)
select bipad,issue_code,yr_term_code from rpt_bipad_issue_0617
order by yr_term_code
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.02 0 0 0 0
Execute 1 5.70 346.11 15813 7967 7909 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 5.71 346.13 15813 7967 7909 20229
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 15696 0.59 340.07
db file scattered read 8 0.19 1.07
SQL*Net message to client 1 0.00 0.00
***********************************************************************
Thanks,
June 18, 2004 - 6:07 pm UTC
no, it was better -- your IO waits were just higher for whatever reason (more people on the machine, whatever)
insert into rpt_fact_retail_blend_0617(bipad,issue_code,yr_term_code)
select bipad,issue_code,yr_term_code from rpt_bipad_issue_0617
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.06 0.06 0 0 0 0
Execute 1 8.11 205.77 14130 19579 55737 20229
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 8.18 205.84 14130 19579 55737 20229
last time, 50% more CPU, less elapses -- but many many more query and current mode gets.
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 29
Rows Row Source Operation
------- ---------------------------------------------------
20229 TABLE ACCESS FULL RPT_BIPAD_ISSUE_0617 (cr=120 r=223 w=0 time=772096
us)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 13911 0.59 197.41
db file scattered read 19 0.12 0.63
latch free 4 0.01 0.01
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 20.62 20.62
last time you waited for IO a little less often and each time you waited on average -- it was less. You just got "unlucky"
A reader, June 22, 2004 - 10:58 am UTC
Thanks for a lot of info.
I have one more question......
After inserting the data into the table that has 8000 partitions, I expect the query to return fast compared to accessing the 900 partition table. It seems it is not.
The old table is partitioned by magazine ID (900 partitions) and the new table is partitioned by magazine ID and monthly code(8000 partitions). So roughly the same magazine ID is distributed to 10 different partitions(on an average) in the new table.
Do you think having the new table is a good idea for query purpose? Our users will always query by magazine and they restrict by time period by saying give me the sales of a magazine for the time period between jan and mar of 2004. Now the new table has data for three months in three different partitions.
Any suggestion in this matter?
Thanks,
June 22, 2004 - 1:53 pm UTC
why did you have this expectation?
two cases:
o you accessed a row by index. You still are accessing the row by index -- index root/branch/leaf -> table access by rowid.
You'll have 2/3/4 LIO's on the index, and one on the table.
After partitioning in N thousand partitions you'll have 2/3/4 LIO's on the index and one on the table!
You might -- may, maybe, might have -- decreased the height of the index (so it now takes 2 LIO's instead of 3) but you won't notice that very much.
o full scan.
before you had N'thousand rows to read
after you have N'thousand rows to read
with partitioning, you are mostly feeling good when you introduce it and performance doesn't go down the tubes! partitioning != "fast=>true". partitioning is a tool, something you need to understand a bit about "what it is doing, what it can do"
when can partitioning speed up stuff?
o by removing a large percentage of a table from consideration during a full scan using partition eliminiation. (so full scan could be faster due to less data processed)
o partitionwise joins
o facilitating parallel update (not necessary but can be used to ensure segregation of segments over physical devices to reduce io contention)
o to reduce contention on heavily inserted into objects (you can have N indexes and N table segments instead of just 1)
when can it not (or have nominal impact)?
o when you are index reading the table. Here you must ensure your partitioning scheme does not negatively impact performance.
o when partition elimination doesn't stand a chance of kicking in.
In your case, I'd guess the former, you are index reading. No big change. That's about the best you could expect.
What is your goal with partitioning? Ease of management? Rolling windows (to age)? What is it you need to achieve. If you are using index access to data, partitioning isn't going to significantly improve your query performance (quite the opposite can be true if you do it wrong).
index partitions in the table partition DML commands
A reader, July 12, 2004 - 8:38 pm UTC
Hi Tom,
In 10g certification outline there is an item:
"Specify storage characteristics for index partitions in the table partition DML commands"
Could you help me give an example? I really can't figure out what it is.
Thanks,
July 12, 2004 - 11:47 pm UTC
pointer to original document you are refering to for context?
From here:
A reader, July 13, 2004 - 1:52 am UTC
July 13, 2004 - 11:24 am UTC
me, no - i'm not certified OCP.
Which index to choose?
Ron, July 16, 2004 - 2:27 pm UTC
Tom:
1)at conditions determine whether to use a Global Index or a Partitioned Index?
2)I have a table that i want to partition by day.Every day i drop the oldest partition and create a new one.Effectively, the table has to have seven partitions(holds a week worth of data).What kind of partitioning method do you recommend and what kind of index
Thanks
July 16, 2004 - 10:51 pm UTC
if you have access to either of my books -- I cover some of the design background you need to pick local or global.
one cannot answer the 2cnd question without understanding
a) the physics of partitioning
b) the needs of your system.
Subpartition later?
A reader, September 08, 2004 - 11:43 pm UTC
Suppose I have range partitioning in my database as per the sizing requirements at the time. The thought was that the ranges would contain similar data volumes (monthly partitions, for example).
But it is later seen that some months have order of magnitude more data than others.
Is it possible to subpartition just these big partitions by using hash partitioning?
Thanks
September 09, 2004 - 8:05 am UTC
all would be subpartitioned (or not) -- however, each partition may have a different number of subpartitions! so some could have 1, others 8, 16, 32, whatever. (power of 2)
Different number?
A reader, September 09, 2004 - 1:38 pm UTC
"however, each partition may have a different number of subpartitions!"
How is this possible, how can I set this up?
Thanks
September 09, 2004 - 2:13 pm UTC
Well how about...?
jc, September 12, 2004 - 12:00 am UTC
Hi, Tom. There's a great deal of good info in this thread! One variation on the partitioning theme - can you partition in 9iR2 by list and then subpartition by hash? Thank you.
Regards.
September 12, 2004 - 11:05 am UTC
composite in 9ir2 always starts with "range" at the top and the subpartitions can be LIST or HASH.
Building LOCAL subpartition index
David, October 25, 2004 - 7:35 pm UTC
Hi Tom, thank you for you answers.
I have a table partitioned by month and subpartitioned by list(user_id)
I'm trying to figure out the syntax for creating local index on that table where I would assign subparitionin segments to diff tablespaces.
here is the table:
(sorry for tablespace syntax, it's relevant to the example)
CREATE TABLE sum_table
(
USER_ID NUMBER NOT NULL
,DATE_OCCURRED DATE
,QTY_OCURRED NUMBER
) PARTITION BY RANGE (DATE_OCCURRED)
SUBPARTITION BY LIST (USER_ID)
SUBPARTITION TEMPLATE
(
SUBPARTITION uid_1165 VALUES (116,117) TABLESPACE tbl_3,
SUBPARTITION uid_1166 VALUES (284) TABLESPACE tbl_4,
SUBPARTITION uid_1167 VALUES (307) TABLESPACE tbl_3,
SUBPARTITION uid_1168 VALUES (3515) TABLESPACE tbl_4,
SUBPARTITION uid_others VALUES (DEFAULT ) TABLESPACE tbl_3
)
(PARTITION PSE_SEP_04 VALUES LESS THAN ('2004-09-01') TABLESPACE tbl_3,
PARTITION PSE_OCT_04 VALUES LESS THAN ('2004-10-01') TABLESPACE tbl_4,
PARTITION PSE_NOV_04 VALUES LESS THAN ('2004-11-01') TABLESPACE tbl_3,
PARTITION PSE_max VALUES LESS THAN (MAXVALUE) TABLESPACE tbl_4)
NOLOGGING NOCACHE PARALLEL;
I can create index only assigning tablespace to partitions like that:
CREATE INDEX sum_PUB_SID_E_po_IDX2 ON sum_PUB_SID_E (USER_ID,DATE_OCCURRED,OFFER_ID) PARALLEL LOCAL
(
PARTITION PSE_SEP_04 TABLESPACE tbl_i3,
PARTITION PSE_OCT_04 TABLESPACE tbl_i4,
PARTITION PSE_NOV_04 TABLESPACE tbl_i3,
PARTITION PSE_max TABLESPACE tbl_i4);
but it won't provide the benefit of splitting the IO for concurrent queries, cause most of the quries run against the latest month. So I want to distribute the users across diff tablespaces by something like this:
CREATE INDEX sum_PUB_SID_E_po_IDX2 ON sum_PUB_SID_E (USER_ID,DATE_OCCURRED,OFFER_ID) PARALLEL LOCAL
(
PARTITION PSE_SEP_04 TABLESPACE tbl_i3,
PARTITION PSE_OCT_04 TABLESPACE tbl_i4,
PARTITION PSE_NOV_04 TABLESPACE tbl_i3,
PARTITION PSE_max TABLESPACE tbl_i4);
SUBPARTITION TEMPLATE
(SUBPARTITION uid_1165 TABLESPACE tbl_i3,
SUBPARTITION uid_1166 TABLESPACE tbl_i4,
SUBPARTITION uid_1167 TABLESPACE tbl_i3,
SUBPARTITION uid_1168 TABLESPACE tbl_i4,
SUBPARTITION uid_others TABLESPACE tbl_i3
);
unfortunately oracle doesn't seem to understand the "SUBPARTITION TEMPLATE" clause in that context.
Is there any valid syntax I can use?
the # of part per #of subpart >800 so I woudn't want to explicitly list every subpartition.
Thank you for your time.
David
October 25, 2004 - 7:54 pm UTC
well -- don't you have a striped filesystem? I mean you have big stuff - isn't your filesystem already striped?
tablespaces are for administrative purposes really -- to spread IO out, you would just add lots of files from different devices to a single tablespace or (better) use striping of the filesystem...?
if you have 800 partitions -- I cannot imagine trying to place them all -- a big nice stripped filesystem is what I would want. Then, segment the indexes into tablespaces based on something that makes your life as an admin easier.
re: Building LOCAL subpartition index
David, October 25, 2004 - 8:33 pm UTC
well, it's getting big but we have pretty cheap setup:
2 JBODs with 4 raid volumes.
Anyway it's not something I can change right now.
The bottom line is, I have to balance the IO using tablespaces.
Just trying to figure out if some syntax exists at all to assign tablespace name to subpartitions of a local index (using subpartition template).
Thank you,
David
October 25, 2004 - 9:00 pm UTC
no, you balance IO with "files"
use small extents (1 meg or something) and 4 files on your 4 volumes and you'll find the index extents "round robin" all over... poor mans striping - even better than "this index partition here, that there and so on"
(haven't played with the subpartition template enough to directly answer -- but, i wouldn't use tablespaces to "spread IO out", files do that)
re: Building LOCAL subpartition index
David, October 26, 2004 - 12:02 pm UTC
OK, I guess I got more than I asked for :)
Thanks a lot Tom,
Your suggestion really makes tablespace-based IO distribution completely unnecessary.
I got it now and will definitely use it not only for that specific issue IÂ’m solving, but also to solve a general problem of IO balancing in my DB.
I only wonder if local tablespaces with auto segment management have any overhead from using large number of small extents? I guess itÂ’s knee-jerk reaction from the days of the dictionary-managed tablespaces but I wonder if 5G segment consisting out of 5000 1M extents will have any noticeable performance overhead for reads, writes and allocation?
Also where can I read to understand how local tablespaces and auto segment management works internally?
Thank you again,
I really appreciate your approach
(getting real practical answer instead of technically correct nonsense :)
David
October 26, 2004 - 12:05 pm UTC
<quote>
Your suggestion really makes tablespace-based IO distribution completely
unnecessary.
</quote>
as it always and forever has been!
if you use system allocated extents -- a 10g object will be in about 350 extents when all is said and done.
when you say 'auto segment management' though -- are you refering to ASSM (automatic segment space managment) or system allocated extent sizes?
(do you have "Effective Oracle by Design" -- i wrote about both)
re: Building LOCAL subpartition index
David, October 26, 2004 - 1:26 pm UTC
Yes, I use ASSM with uniform extent size on 9i r2.
Just read the section in "Effective Oracle by Design" -
found the book almost as useful as your site :D,
I only need to read it more often ;)
Here is my partitioned table copied to the new "RoundRobin" LMT with ASSM:
TABLESPACE_NAME FILE_NAME SIZE_MB
----------------- --------------------------------------- --------
RR_1M_SUM /u01/oradata/prod1/rr_1m_sum_u01_01.dbf 1152
RR_1M_SUM /u02/oradata/prod1/rr_1m_sum_u02_01.dbf 1840
RR_1M_SUM /u03/oradata/prod1/rr_1m_sum_u03_01.dbf 1312
RR_1M_SUM /u04/oradata/prod1/rr_1m_sum_u04_01.dbf 1232
Same if we look at the extents:
select file_id, count(*)
from dba_extents
where segment_name='SUM_PUB_SID'
group by rollup(file_id);
FILE_ID COUNT(*)
---------- ----------
43 1151
44 1839
45 1311
46 1231
Total 5532
The individual partition distribution however look more even:
select file_id,partition_name, count(*)
from dba_extents
where segment_name='SUM_PUB_SID'
group by file_id, partition_name
order by partition_name, file_Id;
FILE_ID PARTITION_NAME COUNT(*)
------- --------------- --------
43 PS_AUG_04_PID_1167 20
44 PS_AUG_04_PID_1167 23
45 PS_AUG_04_PID_1167 22
46 PS_AUG_04_PID_1167 20
43 PS_AUG_04_PID_1168 39
44 PS_AUG_04_PID_1168 38
45 PS_AUG_04_PID_1168 39
46 PS_AUG_04_PID_1168 36
...........................
43 PS_NOV_04_PID_1168 119
44 PS_NOV_04_PID_1168 111
45 PS_NOV_04_PID_1168 114
46 PS_NOV_04_PID_1168 113
1341 rows selected.
The bottom line is, I have 5532 extents this way.
Just wonder if there there any noticeable issue with extents bitmap in file headers
growing too big? Is 5532 too large?
Thank you,
David
October 26, 2004 - 2:17 pm UTC
that is over hundreds of segments though, no problems there.
unevenly spreaded partitions, exchange partitioned table
Marc, December 07, 2004 - 1:42 pm UTC
tom,
I got a big fact table which consists of 4 "sub" - tables, coming from the source which means
you could query star-schema-like like:
select DIM_A.level_1, DIM_B.level_2, sum(revenue)
from (
select DIM_A_SID, DIM_B_SID, revenue from fact_A
UNION ALL
select DIM_A_SID, DIM_B_SID, revenue from fact_B
UNION ALL
select DIM_A_SID, DIM_B_SID, revenue from fact_C
...
), DIM_A, DIM_B
where ...
group by DIM_A.level_1, DIM_B.level_2
source table A : 60 Mio rows ( on average. grows and shrinks --> records move into B or C )
source table B : 2 Mio rows per day (probably only 1 year stored)
source table C : 200 K rows
source table D : 50 Mio rows per year
I'd like to do composite : 1st list ('A', 'B', 'C', 'D')
and then range by date , because I have to exchange or extend "sub"- table (partition) A , add dayly subpartitions to B and so on ...
But that partition design isnt allowed ...
The issue is, that the records are moving between these 4 big tables and the finance-guys like to see a complete
view (union) of all tables at a certain time.
1st question is:
Can I garantuee similar performance on a
statement like the shown Union all compared with
a select on one fact table having "materialized" that union ?
2nd question:
What alternatives are there left ?
I got a merge statement (does the union inline) working, but now I got difficult delta criteria on the source tables to apply , otherwise the update cost of the merge grows heavily. But for source A for example the best way of population to a / the fact table would be a full refresh, especially because we got problems to define a delta ( timestamp > last_load_DateTime) criteria !
hope u can help,
kind regards
marc
December 07, 2004 - 8:49 pm UTC
1) no, not in general. 4 tables, 4 different sets of stats, optimizer optimizing a 4 table union all to be joined -- you'll get radically different plans potentially. they won't be comparable.
2) can't you use a vector here with a range?
ops$tkyte@ORA10GR1> Create Table t
2 (
3 code varchar2(1),
4 dt date
5 )
6 partition by range(code,dt)
7 (
8 partition p1a values less than ( 'A', to_date('01-jan-2004') ),
9 partition p2a values less than ( 'A', to_date('02-jan-2004') ),
10 partition p3a values less than ( 'A', to_date('03-jan-2004') ),
11 partition p1b values less than ( 'B', to_date('01-jan-2004') ),
12 partition p2b values less than ( 'B', to_date('02-jan-2004') ),
13 partition p3b values less than ( 'B', to_date('03-jan-2004') ),
14 partition p1c values less than ( 'C', to_date('01-jan-2004') ),
15 partition p2c values less than ( 'C', to_date('02-jan-2004') ),
16 partition p3c values less than ( 'C', to_date('03-jan-2004') ),
17 partition p1d values less than ( 'D', to_date('01-jan-2004') ),
18 partition p2d values less than ( 'D', to_date('02-jan-2004') ),
19 partition p3d values less than ( 'D', to_date('03-jan-2004') )
20 )
21 /
Table created.
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> declare
2 type vcarray is table of varchar2(1);
3 l_codes vcarray := vcarray('A','B','C','D');
4 begin
5 for i in 1..l_codes.count
6 loop
7 for j in 0..2
8 loop
9 insert into t values ( l_codes(i), to_date('31-dec-2003')+j );
10 end loop;
11 end loop;
12 end;
13 /
PL/SQL procedure successfully completed.
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> select 'p1a', x.* from t partition (p1a) x union all
2 select 'p2a', x.* from t partition (p2a) x union all
3 select 'p3a', x.* from t partition (p3a) x union all
4 select 'p1b', x.* from t partition (p1b) x union all
5 select 'p2b', x.* from t partition (p2b) x union all
6 select 'p3b', x.* from t partition (p3b) x union all
7 select 'p1c', x.* from t partition (p1c) x union all
8 select 'p2c', x.* from t partition (p2c) x union all
9 select 'p3c', x.* from t partition (p3c) x union all
10 select 'p1d', x.* from t partition (p1d) x union all
11 select 'p2d', x.* from t partition (p2d) x union all
12 select 'p3d', x.* from t partition (p3d) x;
'P1 C DT
--- - ---------
p1a A 31-DEC-03
p2a A 01-JAN-04
p3a A 02-JAN-04
p1b B 31-DEC-03
p2b B 01-JAN-04
p3b B 02-JAN-04
p1c C 31-DEC-03
p2c C 01-JAN-04
p3c C 02-JAN-04
p1d D 31-DEC-03
p2d D 01-JAN-04
p3d D 02-JAN-04
12 rows selected.
That is just like a LIST with a RANGE underneath it.
partition auditing
Marc, December 08, 2004 - 5:10 am UTC
Thanks Tom. Good idea.
Another question:
can I see if a partition got updated after a load ?
like :
table partition last_update
A A_1 null / 2004-01-01
A A_2 'Y' / 2004-02-02
...
December 08, 2004 - 10:41 am UTC
we do not track that information in the database.
vector partitioning or composite
Marc, December 14, 2004 - 5:05 am UTC
Tom,
considering your vector-range partition example above ...
alternitively I still could do composite partitioning:
by range (date) and
then subpartition by list (source)
one subpartition for each A,B,C,D
the major load I need to tune is the "exchange" (truncate & re-insert) of all records coming from source B ( approx. 60 Mio ) , so using the vecor I had to truncate all partitions of
"... less than values ('B', 'date1')" (B must be C to get B values categorized in there because of the less than ...)
"... ... ('B', 'date2')"
...
using subpartitions I had to drop the subpartition for 'B'
in every partition.
Will there be a considerable difference in load-time
between these 2 ways ?
I'd like to go for the composite-way - as I can use subpartition template and in general the splitting is more natural.
Could I still get performance problems using too many (> 1000) partitions as J. Lewis found out ( was in 8i ) ? -
on "drop partition" and querying using bind-variables !
thanks a lot,
Kind regards
Marc
December 14, 2004 - 9:03 am UTC
suggest you benchmark using your sample data, to see which is most efficient in your case (it is what I would have to do...)
every release makes managing thousands of extents "faster" than the last release.
IO distribution over partitioned segments
David, January 10, 2005 - 3:05 pm UTC
Tom,
continuing the IO distribution question above,
I'm trying to switch to Round Robin tablespaces striping over all 4 physical raids volumes, however while it should be good once everything is moved, right now RR tablespaces block the most (cause they block on every bottleneck on every volume)
Since the DB is big and very active, I need lots of time for rebuild.
I'm looking for a way to identify the segments that are most active IO-wise and migrate them first to RR.
Are there any apparent way to get that data from the oracle dictionary?
(Statspack only gives the breakdown by file and tablespace)
If not, any other ways?
Thank you very much,
David
January 10, 2005 - 4:21 pm UTC
version? there are segment stats in 9i, none in 8i.
IO distribution over partitioned segments for 9i
David, January 10, 2005 - 6:21 pm UTC
fortunately 9i r2 :)
IO distribution over partitioned segments 9i
David, January 10, 2005 - 6:24 pm UTC
found for statspack:
Level 7 includes the following segment statistics:
Logical reads
Db block changes
Physical reads
Physical writes
Physical reads direct
Physical writes direct
Global cache consistent read blocks served (RAC specific)
Global cache current blocks served (RAC specific)
Buffer busy waits
ITL waits
Row lock waits
Although Statspack captures all segment statistics, it reports only the following statistics that exceed one of the predefined threshold parameters:
Number of logical reads on the segment. The default is 10,000.
Number of physical reads on the segment. The default is 1,000.
Number of buffer busy waits on the segment. The default is 100.
Number of row lock waits on the segment. The default is 100.
Number of ITL waits on the segment. The default is 100.
Number of global cache consistent read blocks served (RAC only). The default is 1,000.
Number of global cache current blocks served (RAC only). The default is 1,000.
thank you!
large list partioning sql
Stanislav, February 24, 2005 - 8:14 pm UTC
Tom, i am trying to partition a giant table say 100M rows using list partitioning by 3 columns; first column (call it A has 50 values, second (B) has 10 and third (C) 20).
to generate sql i need to list all possible combinations which is 50x10x20 = 10,000 combinations;
the resulting sql exceeds the size of varchar2 type (32k). and execute immmediate does not allow to run it due to type error.
how can one execute such a giant sql ?
Stanislav
February 25, 2005 - 4:36 pm UTC
so what is the logic behind such a partitioning scheme? What is driving you to three vectors?
Not that it can be done bear in mind:
(
*
ERROR at line 8:
ORA-14304: List partitioning method expects a single partitioning column
range partitioning maybe, but then you would not need 10,000 combo's
long sql
Stanislav, February 25, 2005 - 5:45 pm UTC
ok, i discovered that oracle 9.2 has an ability
to parse big sql if you split sql into 32k chunks and load them into array of strings each 32k max; that in package documentation for dbms_sql.parse routine
as far a business logic, building a worldwide data warehouse and part of it fact table to store summarized transactions; those are continuouly changing even for prior time periods; thus vector A represents country, B represents snapshot time period and C time period. we store both history and current data views; to enable parallel work done on each country data i am trying to partition it naturally to match existing workstreams. Currently we store the data in one giant flat table and amount of contention generated by parallel workstreams it forcing us to seek partioning schemes
February 25, 2005 - 7:03 pm UTC
but -- you cannot list partition on multiple columns was my point?
Partition by week
atul, June 14, 2005 - 5:12 am UTC
Hi,
i wish to create partion by week,and wish to reuse that partition.
Means for a month i want to create only 4 partitions,last partition will have more days.
Thenfor next month,it should use same partitions we created.
So in short i don't want to create 52 partitions for a year,but just 4 partitions.
I am doing partition on timesatamp column whose dataye is date.
+++++++++++++++++++++
CREATE TABLE TEST ("ID" VARCHAR2(50) NOT
NULL ENABLE, "TRANSACTIONID" VARCHAR2(255) NOT NULL ENABLE,
"SOURCEID" VARCHAR2(255) NOT NULL ENABLE, "DOCUMENTID" VARCHAR2(255)
NOT NULL ENABLE, "TIMESTAMP" DATE NOT NULL ENABLE, "TIMESPENT"
NUMBER, "SERVICEIN" CLOB, "SERVICEOUT" CLOB, "APPNAME" VARCHAR2(50)
NOT NULL ENABLE, "SERVICENAME" VARCHAR2(255) NOT NULL ENABLE,
"MACHINENAME" VARCHAR2(255) NOT NULL ENABLE, "OBJECTTYPE"
VARCHAR2(255), "STATUS" VARCHAR2(10) NOT NULL ENABLE, "VERSION_NUM"
VARCHAR2(10), "MAX_VERSION" VARCHAR2(1), "COMMENTS" VARCHAR2(255))
PCTFREE 25 PCTUSED 40 INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 131072
FREELISTS 1 FREELIST GROUPS 1) TABLESPACE "VRTSD" LOGGING
NOCOMPRESS LOB ("SERVICEIN") STORE AS (TABLESPACE "VRTSD" ENABLE
STORAGE IN ROW CHUNK 8192 PCTVERSION 10 NOCACHE STORAGE(INITIAL
524288 FREELISTS 1 FREELIST GROUPS 1)) LOB ("SERVICEOUT") STORE AS
(TABLESPACE "TESTD" ENABLE STORAGE IN ROW CHUNK 8192 PCTVERSION
10 NOCACHE STORAGE(INITIAL 524288 FREELISTS 1 FREELIST GROUPS 1))
PARTITION BY RANGE ("TIMESTAMP" ) (PARTITION
"CUF_TXN_JUN05_1" VALUES LESS THAN (TO_DATE('JUN-08-2005','Mon-DD-YYYY')) PCTFREE 10 PCTUSED 85
INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 25m NEXT 25m MINEXTENTS 1
MAXEXTENTS 505 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
BUFFER_POOL DEFAULT) TABLESPACE "CUF_TXN_JUN05" LOGGING NOCOMPRESS,
PARTITION "CUF_TXN_JUN05_2" VALUES LESS THAN (TO_DATE('JUN-16-2005','Mon-DD-YYYY')) PCTFREE 10 PCTUSED 85
INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 25m NEXT 25m MINEXTENTS 1
MAXEXTENTS 505 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
BUFFER_POOL DEFAULT) TABLESPACE "CUF_TXN_JUN05" LOGGING NOCOMPRESS,
PARTITION "CUF_TXN_JUN05_3" VALUES LESS THAN (TO_DATE('JUN-23-2005','Mon-DD-YYYY')) PCTFREE 10 PCTUSED 85
INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 25m NEXT 25m MINEXTENTS 1
MAXEXTENTS 505 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
BUFFER_POOL DEFAULT) TABLESPACE "CUF_TXN_JUN05" LOGGING NOCOMPRESS,
PARTITION "CUF_TXN_JUN05_4" VALUES LESS THAN (TO_DATE('JUL-01-2005','Mon-DD-YYYY')) PCTFREE 10 PCTUSED 85
INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 25m NEXT 25m MINEXTENTS 1
MAXEXTENTS 505 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
BUFFER_POOL DEFAULT) TABLESPACE "CUF_TXN_JUN05" LOGGING NOCOMPRESS);
++++++++++++++++++++++++++++
Could you help?
Thanks,
Atul
June 14, 2005 - 9:56 am UTC
why? unless your queries have "where week = 1" or "week = 2", you'll never get partition elimination.
In order to do this, you have to add another column "week", you have to supply a value for that column (via a default, via a trigger, via the application). You can then partition on that column.
Most people would automate the adding and dropping of partitions via a stored procedure -- eg:
procedure
add new partition at end for next week
drop old partition at front to age out the old data
Partitions
atul, June 15, 2005 - 6:52 am UTC
Hi,
say if i want to have 4 partitopns in one month
ie 1-7,8-15,16-23,24-31
Will following logic help
PARTITION BY RANGE ("TIMESTAMP" ) (PARTITION
"CUF_TXN_JUN05_1" VALUES LESS THAN to_number((TO_CHAR(Timestamp,'DD'))
PCTFREE 10 PCTUSED 85
so that i get DD which will helpful for me to keep only 4 partitions only,as its not dependant on month
Thanks
June 15, 2005 - 9:53 am UTC
again, I say "why".
so what, you have 4 partitions. do your users query
where ....
and to_char(timestamp,'dd') <operator> :x
??
if not, no partition elimination and one might ask "why are you partitioning, what is the goal you are hoping to achieve"
if you just want four partitions, hash it on the primary key and be done with it.
reader
A reader, June 24, 2005 - 12:53 pm UTC
In this thread there is an example (june 18,2004)
partition by range and
sub-partition by list
Is it possible to do in 9.2
partition by list and
sub-partition by range
as well
June 24, 2005 - 6:37 pm UTC
range is at the top of the composite scheme, the others can be below it.
you range and then hash|list
reader
A reader, June 27, 2005 - 8:42 pm UTC
"
range is at the top of the composite scheme
"
Does 10g has any enhancement to let composite partitioning
have either range or list as primary partitioning
June 27, 2005 - 8:49 pm UTC
range is at the top.
is there a specific problem you are trying to solve?
Hash partitioned vs. non-partitioned table
Andrey Zamotayev, June 29, 2005 - 8:37 pm UTC
Hi Tom,
We have a table with 38 million rows.
Our core business processes select and update the table by disc_number column which is unique in the table.
So we are considering to hash partition the table by this column to facilitate these selects and updates.
We built two tables to run tests:
1) Non-partitioned table – a copy of original one.
2) Hash partitioned table (64 partitions) with the same set of non-partitioned indexes and with a local unique index on disc_number column.
Somehow update performance on portioned table almost the same (in fact worse) as the one on non-partitioned table even though it uses single partition index unique scan..
So there are a couple of questions:
1) What could be the reason of the behavior above?
2) Would you recommend to go with non-partitioned or partitioned table? Our concerns regarding the second approach are:
a. Higher CPU consumption;
b. Slower data access by non-partitioned index vs. a regular index on non-partitioned table for other DMLs.
Thanks a lot for your help and best regards,
Andrey.
Update test:
Run1 – 5000 updates on non-partitioned table.
Run1 – 5000 updates on partitioned table by disc_number.
Run1 ran in 886 hsecs
Run2 ran in 1177 hsecs
run 1 ran in 75.28% of the time
Name Run1 Run2 Diff
LATCH.post/wait queue 4 3 -1
LATCH.session timer 3 4 1
STAT...calls to get snapshot s 5,004 5,005 1
STAT...cursor authentications 1 0 -1
STAT...deferred (CURRENT) bloc 0 1 1
STAT...cleanout - number of kt 4 5 1
STAT...calls to kcmgcs 3 4 1
STAT...immediate (CURRENT) blo 1 2 1
STAT...redo entries 50,508 50,509 1
LATCH.dummy allocation 2 0 -2
LATCH.job_queue_processes para 2 0 -2
LATCH.transaction allocation 2 0 -2
LATCH.kwqit: protect wakeup ti 0 2 2
LATCH.mostly latch-free SCN 40 42 2
LATCH.list of block allocation 150 148 -2
LATCH.user lock 2 0 -2
STAT...redo synch time 2 0 -2
STAT...consistent gets - exami 15,254 15,256 2
STAT...consistent gets 15,255 15,257 2
STAT...commit txn count during 2 4 2
LATCH.cache buffer handles 5 9 4
STAT...calls to kcmgas 13 8 -5
STAT...write clones created in 11 6 -5
LATCH.channel operations paren 21 27 6
LATCH.lgwr LWN SCN 35 41 6
LATCH.Consistent RBA 34 41 7
STAT...change write time 30 38 8
LATCH.loader state object free 12 24 12
LATCH.FOB s.o list latch 24 11 -13
STAT...consistent changes 511 528 17
LATCH.enqueue hash chains 1,945 1,965 20
STAT...db block changes 101,362 101,392 30
STAT...messages sent 37 85 48
STAT...enqueue releases 5 72 67
STAT...enqueue requests 4 72 68
STAT...recursive cpu usage 268 337 69
LATCH.child cursor hash table 4 118 114
LATCH.session allocation 56 190 134
LATCH.global tx hash mapping 908 757 -151
LATCH.transaction branch alloc 908 757 -151
LATCH.active checkpoint queue 73 242 169
LATCH.redo writing 181 371 190
LATCH.undo global data 299 501 202
STAT...hot buffers moved to he 454 237 -217
LATCH.redo allocation 50,650 50,901 251
LATCH.enqueues 1,890 1,602 -288
LATCH.dml lock allocation 33 323 290
LATCH.library cache pin alloca 1,048 1,340 292
STAT...Elapsed Time 890 1,182 292
LATCH.session idle bit 1,839 2,300 461
LATCH.library cache pin 11,935 12,411 476
LATCH.messages 393 991 598
LATCH.shared pool 6,647 7,320 673
STAT...shared hash latch upgra 3,671 4,378 707
LATCH.library cache 13,627 14,538 911
LATCH.row cache enqueue latch 56,000 55,072 -928
LATCH.row cache objects 56,401 55,364 -1,037
STAT...dirty buffers inspected 505 1,560 1,055
STAT...free buffer inspected 506 1,561 1,055
STAT...physical reads 14,270 17,509 3,239
STAT...free buffer requested 14,623 17,874 3,251
STAT...session uga memory 0 4,264 4,264
LATCH.multiblock read objects 5,313 10,535 5,222
STAT...db block gets 126,535 136,584 10,049
STAT...session logical reads 141,790 151,841 10,051
LATCH.cache buffers lru chain 14,044 24,565 10,521
LATCH.checkpoint queue latch 21,098 47,640 26,542
STAT...session pga memory 60,200 0 -60,200
STAT...redo size 10,106,836 10,241,312 134,476
LATCH.cache buffers chains 572,776 981,340 408,564
Run1 latches total versus runs -- difference and pct
Run1 Run2 Diff Pct
818,412 1,271,503 453,091 64.37%
Non-partitioned index on partitioned table vs. a regular index on non-partitioned table.
Run1: Select 5000 on partitioned table using Non-partitioned index
Run2: Select 5000 on non-partitioned table using the same index
Run1 ran in 435 hsecs
Run2 ran in 1307 hsecs
run 1 ran in 33.28% of the time
Name Run1 Run2 Diff
STAT...enqueue requests 1 2 1
STAT...shared hash latch upgra 4 5 1
LATCH.kwqit: protect wakeup ti 0 2 2
LATCH.session timer 2 4 2
STAT...active txn count during 4 2 -2
STAT...calls to get snapshot s 3 5 2
LATCH.child cursor hash table 4 8 4
STAT...calls to kcmgcs 4 8 4
STAT...deferred (CURRENT) bloc 0 4 4
STAT...db block gets 522 527 5
LATCH.cache buffer handles 0 6 6
STAT...consistent changes 511 517 6
STAT...hot buffers moved to he 0 7 7
STAT...messages sent 4 12 8
STAT...recursive cpu usage 51 60 9
LATCH.channel operations paren 10 29 19
LATCH.active checkpoint queue 6 32 26
LATCH.Consistent RBA 1 28 27
LATCH.lgwr LWN SCN 1 29 28
LATCH.mostly latch-free SCN 1 29 28
STAT...dirty buffers inspected 2 65 63
STAT...free buffer inspected 2 65 63
STAT...buffer is not pinned co 4,900 4,971 71
STAT...buffer is pinned count 5,100 5,029 -71
LATCH.row cache enqueue latch 12 86 74
LATCH.row cache objects 12 86 74
LATCH.redo writing 13 125 112
LATCH.enqueues 8 140 132
LATCH.dml lock allocation 2 206 204
LATCH.messages 28 288 260
LATCH.enqueue hash chains 10 321 311
STAT...no work - consistent re 6,643 6,959 316
LATCH.session allocation 4 326 322
STAT...session logical reads 7,944 7,497 -447
STAT...consistent gets 7,422 6,970 -452
LATCH.undo global data 768 244 -524
LATCH.redo allocation 1,269 736 -533
LATCH.library cache pin alloca 44 704 660
STAT...db block changes 1,780 1,028 -752
STAT...commit txn count during 759 4 -755
STAT...consistent gets - exami 766 9 -757
STAT...cleanout - number of kt 763 4 -759
STAT...immediate (CR) block cl 759 0 -759
STAT...redo entries 1,265 506 -759
LATCH.shared pool 111 902 791
STAT...Elapsed Time 439 1,312 873
LATCH.library cache pin 69 1,077 1,008
LATCH.cache buffers lru chain 17 1,466 1,449
LATCH.session idle bit 28 1,621 1,593
LATCH.library cache 128 1,844 1,716
STAT...free buffer requested 6,369 4,438 -1,931
STAT...physical reads 6,365 4,434 -1,931
LATCH.checkpoint queue latch 684 4,690 4,006
STAT...prefetched blocks 4,654 0 -4,654
STAT...session uga memory -4,768 0 4,768
LATCH.multiblock read objects 250 7,289 7,039
STAT...redo size 110,508 65,108 -45,400
LATCH.cache buffers chains 25,460 188,818 163,358
Run1 latches total versus runs -- difference and pct
Run1 Run2 Diff Pct
28,942 211,136 182,194 13.71%
June 29, 2005 - 8:44 pm UTC
...
So we are considering to hash partition the table by this column to facilitate
these selects and updates.
.....
why? if you are index accessing by disc_number, why would partitioning speed it up -- what was the thoughts behind that?
maybe by HASHING the table into a hash cluster (no index access, just go for the data)
but why partitioning? which is used to speed up things typically by making a full scan full scan less data for example....
Hash partitioned vs. non-partitioned table
Andrey, June 29, 2005 - 9:21 pm UTC
The idea was that data access by local unique index should be faster. Another words single partition unique scan is faster then just unique scan on non-partitioned table.
Is it wrong?
Thanks and regards,
Andrey.
June 30, 2005 - 9:22 am UTC
An index unique scan is pretty much an index unique scan --
root-> branch/branch -> leaf -> table
You would expect N LIO's on the index where N is the distance to the leaf plus 1 LIO to the table.
Now, if by hash partitioning the table and the index, you took 1 level off of the index, you can reduce N by 1.
But..... You've increased the complexity of the retrieval by making the run time plan hash the key to find the right index to retrieve from.
In short, doubtful that is would make any difference on retrieval.
Now on modification -- you've actually added processing in a single user test, why? Because you have to hash the key to find the right partitions.
So in short, without a test case I won't comment on your exact numbers, but the premise of "we'll hash partition for great performance increases" is not going to be true in this case.
Are there reasons to hash partition in OLTP? Sure, ease of administration, possibly higher availability. If you use global indexes intelligently, you'll find you can do it such that you do not NEGATIVELY IMPACT performance. You might even find a small gain in performance scalability wise during modifications as you have N structures instead of just 1 structure (spread the load around different objects -- more concurrent)
If you have access to effective Oracle by Design -- I went over this and more in there with regards to partitioning.
Two level of subpartition
Bala, July 22, 2006 - 6:07 am UTC
Hi Tom,
Is it possible to create two level of subpartition.
I have a requirement like this.
create tabel Inv_transaction(Inv_date DATE,
Dcp NUMBER)
This table would have more than 10 millions of data.
We have to delete records using date and dcp value.
we have decided to create the main partition using month and year(Range partition) and another level as date(List partition) and another level as DCP(List partition).
DCP column will have value from 1 to 19. If we can create parition like this, we can truncate the parition using date and dcp value instead of deleting.
Is it possible to create such level in oracle?
July 23, 2006 - 8:29 am UTC
just 10 million rows? not much.
you get two levels, that is all. A range plus one more level.
I don't get how you would effectively use a "list" on date.
Just RANGE on date
LIST on DCP
you'll get what you need.
Creation of index for particular partition range
Jignesh, September 26, 2006 - 12:05 pm UTC
Tom,
Is it possible to build index for particular partition range. In our warehouse, table grows 60 million rows per day (we have created list partition per day). The retention period is 7 years. We do not have any global index.
There is a requirement for new report which will access last 4 months of data from above table which requires creation of new local index. We do not want to create new local index due to size and growth of the table. Is it possible to create/build index for 'n' partitions? Do you have any suggestions?
Thanks
September 26, 2006 - 4:52 pm UTC
not really, this is not something that is currently supported (or even being thought about as far as i know)
suppose you could create a local index "unusuable" and then just build the bits you want, not really very appealing long term I think.
are you really sure you need an index here? what are you partitioning by?
Tom, if an index is really neeeded...
J. Laurindo Chiappa, September 26, 2006 - 8:09 pm UTC
he can use your "selective index" mecanism - ie, create a FBI index indexing only the records in the desired partitions, no ? This way, the index will be much much smaller AND the desired range of records can be accessed via an index, what appears to be the objective here...
Regards,
Chiappa
September 27, 2006 - 4:24 am UTC
no, he cannot.
he wants to index the last 4 months. That means every second Oracle should be removing "old entries"
In other words, the function would be:
case when dt_of_record < add_months(sysdate,-4) then dt_of_record else null end
but that is not deterministic, dt_of_record is the input and that function returns different values over time.
Creation of index for particular partition range
jignesh, September 27, 2006 - 10:54 am UTC
Tom,
Index is partitioned on "processing date". Query will filter last 4 months data based on processing date and then will look for particular column which is not indexed yet..
i.e. where processing_date between :p_start_date and :p_end_date and ORIGINATOR = :p_originator and........
Here the index is required on Originator. If we do a full table scan of 4 months (60m x 20 days x 4 months ) I dont think this is a good idea. The other alternative we thought is to create local index, run the report and then drop it. However, still thinking for some clever way of handling the situation
September 27, 2006 - 3:33 pm UTC
why not
a) create index
why would you drop it?
disk cheap
your time, not cheap
if you believe the index would be seriously useful.
Creation of index for particular partition range
A reader, September 28, 2006 - 10:30 am UTC
Thanks Tom... we will go with your suggestion.
Alter table hash partition
Yoav, November 14, 2006 - 9:26 pm UTC
Hi Tom,
I have a Syntax problem.
I couldnt found how change the number of hash partition
on a table.
For example:
CREATE TABLE t
(
data varchar2(80)
)
PARTITION BY hash (data) partitions 7
insert into t
select 'usa' || object_name from all_objects;
commit;
I want to change it to 8 hash partition.
Could you please show how to do that ?
Thanks.
November 15, 2006 - 6:57 am UTC
you have to rebuild, all of the data must be rewritten - redistributed. Every row will move. It is not like adding a new empty range partition at the end of a table.
and with hash partitioning, if you use anything other than a power of two:
1, 2, 4, 8, 16, 32, 64, ....
as the number of partitions, you have done it wrong.
Table Partitioning
Prabhat, January 03, 2007 - 6:49 am UTC
I am required to create a table that will hold sales data of an Organization for a given country. The table will be holding multi-million records.
Now, in order to enhance Performance, we have decided to parttition a table , say SalesDat.
Now, what we want is to create table partitions in such a way that the sales for each state in the has its own partition (Table/Data will be partitioned on State-wise basis).
All I want to know from you is how can I create a table in such a way that whenever a new set-up (Branch) in new state is opened (i.e, Partition for new aranch is not existent) the table automaitcally creates, takes care of creating a new partition automatically.
Is it possible? If yest can you please kindly explain how?
Thanx in advance.
January 05, 2007 - 8:07 am UTC
until a future release of Oracle (after 10gR2) you have to do this manually - you have to ensure all of the partitions exist. You would likely use list partitioning for this.
partitioning history data
abz, January 05, 2007 - 10:07 am UTC
I have a table with columns
T_ID NUMBER(10),
CTYPE VARCHAR2(10),
VAL_OF_CTYPE VARCHAR2(80)
and some other columns.
Sample Values of above columns are
T_ID CTYPE VAL_OF_CTYPE
---- ----------- ------------
3 MONTH 200101
3 NAME 'A'
2 MONTH 200101
2 NAME 'B'
4 MONTH 200102
4 NAME 'A'
5 MONTH 200102
1 MONTH 200103
1 NAME 'K'
There can be 10 distinct values for CTYPE column.
All or some values of CTYPE are repeated with every t_id. This is a very large table with about 200 million rows.
The generation of values for T_ID is random, eg. 2 doesnt mean its before 3.
I want to partition it such that historic data can be
seperated from current data on the basis of MONTH.
For example, for the above data set, first 4 rows goes in one partition part_jan2001, then next 3 rows goes
in second partition part_feb2001, the next two rows
goes in third partition part_mar2001, and so on.
January 05, 2007 - 10:51 am UTC
sorry, you have a data model that, while I'm sure you'll say "it is very generic, it rocks!", will never perform, never be query-able
and you will not be partitioning in the manner you request.
there is no way you should be doing this.
generic
abz, January 05, 2007 - 11:32 am UTC
Yes, we know its generic, and we too hate this type
of model, but my problem is, I am using a 3rd party
application which designed this and I have no right to
change it.
Anyway, is there a way (not like I want it) but some
other way to partition this table so that to get
as much performance as we can.
January 06, 2007 - 8:31 am UTC
you won't be partitioning it in the manner you want, that I was clear about.
and partitioning is RARELY about performance in a transactional system. Why do you believe partitioning it would be a performance thing for you?
partitioning
abz, January 08, 2007 - 2:57 am UTC
It is written in documentation that every table
which is greater than 2GB must be considered for
partitioning. It is also written in manual that partitioning
can achieve great performance due to partition pruning,
so how can you say that partition is RARELY about performance.
Also, this table has 200 million rows. Whenever the
optimizer decides for full table scan ignoring
the available indexes, it takes too long to full scan it.
You say many times that "indexes are not always good,
full scns are not always bad". So what is the NUMBER of
rows to be accessed IS on which the optimizer decides
to full scan instead of using index.
January 08, 2007 - 12:25 pm UTC
where do you see that, I'd like to read it in context myself.
If you have access to my books - any of them - I write why over and over.
In a warehouse, partitioning can be a query performing enhancement tool - due to partition elimination during a full scan of a large table predominantly. You have a transactional system - in such a system, care must be taken generally to not NEGATIVELY impact performance.
Now, if you have a transactional system such that you frequently full scan a 200 million row table, it is not likely you have a transactional system.
and if you hinted your queries to force the use of an index - would performance
a) go up
b) go down
c) go sideways.
you don't say.
Transactional
abz, January 09, 2007 - 7:40 am UTC
We have transactional system. The reason of the large data
size is because we have a large no of customers.
About 1.5 million customers each with 10 months of his
history billing data. We cant say its a dataware house
because we are not doing any such multidimensional analysis
or data mining.
OR
can you tell me what size (no. of rows) is it, exceeding
which, your system cannot be called a transactional system.
200 million table is normal in our system because 10 months
history multiply by 1.5 million = about 200 million.
January 11, 2007 - 9:22 am UTC
size has nothing to do with what you are called.
What you DO has everything to do with what you are called.
the link
abz, January 11, 2007 - 6:39 am UTC
abz, January 12, 2007 - 8:48 am UTC
Ok so what we do is "billing our customers" there is not
much like "analysis" so we can call it an OLTP system.
So, does that mean, history of 10 months is too much to
keep online?
The 2BG Thing ...
David Aldridge, January 15, 2007 - 2:18 pm UTC
The 2 GB thing is just wrong. There is almost no table that cannot be considered for some form of partitioning. Even tables of less than 1Mb can be partitioning candidates if, for example, partitioning can be used to simulate multicolumn statistics on that table's data.
The consideration would be, "in what ways might the system benefit or deteriorate due to various forms of partitioning being applied to this data, and what would be the net effect?". Just as you might consider every column of every table as a candidate for indexing.
list list composite partition
Alan, August 03, 2007 - 2:37 am UTC
Hello,Mr. Tom
I have a question abort oracle composite partition,Why oracle don't support list-list composite partition?
If I have a table
t(id int,type varchar2(10)
ID column have (20,30,50,60) values and TYPE column have 'Y' and 'N' values.
How can I make a list-list composte partiton on this table?
Best Regards!
Alan
August 05, 2007 - 1:22 pm UTC
it will, it just doesn't yet (as of 10g Release 2)
You can use range list on your data right now - it'll work as is
Composite Partitioning
A reader, August 10, 2007 - 5:38 am UTC
We have a table which is range-partitioned based on a date field.
Now, We want to add a higher level partition based on status and sub-partition it using date field.
Status field can have two values '01' - Active & '02' - Closed
One possible approach is to do a range partitioning on status field, and hash partitioning on date field.
but hash partitioning on "date" field will restrict us from deleting old data by simply dropping/truncating old partitions.
Any suggestions?
by the way, Your replies are always wonderful!
August 14, 2007 - 10:58 am UTC
you can range on date, list on status (you don't say why you want status FIRST, so I presume it can be second)
you can range/range on status and date
and in 11g, you'll have even more options.
A reader, August 20, 2007 - 7:07 am UTC
Thanks for your reply.
Is range/range partitioning possible in 9i?
August 22, 2007 - 10:28 am UTC
sorry, range/range is 11g...
csk
Sri, May 29, 2008 - 4:48 pm UTC
Hi Tom,
Our fact table has 50000 daily partitions with 2 million records in each partition. We load the fact table every 15 minutes. Around 95% of load data is new/inserts to on average 10 partitions, and remaining 5% of data is adjustments/updates to on average 40 partitions. We have bitmap indexes on the fact table.
We are currently rebuilding the indexes in a ¿for loop¿ sequentially for all the partitions loaded. This is taking more time than we have to meet our SLAs. Your earlier notes suggest we do not have to rebuild the indexes but that is being not accepted within our team :-(. Is there a way to parallelize rebuild index on selected partitions (as in above case 50 partitions out of 50k) using DBMS_PCLXUTIL.BUILD_PART_INDEX () or in any other way.
Thanks
csk
May 29, 2008 - 9:46 pm UTC
... but that is
being not accepted within our team ...
if you load IN BULK (not slow by slow, large array inserts OR direct path loads) you do not.
If they think elsewise, put up the facts...
you can rebuild individual index partitions in parallel
the dbms package just queues rebuilds using dbms_job
you can do that as well.
loop over partitions to rebuild
dbms_job.submit( l_job, 'execute immediate ''alter .....'';' );
end loop;
commit; <<== job_queue_processes will control the 'degree' of parallelism now
Sri, May 29, 2008 - 4:49 pm UTC
hi Tom, we are on 10g. Thanks -csk
A reader, May 30, 2008 - 4:20 pm UTC
Thanks Tom for your quick reply. Very good suggestion. Will post back if I see any issues. Thanks for now.
-csk
Time retention expiring?
Jo Fresh, March 25, 2009 - 7:35 am UTC
in reply to one of your responses. How would you handle creating new partitions and dropping off old partitions when the date passes the time retention period?
e.g. I need to remove types B for the 01-Jan-2004 but keep A a little longer.
------------------------------------------------
Followup December 7, 2004 - 8pm US/Eastern:
ops$tkyte@ORA10GR1> Create Table t
2 (
3 code varchar2(1),
4 dt date
5 )
6 partition by range(code,dt)
7 (
8 partition p1a values less than ( 'A', to_date('01-jan-2004') ),
9 partition p2a values less than ( 'A', to_date('02-jan-2004') ),
10 partition p3a values less than ( 'A', to_date('03-jan-2004') ),
11 partition p1b values less than ( 'B', to_date('01-jan-2004') ),
12 partition p2b values less than ( 'B', to_date('02-jan-2004') ),
13 partition p3b values less than ( 'B', to_date('03-jan-2004') ),
14 partition p1c values less than ( 'C', to_date('01-jan-2004') ),
15 partition p2c values less than ( 'C', to_date('02-jan-2004') ),
16 partition p3c values less than ( 'C', to_date('03-jan-2004') ),
17 partition p1d values less than ( 'D', to_date('01-jan-2004') ),
18 partition p2d values less than ( 'D', to_date('02-jan-2004') ),
19 partition p3d values less than ( 'D', to_date('03-jan-2004') )
20 )
21 /
Table created.
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> declare
2 type vcarray is table of varchar2(1);
3 l_codes vcarray := vcarray('A','B','C','D');
4 begin
5 for i in 1..l_codes.count
6 loop
7 for j in 0..2
8 loop
9 insert into t values ( l_codes(i), to_date('31-dec-2003')+j );
10 end loop;
11 end loop;
12 end;
13 /
PL/SQL procedure successfully completed.
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> select 'p1a', x.* from t partition (p1a) x union all
2 select 'p2a', x.* from t partition (p2a) x union all
3 select 'p3a', x.* from t partition (p3a) x union all
4 select 'p1b', x.* from t partition (p1b) x union all
5 select 'p2b', x.* from t partition (p2b) x union all
6 select 'p3b', x.* from t partition (p3b) x union all
7 select 'p1c', x.* from t partition (p1c) x union all
8 select 'p2c', x.* from t partition (p2c) x union all
9 select 'p3c', x.* from t partition (p3c) x union all
10 select 'p1d', x.* from t partition (p1d) x union all
11 select 'p2d', x.* from t partition (p2d) x union all
12 select 'p3d', x.* from t partition (p3d) x;
'P1 C DT
--- - ---------
p1a A 31-DEC-03
p2a A 01-JAN-04
p3a A 02-JAN-04
p1b B 31-DEC-03
p2b B 01-JAN-04
p3b B 02-JAN-04
p1c C 31-DEC-03
p2c C 01-JAN-04
p3c C 02-JAN-04
p1d D 31-DEC-03
p2d D 01-JAN-04
p3d D 02-JAN-04
12 rows selected.
That is just like a LIST with a RANGE underneath it.
March 29, 2009 - 8:18 pm UTC
... How would you handle creating new partitions
and dropping off old partitions when the date passes the time retention period?
....
by having an automated process most likely, or at least and automated process I could invoke.
eg: by writing code to do it.
your partitioning scheme is a big wrong.
partition by range, SUBPARTITION BY LIST.
Then you can easily identify the partition you want to drop.
ops$tkyte%ORA10GR2> CREATE TABLE t
2 ( dt DATE,
3 code VARCHAR2(1),
4 data varchar2(20)
5 )
6 PARTITION BY RANGE (dt)
7 SUBPARTITION BY LIST (code)
8 SUBPARTITION TEMPLATE
9 (
10 SUBPARTITION a_stuff VALUES('A'),
11 SUBPARTITION b_stuff VALUES('B'),
12 SUBPARTITION c_stuff VALUES('C')
13 )
14 (
15 PARTITION jan1 VALUES LESS THAN (TO_DATE('2-jan-2009','DD-MON-YYYY')),
16 PARTITION jan2 VALUES LESS THAN (TO_DATE('3-jan-2009','DD-MON-YYYY')),
17 PARTITION jan3 VALUES LESS THAN (TO_DATE('4-jan-2009','DD-MON-YYYY'))
18 );
Table created.
ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> insert into t (dt, code, data )
2 select to_date( '01-jan-2009', 'dd-mon-yyyy') + trunc( dbms_random.value( 0, 2.99999999 ) ),
3 chr( ascii('A')+ trunc( dbms_random.value( 0, 2.9999999 ) ) ),
4 'x'
5 from dual connect by level <= 10000
6 /
10000 rows created.
ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> select 'JAN1_A_STUFF', count(*) from t SUBPARTITION(JAN1_A_STUFF) union all
2 select 'JAN1_B_STUFF', count(*) from t SUBPARTITION(JAN1_B_STUFF) union all
3 select 'JAN1_C_STUFF', count(*) from t SUBPARTITION(JAN1_C_STUFF) union all
4 select 'JAN2_A_STUFF', count(*) from t SUBPARTITION(JAN2_A_STUFF) union all
5 select 'JAN2_B_STUFF', count(*) from t SUBPARTITION(JAN2_B_STUFF) union all
6 select 'JAN2_C_STUFF', count(*) from t SUBPARTITION(JAN2_C_STUFF) union all
7 select 'JAN3_A_STUFF', count(*) from t SUBPARTITION(JAN3_A_STUFF) union all
8 select 'JAN3_B_STUFF', count(*) from t SUBPARTITION(JAN3_B_STUFF) union all
9 select 'JAN3_C_STUFF', count(*) from t SUBPARTITION(JAN3_C_STUFF);
'JAN1_A_STUF COUNT(*)
------------ ----------
JAN1_A_STUFF 1117
JAN1_B_STUFF 1126
JAN1_C_STUFF 1109
JAN2_A_STUFF 1150
JAN2_B_STUFF 1104
JAN2_C_STUFF 1108
JAN3_A_STUFF 1107
JAN3_B_STUFF 1071
JAN3_C_STUFF 1108
9 rows selected.
ops$tkyte%ORA10GR2> alter table t drop subpartition jan1_b_stuff;
Table altered.
ops$tkyte%ORA10GR2> select count(*) from t;
COUNT(*)
----------
8874
Time retention expiring?
Jo Fresh, March 25, 2009 - 8:12 am UTC
(forgot to say) and adding new partitions for each of the types also with later dates
e.g. adding 08-JAN-2004 for both Type A and Type B
Time retention expiring?
Jo Fresh, March 30, 2009 - 10:43 am UTC
Thanks for the speedy response, I was kind of hoping I wouldn't have to change the SUBPARTITIONing that I have, but I guess its back to the drawing board.
Thanks again I appreciate the time you spend answering the many questions.
Cheers
Jo
CREATE TABLE mp_revenue_and_pipe2
(reporting_date DATE NOT NULL ,
reporting_cycle VARCHAR2(10) NOT NULL ,
value_type VARCHAR2(20) NOT NULL ,
expectedfees NUMBER,
expenses_usd NUMBER,
actual_revenue NUMBER,
CHECK (value_type IN ('Pipeline','Revenue','FPMS')))
PARTITION BY RANGE (reporting_cycle,reporting_date)
SUBPARTITION BY LIST (value_type)
SUBPARTITION TEMPLATE
(SUBPARTITION pipeline VALUES ('Pipeline') TABLESPACE IRP_DATA,
SUBPARTITION revenue VALUES ('Revenue' ) TABLESPACE IRP_DATA,
SUBPARTITION OTHERS VALUES (DEFAULT) TABLESPACE IRP_DATA)
(
PARTITION h1_2007 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2007','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h1_2008 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2008','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h1_2009 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h1_2010 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h1_2011 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h1_2012 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h1_2013 VALUES LESS THAN ('HALF1' ,TO_DATE('01-JUL-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2007 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2008','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2008 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2009 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2010 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2011 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2012 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2013 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2014','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION h2_2014 VALUES LESS THAN ('HALF2' ,TO_DATE('01-JAN-2015','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2007 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2007','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2008 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2008','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2009 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2010 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2011 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2012 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2013 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q1_2014 VALUES LESS THAN ('QUARTER1',TO_DATE('01-APR-2014','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2007 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2007','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2009 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2010 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2011 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2012 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2013 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q2_2014 VALUES LESS THAN ('QUARTER2',TO_DATE('01-JUL-2014','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2007 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2007','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2008 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2008','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2009 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2010 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2011 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2012 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2013 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q3_2014 VALUES LESS THAN ('QUARTER3',TO_DATE('01-OCT-2014','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2007 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2008','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2008 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2009 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2010 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2011 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2012 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2013 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2014','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION q4_2014 VALUES LESS THAN ('QUARTER4',TO_DATE('01-JAN-2015','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2007 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2008','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2008 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2009','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2009 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2010','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2010 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2011','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2011 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2012','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2012 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2013','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2013 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2014','DD-MON-RRRR')) TABLESPACE IRP_DATA,
PARTITION yr_2014 VALUES LESS THAN ('YEAR' ,TO_DATE('01-JAN-2015','DD-MON-RRRR')) TABLESPACE IRP_DATA
)
/
LIST partition on a DATE data type?
James, August 26, 2009 - 3:35 pm UTC
Hello Tom and thanks for the extremely useful information you provide through this site.
In an earlier post in this thread you said:
"I don't get how you would effectively use a "list" on date. Just RANGE on date."
In a number of databases I have worked with there are columns of type DATE that store a "business" date. These business dates never include a time component (i.e., the time component is always "00:00:00"). Because they are DATE data type they typically get set up with RANGE partitions. I think the actual data in these columns argue for a LIST partition.
It seems a LIST partition in this case would allow the CBO to perform simple partition pruning and be done with the business date filter because it knows everything in the partition matches.
For the RANGE partition, the CBO has additional steps after the partition pruning. The CBO has to look at the statistics for this partition to decide how much data inside the partition matches. And if you don't have good columns stats or histogram showing the terrible data skew in this business date column you could get a bad plan.
That being said going with a LIST partition on a DATE data type seems very unusual and worries me that this may be somewhat untested Oracle waters (fyi, we are running 10.2.0.4).
Assuming a typical query always filters on 1 business date and you could not change the data type (we cannot because it would require changing many tables and many interfaces) how would you partition this type of table? What do you see as the advantages/disadvantages of one approach over the other? Any pitfalls or gotchas to watch out for?
Thank you for your time. James
August 28, 2009 - 4:07 pm UTC
You would have daily parititions? That is the only way this would work.
... The CBO has to look at the statistics for this partition to decide
how much data inside the partition matches. ...
you are hypothesizing something is "bad" when it fact it wouldn't be really - the optimizer is doing that work anyway, regardless.
... nd if you don't have good columns
stats or histogram showing the terrible data skew in this business date column
you could get a bad plan.
...
but that could not happen at the partition level, you have already told us "data is constant, we have a daily partition" - so tell me - how couldn't you have good enough stats (no histograms needed, there is just one value and we know that without histograms), how could a constant value be skewed?
creating NON partitioned index on partitioned table.
adu, April 11, 2011 - 5:00 am UTC
Does creating non partitioned index on a partitioned table hamper performance or manageability in any way...
April 13, 2011 - 8:23 am UTC
Here is the answer:
Yes
No
Maybe
it depends. It might be faster, it might be slower, it might be the same speed. It might be harder to manage, it might be easier to manage, it might be the same trouble to manage.
Think about it - the answer is always going to be "it depends" given a question like that.
How could it hamper performance as a non-partitioned index? Well, the index would have global rowids in it, we have to dereference a more complex rowid format (it has partition information, relative file info, block info and row info - a normal rowid just has file, block and row info). That takes a bit more CPU. The index, since it is non-partitioned, would presumably be larger than a single partition of a partitioned index would be - therefore its height could be larger - resulting in more logical IO's to do a range scan. Therefore, we might do an extra IO or two and have to dereference a more complex rowid.
How could it improve performance as a non-partitioned index? Well, assume the partitioned index was locally partitioned. Assume further, the table was the EMP table and was hash partitioned into 128 partitions on the EMPNO column. Lastly, assume the index was on ENAME. A query such as:
select * from emp where ename = ?
would have to do 128 index range scans using the locally partitioned index. If the index were NOT partitioned, it would have to do one range scan. The non-partitioned index would be orders of magnitude more efficient.
How could it be the same? Well, assume the same table above, but change the locally partitioned ename index into a global range partition index. We'll store A-L in partition 1 and M-Z in partition 2. Assume the height of the global index partitions is 3. So, now a query of the form:
select * from emp where ename = ?
can do global index partition elimination and can index range scan a single partition and will do 3 IO's against the index and a fourth IO against the table, using a global rowid. Now, assume the non-partitioned index also has a height of 3. It too would do 3 IO's against the index and a fourth against the table using a global rowid. They would be the same.
Now you envision a set of circumstances under which
a) the non-partitioned index would be less manageable
b) more manageable
c) no difference really either way.
Just think about what it means to be "more or less manageable" and think up a circumstance whereby a) is true, then b) and then c)
composite Range-Range partition with Interval clause
A reader, August 17, 2011 - 3:59 pm UTC
I would like to create composite Range-Range partition with Interval clause. Is it possible?
CREATE TABLE test_table
(
TABLE_ID1 NUMBER(*,0) NOT NULL ENABLE,
TABLE_ID2 NUMBER(*,0) NOT NULL ENABLE,
OTHER COLUMNS ..............................
TRANS_DATETIME DATE NOT NULL ENABLE,
SETTLEMENT_DATE DATE NOT NULL ENABLE,
)
PARTITION BY RANGE (SETTLEMENT_DATE) INTERVAL(NUMTOYMINTERVAL(1, 'YEAR'))
SUBPARTITION BY RANGE (TRANS_DATETIME) INTERVAL(NUMTOYMINTERVAL(1, 'YEAR'))
SUBPARTITION TEMPLATE
(SUBPARTITION SUB_TH_P1 VALUES LESS THAN (TO_DATE('01-01-2011', 'DD-MM-YYYY')),
SUBPARTITION SUB_TH_P2 VALUES LESS THAN (TO_DATE('01-01-2012', 'DD-MM-YYYY')))
( PARTITION TH_P1 VALUES LESS THAN (TO_DATE('01-01-2011', 'DD-MM-YYYY')),
PARTITION th_p2 VALUES LESS THAN (TO_DATE('01-01-2012', 'DD-MM-YYYY'))
);
August 17, 2011 - 5:40 pm UTC
composite partitioning key vs range-list partitioning
david, February 03, 2012 - 1:17 pm UTC
Hi
I have to partition a table using range partitiong with a date then subpartition by city code (ranges from 1 to 34) using list.
What is best, range partitioning using composite key with date, city code or it is best using range-list subpartitioning? I cant think of any difference really!
Thanks in advance
David