Skip to Main Content
  • Questions
  • Archiving Partition tables size of 3TB+

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, Yash.

Asked: April 19, 2017 - 10:01 pm UTC

Last updated: April 29, 2017 - 7:07 am UTC

Version: Oracle Database 11.2.0.4

Viewed 1000+ times

You Asked

Hi,

I have two Fat Tables, Both are having Composite RANGE-HASH partitions. Table has one Primary RANGE partitioned on Created_Date Column/ Monthly and 4 Subpartitions HASH on Primary Key Column. We have ~3.5 yrs+ data (42 Range Partition/168 Hash Subpartition).

Both the table contains 2 CLOB columns.
Table A has each Partiton average 8M Rows & 5M Blocks.
Table B has each Partiton average 10M Rows & 8M Blocks.

Table A structure look like-

CREATE TABLE TableA
(
  PROVIDER_REQUEST_LOG_ID      NUMBER(19)       NOT NULL, --> Primary Key
  FK_TRANSACTION_ID            NUMBER(19),
  FK_PROVIDER_REQUEST_TYPE_ID  NUMBER(19)       NOT NULL,
  FK_PROVIDER_ID               NUMBER(19)       NOT NULL,
  REQUEST_DATA                 CLOB,
  RESPONSE_DATA                CLOB,
  START_DATE                   TIMESTAMP(6),
  FINISH_DATE                  TIMESTAMP(6),
  CREATED_BY                   VARCHAR2(256 CHAR) NOT NULL,
  CREATED_DATE                 TIMESTAMP(6)     NOT NULL,
  LAST_MODIFIED_BY             VARCHAR2(256 CHAR) NOT NULL,
  LAST_MODIFIED_DATE           TIMESTAMP(6)     NOT NULL
)
TABLESPACE TESTD
PARTITION BY RANGE (CREATED_DATE)
SUBPARTITION BY HASH (PROVIDER_REQUEST_LOG_ID)
SUBPARTITION TEMPLATE
  (SUBPARTITION SP1 TABLESPACE TESTD,
   SUBPARTITION SP2 TABLESPACE TESTD,
   SUBPARTITION SP3 TABLESPACE TESTD,
   SUBPARTITION SP4 TABLESPACE TESTD
  ) 
(
.... 
PARTITION P_OCT
VALUES LESS THAN(TO_DATE('01/11/2016','DD/MM/YYYY')) TABLESPACE TEST1
PARTITION P_NOV
VALUES LESS THAN(TO_DATE('01/12/2016','DD/MM/YYYY')) TABLESPACE TEST2
PARTITION P_DEC
VALUES LESS THAN(TO_DATE('01/01/2017','DD/MM/YYYY')) TABLESPACE TEST3
PARTITION P_JAN
VALUES LESS THAN(TO_DATE('01/02/2017','DD/MM/YYYY')) TABLESPACE TEST4
PARTITION P_FEB
VALUES LESS THAN(TO_DATE('01/03/2017','DD/MM/YYYY')) TABLESPACE TEST5
.....

);


Table B structure look like-
CREATE TABLE TableB
(
  ROS_REQUEST_LOG_ID      NUMBER(19)            NOT NULL,  --> Primary Key
  FK_TRANSACTION_ID       NUMBER(19)            NOT NULL,
  REQUEST_DATA            CLOB,
  RESPONSE_DATA           CLOB,
  START_DATE              TIMESTAMP(6),
  FINISH_DATE             TIMESTAMP(6),
  CREATED_BY              VARCHAR2(256 CHAR)    NOT NULL,
  CREATED_DATE            TIMESTAMP(6)          NOT NULL,
  LAST_MODIFIED_BY        VARCHAR2(256 CHAR)    NOT NULL,
  LAST_MODIFIED_DATE      TIMESTAMP(6)          NOT NULL,
  FK_ROS_REQUEST_TYPE_ID  NUMBER(19)
)
TABLESPACE TESTD
PARTITION BY RANGE (CREATED_DATE)
SUBPARTITION BY HASH (ROS_REQUEST_LOG_ID)
SUBPARTITION TEMPLATE
  (SUBPARTITION SP1 TABLESPACE TEST1,
   SUBPARTITION SP2 TABLESPACE TEST1,
   SUBPARTITION SP3 TABLESPACE TEST1,
   SUBPARTITION SP4 TABLESPACE TEST1
  ) 
(
.... 
PARTITION P_OCT
VALUES LESS THAN(TO_DATE('01/11/2016','DD/MM/YYYY')) TABLESPACE TEST1
PARTITION P_NOV
VALUES LESS THAN(TO_DATE('01/12/2016','DD/MM/YYYY')) TABLESPACE TEST2
PARTITION P_DEC
VALUES LESS THAN(TO_DATE('01/01/2017','DD/MM/YYYY')) TABLESPACE TEST3
PARTITION P_JAN
VALUES LESS THAN(TO_DATE('01/02/2017','DD/MM/YYYY')) TABLESPACE TEST4
PARTITION P_FEB
VALUES LESS THAN(TO_DATE('01/03/2017','DD/MM/YYYY')) TABLESPACE TEST5
.....

);


Now I wanted to Archive those two tables in some other schema, and wanted to keep just 6 month of data....I tried CTAS for just 2 month, but it took more than 6 hours and not completed. unfortunately I had to kill it.

Here is what my query look like for CTAS (for TableA).

CREATE TABLE TABLE_TEMP_A
(
  PROVIDER_REQUEST_LOG_ID     NOT NULL,
  FK_TRANSACTION_ID,          
  FK_PROVIDER_REQUEST_TYPE_ID NOT NULL,
  FK_PROVIDER_ID              NOT NULL,
  REQUEST_DATA,               
  RESPONSE_DATA,              
  START_DATE,                 
  FINISH_DATE,                
  CREATED_BY                  NOT NULL,
  CREATED_DATE                NOT NULL,
  LAST_MODIFIED_BY            NOT NULL,
  LAST_MODIFIED_DATE          NOT NULL
)
TABLESPACE TESTD
PARTITION BY RANGE (CREATED_DATE)
SUBPARTITION BY HASH (PROVIDER_REQUEST_LOG_ID)
SUBPARTITION TEMPLATE
  (SUBPARTITION SP1 TABLESPACE TEST1D,
   SUBPARTITION SP2 TABLESPACE TEST1D,
   SUBPARTITION SP3 TABLESPACE TEST1D,
   SUBPARTITION SP4 TABLESPACE TEST1D
  ) 
(
PARTITION P_NOV16
VALUES LESS THAN (TO_DATE('01/12/2016','DD/MM/YYYY')) TABLESPACE TEST1D,
PARTITION P_DEC16 
VALUES LESS THAN(TO_DATE('01/01/2017','DD/MM/YYYY')) TABLESPACE TEST2D,
PARTITION P_JAN17
VALUES LESS THAN(TO_DATE('01/02/2017','DD/MM/YYYY')) TABLESPACE TEST3D,
PARTITION P_FEB17
VALUES LESS THAN (TO_DATE('01/03/2017','DD/MM/YYYY')) TABLESPACE TEST4D,
PARTITION P_MAR17 
VALUES LESS THAN(TO_DATE('01/04/2017','DD/MM/YYYY')) TABLESPACE TEST5D,
PARTITION P_APR17
VALUES LESS THAN(TO_DATE('01/05/2017','DD/MM/YYYY')) TABLESPACE TEST6D
)
enable row movement nologging 
as select /*+ PARALLEL(a,4)  */ a.* from TableA  a 
where trunc(A.CREATED_DATE)  > trunc(to_date('31/10/2016','DD/MM/YYYY')) ;

...

I tried to look at Exchange partition as well, but seems will not work for me as table has Range-Hash. I got ORA-14292.
I am looking for best and faster approach to do it.

and Connor said...

Take a fresh look at exchange - you should be able to do it. You might need a 2-step process.

See here

https://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:9534224300346144891


Rating

  (1 rating)

Is this answer out of date? If it is, please let us know via a Comment

Comments

invoking functions on partition key columns

Rajeshwaran, May 02, 2017 - 7:09 am UTC

BTW, when you have a function calls on the partition key columns, partition pruning doesn't kick in.

where trunc(A.CREATED_DATE)  > trunc(to_date('31/10/2016','DD/MM/YYYY')) ;


demo@ORA11G> create table t
  2  partition by range(date_range)
  3  (   partition p_2013 values less than ( to_date('01-Jan-2014','dd-mon-yyyy') ),
  4      partition p_2014 values less than ( to_date('01-Jan-2015','dd-mon-yyyy') ),
  5      partition p_2015 values less than ( to_date('01-Jan-2016','dd-mon-yyyy') ),
  6      partition p_2016 values less than ( to_date('01-Jan-2017','dd-mon-yyyy') ),
  7      partition p_2017 values less than ( to_date('01-Jan-2018','dd-mon-yyyy') )     )
  8  as
  9  select a.*, to_date('01-Jan-2013','dd-mon-yyyy') + mod(rownum,1825) date_range
 10  from all_objects a;

Table created.


Lets say the goal is to isolate 2018 data into a different table.
then having a function calls on the partition key column like this, would scan every partitions in the table.

demo@ORA11G> explain plan for
  2  create table t1 as
  3  select *
  4  from t
  5  where trunc(date_range) > to_date('01-Jan-2017','dd-mon-yyyy') ;

Explained.

demo@ORA11G> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------
Plan hash value: 1620526508

-----------------------------------------------------------------------------------------------
| Id  | Operation              | Name | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
-----------------------------------------------------------------------------------------------
|   0 | CREATE TABLE STATEMENT |      | 15276 |  2491K|   414   (1)| 00:00:05 |       |       |
|   1 |  LOAD AS SELECT        | T1   |       |       |            |          |       |       |
|   2 |   PARTITION RANGE ALL  |      | 15276 |  2491K|   350   (1)| 00:00:05 |     1 |     5 |
|*  3 |    TABLE ACCESS FULL   | T    | 15276 |  2491K|   350   (1)| 00:00:05 |     1 |     5 |
-----------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - filter(TRUNC(INTERNAL_FUNCTION("DATE_RANGE"))>TO_DATE(' 2017-01-01 00:00:00',
              'syyyy-mm-dd hh24:mi:ss'))

Note
-----
   - dynamic sampling used for this statement (level=2)

20 rows selected.


when you loose those function calls, things will be done like this.

demo@ORA11G> explain plan for
  2  create table t1 as
  3  select *
  4  from t
  5  where date_range > to_date('01-Jan-2017','dd-mon-yyyy') ;

Explained.

demo@ORA11G> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
-------------------------------------------------------------------------------------------------
Plan hash value: 380553931

------------------------------------------------------------------------------------------------
| Id  | Operation               | Name | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
------------------------------------------------------------------------------------------------
|   0 | CREATE TABLE STATEMENT  |      | 11620 |  1895K|   119   (0)| 00:00:02 |       |       |
|   1 |  LOAD AS SELECT         | T1   |       |       |            |          |       |       |
|   2 |   PARTITION RANGE SINGLE|      | 11620 |  1895K|    70   (0)| 00:00:01 |     5 |     5 |
|*  3 |    TABLE ACCESS FULL    | T    | 11620 |  1895K|    70   (0)| 00:00:01 |     5 |     5 |
------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - filter("DATE_RANGE">TO_DATE(' 2017-01-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

Note
-----
   - dynamic sampling used for this statement (level=2)

19 rows selected.

demo@ORA11G>

More to Explore

VLDB

If you are new to partitioning, check out Connor McDonald's introduction series here.

VLDB

Documentation set on VLDB and Partitioning.