Skip to Main Content
  • Questions
  • PARTITION BY RANGE with timestamp Column

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Hendrik.

Asked: November 13, 2019 - 1:38 pm UTC

Last updated: March 15, 2021 - 3:31 pm UTC

Version: 11.1.0.4

Viewed 10K+ times! This question is

You Asked

Hi Tom,

have the following use case.


DROP TABLE gaga1;

CREATE TABLE gaga1 (abc TIMESTAMP)
PARTITION BY RANGE (abc)INTERVAL ( NUMTODSINTERVAL (1, 'DAY') )( PARTITION OLD_DATA VALUES LESS THAN (TIMESTAMP '2015-01-01 00:00:00.000000'));


INSERT INTO gaga1 select to_timestamp('31/12/9999 23:59:59.000000','DD/MM/YYYY HH24:MI:SS.FF') from dual ;


and get the error

ORA-14300: partitioning key maps to a partition outside maximum permitted number of partitions


so it seems Oracle tries to create all partition until the end of the universe.
is there any ideas how to prevent this?

So in many circumstances "31/12/9999 23:59:59.000000" is marked as the current valid_until.
Thanks
Hendrik


and Chris said...

I hope the universe lasts a bit longer than another 8000 years ;)

When you insert a new partition key value in an interval partitioned table, the database needs to "make room" for all the partitions from this value to the current lower value.

In this case that's:

select date'9999-12-31' - date'2015-01-01' days
from   dual;

DAYS      
  2,916,460 


Nearly 3 million partitions. About three times the upper limit for the number of partitions (2**20 ~ 1 million)!

Some workarounds:

Move the lower or upper bounds

I'm guessing you can't move the lower bound much earlier than 1 Jan 2015. So you'll need to move your "end of the universe" much closer, to around the year 4,000!

In any case, note you can't use the value 31 Dec 9999 for "the end".

Because if you do, the database needs to set the upper bound for the last partition to 1 Jan 10,000. An impossible date in Oracle Database!

Change the granularity

Instead of daily partitions, you use weekly (7 day) you've got enough room to store all the dates you want:

CREATE TABLE gaga1 (abc TIMESTAMP)
  PARTITION BY RANGE (abc) 
  INTERVAL ( NUMTODSINTERVAL (7, 'DAY') ) ( 
    PARTITION OLD_DATA VALUES LESS THAN (TIMESTAMP '2015-01-01 00:00:00.000000')
  );

INSERT INTO gaga1 select to_timestamp('29/12/9999 23:59:59.000000','DD/MM/YYYY HH24:MI:SS.FF') from dual;

SELECT COUNT(*) FROM gaga1;

COUNT(*)   
          1 


Though note here the max value moves back to 29 Dec 9999.

Change the granularity & subpartition

If you must have daily partitions, you could partition by month/week/... Then subpartition by day.

Interval subpartitions are currently unsupported. Instead you can:

- Add a virtual column extracting the day number from the date
- Subpartition by this VC
- Define a subpartition template, which creates a subpartition for each possible day for the top-level granularity

e.g.:

CREATE TABLE gaga1 (
  abc TIMESTAMP,
  abc_day INT as (extract (day from abc) )
) PARTITION BY RANGE (abc) 
  INTERVAL ( INTERVAL '1' MONTH )
  SUBPARTITION BY LIST ( abc_day ) 
  SUBPARTITION TEMPLATE (
    subpartition sp1 values ( 1 ),
    subpartition sp2 values ( 2 )
    -- etc.
  )  ( 
    PARTITION OLD_DATA VALUES LESS THAN (TIMESTAMP '2015-01-01 00:00:00.000000')
  );

Rating

  (5 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

The Universe

Hendrik Stein, November 13, 2019 - 2:22 pm UTC

Thanks Tom,
this is fine for me.
Kind Regards
Hendrik

changing partition key column

Paul, March 10, 2021 - 6:35 am UTC

Greetings!

I have a requirement to change the partition key column in an existing range partitioned table. Currently partitioned on DATE column. We are changing to another column which of TIMESTAMP datatype.

Can you please advise on the best approach (or possible option) how to proceed with this change .?

1) create another table with partition scheme and copy the data from old table.

2) Is there any way i can do this in place partition key column change?

Table has about 100+ billion rows and table size is ~10TB.


Thanks !

Chris Saxon
March 10, 2021 - 9:06 am UTC

If you're on 18c or higher, you can change it with an alter table command:

create table t (
  date_col date, ts_col timestamp
) partition by range ( date_col ) (
  partition p0 values less than ( date'2021-01-01' )
);

alter table t 
  modify partition by range ( ts_col ) (
  partition p0 values less than ( timestamp'2021-01-01 00:00:00' )
) online;


This has an online option.

Before that dbms_redefinition is the best way to go - this allows you to change the table with little/no downtime.

https://oracle-base.com/articles/misc/partitioning-an-existing-table

Changing partition key column

Paul, March 11, 2021 - 7:05 am UTC

Thanks Chris!

Currently the database is on 12.2. So looks like DBMS_REDEFINITION is the best option..

Questions:

1) Spacewise, we need double the space to maintain both current table and interim table, right ?

2) Also, I need to update the new partition key column (which is TIMESTAMP) to UTC time. Is it possible to modify the data with dbms_redefinition on the fly ?


3) With 18c, Does Oracle need to rewrite the data with partition key column change when we execute with alter table command?


Thanks!

Chris Saxon
March 11, 2021 - 9:06 am UTC

1 Yes

2 Yes, you can change the values during the redef, use the col_mapping parameter of start_redef_table

3 - not sure what you're asking here. If you mean - does the alter table to modify partitioning need more space, the answer is yes.

changing partition key

Paul, March 12, 2021 - 7:44 am UTC

Thanks!

1) If i use DBMS_REDEFINITION approach to change the partition key column in Oracle 12,2m

My table has 128 billion rows
13 TB in size
There are 2 local partitioned indexes exist
37586 - total number of daily partitions (Query High compressed)

Questions-

1) Will it blow up undo space and when does dbms_redef commit the rows ? .

2) How do i speed the copy process as quickly as possible and most importantly how to execute without any issue?

3)

To my earliest question
>> - not sure what you're asking here. If you mean - does the alter table to modify partitioning need more space, the answer is yes.

in 18c, Is changing partition key column change is simple dictionary update and how the data gets distributed across the partitions after modifying the partition key column .?


Thanks!
Chris Saxon
March 12, 2021 - 4:37 pm UTC

The idea with dbms_redefinition is:

You create a copy of the original table, changing its properties (e.g. partition key) in the process.

So whether this "blows up undo" depends on how much undo you have available and how you carry out the process!

You copy indexes, constraints, etc. from the original table to the new (there's a handy procedure to do this for you)

Then you sync any changes in the original table with the new (again, via the API); how long this takes depends on the rate of change for you tables and how long you wait to do it.

This is all work you can do as a background process while the application is running. That's the point of dbms_redefinition ;)

Switch the tables over (finish_redef_table)!

This will - briefly - lock the original table. So while you can do this while the application is online, it some cases you may want a brief outage to complete the process.

in 18c, Is changing partition key column change is simple dictionary update and how the data gets distributed across the partitions after modifying the partition key column .?

No, because in general changing the partitioning key means rows will now be in different partitions. The database has to move them to wherever they're going.

changing partition key

Paul, March 15, 2021 - 7:26 am UTC

Thanks Chris!

As i understand, DBMS_REDEFINITION uses incremental materialized views /MVIEW log mechanism to keep track of the rows modifications to the source table since the DBMS_REDEFINITION.START_REDEF_TABLE started.

Questions -
-------------

Database is on Oracle 12.2
1) My partition table has about 128 billion rows.
Also, i believe, DBMS_REDEFINITION users INSERT /*+ APPEND */ behind the covers. So either minimal UNDO or no undo.,
mostly redo generated during this redefinition activity.

1) Can you please shed some light on when the COMMIT happens with dbms_redefinition API.?
Does Oracle issue commit in the end of START_REDEF_TABLE ?
That means, at that point, my 128 billion rows gets inserted into the interim table, right ?

2) Since my partition table has 10000's of partitions present.

Is there any option in DBMS_REDEFINITION, API, i can do batch of partitions at a time to avoid any resource contention like temp, undo, redo issues .?

3)
If for some reason, if tablespace space issue occur or some errors occurs, Do i have option in DBMS_REDEF process to simply restart the process without losing any of the work done until before the error occurred .?

Thanks!

Chris Saxon
March 15, 2021 - 3:31 pm UTC

1 - yes, it'll start the process of copying the rows to the interim table

2 - you redefine a subset of the partitions at a time. You can only do this in some cases though - as you're changing the partitioning column, this is an all-or-nothing job

3 - Redefinition can continue after some errors.

You can check if it's continuable and how by querying:

DBA_REDEFINITION_STATUS

See more on this in the docs https://docs.oracle.com/en/database/oracle/oracle-database/18/admin/managing-tables.html#GUID-58F4E092-A86D-4E00-96F6-56A596C91A53

More to Explore

VLDB

If you are new to partitioning, check out Connor McDonald's introduction series here.

VLDB

Documentation set on VLDB and Partitioning.