Skip to Main Content
  • Questions
  • How to use partial indexes for PK/UK keys

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Petr.

Asked: September 07, 2018 - 11:26 am UTC

Last updated: October 17, 2018 - 10:19 am UTC

Version: 12c

Viewed 1000+ times

You Asked

LiveSQL link: not working post link into LiveSQL link, but link works
https://livesql.oracle.com/apex/livesql/s/g78ij8jxinq4409pp8dbz3owo

Hello all,
we have a BI solution (Oracle 12c + ODI11).
We have a lot of partitioned tables (daily and monthly list partitioning).
Each table has PK key (partitioning key included, index local) - typically DATE_KEY+PK_xx
Each table has UK key (partitioning key included, index local) - typically DATE_KEY+SOURCE_ID+SOURCESYS_ID columns (natural key).

We have a 3y+ history in tables. Index size for old partititons, where no need active constraints (PK and UK), we would like use partial indexing feature.

But...we can't....(no supported partial indexing on unique keys)

I would try to use non unique indexes for PK and UK wth partial indexing...and not works...

We need PK and UK during loading data into target tables, but on query phase we typically use only 1y data and not used PK and UK.

Have you any idea ?

and Chris said...

The issue is you can't have partial constraints. A unique key applies across all rows in the table.

As the docs state:

Unique indexes cannot be partial indexes. This applies to indexes created with the CREATE UNIQUE INDEX statement and indexes that are implicitly created when you specify a unique constraint on one or more columns.

https://docs.oracle.com/en/database/oracle/oracle-database/18/sqlrf/CREATE-INDEX.html#GUID-1F89BBC0-825F-4215-AF71-7588E31D8BFE

If you want the constraint on some rows but not others, you'll need to split into two tables.

But...

What harm is this constraint doing? What benefit are you hoping to gain by disabling the PK/UC on old partitions?

Rating

  (10 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Function Based Index?

Duke Ganote, September 07, 2018 - 8:51 pm UTC

Not a constraint, but FBI could provide "partial indexing", I assume it works across partitions too.
Connor McDonald
September 08, 2018 - 10:48 am UTC

True, but you need the partition key if you want the index to be local...which might defeat the purpose.

With regard to

We need PK and UK during loading data into target tables, but on query phase we typically use only 1y data and not used PK and UK.


On the presuming the keys are to prevent things like erroneous duplicates being loaded etc, it strikes me that perhaps a better way of managing this would be something like:

insert into target
select * from source
where (pk cols) not in ( select pk cols from target )

and then once loaded, set the table or partitions to read only to lock the data.

manual workaround

Racer I., September 10, 2018 - 8:35 am UTC

Hi,

Would this work :

Have those indexes as normal(non-unique) local indexes on your partitions
Exchange the last partition with a table
Activate the unique constraints on that table
do your loads into that table
deactivate the constraints
swap the table with the last partition
maybe : set/keep partitioned table as read only and loading table as r/w

regards,
Chris Saxon
September 10, 2018 - 10:27 am UTC

Assuming splitting the table into two (one with constraints, one without) is viable then some form of partition exchange could work, yes.

Thanks, but...

Petr Simbera, September 10, 2018 - 10:53 am UTC

Hello guys,
thanks for response. I describe problem with PK,UK and partialindexes, but it't only physical implementation.

The idea is eliminate "dead storage consume objects".
PK and UK on tables is about 20% size of data volume.

This keys we don't use after some times, I look how to minimize index storage size (partial indexing is one way, but only for query performance indexes, not for "loading" indexes).

Exchange partition is good way, but we got in one load data for 5 to 45 partitions (based on DATE_KEY), exchange partition has no way. And not all snap, but only increment.
We use MERGE for loading.

Have you any other ideas ?
PS: select where not in select is not way for 100K rows and more, better is not exist, but it's no support for update rows.
Chris Saxon
September 10, 2018 - 4:11 pm UTC

PK and UK on tables is about 20% size of data volume.

Really? That table has many mandatory columns. I'm surprised the PK+UK are 20% the size of the table.

How are you measuring this?

But more importantly: what problem is this space consumption causing?

PS: select where not in select is not way for 100K rows and more, better is not exist, but it's no support for update rows.

I don't understand what you mean by this. Could you clarify please?

simplify (maybe)

Racer I., September 10, 2018 - 12:05 pm UTC

Hi,

What should happen if you get two rows with
a) same PK, different UK
b) same UK, different PK?

I assume only the UK makes sense. So I would leave off the PK (index and constraint).

I would also leave off the UK (again both index and constraint).
Duplicate rows would already be addressed with the MERGE (assuming you JOIN on UK).
You can either leave the WHEN MATCHED clause empty or update existing rows with current values

Make liberal use of ALTER SESSION ENABLE PARALLEL DML and PARALLEL(16)-Hints.

If you can slip in some date filters you should still get partition pruning.

Further assumption : there is no danger of free inserts during the day violating those constraints.

regards,

Chris Saxon
September 10, 2018 - 4:12 pm UTC

I'm not sure what you're getting at here. How can you have two rows with the same PK?

clarify (or not)

Racer I., September 11, 2018 - 6:27 am UTC

Hi,

That question was for the OP and was about trying to add a new row that matches an existing row partially.

If you load the same row multiple times it would be completely the same (same PK, same UK, same rest) so either PK or UK constraints would stop this insert.

If you can have similar rows that match in PK or UK (but not both) and may also differ in the rest then you need to define what should happen.

I think only the UK(natural key)-fields should count for duplicates. So I would leave off the PK since getting two rows with different UKs but the same PK would not be a problem.

If you use the UK-fields for the ON-join of the merge you avoid duplicate UK-rows in the target table without a UK-index or constraint (given no DML other than this load).

There remains to decide what to do with UK-matches : either ignore them (first loaded row wins) or update them (last loaded row and its rest-values wins).

regards,

Chris Saxon
September 11, 2018 - 9:52 am UTC

getting two rows with different UKs but the same PK would not be a problem

That's a bold assertion! If the PK constraint is irrelevant, surely a better way would be to drop this column?

on partial indexing

Rajeshwaran, Jeyabal, September 11, 2018 - 9:09 am UTC


....
Each table has PK key (partitioning key included, index local) - typically DATE_KEY+PK_xx
Each table has UK key (partitioning key included, index local) - typically DATE_KEY+SOURCE_ID+SOURCESYS_ID columns (natural key).
....


Since each key columns have partition key in their PK and FK. so the index supporting those constraints can be defined as local and we can have those un-needed partition index to be set unusable.

something like this.

demo@ORA12C> create table t1
  2  partition by range( dt )
  3  ( partition p_2014 values less than
  4     ( to_date('01-Jan-2015','dd-mon-yyyy') ),
  5    partition p_2015 values less than
  6     ( to_date('01-Jan-2016','dd-mon-yyyy') ),
  7    partition p_2016 values less than
  8     ( to_date('01-Jan-2017','dd-mon-yyyy') ) )
  9  as
 10  select a.*,
 11     to_date('01-Jan-2014','dd-mon-yyyy') + mod(rownum,1080) dt
 12  from all_objects a;

Table created.

demo@ORA12C> create unique index t1_idx on t1(dt,object_id) local(
  2     partition p_2014,
  3     partition p_2015,
  4     partition p_2016);

Index created.

demo@ORA12C> alter table t1 add constraint t1_pk
  2  primary key(dt,object_id)
  3  using index t1_idx;

Table altered.

demo@ORA12C> alter index t1_idx modify partition p_2014 unusable;

Index altered.

demo@ORA12C> alter index t1_idx modify partition p_2015 unusable;

Index altered.

demo@ORA12C> select segment_name,partition_name
  2  from user_segments
  3  where segment_name ='T1_IDX';

SEGMENT_NAME                   PARTITION_
------------------------------ ----------
T1_IDX                         P_2016

demo@ORA12C> create table t2 as select * from t1;

Table created.

demo@ORA12C> alter table t2 add constraint t2_fk
  2  foreign key(dt,object_id)
  3  references t1;

Table altered.

demo@ORA12C> set autotrace traceonly explain
demo@ORA12C> select *
  2  from t1, t2
  3  where t1.object_id = t2.object_id
  4  and t1.dt = to_date('11-sep-2016','dd-mon-yyyy');

Execution Plan
----------------------------------------------------------
Plan hash value: 1348281905

--------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                                      | Name    | Rows  | Bytes | Cost (%CPU)| Time  | Pstart| Pstop |
--------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                               |         |    67 | 18894 |   503   (1)| 00:00:01 |       |       |
|   1 |  VIEW                                          | VW_TE_2 |    68 | 66640 |   503   (1)| 00:00:01 |       |       |
|   2 |   UNION-ALL                                    |         |       |       |            |       |  |       |
|*  3 |    HASH JOIN                                   |         |    67 | 18894 |   503   (1)| 00:00:01 |       |       |
|   4 |     PARTITION RANGE SINGLE                     |         |    67 |  9447 |    70   (0)| 00:00:01 |     3 |     3 |
|   5 |      TABLE ACCESS BY LOCAL INDEX ROWID BATCHED | T1      |    67 |  9447 |    70   (0)| 00:00:01 |     3 |     3 |
|*  6 |       INDEX RANGE SCAN                         | T1_IDX  |    22 |       |     2   (0)| 00:00:01 |     3 |     3 |
|   7 |     TABLE ACCESS FULL                          | T2      | 72364 |  9964K|   432   (1)| 00:00:01 |       |       |
|*  8 |    FILTER                                      |         |       |       |            |       |  |       |
|*  9 |     HASH JOIN                                  |         |    67 | 18894 |   503   (1)| 00:00:01 |       |       |
|  10 |      PARTITION RANGE SINGLE                    |         |    67 |  9447 |    70   (0)| 00:00:01 |     3 |     3 |
|  11 |       TABLE ACCESS BY LOCAL INDEX ROWID BATCHED| T1      |    67 |  9447 |    70   (0)| 00:00:01 |     3 |     3 |
|* 12 |        INDEX RANGE SCAN                        | T1_IDX  |    22 |       |     2   (0)| 00:00:01 |     3 |     3 |
|  13 |      TABLE ACCESS FULL                         | T2      | 72364 |  9964K|   432   (1)| 00:00:01 |       |       |
--------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - access("T1"."OBJECT_ID"="T2"."OBJECT_ID")
   6 - access("T1"."DT"=TO_DATE(' 2016-09-11 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
   8 - filter(NULL IS NOT NULL)
   9 - access("T1"."OBJECT_ID"="T2"."OBJECT_ID")
  12 - access("T1"."DT"=TO_DATE(' 2016-09-11 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

demo@ORA12C> select *
  2  from t1, t2
  3  where t1.object_id = t2.object_id
  4  and t1.dt = to_date('11-sep-2015','dd-mon-yyyy');

Execution Plan
----------------------------------------------------------
Plan hash value: 2573148550

------------------------------------------------------------------------------------------------------
| Id  | Operation                  | Name    | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT           |         |    67 | 18894 |   581   (2)| 00:00:01 |       |       |
|   1 |  VIEW                      | VW_TE_2 |    68 | 66640 |   581   (2)| 00:00:01 |       |       |
|   2 |   UNION-ALL                |         |       |       |            |          |       |       |
|*  3 |    FILTER                  |         |       |       |            |          |       |       |
|*  4 |     HASH JOIN              |         |    67 | 18894 |   581   (2)| 00:00:01 |       |       |
|   5 |      PARTITION RANGE SINGLE|         |    67 |  9447 |   148   (2)| 00:00:01 |     2 |     2 |
|*  6 |       TABLE ACCESS FULL    | T1      |    67 |  9447 |   148   (2)| 00:00:01 |     2 |     2 |
|   7 |      TABLE ACCESS FULL     | T2      | 72364 |  9964K|   432   (1)| 00:00:01 |       |       |
|*  8 |    HASH JOIN               |         |    67 | 18894 |   581   (2)| 00:00:01 |       |       |
|   9 |     PARTITION RANGE SINGLE |         |    67 |  9447 |   148   (2)| 00:00:01 |     2 |     2 |
|* 10 |      TABLE ACCESS FULL     | T1      |    67 |  9447 |   148   (2)| 00:00:01 |     2 |     2 |
|  11 |     TABLE ACCESS FULL      | T2      | 72364 |  9964K|   432   (1)| 00:00:01 |       |       |
------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - filter(NULL IS NOT NULL)
   4 - access("T1"."OBJECT_ID"="T2"."OBJECT_ID")
   6 - filter("T1"."DT"=TO_DATE(' 2015-09-11 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
   8 - access("T1"."OBJECT_ID"="T2"."OBJECT_ID")
  10 - filter("T1"."DT"=TO_DATE(' 2015-09-11 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

demo@ORA12C> set autotrace off
demo@ORA12C>
demo@ORA12C>
demo@ORA12C> select table_name,constraint_name,status
  2  from user_constraints
  3  where table_name in ('T1','T2')
  4  and constraint_type in ('P','R');

TABLE_NAME           CONSTRAINT STATUS
-------------------- ---------- --------
T2                   T2_FK      ENABLED
T1                   T1_PK      ENABLED

demo@ORA12C>
demo@ORA12C> select segment_name,partition_name
  2  from user_segments
  3  where segment_name ='T1_IDX';

SEGMENT_NAME                   PARTITION_
------------------------------ ----------
T1_IDX                         P_2016

demo@ORA12C>


Table expansion optimization will kick in based on the availability of index in each partition.

https://blogs.oracle.com/optimizer/optimizer-transformations:-table-expansion

by this way we can have the constraint available and index only on the certain needed partitions.

kindly advice, if i am getting it wrong.


Chris Saxon
September 11, 2018 - 10:05 am UTC

Good point yes: you could make the old index partitions unusable. This looks like it'll meet their needs.

Impact on library cache

A reader, October 12, 2018 - 4:56 pm UTC

On Olap production environment we load Imtensive and huge amount of data .billions.
Think use the indexing off when loading.
Since the loaded data are queried ahead in the same session in order to build reporting tables so need index on.
So I wonder if there is impact on the library cache pin and lock since this is ddl and notice that other sessions could be loading in same table for another table partition. Notice also that tables are partitioned by load file ID and are subpartitioned by a business code id
Chris Saxon
October 15, 2018 - 12:42 pm UTC

Why exactly are you worried about this? Have you tested the process?

Not all the questions are on a test case basis

A reader, October 15, 2018 - 5:31 pm UTC

Table A is a partitioned table in transactional heavy inserted by many users.
User n can insert only one partition pn at a time.
But one partition is about 4 billions rows commi f from a big XML file. This is step X.
Once the whole file is inserted the same session use table A partition pn to build many other table for reporting this include lit of aggregations ...this is step Y.

My concern is to use this feature of partial index in order to make load faster in the step X (indexing off) and queries faster in step Y (indexing on).

I have a doubt of how this ddl will impact the other sessions who are loading or querying another partitions.

Chris Saxon
October 16, 2018 - 4:19 pm UTC

You've got a well-defined scenario there - sounds to me like you can test it...

I have a doubt of how this ddl will impact the other sessions who are loading or querying another partitions.

Which DDL?

Answer

A reader, October 16, 2018 - 5:50 pm UTC

No cant test leasing 8billions rows.

As i said user 1 schedule a load of billions while user 2 is loading also billions in another partition.

Ddl is the alter table modify all subpartions indexing ON.

will this ddl impact the other partitions who's indexing is Off?
I mean library cache side
Chris Saxon
October 17, 2018 - 10:19 am UTC

You don't need to test loading 8 billion rows. Loading a several million should still take long enough for you test what happens when you enable/disable indexing.

See comments below.

inverse deferral

Racer I., October 17, 2018 - 6:47 am UTC

Hi,

I don't think there is a 'ALL PARTITIONS' syntax. You can only specify individual partitions although sometimes multiple ones in one command.

https://docs.oracle.com/database/121/SQLRF/statements_3001.htm#SQLRF01001

Specify INDEXING ON to change the indexing property for a table partition to ON. This operation has no effect on full indexes on the table. It has the following effects on partial indexes on the table:

Local partial indexes: The table partition is included in the index. The corresponding index partition is rebuilt and marked USABLE.

Global partial indexes: The table partition is included in the index. Index entries for the table partition are added to the index as part of routine index maintenance.

----

I guess a readers question pertains to INDEXING ON for global indexes :

- Index entries for the table partition are added to the index as part of routine index maintenance.

My speculation : this need not be blocking (as presumably the version for local indexes is) but deferred and doesn't mean the global index is unusable in that time (which could be a while for a billion rows...) :
1. add the new partition entries to the global index (probably via SYS.PMO_DEFERRED_GIDX_MAINT_JOB too).
2. while this happens they are ignored during index access, because their partition is still treated as INDEXING OFF, like dropped partitions
3. when finished the partition is marked INDEXING ON and after that the new entries are used.

Or maybe this is blocking, just not locking (making the global index unusable). I also would assume Oracle will first build a local index for the Partition and then merge it into the global one but that will still take time and the above technique may help reduce the impact.

But you would have to actually test and observe.

regards,

Chris Saxon
October 17, 2018 - 10:17 am UTC

Changing indexing is a blocking DDL operation including in 18c. You can verify this by running:

create table t (
  c1 int, c2 int
) partition by list ( c1 ) (
  partition p1 values ( 1 ) indexing off,
  partition p2 values ( 2 ) indexing off,
  partition p3 values ( 3 ) indexing off,
  partition p4 values ( 4 ) indexing on
);

create index i2 on t ( c2 ) indexing partial;


Then in a second session, inserting a row, but not committing it:

insert into t values ( 1, 1 );


Then back in the first:

alter table t modify partition p1 indexing on;

ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired


Though this only applies to DML on the partition you're changing the indexing properties for.

This can also invalidate cursors. Exactly which cursors depends on your version and your queries.

For example, commit in the second session, then run:

select /* c2 */* from t where c2 = 1;

select sql_id, invalidations
from   v$sql
where  sql_text like 'select /* c2 */* from t%';

SQL_ID          INVALIDATIONS   
gzy3k6kscb161                 0 

alter table t modify partition p1 indexing on;

select /* c2 */* from t where c2 = 1;

select sql_id, invalidations
from   v$sql
where  sql_text like 'select /* c2 */* from t%';

SQL_ID          INVALIDATIONS   
gzy3k6kscb161                 1 


The invalidated cursors will be re-parsed. So they need to test their application to see the impact.

But you would have to actually test and observe.

Indeed!

More to Explore

Design

New to good database design? Check out Chris Saxon's full fundamentals class.