Skip to Main Content

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, yamini.

Asked: August 02, 2017 - 3:24 pm UTC

Last updated: July 23, 2020 - 4:29 am UTC

Version: 11g

Viewed 1000+ times

You Asked

Hello,

I need to increase the performance of the following query,Please suggest alternative,this is taking longer time in production.

SELECT *
FROM ( SELECT MAX (IDN_HEADER_DATA_EXCH) AS IDN_HEADER_DATA_EXCH_MAX,
NBR_SSN AS HEADER_SSN
FROM DATA_EXCH.T_EXCH_DATA_HEADER HDR
WHERE LTRIM (RTRIM (CDE_EXCH_IDN)) = '3'
GROUP BY NBR_SSN) MAXHDR
INNER JOIN (SELECT A.IDN_HEADER_DATA_EXCH AS IDN_HEADER,
A.NBR_SSN AS BDX_NBR_SSN,
B.NAM_FIRST AS BDX_NAM_FIRST,
B.NAM_MIDDLE AS BDX_NAM_MIDDLE,
B.NAM_LAST AS BDX_NAM_LAST,
B.DTE_DOB AS BDX_DTE_DOB,
B.CDE_GENDER AS BDX_CDE_GENDER,
B.CDE_ACTION_SSA AS BDX_CDE_ACTION_SSA,
C.CDE_CAT_SSA AS BDX_CAT_SSA,
C.CDE_COMN_STATE AS BDX_CDE_COMN_STATE,
C.CDE_ACTION_STATE AS BDX_CDE_ACTION_STATE,
C.CDE_COMN_MBR AS BDX_CDE_COMN_MBR,
C.CDE_STATUS_MBR AS BDX_CDE_STATUS_MBR,
C.CDE_OWNER AS BDX_CDE_OWNER,
C.DTE_RCVD_LAST AS BDX_DTE_RCVD_LAST,
C.DTE_SENT_LAST AS BDX_DTE_SENT_LAST
FROM DATA_EXCH.T_EXCH_DATA_HEADER A,
DATA_EXCH.T_MBR_HEADER B,
DATA_EXCH.T_MBR_DETAIL C
WHERE A.IDN_HEADER_DATA_EXCH = B.IDN_HEADER_DATA_EXCH
AND A.IDN_HEADER_DATA_EXCH = C.IDN_HEADER_DATA_EXCH
AND A.NBR_SSN > 0
AND NVL (A.CDE_SKPD, 96) NOT IN ('86', '87', '88')
AND NVL (C.CDE_OWNER, 0) <> '2') BDX
ON BDX.IDN_HEADER = MAXHDR.IDN_HEADER_DATA_EXCH_MAX
FULL OUTER JOIN (SELECT NBR_SSN AS CIS_NBR_SSN,
IDN_INDIV AS CIS_IDN_INDIV,
NAM_FIRST AS CIS_NAM_FIRST,
NAM_MI AS CIS_NAM_MI,
NAM_LAST AS CIS_NAM_LAST,
DTE_DOB AS CIS_DTE_DOB,
CDE_GENDER AS CIS_CDE_GENDER,
CDE_ORIGIN_SOURCE AS CIS_CDE_ORIGIN_SOURCE,
CDE_CAT_PROG AS CIS_BUDGET,
CDE_STATUS_PROG AS CIS_PSC
FROM DATA_EXCH_STAGE.T_EXCH_RQ_CIS_EXRCT_STG
WHERE CDE_ORIGIN_SOURCE = 'IVS'
AND CDE_STATUS_BUDGET = '4'
AND NBR_SSN > 0) CIS
ON BDX.BDX_NBR_SSN = CIS.CIS_NBR_SSN
ORDER BY CIS_NBR_SSN;

and Connor said...

Try this - run it with the hint as below

select /*+ gather_plan_statistics */ * 
FROM ( SELECT MAX (IDN_HEADER_DATA_EXCH) AS IDN_HEADER_DATA_EXCH_MAX,
NBR_SSN AS HEADER_SSN
...


select * from table(dbms_xplan.display_cursor(null,null,'ALLSTATS LAST'));


this will give you the plan and the actual/estimate rows comparison - that will give you info on how good a job the optimizer did.

You can post the plan back here in code tags and we can help with a review.

Rating

  (5 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Full outer join taking longer time to execute,how to avoid and improve performance

yamini v, August 03, 2017 - 2:54 pm UTC

Thanks for the reply
I gathered statistics as following .

SQL_ID bjccztvpdrdyv, child number 0
-------------------------------------
SELECT /*OracleDictionaryQueries.ALL_ROLES_QUERY*/ ROLE FROM
SYS.SESSION_ROLES
Plan hash value: 9211853
-----------------------------------------------------------------------------
| Id | Operation | Name | E-Rows | OMem | 1Mem | Used-Mem |
-----------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | | |
| 1 | NESTED LOOPS | | 1 | | | |
|* 2 | FIXED TABLE FULL | X$KZSRO | 1 | | | |
| 3 | TABLE ACCESS CLUSTER| USER$ | 1 | | | |
|* 4 | INDEX UNIQUE SCAN | I_USER# | 1 | 1025K| 1025K| |
-----------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter(("KZSROROL"<>1 AND "KZSROROL"<>USERENV('SCHEMAID')))
4 - access("U"."USER#"="KZSROROL")
filter(("U"."USER#"<>USERENV('SCHEMAID') AND "U"."USER#"<>1))
Note
-----
- Warning: basic plan statistics not available. These are only collected when:
* hint 'gather_plan_statistics' is used for the statement or
* parameter 'statistics_level' is set to 'ALL', at session or system level
Connor McDonald
August 04, 2017 - 1:44 am UTC

That isn't your query. You're picking up the wrong query - so my guess is you're using TOAD or some other similar tool.

Do it in SQL Plus

Tuning sql

yamini v, August 03, 2017 - 3:06 pm UTC

This is the actual execution plan




Plan hash value: 3447042333
-------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Pstart| Pstop |
-------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 52128 | 10M| | 88461 (5)| 00:17:42 | | |
| 1 | SORT ORDER BY | | 52128 | 10M| 14M| 88461 (5)| 00:17:42 | | |
| 2 | VIEW | VW_FOJ_0 | 52128 | 10M| | 86151 (5)| 00:17:14 | | |
|* 3 | HASH JOIN FULL OUTER | | 52128 | 10M| | 86151 (5)| 00:17:14 | | |
| 4 | VIEW | | 621 | 83214 | | 85781 (5)| 00:17:10 | | |
| 5 | NESTED LOOPS | | 621 | 92529 | | 85781 (5)| 00:17:10 | | |
| 6 | NESTED LOOPS | | 1960 | 92529 | | 85781 (5)| 00:17:10 | | |
|* 7 | HASH JOIN | | 1960 | 133K| 3736K| 83608 (5)| 00:16:44 | | |
|* 8 | HASH JOIN | | 68186 | 2929K| 2664K| 59989 (5)| 00:12:00 | | |
| 9 | JOIN FILTER CREATE | :BF0000 | 68186 | 1864K| | 156 (3)| 00:00:02 | | |
|* 10 | TABLE ACCESS STORAGE FULL | T_MBR_DETAIL | 68186 | 1864K| | 156 (3)| 00:00:02 | | |
| 11 | JOIN FILTER USE | :BF0000 | 20M| 311M| | 32409 (7)| 00:06:29 | | |
| 12 | PARTITION HASH ALL | | 20M| 311M| | 32409 (7)| 00:06:29 | 1 | 16 |
|* 13 | TABLE ACCESS STORAGE FULL | T_EXCH_DATA_HEADER | 20M| 311M| | 32409 (7)| 00:06:29 | 1 | 16 |
| 14 | VIEW | | 208K| 5297K| | 23059 (6)| 00:04:37 | | |
| 15 | HASH GROUP BY | | 208K| 3463K| 5848K| 23059 (6)| 00:04:37 | | |
|* 16 | INDEX STORAGE FAST FULL SCAN | NDX_T_EXCH_DATA_HEADER | 212K| 3525K| | 21971 (6)| 00:04:24 | | |
|* 17 | INDEX UNIQUE SCAN | PK_T_MBR_HEADER | 1 | | | 1 (0)| 00:00:01 | | |
| 18 | TABLE ACCESS BY GLOBAL INDEX ROWID| T_MBR_HEADER | 1 | 79 | | 2 (0)| 00:00:01 | ROWID | ROWID |
| 19 | PARTITION LIST SINGLE | | 52128 | 4327K| | 369 (2)| 00:00:05 | 2 | 2 |
| 20 | VIEW | | 52128 | 4327K| | 369 (2)| 00:00:05 | | |
|* 21 | TABLE ACCESS STORAGE FULL | T_EXCH_RQ_CIS_EXRCT_STG | 52128 | 2698K| | 369 (2)| 00:00:05 | 2 | 2 |
-------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("BDX"."BDX_NBR_SSN"="CIS"."CIS_NBR_SSN")
7 - access("A"."IDN_HEADER_DATA_EXCH"="MAXHDR"."IDN_HEADER_DATA_EXCH_MAX")
8 - access("A"."IDN_HEADER_DATA_EXCH"="C"."IDN_HEADER_DATA_EXCH")
10 - storage(NVL("C"."CDE_OWNER",'0')<>'2')
filter(NVL("C"."CDE_OWNER",'0')<>'2')
13 - storage(NVL("A"."CDE_SKPD",'96')<>'86' AND NVL("A"."CDE_SKPD",'96')<>'87' AND NVL("A"."CDE_SKPD",'96')<>'88' AND
"A"."NBR_SSN">0 AND SYS_OP_BLOOM_FILTER(:BF0000,"A"."IDN_HEADER_DATA_EXCH"))
filter(NVL("A"."CDE_SKPD",'96')<>'86' AND NVL("A"."CDE_SKPD",'96')<>'87' AND NVL("A"."CDE_SKPD",'96')<>'88' AND
"A"."NBR_SSN">0 AND SYS_OP_BLOOM_FILTER(:BF0000,"A"."IDN_HEADER_DATA_EXCH"))
16 - storage(LTRIM(RTRIM("CDE_EXCH_IDN"))='3')
filter(LTRIM(RTRIM("CDE_EXCH_IDN"))='3')
17 - access("A"."IDN_HEADER_DATA_EXCH"="B"."IDN_HEADER_DATA_EXCH")
21 - storage("CDE_STATUS_BUDGET"='4' AND "NBR_SSN">0)
filter("CDE_STATUS_BUDGET"='4' AND "NBR_SSN">0)

How to optmize this query running against specific partition

Rich, July 21, 2020 - 3:12 pm UTC

We have partitioned enabled with query high compression.

With serial run, elapsed time: Elapsed: 00:10:34.51
with DOP 8, it runs in Elapsed: 00:01:16.68

This query is accessing specific partition, Just wondering how to tune this query to improve the query execution time ?

SQL> select * from TABLE(dbms_xplan.display_cursor(format => 'ALLSTATS LAST'))
2 /

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID 348rsusgtg97r, child number 0
-------------------------------------
select source, max(last_price_time - last_update_date) as MIN_LAG,
min(last_price_time - last_update_date) as MAX_LAG, avg(extract(second
from last_price_time - last_update_date)+ extract(minute from
last_price_time - last_update_date) * 60 + extract(hour from
last_price_time - last_update_date) * 60 * 60 + extract(day from
last_price_time - last_update_date) * 60 * 60 * 24) AVG_LAG,
median(extract(second from last_price_time - last_update_date)+
extract(minute from last_price_time - last_update_date) * 60 +
extract(hour from last_price_time - last_update_date) * 60 * 60 +
extract(day from last_price_time - last_update_date) * 60 * 60 * 24)
MEDIAN_LAG from test.ffi_prc PARTITION (F_prc_200610)
a group by source

Plan hash value: 1605070224

-------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | E-Rows | OMem | 1Mem | Used-Mem | Used-Tmp|
-------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | | | |
| 1 | SORT GROUP BY | | 13 | 1239M| 23M| 127M (1)| 2432K|
| 2 | PARTITION RANGE SINGLE | | 77M| | | | |
| 3 | TABLE ACCESS STORAGE FULL| FFI_PRC | 77M| 1025K| 1025K| 5142K (0)| |
-------------------------------------------------------------------------------------------------------------

Note
-----
- Warning: basic plan statistics not available. These are only collected when:
* hint 'gather_plan_statistics' is used for the statement or
* parameter 'statistics_level' is set to 'ALL', at session or system level


32 rows selected.

Connor McDonald
July 22, 2020 - 5:30 am UTC

Can you run it (serial) with the gather_plan_statistics hint as then paste your same display_cursor output

Rich, July 22, 2020 - 2:39 pm UTC

Hi Connor,

Ran the query serial along wtth /*+ gather_plan_statistics */ hint,

Elapsed: 00:08:31.25

SQL> select * from TABLE(dbms_xplan.display_cursor(format => 'ALLSTATS LAST'));

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  g0afnk28nnryd, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ source, max(last_price_time -
last_update_date) as MIN_LAG, min(last_price_time - last_update_date)
as MAX_LAG, avg(extract(second from last_price_time -
last_update_date)+ extract(minute from last_price_time -
last_update_date) * 60 + extract(hour from last_price_time -
last_update_date) * 60 * 60 + extract(day from last_price_time -
last_update_date) * 60 * 60 * 24) AVG_LAG, median(extract(second from
last_price_time - last_update_date)+ extract(minute from
last_price_time - last_update_date) * 60 + extract(hour from
last_price_time - last_update_date) * 60 * 60 + extract(day from
last_price_time - last_update_date) * 60 * 60 * 24) MEDIAN_LAG from
test.ffi_prc PARTITION (F_INCPR_200610) a group by source

Plan hash value: 1605070224

------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                   | Name                    | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |                         |      1 |        |     13 |00:07:51.46 |    3152K|   3889K|    588K|       |       |          |         |
|   1 |  SORT GROUP BY              |                         |      1 |     13 |     13 |00:07:51.46 |    3152K|   3889K|    588K|  1239M|    23M|  128M (1)|    2432K|
|   2 |   PARTITION RANGE SINGLE    |                         |      1 |     77M|     77M|00:00:16.37 |    3152K|   3152K|      0 |       |       |          |         |
|   3 |    TABLE ACCESS STORAGE FULL| FFI_PRC        |      1 |     77M|     77M|00:00:09.55 |    3152K|   3152K|      0 |  1025K|  1025K| 5142K (0)|         |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------


26 rows selected.

Connor McDonald
July 23, 2020 - 4:29 am UTC

You can from the timings, that you had no dramas scanning the 70M rows - that was nice and snappy. It was the aggregation that took all the time.

Where you are getting slammed is the calculation of the median.

Typically, a group by can be very efficient via hashing, eg

SQL> create table t tablespace largets nologging
  2  as select
  3  OWNER
  4  ,OBJECT_NAME
  5  ,SUBOBJECT_NAME
  6  ,OBJECT_ID
  7  ,DATA_OBJECT_ID
  8  ,OBJECT_TYPE
  9  ,cast(CREATED as timestamp) created
 10  ,cast(LAST_DDL_TIME as timestamp) LAST_DDL_TIME
 11  ,TIMESTAMP
 12  ,STATUS
 13  ,TEMPORARY
 14  ,GENERATED
 15  ,SECONDARY
 16  ,NAMESPACE
 17  ,EDITION_NAME
 18  ,SHARING
 19  ,EDITIONABLE
 20  ,ORACLE_MAINTAINED
 21  from dba_objects d,
 22   ( select 1 from dual connect by level <= 200 );

Table created.

SQL> select
  2    owner,
  3    max(created - last_ddl_time) as MIN_LAG,
  4    min(created - last_ddl_time) as MAX_LAG,
  5    avg(extract(second from created - last_ddl_time)+
  6        extract(minute from created - last_ddl_time) * 60 +
  7        extract(hour from created -  last_ddl_time) * 60 * 60 +
  8        extract(day from created - last_ddl_time) * 60 * 60 * 24) AVG_LAG
  9  from t
 10  group by owner;

37 rows selected.

Elapsed: 00:00:09.04

---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |    37 |  1036 | 81466   (3)| 00:00:04 |
|   1 |  HASH GROUP BY     |      |    37 |  1036 | 81466   (3)| 00:00:04 |
|   2 |   TABLE ACCESS FULL| T    |    16M|   439M| 80134   (1)| 00:00:04 |
---------------------------------------------------------------------------



but the moment I add the MEDIAN into the mix, then we have to fall back to sorting the data

SQL> select
  2    owner,
  3    max(created - last_ddl_time) as MIN_LAG,
  4    min(created - last_ddl_time) as MAX_LAG,
  5    avg(extract(second from created - last_ddl_time)+
  6        extract(minute from created - last_ddl_time) * 60 +
  7        extract(hour from created -  last_ddl_time) * 60 * 60 +
  8        extract(day from created - last_ddl_time) * 60 * 60 * 24) AVG_LAG,
  9    median(extract(second from created - last_ddl_time)+
 10              extract(minute from created - last_ddl_time) * 60 +
 11              extract(hour from created - last_ddl_time) * 60 * 60 +
 12            extract(day from created - last_ddl_time) * 60 * 60 * 24) MEDIAN_LAG
 13  from t
 14  group by owner;

37 rows selected.

Elapsed: 00:00:56.81


---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |    37 |  1036 | 81466   (3)| 00:00:04 |
|   1 |  SORT GROUP BY     |      |    37 |  1036 | 81466   (3)| 00:00:04 |
|   2 |   TABLE ACCESS FULL| T    |    16M|   439M| 80134   (1)| 00:00:04 |
---------------------------------------------------------------------------



Somewhat counter-intuitively I got better results by ensuring I had a nice large PGA allocation, and then doing as much work with group-by-hash, and then just doing the median with the sort

SQL> with t1 as
  2   ( select /*+ materialize */
  3        owner,
  4        max(created - last_ddl_time) as MIN_LAG,
  5        min(created - last_ddl_time) as MAX_LAG,
  6        avg(extract(second from created - last_ddl_time)+
  7            extract(minute from created - last_ddl_time) * 60 +
  8            extract(hour from created -  last_ddl_time) * 60 * 60 +
  9            extract(day from created - last_ddl_time) * 60 * 60 * 24) AVG_LAG
 10      from t
 11      group by owner
 12  ),
 13  t2 as
 14  ( select /*+ materialize */
 15    owner,
 16    median(created - last_ddl_time ) MEDIAN_LAG
 17  from t
 18  group by owner
 19  )
 20  select *
 21  from t1, t2
 22  where t1.owner = t2.owner;

37 rows selected.

Elapsed: 00:00:19.44

Statistics
----------------------------------------------------------
         30  recursive calls
         16  db block gets
     586443  consistent gets
     586402  physical reads
       1248  redo size
       2907  bytes sent via SQL*Net to client
        969  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
         37  rows processed


Your mileage may vary, but start with a good sized PGA to improve the sort, and see how you go

David D., July 24, 2020 - 12:46 pm UTC

Hello,
I read in your execution plan " TABLE ACCESS STORAGE FULL" : are you using Exadata?

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.