Skip to Main Content

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Nikhil.

Asked: July 05, 2018 - 9:31 am UTC

Last updated: July 10, 2018 - 9:51 am UTC

Version: 11g

Viewed 1000+ times

You Asked

Hi Team,

Could you please have a look at below use case and help to form SQL/PLSQL using which I can get the below report..

Table: order_country : holds order id and country its belong.
There can be 100 and more countries in that but for sample case I have added only 2 india & china here.

For each country there is a separate database table like tab_india, tab_china, tab_us, tab_uk, and so on..
Each table can have different set of columns and can have 100, 200... columns (for sample test i have added only 4 columns here).

I need to get the result -
There will be order_id's as an input (it can be 100 or more)..
Using order_id, get the country from order_country
Then for each country, query the respective table like for 1001-india table would be tab_india, 1002-china table would be tab_china, etc.

Idea is to get coulmns (excluding first 4 columns order_id, order_dt, order_sub_id, order_type) having value as 1 WHERE order_type = 'U' and the count for which particular column is updated for a guven order_id and order_type = 'U'.
i.e. we would search value 1
-- in columns low_value_ind, high_value_ind, credit_value_ind, debit_value_ind for tab_india and
-- in columns lvl_indicator, hgh_indicator, ccy_ind, amt_ind for tab_china etc

Finally below data set to be passed to Frontend..

First 4 columns of each table are used in WHERE to match order_id, order_type, order_dt
NOTE: In each table, we need to check only for order_type = 'U'

Expected result:

ORDER_ID ORDER_DT COLUMNS_UPDATED  UPDATED_COUNT
-------- -------- ---------------  -------------
1001     28-02-18 high_value_ind   3
1001     28-02-18 credit_value_ind 1
1001     28-02-18 debit_value_ind  2
-- Column: low_value_ind will not be considered for 1001 and tab_india as its not contain value 1 for order_type = 'U'
1002     28-02-18 lvl_indicator    3
1002     28-02-18 hgh_indicator    3
1002     28-02-18 amt_ind          2
-- Column: ccy_ind will not be considered for 1002 and tab_china as its not contain value 1 for order_type = 'U'

-- Table structure and sample data..
DROP TABLE order_country PURGE;

DROP TABLE tab_india PURGE;

DROP TABLE tab_china PURGE;

CREATE TABLE order_country
(order_id number(4),
country varchar(50));

INSERT INTO order_country
VALUES (1001, 'india');

INSERT INTO order_country
VALUES (1002, 'china');

COMMIT;

-- tab_india -- 
CREATE TABLE tab_india
(order_id number(4),
order_dt  date,
order_sub_id number(4),
order_type VARCHAR2(1),
low_value_ind number(1),
high_value_ind number(1),
credit_value_ind number(1),
debit_value_ind number(1));

INSERT INTO tab_india
(order_id, order_dt, order_sub_id, order_type, low_value_ind, high_value_ind, credit_value_ind, debit_value_ind)
VALUES 
(1001, TO_DATE('28/02/2018','DD/MM/YYYY'), 1, 'I', 1, null, null,1);

INSERT INTO tab_india
(order_id, order_dt, order_sub_id, order_type, low_value_ind, high_value_ind, credit_value_ind, debit_value_ind)
VALUES 
(1001, TO_DATE('28/02/2018','DD/MM/YYYY'), 1, 'D', null, 1, null,1);

INSERT INTO tab_india
(order_id, order_dt, order_sub_id, order_type, low_value_ind, high_value_ind, credit_value_ind, debit_value_ind)
VALUES 
(1001, TO_DATE('28/02/2018','DD/MM/YYYY'), 2, 'U', null, 1, null,1);

INSERT INTO tab_india
(order_id, order_dt, order_sub_id, order_type, low_value_ind, high_value_ind, credit_value_ind, debit_value_ind)
VALUES 
(1001, TO_DATE('28/02/2018','DD/MM/YYYY'), 2, 'U', null, 1, 1, 1);

INSERT INTO tab_india
(order_id, order_dt, order_sub_id, order_type, low_value_ind, high_value_ind, credit_value_ind, debit_value_ind)
VALUES 
(1001, TO_DATE('28/02/2018','DD/MM/YYYY'), 2, 'U', null, 1, null, null);

COMMIT;

-- tab_china -- 
CREATE TABLE tab_china
(order_id number(4),
order_dt  date,
order_sub_id number(4),
order_type VARCHAR2(1),
lvl_indicator number(1),
hgh_indicator number(1),
ccy_ind number(1),
amt_ind number(1));

INSERT INTO tab_china
(order_id, order_dt, order_sub_id, order_type, lvl_indicator, hgh_indicator, ccy_ind, amt_ind)
VALUES 
(1002, TO_DATE('28/02/2018','DD/MM/YYYY'), 1, 'I', 1, null, null,1);

INSERT INTO tab_china
(order_id, order_dt, order_sub_id, order_type, lvl_indicator, hgh_indicator, ccy_ind, amt_ind)
VALUES 
(1002, TO_DATE('28/02/2018','DD/MM/YYYY'), 1, 'D', 1, 1, null,1);

INSERT INTO tab_china
(order_id, order_dt, order_sub_id, order_type, lvl_indicator, hgh_indicator, ccy_ind, amt_ind)
VALUES 
(1002, TO_DATE('28/02/2018','DD/MM/YYYY'), 2, 'U', 1, 1, null,1);

INSERT INTO tab_china
(order_id, order_dt, order_sub_id, order_type, lvl_indicator, hgh_indicator, ccy_ind, amt_ind)
VALUES 
(1002, TO_DATE('28/02/2018','DD/MM/YYYY'), 2, 'U', 1, 1, null, 1);

INSERT INTO tab_china
(order_id, order_dt, order_sub_id, order_type, lvl_indicator, hgh_indicator, ccy_ind, amt_ind)
VALUES 
(1002, TO_DATE('28/02/2018','DD/MM/YYYY'), 2, 'U', 1, 1, null, null);

COMMIT;

and Chris said...

Ug. That's an ugly database...

Anyway, all you need to do is:

- Union all the country tables together
- For each, select null for the columns that don't exist in it but are in others
- Unpivot the results of this

This will automatically exclude the null valued columns:

with all_values as (
  select order_id , order_dt  , 
         low_value_ind , high_value_ind , credit_value_ind , debit_value_ind ,
         null lvl_indicator , null hgh_indicator , null ccy_ind , null amt_ind
  from   tab_india
  where  order_type = 'U'
  union  all
  select order_id , order_dt  ,
         null low_value_ind , null high_value_ind ,
         null credit_value_ind , null debit_value_ind ,
         lvl_indicator , hgh_indicator , ccy_ind , amt_ind
  from   tab_china
  where  order_type = 'U'
), rws as (
  select * from all_values
  unpivot (
    col for src in (
      low_value_ind,
      high_value_ind,
      credit_value_ind,
      debit_value_ind,
      lvl_indicator ,
      hgh_indicator ,
      ccy_ind ,
      amt_ind
    )
  )
)
  select order_id, order_dt, src, count(*) from rws
  group  by order_id, order_dt, src
  order  by 1, 2, 3;

ORDER_ID   ORDER_DT               SRC                COUNT(*)   
      1001 28-FEB-2018 00:00:00   CREDIT_VALUE_IND            1 
      1001 28-FEB-2018 00:00:00   DEBIT_VALUE_IND             2 
      1001 28-FEB-2018 00:00:00   HIGH_VALUE_IND              3 
      1002 28-FEB-2018 00:00:00   AMT_IND                     2 
      1002 28-FEB-2018 00:00:00   HGH_INDICATOR               3 
      1002 28-FEB-2018 00:00:00   LVL_INDICATOR               3 

Rating

  (6 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

UNPIVOT slows on table with 500+ columns

A reader, July 09, 2018 - 6:18 am UTC

Thanks for your reply Chris!
UNPIVOT gives me expected results, however tables having more than 500 columns taking time almot 2mins to generate result.

Is there ant hint i can use while dealing with UNPIVOT?
Could you please suggest any which i can try.

UNPIVOT slows on table with 500+ columns

A reader, July 09, 2018 - 6:50 am UTC

Correcting previous review comments:

Thanks for your reply Chris!
UNPIVOT gives me expected results, however I have a table having more than 500 columns and no of rowsaround 10000 for given order_id taking time almost 2mins to generate result for a particualr order_id.

Is there ant hint i can use while dealing with UNPIVOT?
Could you please suggest any which i can try.
Chris Saxon
July 09, 2018 - 10:38 am UTC

So your output could have up to (500 * 10,000) = 5,000,000 rows? That's going to take a while to fetch!

For us to help further, we need to see the execution plan for the query. You can learn how to create one in this blog post:

https://blogs.oracle.com/sql/how-to-create-an-execution-plan

UNPIVOT on Values 1 and 0

Nikhil, July 09, 2018 - 7:08 am UTC

Hi Chris,

In addition to above question, in my case the columns used in UNPIVOT are having values either 1 or 0.
Could you please help- how to exclude columns having value 0 from UNPIVOT.
Chris Saxon
July 09, 2018 - 10:38 am UTC

A where clause usually does the trick...

SQL and execution plan after enabling autotrace on

A reader, July 09, 2018 - 2:41 pm UTC

Hi Chris-below query contains 500 columns but due to max limits aded only 3 columns
WITH ATTR_TAB AS (SELECT ORER_ID,ORD_DT,THREE_D_MATCH_LINKING_ID,TRCH_ID,TRDG_BK_INDCTR, . col_500
FROM TAB_1
WHERE ORER_ID IN ('137808') AND ADJMT_OPRN_TYP = 'U'
UNION ALL
SELECT ORER_ID,ORD_DT,THREE_D_MATCH_LINKING_ID,TRCH_ID,TRDG_BK_INDCTR, . col_500
FROM TAB_2
WHERE ORER_ID IN ('137808') AND ADJMT_OPRN_TYP = 'U')
SELECT ORER_ID, NULL, ATTR_NAME, SUM(ATTR_VALUE) ATTR_UPDT_COUNT
FROM ATTR_TAB UNPIVOT EXCLUDE NULLS(ATTR_VALUE FOR ATTR_NAME IN(THREE_D_MATCH_LINKING_ID AS 'Three D Match Linking Identifier',TRCH_ID AS 'Tranche Identifier',TRDG_BK_INDCTR AS 'Trading Book Indicator', .. upto 500 columns))
WHERE ATTR_VALUE <> 0 GROUP BY ORER_ID, ATTR_NAME; -- 99.811 secs

Plan hash value: 2713373401

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | TQ |IN-OUT| PQ Distrib |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 1013 (100)| | | | | | |
| 1 | TEMP TABLE TRANSFORMATION | | | | | | | | | | |
| 2 | PX COORDINATOR | | | | | | | | | | |
| 3 | PX SEND QC (RANDOM) | :TQ10000 | | | | | | | Q1,00 | P->S | QC (RAND) |
| 4 | LOAD AS SELECT | | | | | | | | Q1,00 | PCWP | |
| 5 | UNION-ALL | | | | | | | | Q1,00 | PCWP | |
| 6 | PX BLOCK ITERATOR | | 1 | 57 | 2 (0)| 00:00:01 | KEY | KEY | Q1,00 | PCWC | |
|* 7 | TABLE ACCESS STORAGE FULL | TAB_1 | 1 | 57 | 2 (0)| 00:00:01 | 27 | 27 | Q1,00 | PCWP | |
| 8 | PX BLOCK ITERATOR | | 1 | 100 | 2 (0)| 00:00:01 | 1 | 4 | Q1,00 | PCWC | |
|* 9 | TABLE ACCESS STORAGE FULL | TAB_2 | 1 | 100 | 2 (0)| 00:00:01 | 1 | 4 | Q1,00 | PCWP | |
| 10 | PX COORDINATOR | | | | | | | | | | |
| 11 | PX SEND QC (RANDOM) | :TQ20001 | 1008 | 71568 | 1009 (1)| 00:00:15 | | | Q2,01 | P->S | QC (RAND) |
| 12 | HASH GROUP BY | | 1008 | 71568 | 1009 (1)| 00:00:15 | | | Q2,01 | PCWP | |
| 13 | PX RECEIVE | | 1008 | 71568 | 1009 (1)| 00:00:15 | | | Q2,01 | PCWP | |
| 14 | PX SEND HASH | :TQ20000 | 1008 | 71568 | 1009 (1)| 00:00:15 | | | Q2,00 | P->P | HASH |
| 15 | HASH GROUP BY | | 1008 | 71568 | 1009 (1)| 00:00:15 | | | Q2,00 | PCWP | |
|* 16 | VIEW | | 1008 | 71568 | 1008 (0)| 00:00:15 | | | Q2,00 | PCWP | |
| 17 | UNPIVOT | | | | | | | | Q2,00 | PCWP | |
| 18 | VIEW | | 2 | 13138 | 2 (0)| 00:00:01 | | | Q2,00 | PCWP | |
| 19 | PX BLOCK ITERATOR | | 2 | 156 | 2 (0)| 00:00:01 | | | Q2,00 | PCWC | |
|* 20 | TABLE ACCESS STORAGE FULL| SYS_TEMP_0FD9DD8C1_245F6FB0 | 2 | 156 | 2 (0)| 00:00:01 | | | Q2,00 | PCWP | |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

7 - storage(:Z>=:Z AND :Z<=:Z AND "ADJMT_OPRN_TYP"='U')
filter("ADJMT_OPRN_TYP"='U')
9 - storage(:Z>=:Z AND :Z<=:Z AND ("ORER_ID"='137808' AND "ADJMT_OPRN_TYP"='U'))
filter(("ORER_ID"='137808' AND "ADJMT_OPRN_TYP"='U'))
16 - filter(("unpivot_view_1515"."ATTR_VALUE"<>0 AND "unpivot_view_1515"."ATTR_VALUE" IS NOT NULL))
20 - storage(:Z>=:Z AND :Z<=:Z)
Chris Saxon
July 10, 2018 - 9:48 am UTC

Thanks. But that looks like an explain plan. Not an execution plan. We need to see A-rows and E-rows columns.

Please also place SQL and plans inside < code > tags. Otherwise we lose the formatting. Which makes reading plans a pain...

SQL and execution plan after enabling autotrace on

A reader, July 09, 2018 - 2:51 pm UTC

Typo in previous review comments.
query took 251 secs to execute for one row and 510 columns..

option

Racer I., July 10, 2018 - 7:10 am UTC

Hi,

Maybe you can unpivot each table separately and then union them together? Could be faster. Although it's possible that unpivot doesn't like being in a subquery.

regards,
Chris Saxon
July 10, 2018 - 9:51 am UTC

Possibly.

If the row estimates for tab_1 and tab_2 are about right (57 & 100) and most of the columns are non-null, then the query is fetching ~ 150 * 500 = 75,000 rows. Returning all that to the client will take a while...

More to Explore

SQL

The Oracle documentation contains a complete SQL reference.