Skip to Main Content
  • Questions
  • select multiple columns having non default value

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Dhi.

Asked: August 09, 2016 - 6:56 pm UTC

Last updated: August 12, 2016 - 1:24 pm UTC

Version: Oracle 11g

Viewed 1000+ times

You Asked

Hi,

I have a table like below:
create table test
(
test_id number not null,
employee_id number not null,
type varchar2(10),
col1 number default -1,
col2 number default -1,
col3 number default -1,
col4 number default -1,
primary key(test_id,employee_id)
)

with data in it:
insert into test values (100,999,'Type-1',5,-1,-1,4);
insert into test values (101,999,'Type-2',-1,2,-1,-1);
insert into test values (102,999,'Type-3',-1,2,-1,-1);

I need to select columns which doesn't have default values and their corresponding column name(In my original table i have around 20 columns starting from col1...col20)


Expected output is:
Employee_ID Type column_name value
999 Type-1 col1 5
999 Type-1 col4 4
999 Type-2 col2 2
999 Type-3 col2 2


Would you please let me know who can i achieve this?I will need to write in pl/sql since my table has close to 5000 rows.




and Chris said...

OK, here's one way to do it. All in SQL!

First up, to get the output you requested you'll need to convert the columns to rows. Unpivot helps with this:

select * from test
unpivot (
  val for col in (
    col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4'
  )
);


Add other columns to the unpivot clause as needed.

Once you've done this, you can join it to user_tab_cols. This holds the data_default so you can compare to see if it matches this.

But there's a problem. User_tab_cols.data_default is a long. You can't just join this:

with rws as (     
  select * from test
  unpivot (
    val for col in (
      col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4'
    )
  )
)
  select * from rws
  join   user_tab_cols
  on     col = column_name
  and    val <> data_default
  where  table_name = 'TEST';

ORA-00997: illegal use of LONG datatype


So you need to convert data_default to a usable type. Adrian Billington discusses several methods to do this at:

http://www.oracle-developer.net/display.php?id=430

I've gone with the "convert to XML and extract the values" method:

with xml as (
select dbms_xmlgen.getxmltype(
 'select column_name, data_default
  from   user_tab_cols where  table_name = ''TEST''') x
from dual
), col_defs as (
  select extractValue(xt.object_value, '/ROW/COLUMN_NAME') as col_name,
         extractValue(xt.object_value, '/ROW/DATA_DEFAULT') as d_def
  from   xml, table(xmlsequence(extract(xml.x, '/ROWSET/ROW'))) xt
)
  select * from col_defs;


Put it all together and you've got:

create table test (
test_id number not null,
employee_id number not null,
type varchar2(10),
col1 number default -1,
col2 number default -1,
col3 number default -1,
col4 number default -1,
primary key(test_id,employee_id)
);

insert into test values (100,999,'Type-1',5,-1,-1,4);
insert into test values (101,999,'Type-2',-1,2,-1,-1);
insert into test values (102,999,'Type-3',-1,2,-1,-1);
insert into test values (103,999,'Type-3',default,default,default,default);

with rws as (     
  select * from test
  unpivot (
    val for col in (
      col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4'
    )
  )
), xml as (
select dbms_xmlgen.getxmltype(
 'select column_name, data_default
  from   user_tab_cols where  table_name = ''TEST''') x
from dual
), col_defs as (
  select extractValue(xt.object_value, '/ROW/COLUMN_NAME') as col_name,
         extractValue(xt.object_value, '/ROW/DATA_DEFAULT') as d_def
  from   xml, table(xmlsequence(extract(xml.x, '/ROWSET/ROW'))) xt
)
  select r.* 
  from   rws r
  join   col_defs c
  on     c.col_name = r.col
  and    c.d_def <> r.val;

TEST_ID  EMPLOYEE_ID  TYPE    COL   VAL  
100      999          Type-1  COL1  5    
102      999          Type-3  COL2  2    
101      999          Type-2  COL2  2    
100      999          Type-1  COL4  4

Rating

  (3 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Can you explain?

AndyP, August 11, 2016 - 1:12 pm UTC

I wonder if you can explain the different result I get when trying that approach (tried on 11.2.0.4.0 using sqlplus directly on the linux server, and 12.1.0.2 via sql developer)

When I cut'n'paste exactly your setup and code I get:
  and    c.d_def <> r.val
         *
ERROR at line 22:
ORA-01722: invalid number


Initially I thought numeric-varchar2 data conversion issues, but after some investigation it turned out that the value being returned for d_def (the data_default LONG originally) had a newline and some spaces at the end

DUMPDEF
----------------------------------
Typ=1 Len=6: 45,49,10,32,32,32


and so I could make it work by using this

r.val <> rtrim(rtrim(to_char(c.d_def)),chr(10))


I also tried using the XMLTABLE syntax instead of XMLSEQUENCE, but with the same result and the same fix

The fix I used does work, at least in this case, but relies on inspection of "hidden" characters and feels a bit fragile

Plus it doesn't seem to have been necessary for you

I'm confused. Any ideas?

Chris Saxon
August 11, 2016 - 3:33 pm UTC

Hmmm, not sure.

Are you sure there's no data type conversion issues?

What's your table definition? And what does the raw XML you get from dbms_xmlgen.getxmltype look like?

Test case

AndyP, August 12, 2016 - 7:17 am UTC

Ok, here's the full test case. As I say, nothing fancy - just a straight copy of your code (at least until I tried xmltable instead)

The xml I think points to the data_default value having a newline within it, judging by the way the tags display

col trimdef for a10
set long 20000

create table test
   (test_id number not null
   ,employee_id number not null
   ,type varchar2(10)
   ,col1 number default -1
   ,col2 number default -1
   ,col3 number default -1
   ,col4 number default -1
   ,primary key(test_id,employee_id));

pro The test table
desc test

insert into test values (100,999,'Type-1',5,-1,-1,4);
insert into test values (101,999,'Type-2',-1,2,-1,-1);
insert into test values (102,999,'Type-3',-1,2,-1,-1);
insert into test values (103,999,'Type-3',default,default,default,default);

pro The test data
select * from test;

pro The pivoted test data
select * from test
unpivot ( val for col in ( col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4' ) );

pro This is what we are working with as xml
select dbms_xmlgen.getxmltype('select column_name, data_default from user_tab_cols where table_name = ''TEST''') from dual;

pro The columns and their defaults if any
select col_name
      ,rtrim(d_def) as d_def
  from xmltable('/ROWSET/ROW'
       passing
   (select dbms_xmlgen.getxmltype('select column_name, data_default from user_tab_cols where table_name = ''TEST''') from dual)
       columns col_name varchar2(30) path 'COLUMN_NAME'
              ,d_def varchar2(30) path 'DATA_DEFAULT');

pro The suggested solution using XMLSEQUENCE and extractValue (direct cut'n'paste)
with rws as (
  select * from test
  unpivot (
    val for col in (
      col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4'
    )
  )
), xml as (
select dbms_xmlgen.getxmltype(
 'select column_name, data_default
  from   user_tab_cols where  table_name = ''TEST''') x
from dual
), col_defs as (
  select extractValue(xt.object_value, '/ROW/COLUMN_NAME') as col_name,
         extractValue(xt.object_value, '/ROW/DATA_DEFAULT') as d_def
  from   xml, table(xmlsequence(extract(xml.x, '/ROWSET/ROW'))) xt
)
  select r.*
  from   rws r
  join   col_defs c
  on     c.col_name = r.col
  and    c.d_def <> r.val;

pro The fixed (hacked) solution
with rws as (
  select * from test
  unpivot (
    val for col in (
      col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4'
    )
  )
), xml as (
select dbms_xmlgen.getxmltype(
 'select column_name, data_default
  from   user_tab_cols where  table_name = ''TEST''') x
from dual
), col_defs as (
  select extractValue(xt.object_value, '/ROW/COLUMN_NAME') as col_name,
         extractValue(xt.object_value, '/ROW/DATA_DEFAULT') as d_def
  from   xml, table(xmlsequence(extract(xml.x, '/ROWSET/ROW'))) xt
)
  select r.*
  from   rws r
  join   col_defs c
  on     c.col_name = r.col
  and    r.val <> rtrim(rtrim(to_char(c.d_def)),chr(10));

pro The XMLTABLE version
with data as
(
select * from test
unpivot ( val for col in ( col1 as 'COL1', col2 as 'COL2', col3 as 'COL3', col4 as 'COL4' ) )
)
,coldefs as
(
select col_name
      ,rtrim(rtrim(to_char(d_def)),chr(10)) as trimdef
  from xmltable('/ROWSET/ROW'
       passing
   (select dbms_xmlgen.getxmltype('select column_name, data_default from user_tab_cols where table_name = ''TEST''') from dual)
       columns col_name varchar2(30) path 'COLUMN_NAME'
              ,d_def clob path 'DATA_DEFAULT')
)
select data.test_id
      ,data.employee_id
      ,data.type
      ,data.col
      ,data.val
      ,coldefs.trimdef
--      ,dump(coldefs.trimdef) dumpdef
  from data
      ,coldefs
 where data.col = coldefs.col_name
   and data.val <> coldefs.trimdef
 order by 1,2
/

drop table test purge;

07:47:46 > @coldefs

Table created.

The test table
 Name                                                                     Null?    Type
 ------------------------------------------------------------------------ -------- -------------------------------------------------
 TEST_ID                                                                  NOT NULL NUMBER
 EMPLOYEE_ID                                                              NOT NULL NUMBER
 TYPE                                                                              VARCHAR2(10)
 COL1                                                                              NUMBER
 COL2                                                                              NUMBER
 COL3                                                                              NUMBER
 COL4                                                                              NUMBER


1 row created.


1 row created.


1 row created.


1 row created.

The test data

   TEST_ID EMPLOYEE_ID TYPE             COL1       COL2       COL3       COL4
---------- ----------- ---------- ---------- ---------- ---------- ----------
       100         999 Type-1              5         -1         -1          4
       101         999 Type-2             -1          2         -1         -1
       102         999 Type-3             -1          2         -1         -1
       103         999 Type-3             -1         -1         -1         -1

The pivoted test data

   TEST_ID EMPLOYEE_ID TYPE       COL         VAL
---------- ----------- ---------- ---- ----------
       100         999 Type-1     COL1          5
       100         999 Type-1     COL2         -1
       100         999 Type-1     COL3         -1
       100         999 Type-1     COL4          4
       101         999 Type-2     COL1         -1
       101         999 Type-2     COL2          2
       101         999 Type-2     COL3         -1
       101         999 Type-2     COL4         -1
       102         999 Type-3     COL1         -1
       102         999 Type-3     COL2          2
       102         999 Type-3     COL3         -1
       102         999 Type-3     COL4         -1
       103         999 Type-3     COL1         -1
       103         999 Type-3     COL2         -1
       103         999 Type-3     COL3         -1
       103         999 Type-3     COL4         -1

16 rows selected.

This is what we are working with as xml

DBMS_XMLGEN.GETXMLTYPE('SELECTCOLUMN_NAME,DATA_DEFAULTFROMUSER_TAB_COLSWHERETABLE_NAME=''TEST''')
------------------------------------------------------------------------------------------------------------------------------------
<ROWSET>
  <ROW>
    <COLUMN_NAME>TEST_ID</COLUMN_NAME>
  </ROW>
  <ROW>
    <COLUMN_NAME>EMPLOYEE_ID</COLUMN_NAME>
  </ROW>
  <ROW>
    <COLUMN_NAME>TYPE</COLUMN_NAME>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL1</COLUMN_NAME>
    <DATA_DEFAULT>-1
   </DATA_DEFAULT>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL2</COLUMN_NAME>
    <DATA_DEFAULT>-1
   </DATA_DEFAULT>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL3</COLUMN_NAME>
    <DATA_DEFAULT>-1
   </DATA_DEFAULT>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL4</COLUMN_NAME>
    <DATA_DEFAULT>-1
   </DATA_DEFAULT>
  </ROW>
</ROWSET>


The columns and their defaults if any

COL_NAME             D_DEF
-------------------- ----------
TEST_ID
EMPLOYEE_ID
TYPE
COL1                 -1
COL2                 -1
COL3                 -1
COL4                 -1

7 rows selected.

The suggested solution using XMLSEQUENCE and extractValue (direct cut'n'paste)
  and    c.d_def <> r.val
         *
ERROR at line 22:
ORA-01722: invalid number


The fixed (hacked) solution

   TEST_ID EMPLOYEE_ID TYPE       COL         VAL
---------- ----------- ---------- ---- ----------
       100         999 Type-1     COL1          5
       102         999 Type-3     COL2          2
       101         999 Type-2     COL2          2
       100         999 Type-1     COL4          4

The XMLTABLE version

   TEST_ID EMPLOYEE_ID TYPE       COL         VAL TRIMDEF
---------- ----------- ---------- ---- ---------- ----------
       100         999 Type-1     COL1          5 -1
       100         999 Type-1     COL4          4 -1
       101         999 Type-2     COL2          2 -1
       102         999 Type-3     COL2          2 -1


Table dropped.


Connor McDonald
August 12, 2016 - 7:56 am UTC

There is a small yet critical difference between the test cases.

I put the column commas after the default.
You put the column commas before the name.

It appears the data_default literally reads everything from the default value to the comma for the next column!

Wow!

If you mix and match in one table it's more obvious:

create table test
   (test_id number not null
   ,employee_id number not null
   ,type varchar2(10)
   ,col1 number default -1
   ,col2 number default -1,
    col3 number default -1,
    col4 number default -1
,primary key(test_id,employee_id));

select dbms_xmlgen.getxmltype('select column_name, data_default from user_tab_cols where table_name = ''TEST''') from dual;

<ROWSET>
  <ROW>
    <COLUMN_NAME>TEST_ID</COLUMN_NAME>
  </ROW>
  <ROW>
    <COLUMN_NAME>EMPLOYEE_ID</COLUMN_NAME>
  </ROW>
  <ROW>
    <COLUMN_NAME>TYPE</COLUMN_NAME>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL1</COLUMN_NAME>
    <DATA_DEFAULT>-1
   </DATA_DEFAULT>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL2</COLUMN_NAME>
    <DATA_DEFAULT>-1</DATA_DEFAULT>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL3</COLUMN_NAME>
    <DATA_DEFAULT>-1</DATA_DEFAULT>
  </ROW>
  <ROW>
    <COLUMN_NAME>COL4</COLUMN_NAME>
    <DATA_DEFAULT>-1
</DATA_DEFAULT>
  </ROW>
</ROWSET>


Notice how COL2 & 3 both have their DATA_DEFAULT tag all on one line. And COL4's closing tag is at the start of the line, but COL1's is indented?

So I think you need to go with your solution that trims the column to ensure this always works!

Wow indeed

AndyP, August 12, 2016 - 12:50 pm UTC

Really good problem diagnosis! I felt sure it had to be an xml-related issue and so didn't focus on the data setup. Serves me right for being so set on my preferred line formatting I suppose. Quite interesting though I hope and thanks for solving it
Chris Saxon
August 12, 2016 - 1:24 pm UTC

No worries, I never would have expected something like this to happen!

Just shows the value of providing a test case. It probably would have taken me days to figure out without one ;)

More to Explore

DBMS_XMLGEN

More on PL/SQL routine DBMS_XMLGEN here