Skip to Main Content
  • Questions
  • SQL*Loader into multiple tables depending on column being NULL

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Neil.

Asked: April 23, 2002 - 11:21 pm UTC

Last updated: December 28, 2004 - 3:51 pm UTC

Version: 8.1.7.3.0

Viewed 1000+ times

You Asked

Hi Tom,

I'm trying to use SQL*Loader to load data into two different tables depending on whether the value in a particular field is NULL or not. My *test* data file (NullTest.dat) looks like this:

"Oak","Tree",20,"metres",1
"Elm","Tree",60,"feet",6
"Ash",,20,"feet",2
"Chestnut","Tree",45,"metres",1

Note that the second field in row 3 is null.

My control file looks like this:

LOAD DATA
INFILE 'NullTest.dat'
REPLACE
INTO TABLE tree_test WHEN tree_type <> BLANKS
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(TREE_NAME CHAR,
TREE_TYPE CHAR,
HEIGHT FLOAT EXTERNAL,
UOM CHAR,
MISC CHAR TERMINATED BY WHITESPACE)
INTO TABLE tree_test_nulls WHEN tree_type = BLANKS
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(TREE_NAME POSITION(1) CHAR,
TREE_TYPE CHAR,
HEIGHT FLOAT EXTERNAL,
UOM CHAR,
MISC CHAR TERMINATED BY WHITESPACE)

When I run SQL*Loader I get 3 records loaded into table tree_test (Oak, Elm and Chestnut) but no records are loaded into table tree_test_null. The log file indicates that record 3 was discarded as it failed all WHEN clauses. The relevant section of the log file follows:

Table TREE_TEST, loaded when TREE_TYPE != BLANKS
Insert option in effect for this table: REPLACE

Column Name Position Len Term Encl Datatype
-------------------- ---------- ----- ---- ---- ---------------------
TREE_NAME FIRST * , O(") CHARACTER
TREE_TYPE NEXT * , O(") CHARACTER
HEIGHT NEXT * , O(") CHARACTER
UOM NEXT * , O(") CHARACTER
MISC NEXT * WHT O(") CHARACTER

Table TREE_TEST_NULLS, loaded when TREE_TYPE = BLANKS
Insert option in effect for this table: REPLACE

Column Name Position Len Term Encl Datatype
-------------------- ---------- ----- ---- ---- ---------------------
TREE_NAME 1 * , O(") CHARACTER
TREE_TYPE NEXT * , O(") CHARACTER
HEIGHT NEXT * , O(") CHARACTER
UOM NEXT * , O(") CHARACTER
MISC NEXT * WHT O(") CHARACTER

Record 3: Discarded - failed all WHEN clauses.

Table TREE_TEST:
3 Rows successfully loaded.
0 Rows not loaded due to data errors.
1 Row not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.

Table TREE_TEST_NULLS:
0 Rows successfully loaded.
0 Rows not loaded due to data errors.
4 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.

I could convert all occurrences of ,, in the input data file to '," ",' to workaround the problem but I'm unable to edit the *actual* input data file as it contains sensitive accounts information.

How can I get SQL*Loader to correctly detect a null in the field?

Thanks in anticipation...

Neil W. Garside

PS. Love the "one-on-one" book! I've only had it 2 months and it's nearly worn out!

and Tom said...

I've fought with this issue before (and lost)....

I couldn't figure out a way to do it - BLANKS is not the same as NULL and NULL cannot be used in a WHEN.

My suggestion has been to run one pass on the input -- creating a discard file that you then sqlldr into the other table.

In 9i, this is totally solved with external tables which will allow you to use ANY sql predicate with a multi-table insert to filter and route the data.

Rating

  (6 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Multiple INTO in SQL Loader

Bharath, June 04, 2002 - 11:50 am UTC

Hi,

Is it necessary to have the clause POSITION(1) from the second INTO TABLE onwards when the control file contains more than one INTO TABLE ststement. In the example given below, If I have POSITION(1) then the data is loaded, if the WHEN condition happens to be valid. But if I remove the POSITION(1), the data is not loaded even if the WHEN condition happens to be valid.

I searched about this in your book, but it was not given. Could you explain the use of POSITION clause in sql loader.

Tom Kyte
June 05, 2002 - 6:07 am UTC

If you did not use the POSITITION(1), sqlldr would pick off where it left off in the record for the second INTO section.

So, think of the position(1) like a request to "reset record back to the first character". Without it -- you would pick up where you left off

Multiple INTO in SQL Loader

Bharath, June 04, 2002 - 11:51 am UTC

Hi,

Is it necessary to have the clause POSITION(1) from the second INTO TABLE onwards when the control file contains more than one INTO TABLE ststement. In the example given below, If I have POSITION(1) then the data is loaded, if the WHEN condition happens to be valid. But if I remove the POSITION(1), the data is not loaded even if the WHEN condition happens to be valid.


LOAD DATA
INFILE 'DEPT_LOADER.txt'
BADFILE 'DEPT_LOADER.bad'
DISCARDFILE 'DEPT_LOADER.dis'
TRUNCATE
INTO TABLE dept_loader
WHEN (deptno='10')
(
deptno INTEGER EXTERNAL TERMINATED BY ",",
dname CHAR TERMINATED BY "," ENCLOSED BY '"',
loc CHAR TERMINATED BY "," ENCLOSED BY '"'
)
INTO TABLE dept_loader
WHEN (deptno='20')
(
deptno POSITION(1) INTEGER EXTERNAL TERMINATED BY ",",
dname CHAR TERMINATED BY "," ENCLOSED BY '"',
loc CHAR TERMINATED BY "," ENCLOSED BY '"'
)


I searched about this in your book, but it was not given. Could you explain the use of POSITION clause in sql loader.


Regards,
Bharath

Sagi, October 15, 2003 - 3:22 am UTC

Hi Tom,

I have a problem in using WHEN. I am sorry if this is not the correct place to post.

I tried on ur site and could get no break through. So posting.

I have data like this:

/resource/images/learn/side_nav_dictionary-current.gif
/resource/images/learn/side_nav_links.gif
/resource/images/home/link_request.gif
/resource/jsp/common/internalServerError.jsp
/resource/jsp/common/internalServerError.jsp
/resource/jsp/common/internalServerError.jsp

I want to upload this into a table. But ignore .gif and .jpg records

I tried to use SUBSTR in the column_name where this data goes. But sqlldr gives error.

My control file looks like this

load data
infile 'c:\data.txt'
BADFILE 'c:\data.bad'
DISCARDFILE 'c:\data.dsc'
APPEND into table ACCESS_INFO
WHEN SUBSTR(URL,-4,4) != '.gif' AND
SUBSTR(URL,-4,4) != '.jpg'
fields terminated by whitespace OPTIONALLY ENCLOSED BY '"'
(id SEQUENCE(1,1),
URL CHAR(3000) )

Your help is very much apprieciated.

Thanx in advance.


Tom Kyte
October 15, 2003 - 7:54 am UTC

what version of the database are you using.

Sagi, October 16, 2003 - 1:56 am UTC

Tom,

Sorry forgot to mention about it.

I have this requirement for both 8.1.7 and 9.2.0

Awaiting for your help.

Thanx in advance.

Regards,


Tom Kyte
October 16, 2003 - 10:27 am UTC

don't think you'll be doing it in sqlldr then -- if you are on unix you can load from a named pipe that you fill with a grep on that file to filter out the rows you don't want loaded.

Sagi, October 16, 2003 - 12:13 pm UTC

Tom,

I have to do this on WIN2K.

But let me understand what you meant. If I was on Unix, then I would do the following:

-- Create a named pipe (Say mypipe)
-- use grep to filter. In my case it would be:
grep -v "*.gif" | mypipe | sqlldr control=myctrl.ctl data=mypipe

Is my understanding right?

Secondly, using CYWIN, can i do from windows machine this?

Thanx in advance.

Regards,

Tom Kyte
October 16, 2003 - 5:17 pm UTC

in unix i would:

$ mknod input.dat p
$ egrep -v '(.gif$|.jpg$)' > input.dat &
$ sqlldr .... data=input.dat

on windoze, which doesn't do something as cool as pipes, you may well be plumb out of luck. No idea if cywin supports TRUE pipes

Rick Jolley, December 28, 2004 - 3:51 pm UTC

Great help for filtering loads into a table using sqlldr and the WHEN clause. After reading this article, I added POSITION(1) to the first record of the second INTO TABLE statement and everything worked as expected.

That really should be added to the Oracle documentation, or put into an article on metalink. I've seen lots of questions on this issue with no answer.