Skip to Main Content

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, Stanislav.

Asked: June 02, 2017 - 12:11 pm UTC

Last updated: June 07, 2017 - 4:53 am UTC

Version: server Oracle Database 12c Release 1 (12.1.0.2.0) Enterprise Edition

Viewed 1000+ times

You Asked

Hello,

We have problem with sqlldr, we have server with 32 GB RAM and 24 GB swap.
While loading big table ( 40 mil records with blobs ), free RAM decreases continuostly, and next swap decreases continuostly too.

After some while loading finsihed with errors :

SQL*Loader-556: unable to allocate read buffer
SQL*Loader-509: System error: Not enough space
SQL*Loader-502: unable to open data file '/LOAD_FS/UNLOAD10099/VS/vvs_msg.8_284212727.1.blob' for field CONTENT table VVS_MSG

Why sqlldr consumes so much RAM ( all RAM) if it is simple loading withot sort or another operations.

Sqlldr copmmands looks like :
sqlldr / LOG=/INST/R10100/rvsd1/VS/UTIL/LOG/vvs_msg.log.8 CONTROL=/LOAD_FS/UNLOAD10099/VS/CTL/vvs_msg.ctl.8 ROWS=5000 >/INST/R10100/rvsd1/VS/UTIL/LOG/vvs_msg.out.8 2>&1

We load data with 8 simultaneous sqlldr processes.

We found, when execute one only process
sqlldr / LOG=/INST/R10100/rvsd1/VS/UTIL/LOG/vvs_msg.log.1 CONTROL=/LOAD_FS/UNLOAD10099/VS/CTL/vvs_msg.ctl.1 ROWS=100 BINDSIZE=500000 >/INST/R10100/rvsd1/VS/UTIL/LOG/vvs_msg.out.1 2>&1

that reaches about 3.5 GB RAM and falls down with core dump and ORA errors
32243932 Segmentation fault(coredump)

SQL*Loader-502: unable to open data file '/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_284218096.1.blob' for field CONTENT table VVS_MSG
SQL*Loader-556: unable to allocate read buffer
SQL*Loader-509: System error: Not enough space

Amount of RAM varies but evry is about 3.5 GB
3391568
3402968
3408636
etc

So it looks like sqlldr reach some limit of RAM and fall down.
User, which executes sqlldr has no limit in OS.

But why it does not release RAM ?
Why RAM still rises ?

Table is truncated every before executing sqlldr.

Command line is :
sqlldr / LOG=/INST/R10100/rvsd1/VS/UTIL/LOG/vvs_msg.log.1 CONTROL=/LOAD_FS/UNLOAD10099/VS/CTL/vvs_msg.ctl.1 ROWS=100 BINDSIZE=500000 >/INST/R10100/rvsd1/VS/UTIL/LOG/vvs_msg.out.1 2>&1

Control file :

LOAD DATA
INFILE '/LOAD_FS/UNLOAD10099/VS/vvs_msg.unl.1' "str '|\n'"
BADFILE '/LOAD_FS/UNLOAD10099/VS/BAD/vvs_msg.bad.1'
DISCARDFILE '/LOAD_FS/UNLOAD10099/VS/DSC/vvs_msg.dsc.1'
APPEND
INTO TABLE vvs_msg
FIELDS TERMINATED BY '|'
TRAILING NULLCOLS
(
id,
content_filename FILLER CHAR(100),
content LOBFILE(content_filename) TERMINATED BY EOF,
code_page
)

First 10 records from unload :

23453280|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_23453280.1.blob|U|
28770528|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28770528.1.blob|U|
28847952|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28847952.1.blob|U|
28945056|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28945056.1.blob|U|
28952016|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28952016.1.blob|U|
39405984|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_39405984.1.blob||
44795424|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_44795424.1.blob|U|
47572848|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_47572848.1.blob|U|
50474736|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_50474736.1.blob||
50504208|/LOAD_FS/UNLOAD10099/VS/vvs_msg.1_50504208.1.blob||


List of first 10 blob files :
-rw-r--r-- 1 zaved uziv_dis 187 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_23453280.1.blob
-rw-r--r-- 1 zaved uziv_dis 358 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28770528.1.blob
-rw-r--r-- 1 zaved uziv_dis 187 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28847952.1.blob
-rw-r--r-- 1 zaved uziv_dis 427 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28945056.1.blob
-rw-r--r-- 1 zaved uziv_dis 326 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_28952016.1.blob
-rw-r--r-- 1 zaved uziv_dis 567 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_39405984.1.blob
-rw-r--r-- 1 zaved uziv_dis 187 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_44795424.1.blob
-rw-r--r-- 1 zaved uziv_dis 127 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_47572848.1.blob
-rw-r--r-- 1 zaved uziv_dis 1760 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_50474736.1.blob
-rw-r--r-- 1 zaved uziv_dis 56002 29 máj 17:31 /LOAD_FS/UNLOAD10099/VS/vvs_msg.1_50504208.1.blob

Usually blob files have up to 500B, but some of them have 200KB .

One unload has 6127708 records.

Very thank you for your answer.

and Connor said...

You might have hit a bug there.

I've replicated your issue, I was just loading the same few blobs over and over with a data file like this:

1|/tmp/blob1.jpg|U|
2|/tmp/blob2.jpg|U|
3|/tmp/blob3.jpg|U|
4|/tmp/blob4.jpg|U|
5|/tmp/blob0.jpg|U|
6|/tmp/blob1.jpg|U|
7|/tmp/blob2.jpg|U|
8|/tmp/blob3.jpg|U|
9|/tmp/blob4.jpg|U|
10|/tmp/blob0.jpg||
...


with a table as

create table vvs_msg
(  id int,
   content blob,
   code_page varchar2(5)
);


LOAD DATA
INFILE '/tmp/load.dat' 
BADFILE '/tmp/load.bad'
DISCARDFILE '/tmp/load.dsc'
TRUNCATE
INTO TABLE vvs_msg
FIELDS TERMINATED BY '|'
TRAILING NULLCOLS
(
id,
content_filename FILLER CHAR(100),
content LOBFILE(content_filename) TERMINATED BY EOF,
code_page
)


and when I look at the sqlldr process, its memory size continues to grow and grow

[oracle@vbgeneric ~]$ cat /proc/5310/status | grep ^Vm
VmPeak:   191088 kB
VmSize:   191088 kB
VmLck:        0 kB
VmPin:        0 kB
VmHWM:    80736 kB
VmRSS:    80736 kB
VmData:    77840 kB
VmStk:      472 kB
VmExe:     1448 kB
VmLib:    70756 kB
VmPTE:      332 kB
VmPMD:       12 kB
VmSwap:        0 kB

[oracle@vbgeneric ~]$ cat /proc/5310/status | grep ^Vm
VmPeak:   191352 kB
VmSize:   191352 kB
VmLck:        0 kB
VmPin:        0 kB
VmHWM:    81000 kB
VmRSS:    81000 kB
VmData:    78104 kB
VmStk:      472 kB
VmExe:     1448 kB
VmLib:    70756 kB
VmPTE:      332 kB
VmPMD:       12 kB
VmSwap:        0 kB

[oracle@vbgeneric ~]$ cat /proc/5310/status | grep ^Vm
VmPeak:   191484 kB
VmSize:   191484 kB
VmLck:        0 kB
VmPin:        0 kB
VmHWM:    81000 kB
VmRSS:    81000 kB
VmData:    78236 kB
VmStk:      472 kB
VmExe:     1448 kB
VmLib:    70756 kB
VmPTE:      332 kB
VmPMD:       12 kB
VmSwap:        0 kB

[oracle@vbgeneric ~]$ cat /proc/5310/status | grep ^Vm
VmPeak:   201384 kB
VmSize:   201384 kB
VmLck:        0 kB
VmPin:        0 kB
VmHWM:    91032 kB
VmRSS:    91032 kB
VmData:    88136 kB
VmStk:      472 kB
VmExe:     1448 kB
VmLib:    70756 kB
VmPTE:      352 kB
VmPMD:       12 kB
VmSwap:        0 kB

[oracle@vbgeneric ~]$ cat /proc/5310/status | grep ^Vm
VmPeak:   202572 kB
VmSize:   202572 kB
VmLck:        0 kB
VmPin:        0 kB
VmHWM:    92088 kB
VmRSS:    92088 kB
VmData:    89324 kB
VmStk:      472 kB
VmExe:     1448 kB
VmLib:    70756 kB
VmPTE:      352 kB
VmPMD:       12 kB
VmSwap:        0 kB


I tried some variations (direct, different read/bind sizes etc) and none seemed to help significantly.

So get in touch with Support - you can link to the this question as supporting evidence. As a workaround, look at processing the files in smaller batches, since it seems proportional to the number of the blobs loaded.

Rating

  (1 rating)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Stanislav Hricko, June 07, 2017 - 6:28 am UTC


More to Explore

CLOB/BLOB/etc

Complete documentation on Securefiles and Large Objects here