Skip to Main Content

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, Achesh.

Asked: July 03, 2019 - 10:15 am UTC

Last updated: November 12, 2020 - 3:47 am UTC

Version: 12c

Viewed 50K+ times! This question is

You Asked

I have a requirement to insert records (or any other DML) into a table - but in case of encountering error records, the inserts must continue and the errors must be logged.

Logically I thought of using the LOG ERRORS INTO a table.

I have a few questions pertaining to this -

Can a collection or a temporary table be used instead of a error log table (which is created using DBMS_ERRLOG.create_error_log or a manual table)

Is LOG ERRORS INTO the only way to "continue insert in case of erred records and return log info for erred records" scenario?

3.Will LOG ERRORS INTO still work if the RETURNING INTO clause is used in the DML statement?

Thank you!

and Chris said...

You can create the error log as a global temporary table. But you must do this with the "on commit preserve rows" option:

create table t (
  c1 int primary key, c2 varchar(5)
);

create global temporary table err$_t ( 
  ora_err_number$ number, 
  ora_err_mesg$ varchar2(2000), 
  ora_err_rowid$ urowid (4000), 
  ora_err_optyp$ varchar2(2), 
  ora_err_tag$ varchar2(2000), 
  c1 varchar2(4000), 
  c2 varchar2(4000)
);

insert into t 
  with rws as (
    select level x from dual
    connect by level <= 10
  )
    select x, lpad ( 'x', x, 'x' ) 
    from   rws
  log errors into err$_t
  reject limit unlimited;

select c1, c2 from err$_t;

no rows selected
rollback;

truncate table err$_t;
drop table err$_t 
  cascade constraints purge;
  
create global temporary table err$_t ( 
  ora_err_number$ number, 
  ora_err_mesg$ varchar2(2000), 
  ora_err_rowid$ urowid (4000), 
  ora_err_optyp$ varchar2(2), 
  ora_err_tag$ varchar2(2000), 
  c1 varchar2(4000), 
  c2 varchar2(4000)
) on commit preserve rows;

insert into t 
  with rws as (
    select level x from dual
    connect by level <= 10
  )
    select x, lpad ( 'x', x, 'x' ) 
    from   rws
  log errors into err$_t
  reject limit unlimited;

select * from t;

C1   C2      
    1 x        
    2 xx       
    3 xxx      
    4 xxxx     
    5 xxxxx  

select c1, c2 from err$_t;

C1   C2           
6     xxxxxx        
7     xxxxxxx       
8     xxxxxxxx      
9     xxxxxxxxx     
10    xxxxxxxxxx    

rollback;


When using bulk processing in PL/SQL, you can also the SAVE EXCEPTIONS clause of FORALL to collect an array of the failing rows.

declare
  cursor cur is 
    with rws as (
      select level x from dual
      connect by level <= 10
    )
      select x c1, lpad ( 'x', x, 'x' ) c2
      from   rws;
  
  type cur_rec is table of cur%rowtype
    index by pls_integer;
    
  recs cur_rec;
begin
  open cur;
  fetch cur bulk collect into recs;
  
  forall i in 1 .. recs.count 
    save exceptions
    insert into t 
    values recs( i );
    
  close cur;
end;
/

ORA-24381: error(s) in array DML

select * from t;

C1   C2      
    1 x        
    2 xx       
    3 xxx      
    4 xxxx     
    5 xxxxx    


To see what the errors are, you need to handle ORA-24381 in an exception block. Read more about this at:

https://livesql.oracle.com/apex/livesql/file/content_CMSE3VBYFCO9IX3PGE4S5C1Q8.html
https://blogs.oracle.com/oraclemagazine/bulk-processing-with-bulk-collect-and-forall

There is another method to continue past errors. The ignore_row_on_dupkey_index hint. This silently bypasses any rows which have a primary or unique key clash:

insert into t values ( 1, 'new' );

ORA-00001: unique constraint (CHRIS.SYS_C008328) violated

insert /*+ ignore_row_on_dupkey_index ( t ( c1 ) ) */into t 
values ( 1, 'new' );

0 rows inserted.


But unlike LOG ERRORS/SAVE EXCEPTIONS you have no record of the ignored rows. You need to do comparisons afterwards to see what these are.

Will LOG ERRORS INTO still work if the RETURNING INTO clause is used in the DML statement?

Yes

declare
  val int;
begin
  insert into t values ( 99, 'fine' )
  returning c1 into val
  log errors into err$_t;
  
  dbms_output.put_line ( 'Added ' || val );
end;
/

Added 99

Rating

  (9 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Thank you! This helps, but..

Achesh Shetty, July 04, 2019 - 1:36 pm UTC

Thank you so much for the response, Chris! You are a savior.

The only question I have now is, should it be okay if there are a set of Procedures in a Package, can I create GTTs for each Procedure in the package? Will the GTT be only procedure specific, or should an explicit drop be performed in the procedure? Will this be too much of an overload on the Package?

Please note : I would have created a single Error Logging GTT in the package itself - but the invoker is interested in triggering certain procedures at certain times only and not the package as a whole.
Please find the code skeleton below :

Package 

     Procedure1
         Create Procedure1 specific GTT
         DML Statements - Log error into the GTT
         return a cursor with the GTT records (if any)
     End Procedure1

     Procedure2
         Create Procedure1 specific GTT
         DML
         return a cursor with the GTT records (if any)
     End Procedure2

     Procedure3
          Create Procedure3 specific GTT
          DML
          return a cursor with the GTT records (if any)
     End Procedure1


End Package



Thank you again!!
Chris Saxon
July 04, 2019 - 3:17 pm UTC

You should have an error logging table for each TABLE you do DML on. Otherwise you can't store the column details for the failing rows.

If you have several procedures all changing the same table, you could create an error logging GTT for each. But I don't see the point.

Also... is there any specific reason you want to use a GTT? The rows in these are session specific. Once you disconnect they're gone!

Remember you can add a tag to rows in the error log. So you can capture when/where the statement raised the problems:

log errors into err$_t (
  'proc 1 @ ' || to_char ( sysdate, 'yyyy-mm-dd hh24:mi:ss' ) 
)

For The Generating Error one Log Table In Entire Database

Avinash Narayan Mishra, November 12, 2019 - 2:13 am UTC

Dear Sir,
I have seen everything regarding this topic from top to bottom but i have some questions.
1.Is this possible to create only one table for capturing error
log for the Database.

It meant ,Suppose we have 5 tables in the database but i need to capture the error log for these 5 tables in one single error log table.

I need to implement for all the tables.

Sir Please help me out from this situation.




Chris Saxon
November 12, 2019 - 11:36 am UTC

You can create an error log for each table:

create table t1 (
  t1c1 int, t1c2 int
);
  
create table t2 (
  t2c1 date  
);

begin 
  dbms_errlog.create_error_log (dml_table_name => 't1');
  dbms_errlog.create_error_log (dml_table_name => 't2');
end;
/

desc err$_t1

Name              Null?   Type             
ORA_ERR_NUMBER$             NUMBER            
ORA_ERR_MESG$               VARCHAR2(2000)    
ORA_ERR_ROWID$              UROWID            
ORA_ERR_OPTYP$              VARCHAR2(2)       
ORA_ERR_TAG$                VARCHAR2(2000)    
T1C1                        VARCHAR2(4000)    
T1C2                        VARCHAR2(4000) 

desc err$_t2

Name              Null?   Type             
ORA_ERR_NUMBER$             NUMBER            
ORA_ERR_MESG$               VARCHAR2(2000)    
ORA_ERR_ROWID$              UROWID            
ORA_ERR_OPTYP$              VARCHAR2(2)       
ORA_ERR_TAG$                VARCHAR2(2000)    
T2C1                        VARCHAR2(4000)  

For The Generating Error one Log Table In Entire Database

Avinash Narayan Mishra, November 25, 2019 - 11:21 pm UTC

Sir,
Thanks for your feedback but i already told that i want to
create a table for error tracking for entire database.
Suppose i have list table t1,t2,..n.I don't wanna generate
table wise error log i.e any error will happen while doing dml operations
for all the tables then log data should be inserted in
in one target error log table.

Is this possible ???????
Waiting for your meaningful response...........
Chris Saxon
November 26, 2019 - 11:18 am UTC

The error logging tables have a column for each column in the table you're capturing errors for.

You can use an error logging table which has more columns than the target table (but not vice-versa).

So you could create an error log table with 1,000 columns (the limit in Oracle Database). And log everything to that.

I wouldn't recommend this though - it's better to have 1:1 mapping between target and error tables.

A reader, November 26, 2019 - 5:07 pm UTC

You can use pragma autonomous transaction in the PLSQL and insert and commit the errors into log table and rollback rest of the data
Chris Saxon
November 26, 2019 - 5:56 pm UTC

Indeed you can! That's more of a roll-your-own solution than something out of the box though.

Jess, November 03, 2020 - 11:13 pm UTC

Hi Chris,

Have you seen a case where error logging doesn't work for a seemingly non-obvious reason?

We have 2 tables, parent/child. No deferred constraints, no clobs, pretty basic. Delete from parent, log errors, reject unlimited.

The error logging is completely ignored, like it's not even there, and the statement fails with "child records found".

Take the DDL from the tables and create identical tables using the same DDL (bar names). Same schema, same constraints, keys, indexes, everything.

Run the same delete on the parent table against the new pair with error logging, and it works like a charm. We are completely out of ideas as to what to attribute this discrepancy to...

Tried it both with create_error_log and with a hand-rolled table. Nothing works with existing tables, everything works with the new ones created from the same DDL.

Any ideas would be much appreciated...

Connor McDonald
November 04, 2020 - 2:43 am UTC

Could perform a 10046 trace at level 12 on the working and non working examples and send them along to asktom_us@oracle.com.


Jess, November 04, 2020 - 4:55 pm UTC

Thanks ever so much, Connor, I emailed you the trace file.
Connor McDonald
November 05, 2020 - 2:29 am UTC

I found this in the trace file

"INSERT INTO ORBIT_LOG VALUES('D', 'H_PA_PROFILE',:B3 ,:B2 , :B1 )"

Is it possible you have a statement level trigger on the table?

Its not documented, but if memory serves, they can cause some dramas with dml error logging

Jess, November 05, 2020 - 7:09 am UTC

Hi Connor,
Thank you for having a look, that's really something!

The table indeed has a trigger on it, but it's a row-level one:

CREATE OR REPLACE TRIGGER schema.table
AFTER DELETE ON table
FOR EACH ROW

DECLARE
   <vars>
BEGIN
 INSERT INTO log_table
 VALUES('D', 'table', <other vars>);

 INSERT into history_table <individual column values>
END;


But having read the documentation (and having used DML logging a fair bit before), I haven't seen anything mentioned about triggers being on the list of things that are not supported... When I said we're creating the table using the same DDL, we were, but we weren't adding the triggers because there was no reason to. Now that I tried adding it to the new "comparison" tables, I'm seeing the same failures... It doesn't even matter what the trigger does; I replaced the body with "select 1 from dual", and it still causes error logging to go up in flames...

So what can we do? Disabling the trigger works, but we cannot do that for real because the code we want to log the errors runs during regular hours...

If there are undocumented dramas with triggers, as you say, is there also something undocumented you could advise to get around these row level bits of joy (short of disabling)?

Thank you as ever...

So
Connor McDonald
November 05, 2020 - 7:47 am UTC

It looks like the very existence of the trigger is enough to stop error logging for a foreign key. For example, if I code up a trigger with

WHEN (1=0)

then I'll still get the -2292, so there isn't really anything we can put *into* the trigger to work around this, because we don't even get to the trigger code. Check constraints etc seem to be fine based on my testing, so currently the only workaround I could think of is explicitly tweaking the SQL ie

delete from parent
where  not exists ( select from child )
.,...


I'll ask around internally and see what else I can find out

Jess, November 05, 2020 - 12:02 pm UTC

Thanks Connor!

I can't believe this is not documented.... :(
We cannot change the SQL because we need to delete a lot of data from a lot of tables (and we expect 100% to succeed). This way if individual records fail, we know what they are easily and can investigate. Otherwise we would need to requery everything, which gets very messy...

Any workaround would be amazing, else we're going to have to rework with bulk collect save exceptions....

Connor McDonald
November 06, 2020 - 3:52 am UTC

OK, here's a workaround a little out of left field :-)

First I'll recreate the problem with some simple tables

par = parent
chd = child
par_log = logging table


SQL>
SQL> create table par ( p int primary key );

Table created.

SQL> create table chd ( c int primary key, p int references par ( p ));

Table created.

SQL>
SQL> create table par_log
  2   (    mod_type varchar2(1),
  3        mod_seqno number,
  4        mod_edit_date date,
  5        transaction_id varchar2(50),
  6        mod_action varchar2(1),
  7        parent number);

Table created.

SQL>
SQL>
SQL> insert into par values (1);

1 row created.

SQL> insert into par values (2);

1 row created.

SQL> insert into par values (3);

1 row created.

SQL>
SQL> insert into chd
  2  select rownum,1 from dual
  3  connect by level <= 100;

100 rows created.

SQL>
SQL> insert into chd
  2  select rownum+100,2 from dual
  3  connect by level <= 100;

100 rows created.

SQL>
SQL> exec dbms_errlog.create_error_log('PAR')

PL/SQL procedure successfully completed.

SQL>
SQL> create or replace
  2  trigger PAR_TRG
  3  before delete on par
  4  for each row
  5  begin
  6      insert into par_log
  7      values ('N',1,sysdate,dbms_transaction.local_transaction_id,'D',:old.p);
  8  end;
  9  /

Trigger created.

SQL> delete from par
  2  where p = 1
  3  log errors reject limit unlimited;
delete from par
*
ERROR at line 1:
ORA-02292: integrity constraint (MCDONAC.SYS_C0029183) violated - child record found

SQL>



Now I'll drop the trigger and recreate it to do the same work but as a *compound* trigger

SQL> drop trigger par_trg;

Trigger dropped.

SQL> create or replace trigger par_trg
  2    for insert or update or delete on par
  3    compound trigger
  4
  5    before each row is
  6    begin
  7      insert into par_log
  8      values ('N',1,sysdate,dbms_transaction.LOCAL_TRANSACTION_ID,'D',:old.p);
  9    end before each row;
 10
 11  end;
 12  /

Trigger created.

SQL>
SQL> delete from par
  2  where p = 1
  3  log errors reject limit unlimited;

0 rows deleted.



Clearly this shouldn't be a differentiating factor, so I'll be logging a bug but this should get you moving.

Also...to get this work done, here's an easy justification for you :-) Using a compound trigger you can improve the performance.

Your current trigger probably looks like

- get sysdate
- get a sequence number
- insert log header
- insert log detail

on a row by row basis.....ugh :-)

A compound trigger lets you do this:

create or replace trigger par_trg
  for insert or update or delete on par    
  compound trigger

  type list_of_header is table of header_table%rowtype
    index by pls_integer;

  type list_of_detail is table of detail_table%rowtype
    index by pls_integer;
    
  l_header list_of_header;
  l_detail list_of_detail;

  before statement is
  begin
    l_header.delete;
    l_detail.delete;
  end before statement;

  before each row is
  begin
    l_header(l_header.count+1).co11 := ...;
    l_header(l_header.count).co12 := ...;
    l_header(l_header.count).co13 := ...;

    l_detail(l_detail.count+1).col1 := ....;
    l_detail(l_detail.count).col2 := ....;
    l_detail(l_detail.count).col3 := ....;
  end before each row;

  after statement is
  begin
    forall i in 1 .. l_header.count
      insert into header_table values l_header(i)

    forall i in 1 .. l_detail.count
      insert into detail_table values l_detail(i)
      
  end after statement;

end;
/


and that code obviously does not need to be hand written - you can simply query the dictionary to auto-generate it. See examples in my git repos for auditing for this

https://github.com/connormcd/audit_utility

Jess, November 06, 2020 - 11:35 am UTC

Hi Connor,
This is sheer ~trickery~ brilliance!

I knew of compound triggers as a thing, but never seen on in the wild. This definitely lets us move forward with the original desgin.

Thank you again for persisting with it (and for logging a bug!), we are in your debt.
Connor McDonald
November 12, 2020 - 3:47 am UTC

Just to add - there is a debate going on inside the development group about whether we should be allowing this.


More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library