well, I don't get it - that merge would basically update every single row in the table - which would be slightly "expensive"
when the update runs forever - is it doing work or is it perhaps blocked? when it runs forever - is the plan the same as the "normal" plan? What have you done to diagnose this so far?
Assuming a unique constraint on ROW_ID, you can try:
update
( select tbl.flg_col1
from tbl, (select * from err where key_col1='A') err
where tbl.key_col1 = 'A'
and tbl.rowid = err.row_id(+)
and err.row_id is null )
set flg_col1 = 'Y'
/