Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question.

Asked: May 13, 2002 - 9:39 am UTC

Last updated: September 06, 2006 - 3:36 pm UTC

Version: 8.1.7.0

Viewed 1000+ times

You Asked

Hello Tom,

We have a java application which reads a flat file and loads the data
into multiple tables. If there are more than one flat file in the directory then multiple thread processes work on those flat files in parallel.

The logic is somewhat like:

1. select temp_seq.nextval into l_value;

2. Insert into main_table(l_value, other columns);

process other details

3. Insert into detail_tables(l_value, other columns);

4. continue the same for other records in the file.

Selecing a sequence's next value each time is time consuming over JDBC. So the logic was

to create a sequence with start value 1 increment by 1000;

1. Fetch the sequence.next val in to l_value

2. for i in 1..1000

insert into main_table (l_value,other's);
process other details;
insert into detail_table (l_value,other's);

l_value:=lvalue+1;
end loop;

The problem here is, if the flat file has less records then all the unused numbers will go waste.

Since we have to process large number of files with varying record count and hold these over a period of 16 months do you think the above logic is feasible?

Thanks











and Tom said...

Just code


insert into main_table( temp_seq.nextval, other-columns );
insert into detail_tables( temp_seq.CURRVAL, other-columns );


you need never fetch the sequence value BACK. Actually, I would look to do this in a stored procedure or even better yet -- sqlldr.

Sequence numbers are never "wasted", there is a huge space for these numbers. Losing 1,000's of them won't affect you in the long run.

Rating

  (10 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

creating sequence/p.k base on selected dept column of a table

Godwin, May 13, 2002 - 1:59 pm UTC

Please help me with this problem given to me by my boss.
''I need to create sequence/primary key which will pick the first 4 characters of a column called dept and count the number of records in that same dept to generate the p.k. the sequence must restart at every new dept. eg.
select dept,grade from staffrecord
insert into mytable
values (dept,grade,seq)
Thus the records in mytable should look like
Zoology Senior Clerk ZOOL001
Zoology Junior Clerk ZOOL002
Mathematics Lecturer MATH001
Mathematics T.A MATH002
............ ..... ......
in that order.
Will plsql rather be okay?
Thanks

Tom Kyte
May 13, 2002 - 2:25 pm UTC

YUCK. What a terrible idea.

You can generate a blazing 1 key at a time!!! And still hit dups (make sure your app is ready to deal with "dup_val_on_index" (you need a unique constraint on seq)

you can:


insert into mytable
select :dept, :grade, upper(substr(:dept,1,4)) || to_char(count(*)+1,'fm0000')
from mytable
where substr(seq,1,4) = upper(substr(:dept,1,4));

But it'll full scan mytable for every key (unless you use a function based index on substr(seq,1,4)

Terrible idea.



A reader, August 01, 2003 - 6:11 am UTC

select dept,grade,upper(substr(dept,1,4)) ||
to_char(rank () over(partition by dept order by grade ) ,'fm0000')
from mytable;



Tom Kyte
August 01, 2003 - 8:00 am UTC

you would want to use row_number() to avoid dups -- and that works on the way "out" nicely

About Sequence...

reader, March 12, 2004 - 10:02 am UTC

Hi Tom,

I think this relevant here. I would like to use a
sequence to in this way. I need to keep the current
value of the sequence in order to populate my
child table. So my question is how can I keep(preserve)
the current value so that I can use it for my child
table?

Tom Kyte
March 12, 2004 - 11:01 am UTC

currval returns your sessions last returned value from nextval

insert into parent .... values ( s.nextval, .... )
insert into child ..... values ( ..., s.CURRVAL, .... )

Auto incremental field in child table

A reader, March 12, 2004 - 12:23 pm UTC

Hi Tom, 

I have a table:

SQL> desc BUDGETITEMAMOUNTS
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 BUDGETITEMID                              NOT NULL NUMBER(14)
 BUDGETITEMAMTSEQID                        NOT NULL NUMBER(1)
 BUDGETITEMAMTAMT                          NOT NULL NUMBER(20,4)

The requirement is to incremental budgetitemamtseqid field automatically for each new record of the same budgetitemid.

For example:
BUDGETITEMID           BUDGETITEMAMTSEQID     
1                      1
1                      2
2                      1
1                      3
2                      2
2                      3

I write a trigger like this one:

CREATE OR REPLACE TRIGGER budgetitemamounts_seq_trg
BEFORE INSERT ON budgetitemamounts
FOR EACH ROW
BEGIN
  IF (:new.BUDGETITEMAMTSEQID) IS NULL THEN
    SELECT nvl(max(BUDGETITEMAMTSEQID),0) + 1
    INTO :new.BUDGETITEMAMTSEQID
    FROM budgetitemamounts 
    WHERE BUDGETITEMID = :NEW.BUDGETITEMID;
  END IF;
END;
/


But I realized that it may have a concurrency problem that multiple users may try to insert the same seqid. Do you have a better idea on how to do it?

Thanks.
 

Tom Kyte
March 12, 2004 - 5:22 pm UTC

you would have to serialize at the budgetitemid.


Why do you need "1,2,3,4", what is wrong with "100,1032213, 2432425353, 5215252153125325" for example? it gives you the order you want.

(my answer to this is always -- rethink the "requirement", it'll affect your performance in a miserable way)


besides your trigger is one of those "mutating" ones if you ever attempt to insert as select

Related with Sequences and Locking

A reader, November 17, 2004 - 9:10 pm UTC

Hi Tom,
I have a requirement where in I need to "lock" a finite set of sequences. I am not satisfied with the requirement itself but before I say so to the developers, I wanted to know if there is a way out.

We have a c++ program that calls a pl/sql stored procedure.
The SP calls using a loop another SP to "reserve" a block of sequences. The 2nd SP looks like below:

CREATE OR REPLACE PROCEDURE LYGETBLOCKSEQUENCESP
(
po_resultcur OUT ETS_PKG.refcur,
pi_seq_name IN VARCHAR2,
pi_reserve_value IN NUMBER,
po_returnstatus OUT NUMBER
)
IS
PRAGMA AUTONOMOUS_TRANSACTION;

v_msg VARCHAR2(1000);
v_procid VARCHAR2(255) := ' [LYGETBLOCKSEQUENCESP]';
v_sqlstatement VARCHAR2(4000);
v_startval NUMBER;
v_endval NUMBER;

BEGIN

-- This inital selection of NEXTVAL is to initialize the sequence within a session.
EXECUTE IMMEDIATE ' SELECT ' || pi_seq_name || '.NEXTVAL ' ||
'FROM DUAL '
INTO v_startval;

EXECUTE IMMEDIATE 'ALTER SEQUENCE ' || pi_seq_name || ' INCREMENT BY ' || TO_CHAR(pi_reserve_value+1);

EXECUTE IMMEDIATE ' SELECT ' || pi_seq_name || '.NEXTVAL ' ||
'FROM DUAL '
INTO v_endval;

EXECUTE IMMEDIATE 'ALTER SEQUENCE ' || pi_seq_name || ' INCREMENT BY 1';


OPEN po_resultcur
FOR 'SELECT :b_startval AS "StartValue", '
||' :b_endval AS "EndValue" '
||'FROM DUAL '
USING v_endval - pi_reserve_value + 1,
v_endval;

po_returnstatus := 0;

EXCEPTION
WHEN OTHERS THEN
IF SQLCODE = -20999 THEN
v_msg := SUBSTR(SQLERRM,12);
ELSE
v_msg := SUBSTR(SQLERRM,1,255) || v_procid;
END IF;

-- Return -1 to the calling program to indicate failure.
po_returnstatus := -1;
RAISE_APPLICATION_ERROR(-20999,v_msg);
END;


Let us say the pi_reserve_value is 5. Based on the v_startval and v_endval the app uses after a certain amount of time, the intermediate values for a key field (it can decide to rollback the whole transaction too). Multiple users can be executing the same application concurrently. The requirement is that for two different sets of values there should be no overlap. For example if user A gets (100,106), (117,123)... then user B should not get (104,110), (111,117) etc.(104,105 and 106 will be used by 2 users wrongly).

Is there any way to overcome the above problem using the Oracle Sequences? Or will this be better off taken care by C++ itself? Gaps in the sequences is not a problem.

Thanks a lot for all the help I get from your site.

Manjunath




Tom Kyte
November 18, 2004 - 10:17 am UTC

not going to happen in a concurrent environment -- DDL like that on a regular basis = worst idea you could ever have.


pick a number, say "100"

create sequence S increment by 100;


now, every select s.nextval reserves 100 values for ANY session -- if they just wanted 4, so what, if they want 200, have them call nextval two times. if they wanted 42, great.


Even if they call this nextval millions of times per second, it would take billions of years to exhaust the values.

(TO_NUMBER(RPAD('9',27,'9'))/100000000)/60/60/24/366
----------------------------------------------------
3.1623E+11


well, more than billions actually.

Thanks!

Manjunath Kottur, November 18, 2004 - 12:24 pm UTC

Hi Tom,

THanks, this gave me the necessary info.

Manjunath


conditional sequence, is it possible ?

A reader, September 05, 2006 - 9:47 am UTC

In the production system, I am (in some cases) using auto generate key take from
FUNCTION (PL/SQL). I know that this is not the best solution because it will not scale well.

The function I discussed here, implemented using AUTONOMOUS TRANSACTION, for expectation reducing overhead (locking/serialization) on the table that maintain the key generation.

What I do here, caused that the SEQUENCE cannot satisfy my purpose
(this conclusion base on my knowledge now).

This is the scenario:

The key that I alway generate have the format
two characters,two digits explaining current year, six digits counter,
so it may look like : AA06001234 (AA - application identifier, 06 - current year and 001234 - current counter for that account)

When the year part change (e.g : 07 - next year), the counter starting again from 000001.
For this purpose, I implemented key generation by using FUNCTION not SEQUENCE.
What I am asking here, is there any solution propose by using SEQUENCE ?
Please give me the light Tom.



Tom Kyte
September 05, 2006 - 5:12 pm UTC

it seems to me you have made the cardinal sin of database stuff here.

You have encoded three fields into one. Stop doing that, nuff said, you really meant to have three fields :)

and that cycle around bit - that cannot be a real life business (we need to make money) requirement?

sequence..

A reader, September 06, 2006 - 9:32 am UTC

So you think, I must breaking down the key into three different piece and
making them to be composite primary key?

To make that changes, I will rebuild application totally. I am not the
system architect that disign the system and I enter the project when the analysis phase was finished and start to coding. And now it is in production !!!

When I ask him (system architect), why we must encode three fields into one, not separating it (as your advice), so we can employ SEQUENCE on the generation of counter part.
He said that when the counter part is separated, it will grow to very huge number that we can not control it. Say the maximum counter for last year is 986432, and when we added from transactions this year, it will exploded (more than just six digits).
Human being could remember number aproximately below 10 digit (as the theory of software engineering he said), and when the number (counter part) becoming huge, human will difficult to remember it in temporary time.

Questions:
o Do you have best practice in genereting key automatically. Just as my requirement that I discussed in the previous question. Please elaborate your solutions.
o When I am stay to using the first approach (encoding it into one field), do you think AUTONOMOUS_TRANSACTION will help in the function generation ?

Thanks Tom.

Tom Kyte
September 06, 2006 - 3:36 pm UTC

so why do you care of the human cannot remember it. trust me, if you have hundreds of thousands of things - no one is going to remember anything like that anyway. That is just "not a smart reason" if you ask me.

if you are thinking your end users will remember aa7777nnnnnn for any period of time, well, good luck to you all.


there is no such thing as a "best practice" here. If you have a surrogate key, just use a sequence. Look at my keys in my url's. I'm not really concerned with you remembering them (and they are not sequence generate - they are a combination of a sequence, random numbers and timestamps.) My requirement was to have them not be guessable.


select :app_code || to_char(sysdate,'yy') || to_char(seq.nextval,'fm00000')
from dual;


create sequence seq maxvalue 999999 cycle;


you have no need to reset it "yearly" based on your "stated requirement", you just want to keep it six digits. So, every million times it'll roll over and start over - just hope you don't do more than a million a year.

Thanks Tom

A reader, September 06, 2006 - 7:09 pm UTC


CURRVAL increments?

Nengbing, January 11, 2007 - 10:51 am UTC

I noticed the following behavior while I was using Oracle sequence in an analytic function such that it appears CURRVAL is returning a number incremented.

select l,
case when l<3 then '<3' else '>=3' end case_char,
case when l<3 then seq1.nextval else seq1.currval end case_seq
from ( select level l from dual connect by level <= 5)

L CAS CASE_SEQ
---------- --- ----------
1 <3 446
2 <3 447
3 >=3 448
4 >=3 449
5 >=3 450


Is there an explanation for this?

Thanks a lot, Tom, for your tremendous contribution to the Oracle community.


More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library