Skip to Main Content
  • Questions
  • PGA memory versus TEMP space versus LARGE_POOL

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, Stewart.

Asked: May 11, 2009 - 3:22 pm UTC

Last updated: May 25, 2021 - 2:53 am UTC

Version: 10.2.0.4

Viewed 10K+ times! This question is

You Asked

This is not a specific test-case question... but really more of a general database architecture question. I'm trying to get a handle on what kinds of operations the database does in TEMP space versus what is does in the PGA memory. A consistent adjective I hear for the kinds of operations that occur in both areas is "sorting"... so I'm trying to understand what differentiates these operations. I know that GROUP BY operations, as well as hash joins and analytic functions occur in the PGA. What does that leave for TEMP space?

As a companion to this question... assuming a query is parallelized... what is it that is actually going on in the LARGE_POOL. I realize it's the parallel execution messages... but I'm not certain what that really means.

Thanks, as usual, for taking my question.

and Tom said...

In older releases - we used to use sort_area_size/hash_area_size parameters to control how much PGA memory could/would be used before swapping data out to disk (temp). I'm not going to talk about that - we'll assume automatic pga memory management - the story is the same, but sort/hash_area_sizes were one size fits all - everyone uses the same size, automatic pga memory management is more dynamic.

Ok, so you run some sql, this sql is going to do some operation like

o hashing
o sorting (order by, analytics, sort distinct, group by sort, etc...)
o etc - anything that needs "memory"

Oracle will allocate a workarea (a sort area, a hash area, whatever). The size of this workarea will be determined by Oracle based on the current workload on the system.

If this workarea (in your pga) fills up - and we are not done with the operation (the sort, the hash, whatever) we will swap the contents of that out to disk - a write to temp. Later, we'll read it back in. Temp works like "virtual memory" in a sense - we page your pga out to it and page it back in (it is not true virtual memory, that is just an analogy)

All the operations you mentioned above - group by, hash joins, analytic functions occur in the pga - but could definitely be swapped out to temp.


As for the large pool, we use that in parallel query to send messages back and forth, the parallel execute servers are in different processes - they cannot talk directly to eachother since they each have their own address space. Hence, one process will put some data into the large pool (a piece of SHARED memory that everyone can access) and another will read it from there. It is just a shared memory structure that everyone - every oracle process - can read/write.

Rating

  (15 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Fantastic

Vic010, May 13, 2009 - 3:55 am UTC

This is the best explanation of this I have heard.

Awesome Tom.

PGA sizing

ldsandon, May 13, 2009 - 10:23 am UTC

"The size of this workarea will be determined by Oracle based on the current workload on the system."
How Oracle does this, exactly? In my experience it likes to size it on the smaller side - often I get queries swapping out to TEMP because they get many records to sort (especially when using Advanced Queues, which queries I can't control).
Tom Kyte
May 13, 2009 - 2:24 pm UTC

If you are interested in some of the details and how to measure this - I spent a lot of pages in Expert Oracle Database Architecture describing it.

Basically, an individual session will be allowed to use some percent of the pga aggregate target - say 5%.

They will not use all 5% for the FIRST statement they do (for you typically have many statements opened concurrently). So, you get a percent of this percent.

... often I get queries swapping out to TEMP because they get many
records to sort ...

that in no way means that is "likes to size it on the smaller side". False cause and effect here.

Look you have many users.
You have a finite amount of memory you have told us to use.
We need to allocate memory in such a way that the many users can simultaneously sort more than one query each.

We need to allocate memory in such a way that the many users can simultaneously sort more than one query each.


If you are writing to temp - it would be because you either

a) have insufficient memory - you have allocated the SGA and told us to use the rest of the memory for the PGA - you don't have enough to hold everything in memory

b) have underallocated the PGA target - you have allocated the SGA and then held back some RAM and told us to use some small percentage of the remaining free memory.


In either case, you can use the pga advisor (statspack or AWR reports) to see what impact various sizes of the PGA target would have on you based on your actual workload.

Agree with Vic010 from UK!

Robert, May 13, 2009 - 10:31 am UTC


Thanks Tom... but a follow-up

Stewart Bryson, May 13, 2009 - 10:37 am UTC

This was a great answer Tom... but I have a follow-up.

So, the database will decide to use TEMP space for situations where the PGA memory allocated (either from pga_aggregate_target or *_area_size depending on workarea_size_policy) is insufficient, correct? So then, what am I looking at when I look at:

SQL> SELECT CASE WHEN low_optimal_size < 1024*1024
  2         THEN to_char(low_optimal_size/1024,'999999') ||
  3         'kb <= PGA < ' ||
  4         (high_optimal_size+1)/1024|| 'kb'
  5         ELSE to_char(low_optimal_size/1024/1024,'999999') ||
  6         'mb <= PGA < ' ||
  7         (high_optimal_size+1)/1024/1024|| 'mb'
  8         END pga_size,
  9         optimal_executions,
 10         onepass_executions,
 11         multipasses_executions
 12    FROM v$sql_workarea_histogram
 13   WHERE total_executions <> 0
 14   ORDER BY low_optimal_size
 15  /

PGA_SIZE                     | OPTIMAL_EXECUTIONS | ONEPASS_EXECUTIONS | MULTIPASSES_EXECUTIONS
---------------------------- | ------------------ | ------------------ | ----------------------
      2kb <= PGA < 4kb       |             466263 |                  0 |                      0
      4kb <= PGA < 8kb       |                 24 |                  0 |                      0
      8kb <= PGA < 16kb      |                  0 |                 14 |                     14
     16kb <= PGA < 32kb      |                 28 |                  0 |                      0
     32kb <= PGA < 64kb      |                 38 |                  4 |                      0
     64kb <= PGA < 128kb     |               9260 |                  0 |                      0
    128kb <= PGA < 256kb     |              20424 |                  0 |                      0
    256kb <= PGA < 512kb     |              18240 |                  0 |                      0
    512kb <= PGA < 1024kb    |              21731 |                  0 |                      2
      1mb <= PGA < 2mb       |               8547 |                  0 |                      8
      2mb <= PGA < 4mb       |               1425 |                  2 |                      0
      4mb <= PGA < 8mb       |               5608 |                170 |                      0
      8mb <= PGA < 16mb      |                923 |                  8 |                      0
     16mb <= PGA < 32mb      |                707 |                 49 |                      8
     32mb <= PGA < 64mb      |                327 |                 22 |                      0
     64mb <= PGA < 128mb     |                483 |                 66 |                     16
    128mb <= PGA < 256mb     |                120 |                124 |                     10
    256mb <= PGA < 512mb     |                 12 |                 91 |                      6
    512mb <= PGA < 1024mb    |                 64 |                 66 |                      0
   1024mb <= PGA < 2048mb    |                  0 |                135 |                      8
   2048mb <= PGA < 4096mb    |                  0 |                  2 |                      0
   4096mb <= PGA < 8192mb    |                109 |                  1 |                      0
   8192mb <= PGA < 16384mb   |                 30 |                  0 |                      0
  16384mb <= PGA < 32768mb   |                  0 |                  3 |                      0

24 rows selected.

Elapsed: 00:00:00.01
SQL> 

Are the ONEPASS_EXECUTIONS and the MULTIPASSES_EXECUTIONS columns indicative of a movement to TEMP space... or something else?

I realize I sort of stuck two questions together... but a follow-up on the large_pool question as well. These shared memory structures... are they cached blocks, are they table-like structures, or something else? I'm curious about the size of these structures, in comparison to the size of the parallel process being executed. And are these structures comparable to what is being created in PGA? For instance, when an individual parallel thread has completed a piece of work ( a sort, or a hash, etc.), is that piece then transferred to the large_pool to be available to other parallel processes? Is that the right way to think about it?

Tom Kyte
May 13, 2009 - 2:54 pm UTC

... Are the ONEPASS_EXECUTIONS and the MULTIPASSES_EXECUTIONS columns indicative of
a movement to TEMP space... ..

yes, they are.


If you want the in's and out's of what the bits and bytes of memory are used for in my words, I'll have to recommend Expert Oracle Database Architecture - I spent a full chapter on the memory structures - many many pages.

... These shared memory structures... are they cached
blocks, are they table-like structures, or something else? ...

something else - they are responses, they are data structures full of information.


ops$tkyte%ORA10GR2> show parameter mess%size

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
parallel_execution_message_size      integer     2148


if a message exceeds that, it is broken in to smaller messages.

... when an individual parallel thread has completed a piece of
work ( a sort, or a hash, etc.), is that piece then transferred to the
large_pool to be available to other parallel processes? Is that the right way
to think about it? ...

yes.

http://docs.oracle.com/docs/cd/B19306_01/server.102/b14223/usingpe.htm#sthref2033

Perfect

Stewart Bryson, May 13, 2009 - 6:05 pm UTC

Thanks Tom.

Suraj Sharma, May 13, 2009 - 10:22 pm UTC

Hi Tom,

"If this workarea (in your pga) fills up - and we are not done with the operation (the sort, the hash, whatever) we will swap the contents of that out to disk - a write to temp. Later, we'll read it back in. Temp works like "virtual memory" in a sense - we page your pga out to it and page it back in (it is not true virtual memory, that is just an analogy)"

I saw many times that the actual PGA used more than the parameter pgs_aggregate_target (Even I put this question in one of your forum and I got its reply as "Yes" http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:152388100346367265 )

So if the PGA memory can grow more than pga_aggregate_target and consider that we have enough physical memory on the server, in this circumstances, when all the contents will be written to disk (temp file)

Suraj Sharma,
Tom Kyte
May 14, 2009 - 10:27 am UTC

forget the pga aggregate target for the question:

... and consider that
we have enough physical memory on the server, in this circumstances, when all
the contents will be written to disk (temp file) ...

that we can exceed the target is not relevant to answering that question.


You could write to temp on a machine with 1,000 gb of free memory and you are the only user.

You get allocated a work area (using manual memory management, that would be your sort_area_size init.ora setting for a sort for example, using auto memory management that might be 50% of 5% of pga_aggregate_target). So you get this workarea of Xmb in size.

If you need (X+1)mb of ram to sort, you will 'swap to disk' (even though there are gb's and gb's of free memory - they are NOT your free memories - you have your workarea of size Xmb).

If you need (X-1)mb of ram to sort, you will NOT 'swap to disk'

PGA sizing

ldsandon, May 14, 2009 - 6:05 am UTC

We'll look in the book.

> Look you have many users.

Actually, I haven't. Does Oracle (10gR2) use the actual number of connected users, sizing areas dynamically, or the number of maximum connections allowed?
Because I see that even when there are few users connected (i.e. at night), unless I let those processes to size their own sort area, they end up with a too small one, while there is unused memory.
My issue is I have a process that can enqueue lots of data in a short time, and have to dequeue and process them in order. The dequeue processes may become very slow due to sorts swapping out if automatic PGA management is enabled.
Tom Kyte
May 14, 2009 - 10:55 am UTC

Ok, look in general you have many users. automatic pga memory management is designed for an environment where you have between 0 and infinity users and is designed to allocate memory up to the TARGET level in a "fair" and "sharing way"


If you have only one user, they use manual memory management.


the goal is not to use all of the memory - the goal is to hold some in reserve - for the very possible condition of "more users need some right now" and avoid - "but we've given it all to Joe over there"


If you have a batch window where you know "I am the only game in town", then you can certainly use manual memory management for them (they can alter session and set the memory management to manual and set their own sort/hash area sizes)


Here is a quote from Expert Oracle Database Architecture

<quote>

Choosing Between Manual and Auto Memory Management

So, which method should you use: manual or automatic? My preference is to use the automatic PGA memory management by default.

Begin Caution
Caution I'll repeat this from time to time in the book: please do not make any changes to a production system - a live system - without first testing for any side effects. For example, please do not read this chapter, check your system, and find you are using manual memory management, and then just turn on automatic memory management. Query plans may change, and performance may be impacted. One of three things could happen:
* Things run exactly the same.
* Things run better than they did before.
* Things run much worse then they did before.
Exercise caution before making changes, test the proposed change first.
End Caution


One of the most perplexing things for a DBA can be setting the individual parameters, especially parameters such as SORT|HASH_AREA_SIZE and so on. Many times, I see systems running with incredibly small values for these parameters-values so small that system performance is massively impacted in a negative way. This is probably a result of the fact that the default values are very small themselves: 64KB for sorting and 128KB for hashing. There is a lot of confusion over how big or small these values should be. Not only that, but the values you would like to use for them might vary over time, as the day goes by. At 8:00 am, with two users, a 50MB sort area size might be reasonable for the single user logged in. However, at 12:00 pm with 500 users, 50MB might not be appropriate. This is where the WORKAREA_SIZE_POLICY = AUTO setting and the corresponding PGA_AGGREGATE_TARGET come in handy. Setting the PGA_AGGREGATE_TARGET, the amount of memory you would like Oracle to feel free to use to sort and hash, is conceptually easier than trying to figure out the perfect SORT|HASH_AREA_SIZE, especially since there isn't a perfect value for these parameters; the perfect value varies by workload.

Historically, the DBA configured the amount of memory used by Oracle by setting the size of the SGA (the buffer cache; the log buffer; and the shared, large, and Java pools). The remaining memory on the machine would then be used by the dedicated or shared servers in the PGA region. The DBA had little control over how much of this memory would or would not be used. She could set the SORT_AREA_SIZE, but if there were 10 concurrent sorts, then Oracle could use as much as 10 * SORT_AREA_SIZE bytes of RAM. If there were 100 concurrent sorts, then Oracle would use 100 * SORT_AREA_SIZE bytes; for 1,000 concurrent sorts, 1,000 * SORT_AREA_SIZE; and so on. Couple that with the fact that other things go into the PGA, and you really don't have good control over the maximal use of PGA memory on the system.

What you would like to have happen is for this memory to be used differently as the memory demands on the system grow and shrink. The more users, the less RAM each should use. The fewer users, the more RAM each should use. Setting WORKAREA_SIZE_POLICY = AUTO is just such a way to achieve this. The DBA specifies a single size now, the PGA_AGGREGATE_TARGET or the maximum amount of PGA memory that the database should strive to use. Oracle will distributes this memory over the active sessions as it sees fit. Further, with Oracle9i Release 2 and up, there is even PGA advisory (part of statspack, available via a V$ dynamic performance view and visible in Enterprise Manager), much like the buffer cache advisor. It will tell you over time what the optimal PGA_AGGREGATE_TARGET for your system is to minimize physical I/O to your temporary tablespaces. You can use this information to either dynamically change the PGA size online (if you have sufficient RAM) or decide whether you might need more RAM on your server to achieve optimal performance.


Are there times, however, when you won't want to use it? Absolutely,
and fortunately they seem to be the exception and not the rule. The automatic memory management was designed to be multiuser 'fair.' In anticipation of additional users joining the system, the automatic memory management will limit the amount of memory allocated as a percentage of the PGA_AGGREGATE_TARGET. But what happens when you don't want to be fair, when you know that you should get all of the memory available? Well, that would be time to use the ALTER SESSION command to disable automatic memory management in your session (leaving it in place for all others) and to manually set your SORT|HASH_AREA_SIZE as needed. For example, that large batch process that takes place at 2:00 am and does tremendously large hash joins, some index builds, and the like? It should be permitted to use all of the resources on the machine. It does not want to be 'fair' about memory use -- it wants it all, as it knows it is the only thing happening in the database right now. That batch job can certainly issue the ALTER SESSION commands and make use of all resources available.


So, in short, I prefer to use automatic PGA memory management for end user sessions -- for the applications that run day to day against my database. Manual memory management makes sense for large batch jobs that run during time periods when they are the only activities in the database.
</quote>

Using the LARGE_POOL

Stewart Bryson, July 07, 2009 - 3:45 pm UTC

I have an environment where I am running out space for the PX Msg Pool:

ORA-04031: unable to allocate 131096 bytes of shared memory ("shared pool","unknown object","sga heap(1,0)","PX msg pool")

As you can see, the Px Msg Pool is actually being allocated out of the Shared Pool instead of the Large Pool. Here are my parameters:
SQL> show parameter parallel

NAME                                 | TYPE        | VALUE
------------------------------------ | ----------- | ------------------------------
fast_start_parallel_rollback         | string      | LOW
parallel_adaptive_multi_user         | boolean     | TRUE
parallel_automatic_tuning            | boolean     | FALSE
parallel_execution_message_size      | integer     | 65536
parallel_instance_group              | string      |
parallel_max_servers                 | integer     | 50
parallel_min_percent                 | integer     | 0
parallel_min_servers                 | integer     | 0
parallel_server                      | boolean     | FALSE
parallel_server_instances            | integer     | 1
parallel_threads_per_cpu             | integer     | 2
recovery_parallelism                 | integer     | 4
SQL> show parameter large

NAME                                 | TYPE        | VALUE
------------------------------------ | ----------- | ------------------------------
large_pool_size                      | big integer | 304M
SQL> show parameter shared

NAME                                 | TYPE        | VALUE
------------------------------------ | ----------- | ------------------------------
hi_shared_memory_address             | integer     | 0
max_shared_servers                   | integer     |
shared_memory_address                | integer     | 0
shared_pool_reserved_size            | big integer | 26843545
shared_pool_size                     | big integer | 512M
shared_server_sessions               | integer     |
shared_servers                       | integer     | 0
SQL> show parameter sga

NAME                                 | TYPE        | VALUE
------------------------------------ | ----------- | ------------------------------
lock_sga                             | boolean     | FALSE
pre_page_sga                         | boolean     | FALSE
sga_max_size                         | big integer | 1520M
sga_target                           | big integer | 0
SQL>

The only thing I can find about how to dictate the use of the Large Pool is dated references to PARALLEL_AUTOMATIC_TUNING. This is a 10g environment, so PARALLLEL_AUTOMATIC_TUNING is deprecated (or so I thought) in favor of just using PARALLEL_ADAPTIVE_MULTI_USER.

How do I dictate using the Large Pool?

Thanks.

Tom Kyte
July 07, 2009 - 7:16 pm UTC

setting PARALLEL_AUTOMATIC_TUNING tuning = true will get it to go to the large pool, there is also an undocumented init.ora parameter that you can consider under the guidance of support. You would have to contact them regarding using it safely.

A reader, July 07, 2009 - 9:00 pm UTC

Do you recommend using a deprecated parameter ?
( aside from the undocumented parameter under the guidance of Oracle Support)

Here in this case , PARALLEL_AUTOMATIC_TUNING

<quote>
This parameter is deprecated. PARALLEL_AUTOMATIC_TUNING is retained for backward compatibility only
</quote>


Tom Kyte
July 08, 2009 - 3:01 pm UTC

it should accomplish what you want - to have the messages allocated from the large pool instead of the shared pool

the other alternative would be to increase your shared pool size.

PARALLEL_AUTOMATIC_TUNING deprecated

Stewart Bryson, July 08, 2009 - 9:50 am UTC

As I mentioned, as well as the reader above me, PARALLEL_AUTOMATIC_TUNING is documented as a deprecated parameter. That apparently seems false... or, the use of the LARGE_POOL for the PX msg pool without using SGA_TARGET is only supported for backward compatibility. Although I have PARALLEL_ADAPTIVE_MULTI_USER set to TRUE (which I thought took over all functionality from PARALLEL_AUTOMATIC_TUNING), I'm concerned this might have other effects that are unknown to me.

The performance benefits of having the Px msg pool in the LARGE_POOL are discussed in-depth here on your site. Is the thought now that this only makes sense in an automatic-SGA environment?

Thanks, as always.

mvernon, August 09, 2012 - 11:09 am UTC

I have a question regarding how to execute the strategy described above and repeated here:

"...But what happens when you don't want to be fair, when you know that you should get all of the memory available? Well, that would be time to use the ALTER SESSION command to disable automatic memory management in your session (leaving it in place for all others) and to manually set your SORT|HASH_AREA_SIZE as needed. For example, that large batch process that takes place at 2:00 am and does tremendously large hash joins, some index builds, and the like? It should be permitted to use all of the resources on the machine. It does not want to be 'fair' about memory use -- it wants it all, as it knows it is the only thing happening in the database right now. That batch job can certainly issue the ALTER SESSION commands and make use of all resources available. "

This is our situation - we want to do hash joins between two large tables, about 70G each. We are at 11.1.0.7 and we have 64GB of physical memory. As best as I can determine, if I use ALTER SESSION to disable PGA memory management, I would want to set hash_area_size to a large number, say 32GB for purposes of discussion. But hash_area_size appears to be an Integer type (not Big Integer) so I am unable to use more than ~2GB for the hash join.

Is my understanding correct that I cannot specify an amount of hash_area_size after disabling pga memory management to use > 2GB of memory?

Is there a way to assign > 2GB memory for hash operation to a single session?

Thanks

Tom Kyte
August 17, 2012 - 1:31 pm UTC

Temp-usage and internal Temp-optimization

A reader, February 05, 2013 - 9:11 am UTC

Hi Tom,

DBAs usually preach to developers not to implement their own fancy joins on the client side but instead leave joining to the database server because in most cases the database will do a much better job. DBAs also use to tell them about oracles very smart optimizer and other stuff like that. Yet there is this very annoying thing with the temp usage. It seems that oracle just expands all data in the temp when processing it, which in some cases results in (unnecessary) sorts on disks and hence becomes incredibly slow. As far as I know there are algorithms around which keep the memory for sorting small (working similar to pointers) and for a long time I thought Oracle would implement some sort of temp-optimization. We know about the optimization done by oracle-net and we can easily verify that with a network sniffer (I think Jonathan Lewis once wrote an article about that, also concluding to leave joins to the database server).

This problem with temp can be demonstrated with a detail-table (containing many, but rather small records) joining just a few records from a very "fat" master:

-- create a "fat" master
create table tmaster
(
id number primary key,
t1 varchar2(4000),
t2 varchar2(4000),
t3 varchar2(4000),
t4 varchar2(4000),
t5 varchar2(4000),
t6 varchar2(4000),
t7 varchar2(4000),
t8 varchar2(4000),
t9 varchar2(4000)
);

-- We will reference only ONE master-record from this detail-table :
create table tdetail_ref_one
(
id number primary key,
masterid number references tmaster(id),
t varchar2(40)
);

-- We will reference MANY master-records from this detail-table :
create table tdetail_ref_many
(
id number primary key,
masterid number references tmaster(id),
t varchar2(40)
);


-- generate some data:
declare
i number;
tmp varchar2(4000);
begin
for i in 1..3000 loop
tmp := rpad(to_char(i),4000,'M');
insert into tmaster values (i,tmp,tmp,tmp,tmp,tmp,tmp,tmp,tmp,tmp);
for d in 1..30 loop
insert into tdetail_ref_one values (i*1000+d, 1 , rpad(to_char(i),40,'D')); -- all detail-records refer to the same master-record
insert into tdetail_ref_many values (i*1000+d, i , rpad(to_char(i),40,'D')); -- varying master-records
end loop;
end loop;
dbms_stats.gather_table_stats(user,'TMASTER',cascade=>true);
dbms_stats.gather_table_stats(user,'TDETAIL_REF_MANY',cascade=>true);
dbms_stats.gather_table_stats(user,'TDETAIL_REF_ONE',cascade=>true);
end;

commit;

select * from tmaster m , tdetail_ref_many d where m.id = d.masterid order by d.t;
--> ca. 3 Gig of Temp required. (Which is not too surprising)

select * from tmaster m ,tdetail_ref_one d where m.id = d.masterid order by d.t;
--> Again ca. 3 Gig of Temp required. But we always join the SAME "fat" master-record. So ideally it shouldn't use much more temp than the next query:

select m.id, d.* from tmaster m ,tdetail_ref_one d where m.id = d.masterid order by d.t;
--> No temp at all (because we don't select the "fat" part of the master record)

In similar scenarios it seems we are much better off with "do-it-yourself" - joins on the client. (And that is not really satisfying.)

As always your advise on this topic is much appreciated.
Tom Kyte
February 06, 2013 - 1:47 pm UTC

can you tell me a real world case where this is true? I mean how many times do you have a large master table that every record in a dimension joins to the same master? the rest of the master records have no representation in the dimension?


this doesn't seem like it would really happen - why don't the any of the masters have a dimension record?


there are exceptions to every rule. What your DBA's have taught you is correct in almost every single case. Are there times, special edge cases, strange conditions whereby the general rule of thumb is not true? Of course - that is true in all situations when dealing with software, algorithms and the like.

Your case is a very special, if not unique, one. You have a fact table table (you call detail) that points to a single row in a large dimension table (you call a master). It is very unusual to say the least. This is one of those extremely rare cases where a query to select back ordered data from the detail, followed by a query to pick up the single master record would perhaps make sense - but it is an edge case/special case.

Meaning - almost all of the time, just do it in one big bulk sql statement - there are rare cases where this might not be true (just like everything in software) - and those cases should be clearly justified (proven with math) to necessitate a different approach. The problem I see happening is - a developer one time finds a case like this and then says "all cases must be like this, I'll never use the database again". These are truly the rare exceptions - not the rule.

Re: Temp-usage and internal Temp-optimization

A reader, February 07, 2013 - 5:28 am UTC

> can you tell me a real world case where this is true?
Sure, such real world cases do exist. It regularly happens with time-series-data: For example the temperature (for New York, LA, etc.) over an extended period of time. Another example: Finance data, say securities and their charts over time.

> I mean how many times do you have a large master table that every record in a dimension joins to the same master? the rest of the master records have no representation in the dimension?
Well, the previous example goes to the extremes in order to answer the question whether temp-optimization is going on or not. And it seems it isn't. The real world cases would be of course more complex, for example queries like "the temperature over time for some east-coast-cities". Or "The chart for those securities which defaulted during the last 3 month". We always have the same pattern: Many (small) rows from a detail (fact) table joining comparatively few (but potentially "fat") rows from a master (dimension). When analysing where such real world queries spent too much time it turned out that they "got lost in the temp". As algorithms exist which would keep the temp small enough the question was: Does Oracle implement some kind of temp-optimization (and if yes -> understand why it apparently doesn't happen in our queries).

> there are exceptions to every rule [...] Of course - that is true in all situations when dealing with software, algorithms and the like.
Yes. But for the average humble developer or DBA there is also the question: "Is it just us not knowing about a certain oracle-feature or does it really not exist?". Of course: If Tom Kyte doesn't know about it then we can safely assume it doesn't exist, at least not in the current version :-)

Tom Kyte
February 11, 2013 - 8:32 am UTC

with time series data, this would not be true:


I mean how many times do you have a large master table that every record in a dimension joins to the same master? the rest of the master records have no representation in the dimension?


I see now you were setting up a super exaggerated example. Your real world case would be "a single very fat dimension record is joined to thousands of fact records and we don't need the fat dimension record in processing the fact records - but since we join they get thrown in to the sort anyway and blow out temp"

Here is a potential 'technique' for such a thing, scalar subqueries. We can process tdetail_ref_many and then pick up the master row after processing and sorting it. Here is a comparision of the estimated temp space for "just a join" and "scalar subquery"

ops$tkyte%ORA11GR2> create or replace type myScalarType as object
  2  (
  3  id     number ,
  4  t1      varchar2(4000),
  5  t2      varchar2(4000),
  6  t3      varchar2(4000),
  7  t4      varchar2(4000),
  8  t5      varchar2(4000),
  9  t6      varchar2(4000),
 10  t7      varchar2(4000),
 11  t8      varchar2(4000),
 12  t9      varchar2(4000)
 13  );
 14  /

Type created.

ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> set autotrace traceonly explain
ops$tkyte%ORA11GR2> select * from tmaster m ,tdetail_ref_one d where m.id = d.masterid order by d.t;

Execution Plan
----------------------------------------------------------
Plan hash value: 3990881030

----------------------------------------------------------------------------------------------
| Id  | Operation                     | Name            | Rows  | Bytes |TempSpc| Cost (%CPU)|
----------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT              |                 | 90000 |  3095M|       |   678K  (1)|
|   1 |  SORT ORDER BY                |                 | 90000 |  3095M|   703M|   678K  (1)|
|   2 |   MERGE JOIN                  |                 | 90000 |  3095M|       |  4324   (1)|
|   3 |    TABLE ACCESS BY INDEX ROWID| TMASTER         |  3000 |   103M|       |  3008   (1)|
|   4 |     INDEX FULL SCAN           | SYS_C0047219    |  3000 |       |       |     7   (0)|
|*  5 |    SORT JOIN                  |                 | 90000 |  4394K|    10M|  1316   (1)|
|   6 |     TABLE ACCESS FULL         | TDETAIL_REF_ONE | 90000 |  4394K|       |   205   (1)|
----------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   5 - access("M"."ID"="D"."MASTERID")
       filter("M"."ID"="D"."MASTERID")

ops$tkyte%ORA11GR2> select d.*, (select myScalarType( id, t1, t2, t3, t4, t5, t6, t7, t8, t9) from tmaster where tmaster.id = d.id)
  2    from tdetail_ref_many d
  3   order by d.t
  4  /

Execution Plan
----------------------------------------------------------
Plan hash value: 2022710602

---------------------------------------------------------------------------------------------
| Id  | Operation                   | Name             | Rows  | Bytes |TempSpc| Cost (%CPU)|
---------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |                  | 90000 |  4482K|       |  1350   (1)|
|   1 |  TABLE ACCESS BY INDEX ROWID| TMASTER          |     1 | 36013 |       |     2   (0)|
|*  2 |   INDEX UNIQUE SCAN         | SYS_C0047219     |     1 |       |       |     1   (0)|
|   3 |  SORT ORDER BY              |                  | 90000 |  4482K|  5680K|  1350   (1)|
|   4 |   TABLE ACCESS FULL         | TDETAIL_REF_MANY | 90000 |  4482K|       |   205   (1)|
---------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("TMASTER"."ID"=:B1)

ops$tkyte%ORA11GR2> set autotrace off


of course, if you needed some attributes from tmaster to process tdetail-ref-many by - you would/could do this:


ops$tkyte%ORA11GR2> select d.*, (select myScalarType( id, t1, t2, t3, t4, t5, t6, t7, t8, t9) from tmaster where tmaster.id = d.id)
  2    from tdetail_ref_many d, tmaster m
  3   where d.id = m.id
  4     and m.t3 = 'abc...'
  5   order by d.t
  6  /

Execution Plan
----------------------------------------------------------
Plan hash value: 1760991612

---------------------------------------------------------------------------------------
| Id  | Operation                     | Name             | Rows  | Bytes | Cost (%CPU)|
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT              |                  |     1 |  4056 |  3214   (1)|
|   1 |  TABLE ACCESS BY INDEX ROWID  | TMASTER          |     1 | 36013 |     2   (0)|
|*  2 |   INDEX UNIQUE SCAN           | SYS_C0047219     |     1 |       |     1   (0)|
|   3 |  SORT ORDER BY                |                  |     1 |  4056 |  3214   (1)|
|   4 |   NESTED LOOPS                |                  |       |       |            |
|   5 |    NESTED LOOPS               |                  |     1 |  4056 |  3213   (1)|
|   6 |     TABLE ACCESS FULL         | TDETAIL_REF_MANY | 90000 |  4482K|   205   (1)|
|*  7 |     INDEX UNIQUE SCAN         | SYS_C0047219     |     1 |       |     0   (0)|
|*  8 |    TABLE ACCESS BY INDEX ROWID| TMASTER          |     1 |  4005 |     1   (0)|
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("TMASTER"."ID"=:B1)
   7 - access("D"."ID"="M"."ID")
   8 - filter("M"."T3"='abc...')






Of course: If Tom Kyte doesn't know about it then we can safely assume it doesn't exist, at least not in the current version :-)

that is FALSE :) I learn something new about Oracle just about every single day.

Should an increase in PGA_AGGREGATE_TARGET cause an increase in Used-tmp for a query?

Andrew Markiewicz, May 24, 2021 - 4:28 pm UTC

On the theme of this topic, I have a query that was going to disk for HJ and a sort. I was considering increasing the PGA for the instance and expecting better performance. It actually got worse by using more TMP space for a sort.
The plan hash is the same. The number of rows processed is the same.
Is there a reason for this?

Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production

PGA_AGGREGATE_TARGET=8G
Plan hash value: 3393810516
 
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                                         | Name                        | Starts | E-Rows |E-Bytes|E-Temp | Cost (%CPU)| E-Time   | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | CREATE TABLE STATEMENT                            |                             |      1 |        |       |       |   879K(100)|          |      0 |00:15:54.87 |    3017K|   3391K|    535K|       |       |          |         |
|   1 |  LOAD AS SELECT                                   | AMA_TMP_178348_12           |      1 |        |       |       |            |          |      0 |00:15:54.87 |    3017K|   3391K|    535K|  2070K|  2070K| 2070K (0)|         |
|   2 |   OPTIMIZER STATISTICS GATHERING                  |                             |      1 |  87072 |    20M|       |   878K  (1)| 00:00:35 |   3485K|00:15:42.27 |    2900K|   3391K|    495K|   256K|   256K|  768K (0)|         |
|   3 |    SORT GROUP BY                                  |                             |      1 |  87072 |    20M|    21M|   878K  (1)| 00:00:35 |   3485K|00:15:38.73 |    2900K|   3391K|    495K|   209M|  6788K|   97M (1)|     376M|
|*  4 |     HASH JOIN                                     |                             |      1 |  87072 |    20M|       |   874K  (1)| 00:00:35 |   6971K|00:15:05.29 |    2900K|   3343K|    447K|  1222K|  1222K| 1650K (0)|         |
|*  5 |      INDEX RANGE SCAN                             | INX_FBI_PC_CAT_UPCODE_ID    |      1 |     45 |   720 |       |     2   (0)| 00:00:01 |     33 |00:00:00.04 |       2 |      2 |      0 |       |       |          |         |
|*  6 |      HASH JOIN                                    |                             |      1 |  88757 |    19M|       |   874K  (1)| 00:00:35 |   6971K|00:15:19.31 |    2900K|   3343K|    447K|  1123K|  1123K| 1632K (0)|         |
|   7 |       VIEW                                        | index$_join$_040            |      1 |    331 |  6951 |       |    46   (0)| 00:00:01 |    320 |00:00:00.04 |      56 |     30 |      0 |       |       |          |         |
|*  8 |        HASH JOIN                                  |                             |      1 |        |       |       |            |          |    320 |00:00:00.04 |      56 |     30 |      0 |  1572K|  1572K| 2180K (0)|         |
|   9 |         INDEX FAST FULL SCAN                      | PK_PROTOCOL                 |      1 |    331 |  6951 |       |    21   (0)| 00:00:01 |   6654 |00:00:00.01 |      22 |      0 |      0 |       |       |          |         |
|* 10 |         INDEX FAST FULL SCAN                      | UK_PROTOCOL                 |      1 |    331 |  6951 |       |    36   (0)| 00:00:01 |    320 |00:00:00.03 |      34 |     30 |      0 |       |       |          |         |
|* 11 |       HASH JOIN                                   |                             |      1 |    879K|   172M|  6320K|   874K  (1)| 00:00:35 |   7774K|00:15:13.90 |    2900K|   3342K|    447K|    20M|  3667K|   19M (0)|         |
|  12 |        VIEW                                       | index$_join$_037            |      1 |    294K|  2871K|       |  1916   (1)| 00:00:01 |    294K|00:00:01.07 |    1179 |    824 |      0 |       |       |          |         |
|* 13 |         HASH JOIN                                 |                             |      1 |        |       |       |            |          |    294K|00:00:01.07 |    1179 |    824 |      0 |    18M|  4191K|   17M (0)|         |
|  14 |          INDEX FAST FULL SCAN                     | INX_SPCS_SPONSOR_PCLID      |      1 |    294K|  2871K|       |   579   (1)| 00:00:01 |    294K|00:00:01.29 |     471 |    463 |      0 |       |       |          |         |
|  15 |          INDEX FAST FULL SCAN                     | PK_SMRS_PR_SUBJECT          |      1 |    294K|  2871K|       |   871   (1)| 00:00:01 |    294K|00:00:01.19 |     708 |    361 |      0 |       |       |          |         |
|* 16 |        HASH JOIN                                  |                             |      1 |    880K|   164M|    10M|   863K  (1)| 00:00:34 |   7774K|00:15:08.44 |    2899K|   3342K|    447K|    20M|  3332K|   20M (0)|         |
|  17 |         TABLE ACCESS FULL                         | BSMS_PCS_CASE               |      1 |    273K|  7481K|       |  1206   (1)| 00:00:01 |    273K|00:00:00.04 |    4385 |   4356 |      0 |       |       |          |         |
|* 18 |         HASH JOIN                                 |                             |      1 |    880K|   140M|   120M|   854K  (1)| 00:00:34 |   7774K|00:15:02.52 |    2895K|   3337K|    447K|  1900M|    32M| 1032M (1)|    1297M|
|* 19 |          HASH JOIN                                |                             |      1 |    880K|   109M|   117M|   806K  (1)| 00:00:32 |   7774K|00:13:33.89 |    2775K|   3055K|    283K|  1797M|    32M|  897M (1)|    1233M|
|* 20 |           HASH JOIN                               |                             |      1 |    942K|   106M|   106M|   785K  (1)| 00:00:31 |   7776K|00:12:21.37 |    2752K|   2876K|    128K|  1600M|    33M|  747M (1)|    1022M|
|* 21 |            HASH JOIN                              |                             |      1 |    942K|    96M|       |   727K  (1)| 00:00:29 |   7776K|00:15:21.27 |    2657K|   2653K|      0 |   850K|   850K|  568K (0)|         |
|  22 |             NESTED LOOPS                          |                             |      1 |      1 |    71 |       |    20   (0)| 00:00:01 |      2 |00:00:00.04 |      21 |     13 |      0 |       |       |          |         |
|  23 |              NESTED LOOPS                         |                             |      1 |     22 |    71 |       |    20   (0)| 00:00:01 |     44 |00:00:00.03 |      14 |      8 |      0 |       |       |          |         |
|  24 |               NESTED LOOPS                        |                             |      1 |      2 |    62 |       |    16   (0)| 00:00:01 |      4 |00:00:00.01 |       8 |      6 |      0 |       |       |          |         |
|  25 |                TABLE ACCESS BY INDEX ROWID        | EDC_FORM                    |      1 |      1 |    24 |       |     2   (0)| 00:00:01 |      1 |00:00:00.01 |       3 |      2 |      0 |       |       |          |         |
|* 26 |                 INDEX UNIQUE SCAN                 | UK_EDC_FORM_NAME_VERSION    |      1 |      1 |       |       |     1   (0)| 00:00:01 |      1 |00:00:00.01 |       2 |      2 |      0 |       |       |          |         |
|  27 |                TABLE ACCESS BY INDEX ROWID BATCHED| EDC_FORM_ELEMENT            |      1 |      2 |    14 |       |    14   (0)| 00:00:01 |      4 |00:00:00.01 |       5 |      4 |      0 |       |       |          |         |
|* 28 |                 INDEX RANGE SCAN                  | INX_EFE_FORM_ID             |      1 |     21 |       |       |     1   (0)| 00:00:01 |      4 |00:00:00.01 |       2 |      1 |      0 |       |       |          |         |
|* 29 |               INDEX RANGE SCAN                    | INX_EFE_SECTION_ID          |      4 |     11 |       |       |     1   (0)| 00:00:01 |     44 |00:00:00.02 |       6 |      2 |      0 |       |       |          |         |
|* 30 |              TABLE ACCESS BY INDEX ROWID          | EDC_FORM_ELEMENT            |     44 |      1 |    40 |       |     3   (0)| 00:00:01 |      2 |00:00:00.04 |       7 |      5 |      0 |       |       |          |         |
|  31 |             TABLE ACCESS FULL                     | EDC_FIELD_VALUE             |      1 |    252M|  8677M|       |   726K  (1)| 00:00:29 |    252M|00:03:02.56 |    2657K|   2653K|      0 |       |       |          |         |
|  32 |            INDEX FAST FULL SCAN                   | INX_ESI_FIID_SIID           |      1 |     23M|   266M|       | 25727   (1)| 00:00:02 |     23M|00:00:26.26 |   95123 |  93929 |      0 |       |       |          |         |
|  33 |           TABLE ACCESS FULL                       | ONC_BSM_ANNOTATION_INSTANCE |      1 |   7723K|    88M|       |  6498   (1)| 00:00:01 |   7723K|00:00:04.09 |   23643 |  23612 |      0 |       |       |          |         |
|  34 |          TABLE ACCESS FULL                        | BSMS_PCS_SPECIMEN           |      1 |   3888K|   137M|       | 32542   (1)| 00:00:02 |   3888K|00:00:08.60 |     119K|    119K|      0 |       |       |          |         |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


PGA_AGGREGATE_TARGET=20G

Plan hash value: 3393810516
 
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                                         | Name                        | Starts | E-Rows |E-Bytes|E-Temp | Cost (%CPU)| E-Time   | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | CREATE TABLE STATEMENT                            |                             |      1 |        |       |       |   879K(100)|          |      0 |00:19:26.05 |    3018K|   4549K|   1936K|       |       |          |         |
|   1 |  LOAD AS SELECT                                   | AMA_TMP_178348_12           |      1 |        |       |       |            |          |      0 |00:19:26.05 |    3018K|   4549K|   1936K|  2070K|  2070K| 2070K (0)|         |
|   2 |   OPTIMIZER STATISTICS GATHERING                  |                             |      1 |  87072 |    20M|       |   878K  (1)| 00:00:35 |   3485K|00:19:12.57 |    2901K|   4549K|   1895K|   256K|   256K|  768K (0)|         |
|   3 |    SORT GROUP BY                                  |                             |      1 |  87072 |    20M|    21M|   878K  (1)| 00:00:35 |   3485K|00:19:09.63 |    2901K|   4549K|   1895K|  1992M|    36M|  110M (1)|      11G|
|*  4 |     HASH JOIN                                     |                             |      1 |  87072 |    20M|       |   874K  (1)| 00:00:35 |   6971K|00:13:30.03 |    2901K|   3046K|    393K|  1222K|  1222K| 1649K (0)|         |
|*  5 |      INDEX RANGE SCAN                             | INX_FBI_PC_CAT_UPCODE_ID    |      1 |     45 |   720 |       |     2   (0)| 00:00:01 |     33 |00:00:00.01 |       2 |      0 |      0 |       |       |          |         |
|*  6 |      HASH JOIN                                    |                             |      1 |  88757 |    19M|       |   874K  (1)| 00:00:35 |   6971K|00:13:27.73 |    2901K|   3046K|    393K|  1123K|  1123K| 1648K (0)|         |
|   7 |       VIEW                                        | index$_join$_040            |      1 |    331 |  6951 |       |    46   (0)| 00:00:01 |    320 |00:00:00.01 |      56 |      0 |      0 |       |       |          |         |
|*  8 |        HASH JOIN                                  |                             |      1 |        |       |       |            |          |    320 |00:00:00.01 |      56 |      0 |      0 |  1572K|  1572K| 2180K (0)|         |
|   9 |         INDEX FAST FULL SCAN                      | PK_PROTOCOL                 |      1 |    331 |  6951 |       |    21   (0)| 00:00:01 |   6654 |00:00:00.01 |      22 |      0 |      0 |       |       |          |         |
|* 10 |         INDEX FAST FULL SCAN                      | UK_PROTOCOL                 |      1 |    331 |  6951 |       |    36   (0)| 00:00:01 |    320 |00:00:00.01 |      34 |      0 |      0 |       |       |          |         |
|* 11 |       HASH JOIN                                   |                             |      1 |    879K|   172M|  6320K|   874K  (1)| 00:00:35 |   7774K|00:13:25.43 |    2901K|   3046K|    393K|    20M|  3667K|   19M (0)|         |
|  12 |        VIEW                                       | index$_join$_037            |      1 |    294K|  2871K|       |  1916   (1)| 00:00:01 |    294K|00:00:00.30 |    1179 |      0 |      0 |       |       |          |         |
|* 13 |         HASH JOIN                                 |                             |      1 |        |       |       |            |          |    294K|00:00:00.30 |    1179 |      0 |      0 |    18M|  4191K|   17M (0)|         |
|  14 |          INDEX FAST FULL SCAN                     | INX_SPCS_SPONSOR_PCLID      |      1 |    294K|  2871K|       |   579   (1)| 00:00:01 |    294K|00:00:00.05 |     471 |      0 |      0 |       |       |          |         |
|  15 |          INDEX FAST FULL SCAN                     | PK_SMRS_PR_SUBJECT          |      1 |    294K|  2871K|       |   871   (1)| 00:00:01 |    294K|00:00:00.03 |     708 |      0 |      0 |       |       |          |         |
|* 16 |        HASH JOIN                                  |                             |      1 |    880K|   164M|    10M|   863K  (1)| 00:00:34 |   7774K|00:13:21.97 |    2900K|   3046K|    393K|    20M|  3332K|   20M (0)|         |
|  17 |         TABLE ACCESS FULL                         | BSMS_PCS_CASE               |      1 |    273K|  7481K|       |  1206   (1)| 00:00:01 |    273K|00:00:00.04 |    4385 |      0 |      0 |       |       |          |         |
|* 18 |         HASH JOIN                                 |                             |      1 |    880K|   140M|   120M|   854K  (1)| 00:00:34 |   7774K|00:13:18.57 |    2895K|   3046K|    393K|  1900M|    32M| 1240M (1)|    1173M|
|* 19 |          HASH JOIN                                |                             |      1 |    880K|   109M|   117M|   806K  (1)| 00:00:32 |   7774K|00:11:56.91 |    2775K|   2898K|    245K|  1797M|    32M| 1165M (1)|    1137M|
|* 20 |           HASH JOIN                               |                             |      1 |    942K|   106M|   106M|   785K  (1)| 00:00:31 |   7776K|00:10:40.89 |    2752K|   2755K|    102K|  1600M|    33M| 1055M (1)|     813M|
|* 21 |            HASH JOIN                              |                             |      1 |    942K|    96M|       |   727K  (1)| 00:00:29 |   7776K|00:09:43.89 |    2657K|   2653K|      0 |   850K|   850K|  568K (0)|         |
|  22 |             NESTED LOOPS                          |                             |      1 |      1 |    71 |       |    20   (0)| 00:00:01 |      2 |00:00:00.01 |      21 |      0 |      0 |       |       |          |         |
|  23 |              NESTED LOOPS                         |                             |      1 |     22 |    71 |       |    20   (0)| 00:00:01 |     44 |00:00:00.01 |      14 |      0 |      0 |       |       |          |         |
|  24 |               NESTED LOOPS                        |                             |      1 |      2 |    62 |       |    16   (0)| 00:00:01 |      4 |00:00:00.01 |       8 |      0 |      0 |       |       |          |         |
|  25 |                TABLE ACCESS BY INDEX ROWID        | EDC_FORM                    |      1 |      1 |    24 |       |     2   (0)| 00:00:01 |      1 |00:00:00.01 |       3 |      0 |      0 |       |       |          |         |
|* 26 |                 INDEX UNIQUE SCAN                 | UK_EDC_FORM_NAME_VERSION    |      1 |      1 |       |       |     1   (0)| 00:00:01 |      1 |00:00:00.01 |       2 |      0 |      0 |       |       |          |         |
|  27 |                TABLE ACCESS BY INDEX ROWID BATCHED| EDC_FORM_ELEMENT            |      1 |      2 |    14 |       |    14   (0)| 00:00:01 |      4 |00:00:00.01 |       5 |      0 |      0 |       |       |          |         |
|* 28 |                 INDEX RANGE SCAN                  | INX_EFE_FORM_ID             |      1 |     21 |       |       |     1   (0)| 00:00:01 |      4 |00:00:00.01 |       2 |      0 |      0 |       |       |          |         |
|* 29 |               INDEX RANGE SCAN                    | INX_EFE_SECTION_ID          |      4 |     11 |       |       |     1   (0)| 00:00:01 |     44 |00:00:00.01 |       6 |      0 |      0 |       |       |          |         |
|* 30 |              TABLE ACCESS BY INDEX ROWID          | EDC_FORM_ELEMENT            |     44 |      1 |    40 |       |     3   (0)| 00:00:01 |      2 |00:00:00.01 |       7 |      0 |      0 |       |       |          |         |
|  31 |             TABLE ACCESS FULL                     | EDC_FIELD_VALUE             |      1 |    252M|  8677M|       |   726K  (1)| 00:00:29 |    252M|00:01:54.22 |    2657K|   2653K|      0 |       |       |          |         |
|  32 |            INDEX FAST FULL SCAN                   | INX_ESI_FIID_SIID           |      1 |     23M|   266M|       | 25727   (1)| 00:00:02 |     23M|00:00:04.05 |   95123 |      0 |      0 |       |       |          |         |
|  33 |           TABLE ACCESS FULL                       | ONC_BSM_ANNOTATION_INSTANCE |      1 |   7723K|    88M|       |  6498   (1)| 00:00:01 |   7723K|00:00:00.71 |   23762 |      0 |      0 |       |       |          |         |
|  34 |          TABLE ACCESS FULL                        | BSMS_PCS_SPECIMEN           |      1 |   3888K|   137M|       | 32542   (1)| 00:00:02 |   3888K|00:00:01.68 |     120K|     15 |      0 |       |       |          |         |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Connor McDonald
May 25, 2021 - 2:53 am UTC

The "used-tmp" column suggests the reason for the increased execution time - the dumping of 11GB to temporary storage.

In terms of "why", we probably need to dig deeper. If this is reproducible, can you run these in different sessions, each preceded by

alter session set events '10032 trace name context forever';
alter session set events '10033 trace name context forever';

and send the trace file to asktom_us@oracle.com with subject:

"PGA 1709080600346973208"

Andrew Markiewicz, May 25, 2021 - 10:14 pm UTC

Thanks Connor.
It is reproducible. Ran the traces as requested and sent the files.