Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Muhammad Riaz.

Asked: December 27, 2005 - 12:02 am UTC

Last updated: April 18, 2012 - 8:07 am UTC

Version: 9.2.0

Viewed 1000+ times

You Asked

Hi Tom,

I am building a list of "Don't do this" things specially w.r.t SQL Performance Tunning. On the basis of your experience, would you like to add something to this list please?

and Tom said...

o don't accept string literals from end users and concatenate them into your SQL (eg: DO use binds in almost all cases)

o don't test on an empty database or a database with a small percentage of the real system. importing statistics from a "real" database doesn't work. You need real data volumes if you want to see what will actually happen in real life.

o don't test with a single user, scalability issues will never make themselves apparent.

o DO use a source code control system if you have more than 5 or so developers working. Consider EVERYTHING that starts with CREATE, GRANT, ALTER, DROP (eg: DDL - all dll - create procedure, create table) as source code. Do to it what you would do to source code (because it is SOURCE CODE)

o don't say "we'll just whip it out and see what sticks". Meaning DO design, DO think about security from the beginning, DO think about scalability from day 1. DO design your system. DO NOT just wing it as you go along. That might sound like fun, but you'll be working on that same system forever fixing it, patching it, trying to make it work.

o don't take any advice from experts unless you see compelling evidence that what is being suggested actually applies to you. Eg: if you see a book that just "says"
- commit frequently to save resources
- take it easy on the database, don't join, do it yourself in code
- etc....

and that is all they say (a bunch of advice with no reasoning behind it, no assumptions, no examples, no evidence - in short - no proof), ignore it. Find someone willing to explain "why" - so you can increase your knowledge but also understand when something MIGHT NOT APPLY (because nothing is ever 100% true all of the time)

o don't try to optimize by hypothesize. (it rhymes :) I "feel this will be slow" - nope, don't "feel" - show it to be slow, then optimize it. Point I'm trying to make is I see people "feeling that doing 'X' will be slow" and trying to design a system around not doing 'X'. Show us that 'X' is too slow for you (typically trying to avoid the use of a database feature because they have heard "it is slow" - eg: auditing. Show

a) it is slow
b) you can do it faster

don't just do it yourself - do step a and b, and then you can do it yourself if b is true)


I'm sure this'll become a "hot" thread for some reason.

Rating

  (34 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, December 27, 2005 - 11:13 am UTC


This should be a hot thread !

Michel Cadot, December 27, 2005 - 11:24 am UTC


More thoughts...

Craig, December 27, 2005 - 12:40 pm UTC

- Don't reinvent the wheel. There's a lot of built-in functionality that goes overlooked because developers aren't aware of what tools Oracle includes with their product. I started working with my current employer, observed their processes and found that there are already features available which are better suited to requirements.

- Don't ignore the Oracle documentation. It really does answer your questions, such as: "Do I need to build this myself, or can Oracle already do it?". I love when Tom answers a posted question with a link to another thread or a link to Oracle documentation.

Regards,
Craig

Tom Kyte
December 27, 2005 - 2:13 pm UTC

- about the wheel, yes that was the point I was trying to make with "don't try to optimize by hypothesize." The most common reason I hear for not using a builtin feature "we heard it was slow"



Technologies

Wolfgang, December 27, 2005 - 3:58 pm UTC

Don't use technologies just because they are cool. Think practical (e.g. not every file has to be written in XML).

Don't hesitate to throw away bad code - rewriting is often easier then maintaining bad programs.


Document !!!

Gleisson Henrique, December 27, 2005 - 5:06 pm UTC

o don't write code without documentation/explaination.

I inherited an application designed by three developers, none of which would document/explain their code. In order to debug, I spent more time interpreting the code than actually fixing the it.

o don't name your variable arbitrarily

The same three developer that I mentioned before would use variable names out of this world. If it is a counter, then should be cnt. If it's a global variable then g_meaninful_name.

Those two little concepts seems silly and very novice, but they sure make a difference when you have to debug and interpret someone else's code.

Documentation

Johan Snyman, December 28, 2005 - 2:34 am UTC

Just one remark regarding documentation:

The documentation should not indicate what is being done, it should indicate why.

The "what" can be seen from the code (especially if meaningful variable names are used, etc.). It is usually the "why" that is missing and often lead to problems with maintainability, etc.

Have a look at </code> http://tlug.up.ac.za/old/htwuc/unmain.html <code>
and do the opposite!


comments

Karthick, December 28, 2005 - 3:14 am UTC

Tom which is good..

Comment your code

or

Code what you comment.

Tom Kyte
December 28, 2005 - 9:37 am UTC

both

meaning, it is somewhat circular.

you start by coding what you commented, then you maintain them both as you fix bugs.

some more ideas for DOs:

Sandy Mamoli, December 28, 2005 - 4:12 am UTC

Agree, and some more ideas for a DO list:

- Refactor: If you see bad code don't hesistate to refactor it. Maybe something was even good code a while ago but the requirements have changed - just change and improve it.

- Unit test: feel more secure about making changes, be sure not to break any dependencies you might not even be aware of (utplsql is great).

- Use source control even if your are the only developer on the project: Make sure you don't lose any of your work, you have a history, you can revert to older code

RE: Sandy, Well, it depends....

Bill S., December 28, 2005 - 9:32 am UTC

[QUOTE]Sandy said....- Refactor: If you see bad code don't hesistate to refactor it. Maybe something
was even good code a while ago but the requirements have changed - just change
and improve it.[/QUOTE]

Well, that depends. I once worked on an old program that had a source listing that stood nearly 3 feet high when printed. I was asked to implement a minor change which required adding a small module. While in there, I noticed some pretty horrific spaghetti code that could use some untangling. Long story short I spent the next 5 days working with 2 other programmers to put it back the way it had been, because untangling it was such a mess. The boss? He said, "Let this be a lesson to you - it may look horrible but if it ain't broke don't fix it unless you have the next wekk entirely free.". Sometimes it is better to let old code lie until you can re-write the entire thing. :-D

My "Not To Do":
Don't do ANYTHING without a spec. Period. Without a spec, you can't possibly argue that your deliverable is as requested. :-D

SQL Performance - Not to do Things!!

Magesh, December 28, 2005 - 12:00 pm UTC

With regard to SQL Performance, I would say, don't comment your sql to change the optimizer plan unless you are sure. Meaning, don't include comments like /*+ INDEX(XXX) */ just because you notice that the explain plan is not using the index you created. Sometimes using index may be more expensive than not using it. And ofcourse, do your testing on Production data but not on Production environment.

David Aldridge http://oraclesponge.blogspot.com, December 28, 2005 - 1:10 pm UTC

>> Don't do ANYTHING without a spec. Period. Without a spec, you can't possibly argue that your deliverable is as requested. :-D <<

Hah hah hah ... *wipes eyes* ... specifications, yes I remember those.

My 2 cts

Frank, December 28, 2005 - 3:42 pm UTC

My advice is to use a source revision control system. When you store old versions, there is no more need to comment old code. You can safely remove it from your source. If it turns out that the new code does not work anymore, you won't need 3 people for 5 days to get it back the way it was; you just restore the last good version.

Another advice: either keep your in-source documentation up-to-date or remove it. Few things are more annoying than out-of-date comments.


Tom Kyte
December 28, 2005 - 6:08 pm UTC

the old code would have been commented way before it became old code!

Few things are as annoying....

Bill S., December 28, 2005 - 4:12 pm UTC

[QUOTE FRANK]Few things are more annoying than out-of-date comments.[/QUOTE]

Except for NO COMMENTS. How am I (who has now inherited code you wrote last year and I am brand spankin' new to the company) supposed to know what you MEANT to do if it isn't commented? Maybe (and it IS possible you know) the code doesn't do EVERYTHING you intended it to do, or maybe has a nasty side effect in a certain circumstance you haven't touched on yet.
Do an old guy a favor, and give me a hint at least. :-D

Prashant Pathak, December 28, 2005 - 10:50 pm UTC

*)do not forget to provide parameter info and do maintain history info for a perticular code.this helps if the code is bugging.
*)If possible main a tablular format at begining of code which will specify operating being performed on a set of table and what kind of operation(SELECT ,INSERT,DELETE or UPDATE) is being done.

Instrument your code

Dinesh Velhal, December 28, 2005 - 11:40 pm UTC

Thanks everybody for posting all the Do's and Dont's list. This is going to be my home page in browser!

Just to add a bit to the Do list...
* Please INSTRUMENT your code.

I have been working on oracle for past 4 yars. I have seen many situatioans where once the code goes live, and suddenly there is some bug starts making rounds. Without proper instrumentation, there is no easy way to know exactly what part of your code is going wrong.
Instrumentation does add to the coding efforts. but looking at the returns in the long run, it actually saves many man-hours required in maintaining the system.



Tom Kyte
December 29, 2005 - 10:17 am UTC

Nice one - definitely.

RE: My 2 cts

Frank, December 29, 2005 - 4:59 am UTC

[quote]there is no more need to comment old code
<snip>
Followup:

the old code would have been commented way before it became old code! [/quote]

Ah the problem of a non-native speaker (me)
I'm afraid I did not translate my intended idea too well.
I meant to say there is no need to put /* ... */ around existing code. (should that have been 'comment out'?)

Tom Kyte
December 29, 2005 - 10:23 am UTC

Ahh, got it - yes, you would not comment OUT old code, you would fix old code and fix old comments to reflect new code :)

RE: Frank - Well, OF COURSE! Should have figured that out.

Bill S., December 29, 2005 - 8:39 am UTC

[QUOTE Frank]Ah the problem of a non-native speaker (me)
I'm afraid I did not translate my intended idea too well.[/QUOTE]

Maybe we were too quick to read your statement! When I go back and look at it again, I can see what you meant quite easily. I should slow down and look at the WHOLE statement before commenting.

Thanks for clarifying that, sometimes I tend to forget that not everyone here speaks English as a native language. I need to keep that in mind. :-D

Another item for comments

DavidS, December 29, 2005 - 9:12 am UTC

Something else you may find useful in the comments are references to any dependencies. For example:
-- this procedure is designed for BATCH MODE only, and should not be run during 'client' business hours.
-- this item expects 'abc' values to be available for the lookup. Otherwise, check the XYZ table for incorrect entries.

This can save a lot of time...

A slightly different direction

Charlie B., December 29, 2005 - 9:25 am UTC

DON'T expect the same db on two different hosts to work the same. Maybe one's a 4-way Sun and the other's two dual Intels. Or one is Sun and the other's Linux. Or one's production and the other's an import of production. There will be differences.

DON'T change init.ora parameters without documenting what the old ones were.

DON'T be afraid of collecting histogram data when you run stats. Histograms can be one of your best friends.

Emiel, December 30, 2005 - 8:11 am UTC

you Said: " importing statistics from a "real" database doesn't work. You need real data volumes if you want to see what will actually happen in real life."

What benefit would you gain from importing statitistics then ?

Tom Kyte
December 30, 2005 - 9:53 am UTC

The are useful to

a) give to support to show "see, with these stats, we get the wrong plan" without giving support all 10 billion rows of data to reproduce the issue

b) after you restore production to QA (to prove you can restore it), you might gather statistics there (on QA) and test your application with the new statistics - to verify that the changed query plans (that is mostly why we gather statistics after all - to cause plans to change!) are for the better - not for the worse. And after you verify that they are for the good - you export from QA and import into production (after saving productions values, just in case)

for example.

INSTRUMENT your code

karthick, January 02, 2006 - 5:34 am UTC

what does INSTRUMENT your code means

Tom Kyte
January 02, 2006 - 10:00 am UTC

Just a question

A reader, January 05, 2006 - 6:16 am UTC

Hi Tom,
In your blog, you've mentioned using a database table to hold a flag that can be set when you wanted to turn on instrumentation.
In this case, I'm assuming that you would have a stored procedure that would check this flag value and if it is set, then it would log all information as required. This procedure would be called from within your appl. My question is, would it be expensive to keep reading from this table?
If possible, could you give me a small example of how you would do it using a table. Thanks a lot

Tom Kyte
January 05, 2006 - 10:48 am UTC

dual is read alot :)

every page on htmldb applications - read your session state from .... a table.

I use select to read from tables...

You can set the flag in a table and read it..
You can pass it in a URL...


Use of an appropriate data structure can be very useful here. Say you wanted a table - a simple:

select parameter_value from my_parameter_table where parameter_name = :bind;

would likely work BEST as a single table hash cluster - we'll hash the bind variable, get the database block address (DBA) and get that block - no index range scan, one LIO.

Please explain

Dinesh Velhal, January 06, 2006 - 7:59 am UTC

Dear Tom

The last part of your review...

"would likely work BEST as a single table hash cluster - we'll hash the bind
variable, get the database block address (DBA) and get that block - no index
range scan, one LIO"

Would you please explain what you meant by hash the bind variable and get block address?

Thanks

Dinesh

Tom Kyte
January 06, 2006 - 1:59 pm UTC

Hashing hopefully is something known to us - this is just like a "hash table" in a data structures class - it is a typical "hashing function". We take the "key" for a row (you designate a hash key, a column/columns from the table). We HASH that key (typically hash function - just like dbms_utility might do with the hash function in that package). That HASHed value is really the address of the row.

When you insert - we hash the key to figure out where to PUT the row.

When you query, we hash the key you provide again to find out where we PUT the row and go there to GET the row.

<quote src = Expert Oracle: Database Architecture>

Hash Clustered Tables

Hash clustered tables are very similar in concept to the index clustered tables just described with one main exception: the cluster key index is replaced with a hash function. The data in the table is the index; there is no physical index. Oracle will take the key value for a row, hash it using either an internal function or one you supply, and use that to figure out where the data should be on disk. One side effect of using a hashing algorithm to locate data, however, is that you cannot range scan a table in a hash cluster without adding a conventional index to the table. In an index cluster, the query

select * from emp where deptno between 10 and 20

would be able to make use of the cluster key index to find these rows. In a hash cluster, this query would result in a full table scan unless you had an index on the DEPTNO column. Only exact equality searches (including IN lists and subqueries) may be made on the hash key without using an index that supports range scans.

In a perfect world, with nicely distributed hash key values and a hash function that distributes them evenly over all of the blocks allocated to the hash cluster, we can go straight from a query to the data with one I/O. In the real world, we will end up with more hash key values hashing to the same database block address than fit on that block. This will result in Oracle having to chain blocks together in a linked list to hold all of the rows that hash to this block. Now, when we need to retrieve the rows that match our hash key, we might have to visit more than one block.

Like a hash table in a programming language, hash tables in the database have a fixed “size.” When you create the table, you must determine the number of hash keys your table will have, forever. That does not limit the amount of rows you can put in there.
</quote>

You are the best!!!!!

A reader, January 06, 2006 - 8:25 am UTC


Excellant Information

Dinesh Velhal, January 09, 2006 - 2:39 am UTC

Hi Tom,

Many thanks for the follow up. Every time I visit your site or read your feedback, I learn something new.


Regards
Dinesh

DO use binds in almost all cases

Vadim Bobrov, January 30, 2006 - 3:43 pm UTC

speaking about:
don't take any advice from experts unless you see compelling evidence that what is being suggested actually applies to you..

in some cases using binds vs. literals can be disastrous. Here is an example. You have a huge table with an indexed column. You select from this table searching by that column. The cardinality of the indexed column is rather low but the values distribution for this column is very skew, let's say you have only one row with some value and you need THAT row, you have a histogram and you want to use the index - CBO will ignore the index and go for full table scan if you use binds. If you use a literal in this case CBO will consult the histogram and figure out it had better use index

Question regarding the Database Block Address

Neil, February 02, 2006 - 5:12 pm UTC

Tom,
If Oracle uses the Data Block Address to determine whether or not a block is in the buffer cache, and thus whether or not to waste a physical read going to get it, how does it determine the data block addresses of all the blocks it needs to satisfy a given query? It seems a bit chicken-and-egg to me: there's no way to tell the data block address without reading the block, and no way of reading the block from the cache without the data block address.
I can see that a big clue to go looking in the cache rather than on disk would be the fact that the statement in question has previously been executed, and thus there's a possibility of some or all of the required blocks being in the cache, but I still don't understand how the data block addresses are calculated...

Tom Kyte
February 03, 2006 - 1:42 pm UTC

full scan: we know all of the extents - which have a file# and start block/end block. We know ALL of the blocks we need to read (a data block address is a file.block)

index access: well, we sort of get the rowid from the index, the rowid contains the file.block.


I don't see how this is chicken and egg at all???


We always have the data block address before going after the block - else we'd never know what to go after!

The data block address question

Neil, February 03, 2006 - 5:19 am UTC

Tom - No need to answer the above.
I've been looking further into this, and it transpires that Oracle can work out the data block addresses of the blocks needed to satisfy a query. If an index can be used, the root block address will be in the data dictionary. This will yield the DBA of that root block. If it's not in the cache, it's read from disk. Then it's used to get to the leaf node blocks from which rowids and therefore DBAs of the data blocks can be ascertained. If no index can be used, the address of the first block in a table (also in the data dictionary) is used as a starting point.
Thanks to "Scaling Oracle 8i" by James Morle and "Practical Oracle 8i" by Jonathan Lewis.

One more -- just encountered today (again)

Andrew, February 03, 2006 - 3:09 pm UTC

Do not mask oracle error messages in you application. For instance, user application returns message -- 'Login Failed'. So call the DBA and tell him this. Now where does he start to look for a solution? Is it TNS, is it bad password, is it time to change your password, is database down, is it bad userid, is account locked? Pass through the oracle exception code, at least, to the user so that he can pass it on when he encounters a problem.

Likewise for 'Update Failed' , 'Lookup Failed', and so forth. I am sure more than one other DBA has encountered this bit of helplesness from applications.

To Andrew

Loz, February 06, 2006 - 8:18 pm UTC

I don't agree with showing all the detail of an error. Give your users a nice friendly message telling them their logon failed and to contact support. Put the gory details of it in the application error log somewhere. Users do not tend to be able to (or want to) accurately record error messages. Then best you usually get is "it errored" and they've lost the detail. Plus I'm always slightly reluctant to reveal the technology behind an app for some reason but I'm not sure there's a good reason for this.

Not to Do

SrinivasaRao Kandula, March 23, 2006 - 12:10 am UTC

Tom,
Your answers are always en-lightening me in one or other way.
Thanks for such a good forum and your Patience.
Great job.

Regards
srinivas

Not to do

???????, May 15, 2006 - 6:57 pm UTC

this thing sucked

Tom Kyte
May 15, 2006 - 8:25 pm UTC

but, and this is important, was it good for you?

;)

One I have seen far too often

Chris, May 18, 2006 - 10:19 am UTC

Do not store multiple bits of data in a single field, a classic has to be developers who use a varchar2(2000) field entitled "FREE_TEXT" and then insert pipe delimited data into it, usually something like this.

123|10-JAN-06|SOLD|BUSINESS|400|WIDGETS
124|10-JAN-06|BUY|CONSUMER|CANS|500

All in one field, to make it worse usually there will be differences in the number of bits of data stored in the field, the order they are stored in etc.

The reason usually given for this is its too hard for developers to get changes made to production database designs so these fields are inserted just in case we need to make a small change later. Of course you end up with insane substring queries to get information back out.

The worst cas eof this i ever saw was an application which recieved customer name and address details as an XML file and imported each XML record as a single varchar2 Field then parsed the xml on the way out of the database to get the fields. This worked rigth up untill anyone asked a qustion like which postal regions are our advertising campaigns most successful in.

Real Data

A reader, April 18, 2012 - 7:15 am UTC

Hi Tom

To tune you required real data volumes, but also you need real data, right ?

Tom Kyte
April 18, 2012 - 8:07 am UTC

you need representative data, yes.

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library