Mac, April 07, 2005 - 4:03 pm UTC
Asserting something without backing it up and expecting people to toe your line used to be the norm under communism and feudalism. When most of those people are waking up to reality, it is strange to see people expect compliance based on only their word.
Excellent Article
Warren McCall, April 07, 2005 - 4:45 pm UTC
Excellent article Tom. Hopefully the "hand-wavers" out there will actually take some time to read, digest and understand what you are really trying to say. As you say, a statement without evidence to back it up is, well, just a statement. I can't know whether it is true, or in what situations it may be true, since I do not have the evidence to review/test the statement.
I doubt you will ever convince the "hand-wavers," but here's to hoping! They are relatively smart guys, it is just their method that is lacking. I'm sure they would "learn something new about Oracle every day" if they followed an evidence based process as well. I know that I learn something every single day, and that is one of the reasons I love this stuff!
Great article
Ajay, April 07, 2005 - 5:05 pm UTC
Great article. I don't quite agree with Warren's statement "They are relatively smart guys, it is just their method that is lacking." (Do you have any proof of that,Warren? :-)). The hand-wavers actually come across as insular and unreliable.
Thanks for the article, Tom, and good luck getting an answer from Mike.
Lovely article!
Menon, April 07, 2005 - 5:07 pm UTC
Well, the link "recent thread on their website" is
not working (perhaps disabled by the site owner.)
I suspect some other links would also follow
suit!;)
Your example at the end also shows your humility.
Out of countless examples where
you have debunked myths and pointed out mistakes
made by others, you chose the example in which Jonathan
did it to you!:)
Excellent article and I think most people who
visit and learn from this forum would completely
agree with its contents.
Alberto Dell'Era, April 07, 2005 - 6:05 pm UTC
Well, you just described the "Method of Science" applied to Oracle ... i agree 100%, especially since I committed to that Method since i was a 10-year-old naive guy, after an hot debate with my "Catholic Drill Instructor" of that time.
Thanks to that method, and *especially* reproducibile examples and "peer scrutiny", we now have, after less than 200 years, electricity, airplanes, drugs, computers, after around 30,000 years since the start of mankind history ... looks like a terrific productive method, isn't it ?
It seems obvious to me that applying it to Oracle should be productive as well - and in fact, what method the top-notch and super-productive Oracle experts use ?
I've seen above that Warren McCall don't think that you will convince the hand-wavers; I agree, but I would like to quote something you said here : "facts historically have never ever won out over religion", but adding at the end "in the short run". In the long run, exactly the opposite is true; it took a bit of time, but now we know that planet Earth circles around the Sun, don't we ?
Great Article
rv, April 07, 2005 - 6:14 pm UTC
Empirical Evidence that Tom Rocks
Old Ada Guy, April 07, 2005 - 6:26 pm UTC
I know this goes against the theme of the piece, but I have to say that, as a traveling consultant I am frequently under pressure to solve some Oracle problem *immediately*. I have blindly copied solutions from all the "experts" mentioned here; only Tom Kyte is worth trying that approach with - the others will more often than not get you into trouble. As you may expect, looking for a quick fix I have occasionally applied a Tom solution out of context and hosed myself, but every time, I have gone back and figured out where I went wrong, and found that Tom's approach works.
April 07, 2005 - 7:06 pm UTC
Ada, the mother of PLSQL ;)
Great Appreciation
Michael, April 07, 2005 - 7:04 pm UTC
Tom,
Just wanted to let you know that I appreciate your website and the services you provide to the Oracle community. It seems that there has been a lot of Tom bashing from Don and company for no good reasons.
You have a loyal fan and follower here.
April 07, 2005 - 7:14 pm UTC
But -- don't trust me please, make me show you, convince you.
It is funny, there is a review of my book Effective Oracle by Design here:
</code>
http://www.bookpool.com/.x/6rrhp2aq56/sm/0072230657 <code>
Now the review states:
<quote>
Many love Tom Kyte for through Oracle examples, but the organisation of this book is wanting and the examples are long-winded and obtuse.
I trust the author and do not need long-winded code example to prove a point. Content is good, but the same book could have been done with half the pages.
</quote>
Now, everyone is entitled to their opinion -- that is fine. But I really do disagree with the "I trust the author". I don't trust myself sometimes.
I could have done the book with half the pages, but much of it would have been only half right, some of it downright wrong and the rest of it OK.
Proof and Evidence
A reader, April 07, 2005 - 7:12 pm UTC
I think "Proof" and "Evidence" are quite big words to throw around. I think terms like "Examples", "Experiments" or "Tests" would fit much better. I think such tests are called "artificial tests" since they try to isolate one thing to test. The important things are that the tests are small (ie. minimal) and complete (ie. self-contained) so they are reproduceable and that they are accompanied by commentary that tells us what they (are ment to) show and pointers to the documentation that tells us why they show what they show.
Another kind of test is the "real world test", where a system that is complete (in the sense that is as complex as a real system would be) is set up. The problem with these kind of test is that they only show one example of the "real world" and it is harder to understand what you see or why you see what you see. Therefore they might only be usefull to show that you are wrong (eg. stress tests) not that you are right.
None of the types of tests give any proof. They both give examples of what the software did in one specific run of the test. Both kinds of tests are usefull: The "artificial test" as an aid to understanding and the "real world test" to try to see if your model was to simplistic (and if so, you need to simplify it to find out why).
(I think this is on topic, but I confess that I have not read all of the threads. Also she whole "scientist"/"engineer"-thing seems to be way out of proporsion - it is just software.)
April 07, 2005 - 7:22 pm UTC
but how are they "artificial"? I don't like those labels (see: </code>
http://asktom.oracle.com/pls/ask/z?p_url=http%3A%2F%2Fwww.jlcomp.demon.co.uk%2Fthinking.html&p_cat=DEBATE_2005&p_company=10 for why. "artificial" has a negative connotation and "real world" has a positive one. They are both positive)
They are in fact both evidence (def: A thing or things helpful in forming a conclusion or judgment). Here is what happens when you do this. This is how much LIO you can expect when you do that. This approach latches like this, this other way does this.
Both tests give you "proof" -- not proof like "the square root of two is not rational proof", but proof as in evidence. "prove it to me", "make me believe", "let me see it for myself". Using the common defintion of proof out of the dictionary.
http://dictionary.reference.com/search?q=proof <code>
The evidence or argument that compels the mind to accept an assertion as true
I wholly agree with your last statement. I just want people to understand how the thing called Oracle actually works. Factually.
Tests
Ajay, April 07, 2005 - 8:23 pm UTC
"A reader" above said "and it is harder to understand what you see or why you see what you see."
Thats half the question right there, I think. What is it that you see? What tangible thing do you look at before you can say "I know what the problem is". Is it a trace file, is it some instrumentation that you add to the code, some output from a profiler, some script you run?
Its not enough to say "it happens on real world systems". What happens on real world systems? How do you know what is happening? Everybody here works on a real-world system. Tell us. What do you see, how do you see it, and why won't your methods work for us?
Jay, April 08, 2005 - 12:43 am UTC
Tom,
The entire Oracle community respects you for your hard work, time, efforts and of course most important "proofs" :-)
Thanks.
b.s. repelant
scott, April 08, 2005 - 2:02 am UTC
We have been honored to have one of our database installations evaluated by a "tips and tricks" author. The evaluation was based on output from scripts that highlighted exceptions to "rules". Exceptions were used as basis for change recommendations. The DBAs predicted the changes would make little to no difference. However, we could not communicate the importance of "user response time" and "resource utilization".
We went on to implement the recommended changes and realized NO performance gain (as measured by user response time, which was measured by a clock).
This is how I learned understanding what is correct is only correct if you can communicate it is correct. In cases such as these, I often communicate with individuals who are not well versed in database "speak", or worse yet they are in management ;).
In our experience, graphs are a good way to send the message. We've had some success with using OS performance monitoring tools, such as perfmon on NT. Our graphs outline hardware CPU utilization, Disk IO, Memory utilization, network etc.
My point is...if you are responsible for Oracle database(s), turn on OS performance monitoring, and keep reasonable history. This way, after implementing "tips and tricks author recommendations" you can prove they caused positive, negative or nil impact on hardware performance. Better yet, base your own conclusions and recommendations on the findings from proactive performance monitoring.
Another thought....get a stopwatch and time key application processes with representative load before and after implementing a change.
Proof is very enlightening.
April 08, 2005 - 6:55 am UTC
<quote>#Band keep reasonable history</quote>
And do that for the *database* as well, or even more so.
That is one of the things I really like about 10g, it has a long term memory, it retains this history and develops trends for you.
Scientific Method
Winston, April 08, 2005 - 2:18 am UTC
Most of people I worked with trust test cases vs claims. They like to see the proof/evidence vs just being told "it's a fact".
I believe in the Scientific Method you're using. I highly appreciate your time and energy spent on providing such a wonderful web site!
I believe Scientific Method is the only right method to solve database (computer software) problems.
Are there circumstances in which the Scientific Method is not applicable?
There are, of course, circumstances when one cannot isolate the phenomena or when one cannot repeat the measurement over and over again. In such cases the results may depend in part on the history of a situation. This often occurs in social interactions between people. For example, when a lawyer makes arguments in front of a jury in court, she or he cannot try other approaches by repeating the trial over and over again in front of the same jury. In a new trial, the jury composition will be different. Even the same jury hearing a new set of arguments cannot be expected to forget what they heard before.
Are there circumstances in database field (part of Computer Science) in which the Scientific Method is not applicable?
None
Carl Rogers Quote
A reader, April 08, 2005 - 3:00 am UTC
"The facts are always friendly, every bit of evidence one can acquire, in any area, leads one that much closer to what is true."
I wonder
Invisible, April 08, 2005 - 5:52 am UTC
Speaking about proof...
Just about any time somebody asks something on this website, you seem to be able to come up with printouts and numbers and so forth. Question: Do you have a HUGE stack of DBs sitting around to do all these tests with? You seem to be able to answer questions even about pretty old versions of Oracle - do you have one of each version or something??
Also... it must take you absolutely *ages* to do all these tests. And to write the miles of text to go with it. Do you actually get paid to do this forum, or do you have a "day job" on top of that as well? Either way, you must be one pretty busy puppy!
It's nice to be able to speak to somebody who reall KNOWS about stuff though. I always enjoy reading your replies. (Even to stuff that's nothing to do with me!)
Keep up the good work! :-D
April 08, 2005 - 7:55 am UTC
I have 7.3, 8.0.6, 8.1.5, 8.1.7, 9.0, 9.1, 10.1 on solaris (multi-cpu e450)
9.0, 9.1, 10.1 on linux (single cpu)
8.1.7, 9.1, 10.1 on windows (single cpu, vmware, for answering windows specific)
10.1 on linux (multi-cpu dell poweredge)
9.1, 10.1 on linux (single cpu, vmware, for demos on the road)
I use them all as needed.
As I said in the article:
<quote>
I truly believe that people wanting to provide information should back it up. Give evidence. Yes, I know,
we are all busy people and providing the evidence is hard, but if you want to be a published author, if you want people to accept your advice, you have to go that extra mile. The information you use should be real, not made up. Youll find you make far far fewer errors that way. You should be able to explain everything in your findings not most things and you cannot just ignore things that dont fit your hypothesis.
</quote>
It actually doesn't take ages (it used to, in 1995 and so on when I first started answering the questions. Now it takes very little actual time. If you understand how it works, you start to understand how to show how it works). It takes longer than just saying "it works like this, trust me". But if I did that, I'd be wrong much more often than I already am.
Here is the background behind this site, why it even exists.
</code>
http://groups-beta.google.com/group/comp.databases.oracle.misc/msg/d04bd2c3492776d0?dmode=source <code>
(someone updating the dictionary). We had an offline conversation (i'm 99% sure it was this thread, there are about 2 or 3 'update the dictionary to rename a column' threads around that time but this is the one I remember). I was told they got the idea from....
oracle.com
of all places. It was a user 'tip' submitted to Oracle Magazine. I got in touch with them and asked about it. Turns out they got tips but no one to check them out. We got this one pulled right away and I started reviewing the tips.
After a while they said, you know, we get questions too. So, I started looking at them. That was the beginning of the end as they say. January 2000 we started this and by April 2000 it was big enough to be a web site (used 100% email back then, no database even). That worked for about two months, did not scale very well (no way to turn it off!) So end of May beginning of June 2000 the first version of this site was truly born.
In addition to doing this, yes, I have a day job. I work with customers on various problems. I help sell the software. I try to promote the best way to use the software (which always starts with "understand it").
Excellent article
A reader, April 08, 2005 - 5:58 am UTC
This is *not* a true story.
One of my friends once passed me a procedure that returns 100 when 10 is passed in. He has been successfully using it
all over the world for 10 years to convert 10 to 100. He didn't tell me what it does but I assumed that it mulplies the input number by 10. I trust him and have also started using it all over the world to multiply a number by 10. I wanted to keep it a secret but some of my clients have started complaining that they are getting 4 when they are expecting 20.
:-)
are there more questions on asktom
A reader, April 08, 2005 - 6:17 am UTC
asked by tom,
or is this the 1st one ?
April 08, 2005 - 8:06 am UTC
No, I've asked questions of you guys in the past. (inputs from you). There should be at least two or three others floating out there.
More of a limitation of my software than by design :) That I have to ask myself a question, and answer it, to get a discussion going.
And about time too ...
Doug, April 08, 2005 - 6:38 am UTC
Tom, thanks for an excellent response to the ongoing arguments about the importance of proof. Having followed the threads that you referenced in your article for a number of weeks now, it beggars belief that people persist with the 'Honestly, I know, I've seen it happen and I'm a *really big* authority on Oracle, so you'll just have to trust me' approach. Long may the likes of you and Jonathan Lewis challenge apparent truths. I think we all learn by your example.
I'm not so sure about this statement, though :-
"Are there circumstances in database field (part of Computer Science) in which the Scientific Method is not applicable?
None"
While I recognise the underlying truth in this statement and think it's *particularly* important when writing books or publishing on websites, I'm not sure how much of my time doing real work on real databases used by real companies I apply the scientific method. There are times when I bypass such important steps because *I'm not given the time to use the correct approach*. For example, I'm afraid there have been a couple of sites where, rather than wade through the nightmare that is a particular database's shared pool problems (application-generated, of course!), I've gone with flushing the shared pool periodically to get around those problems. I'm fully aware that this isn't the best approach, and I wouldn't write articles suggesting others do this, but if it fixes the problem and (crucially) doesn't cause new problems, the business may be more satisfied with that than me coming up with a proper solution. Sometimes it's about delivering solutions that are less than ideal. But I emphasise, I wouldn't start recommending those solutions as 'silver bullets' for others to try.
But, in the interests of not giving any self-proclaimed experts the opportunity to say 'look - someone else agrees with us!', I'd better leave it there.
Long may dedicated professionals such as yourself improve the quality of everyone's work!
April 08, 2005 - 8:23 am UTC
Never say Never.
Never say Always.
I Always say.
Re:And about time too :Doug from Edinburgh
A reader, April 08, 2005 - 7:06 am UTC
Hi Doug,
<quote> I wouldn't write articles suggesting others do this, but if
it fixes the problem and (crucially) doesn't cause new problems </quote>
I think thats the important bit. I am sure there are lots of time when Don goes to rescue a project when its in complete mess. He uses some tricks to solve the problem quickly , tricks that he has learnt by experience by fixing similar problems in the past. I dont see anything wrong with that. That should buy him some time to analyse a few things like
Has it actually fixed the problem?( You have to isolate the actual problem to determine this)
Has it created some other problems?( You need to know what could go wrong with the approach)
Is it a temporary fix?
So before applying something you need to know what it does,what could go wrong with it and for that you have test and find out.. And also while writing an article you need to tell people about side effects of some approach rather than saying 'if you see this apply fix a','if you see this apply fix b'... If it was that simple I think the database developers would have included all that in the database itself..
Tom,
Is there anyway we can see the whole thread rather than the first question and answer while following up?
Regards,
Kal
April 08, 2005 - 8:26 am UTC
I'll put that request on my growing list of things to add ;)
I so use tabbed browsing that I generally open links in new tabs -- but I see what you mean.
Outstanding!
Invisible, April 08, 2005 - 8:15 am UTC
So... T. Kyte has a whole job to do *and* still manages to find time to write lucid, clear and logical followups to the questions posed here? Gees, if only Windoze could multitask as efficiently! ;-)
And wow - that is one impressive stack of DBs you got there! I couldn't even FIT all those computers in my office. ;-)
Interesting to read about how it all started... and I see from that thread, that even way back then, you were already writing in the beautiful clear style we all know and love.
Microsoft has the concept of "most valued professionals" (MVPs). If Oracle corp had this concept, I think T. Kyte would have to be pretty high up that list. :-)
Thanks!
Oracle ACE's
Dave, April 08, 2005 - 8:35 am UTC
Good article
Peter Scott, April 08, 2005 - 8:41 am UTC
Tom, to you have to wait for Tom to be not busy before submitting a question? ;-)
Your article, and some of the follow-ups above are well measured
Key points (to me) are:
We are not dealing with absolute MATHEMATICAL proofs, they are not really applicable to the types of problems we are confronted with in the real world, rather we concerned that our ideas are validated beyond reasonable doubt by empirical observation (try it, but measure it).
The world moves on, truth may only be transiant, bound by a set of conditions. In the middle ages there were proofs of geocentric 'solar' system, then a proof came that we circled the sun, and then we were just part of the universe... likewise, what I did on Oracle 7 may not be most sensible for 10g.
Rules change, but the method of "challenge, validate but always measure" remains constant
April 08, 2005 - 8:54 am UTC
No, I have his ear :)
Your paragraph resonates well with me. What I had the most problem with Don's "myth debunking" was that he was fighting paper dragons.
In his article </code>
http://dba-oracle.com/oracle_news/2005_3_28_false_proofs.htm he is claiming there are a bunch of mad scientists out there that have:
<quote>
the mistaken belief that SQL*Plus scripts and benchmarks somehow "prove" Oracle performance behavior
</quote>
Really, I wish he would name names and point to examples.
Also, in a article entitled "debunking", you would hope to see *some* debunking.
(note: if you cannot read his thread pointed to by that article, if you get a page from his forum that says:
<quote>
Sorry, an error occurred. If you are unsure on how to use a feature, or don't know why you got this error message, try looking through the help files for more information.
The error returned was:
Sorry, you are not permitted to use this board
</quote>
then you (like Oracle!) have been "banned" from his forums. When I'm behind the Oracle firewall, I cannot get to those pages. When outside, on the "public" internet I can. Used to be able to get there from inside Oracle. So, if you cannot read his material...)
http://www.dizwell.com/BB/viewtopic.php?t=112 <code>
Fantastic!
Invisible, April 08, 2005 - 8:44 am UTC
Wahaaaay - Tom is an ACE! 8-D
(And even the capitals are justified. ;-)
David Aldridge, April 08, 2005 - 8:49 am UTC
>> Also she whole "scientist"/"engineer"-thing seems to be way out of proporsion - it is just software <<
Yeah, my bad. The point I was trying to make was to refute this idea that you cannot decompose the "problem" of database performance into small sub-problems, and that this doesn't have to be some "hard" science in order to do so. Point was wasted though -- all the DKB had to do to make his point was to show that the scripts I tested large blocksize with weren't valid in larger systems, or even to give an idea of why.
The thread was a waste of time in the end -- I stopped getting any responses about half way through. neither dignified nor respectable, I thought.
those links work for me..
Peter, April 08, 2005 - 9:25 am UTC
...but only in the sense of having access to the Oracle DBA forum.
As aside on the subject of citation, I made a similar comment in an email comment to Jonathan Lewis on feed back on his 'Debate' piece!
Did you know...
Tony Andrews, April 08, 2005 - 9:36 am UTC
I don't know whether you have followed some recent discussions on the "Oracle DBA Forum":
</code>
http://dba.ipbhost.com/index.php?showforum=2 <code>
If not, you may not be aware that they have now outlawed links to AskTom in posts. The forum software has been tweaked to automatically change all occurences of "asktom" and "kyte" to "oracle", so that you see text like "As Tom Oracle says at oracle.oracle.com..."!
Apprently it is because you are so rude and obnoxious to your readers! This led to some lively debate, which has now of course disappeared from the site!
April 08, 2005 - 9:46 am UTC
and I've had reports that I'm no longer a banned word on that site.
But many people are banned from the site.
;-)
Invisible, April 08, 2005 - 10:07 am UTC
*Tom* is too rude??
Kettle. Black. Pot. Permute as necessary. :-P
Ever heard of a psychology concept known as "transference"? ;-)
http://dba.ipbhost.com/index.php?showtopic=1473
A reader, April 08, 2005 - 10:12 am UTC
An absolutely fabulous article
Robyn, April 08, 2005 - 10:14 am UTC
Way back when, I realized I needed to figure out a way to learn about things that I didn't even know to ask yet. It's relatively easy to track down an answer to a question you know to ask, but how do you know what you might not even think to question? By listening in on some of the better discussions groups, it's possible to see what other DBA's are concerned about and add those subjects to your list of topics to research. When those other DBA's provide proofs and the details of their testing, it becomes possible to run through the test and actually learn more about how Oracle works. And that is especially important for the beginners!
As Oracle changes, of course the results may change and what helps a database today could hurt it tomorrow. But, if a DBA can learn why something works, the knowledge can be applied to new systems where it is still appropriate. I read a lot and I buy a lot of books, and if the author provides the tests and the proofs, I can run my tests and apply what is appropriate for my systems. Cary Millsap's OFA paper was one of my favorites back in the 90's, and maybe the same rules don't apply when most of the systems I'm given to work with are SANs, but I learned that consistency and organization are key to a well managed system. And my god - when method R was published and I could document the expected results and actually deliver them - what a breakthrough!! I remember my early days of 'Well this worked for system A, maybe it will help system B.' and a scientific approach is infinitely better.
One of my former bosses used to describe Oracle performance tuning as half science and half black magic. But then again, magic is usually how humans describe what they dont understand.
Thanks Tom - for continuing to provide useful, documented tests that help all DBAs, especially the beginners!
Let's crash test Jumbo Jet
SFT, April 08, 2005 - 11:55 am UTC
Tom,
thank you very much for really good articles of yours and JL and for the opportunity to participate in the thread (without extra cost of registration and censorship:)
In my humble opinion the question(s) being discussed here is much wider. It concerns the whole software industry, not just Oracle and Oracle community. There are a lot of folks out there that deliberately promote the view that nowadays software is too complex to be dealt with in scientific way. Just look into MS documentation nothing near to the clear and comprehensive way the Oracle docs are built. The usual message from the guys on thats too complex side is I know how to tackle that, follow me, buy my stuff. And there are a lot of other guys that are too lazy to think on their own that believe they can buy any knowledge from an expert. Search for marriage on Amazon resulted in 21342 hits! How many poor creatures have bought and read this (mostly) junk?
Don on his site complained that most of supporters of scientific method are not native Americans. May be there is a problem with Americans? (Sorry, Tom, do not intend to hurt you). It looks like the people with math or engineering background tend to endorse the method as something obvious. Read e.g. a post from David Aldridge on Dons thread (why he did not put it here?). Ive heard that education in US is quite pragmatic and may be thats why US software community at large lacks the understanding of importance of proof and tends to do everything by textbook which is a good thing when you driving a car and not so good when you driving or even worse building a jumbo jet.
BTW I like this analogy with jumbo jet. (Again refer to David Aldridge post.) Why Tom and JL try to explain that their proof is not mathematical? Of coarse not. But take physics or better yet aerospace industry. No one would build jumbo jet without applying respective theory, sometimes very artificial indeed, and without unit testing. They learned it the hard way. No one in modern days would say heavier things tend to fall faster, Ive seen it many times because the guy called Galilei proved just opposite by very simple test some 400 years ago. Neither would Galilei have to apologize to another guy called Isaac that proved that heavier things really fall faster. Where the physics and aerospace industry along with it would now be without these small and simple proofs? Just imagin jumbo jet packed with crash dummies being crash tested to see how good security belts work.
Napoleon is said to have a motto Lets engage into the battle first and then see what will happen. Poor guy did not have test battlefields to try it first with a couple of platoons. He ended up with Waterloo. We have our test battlefields, lets use them
April 08, 2005 - 12:17 pm UTC
Well, I'm just a poor math major with a BA degree from the University of Pittsburgh (now no one has to wonder why Don picked that school in his "are all scientists created equal" </code>
http://dba-oracle.com/oracle_tips_credence_qualifications.htm <code>pitch :) Abstract Algebra. It was four years of mathematical proofs.
Guess the prove it to me thing just comes with the territory.
Not so sure I agree with your judgements of higher education in the US, but that is a subjective feeling on my part (not really qualified to comment, so I won't).
can you hear me now . . . . can you hear me now . . . . can you hear me now . . .
AJ Allen, April 08, 2005 - 12:08 pm UTC
Our daily lives are filled with proofs. When we get our cars back from the repair shop the mechanic tells us We found the problem and fixed it. Here is the busted part and your car no longer makes that funny noise since we replaced it (proof). If someone goes into the hospital complaining of chest pains, the doctor had better do some testing and prove his diagnosis before sending his patient to intensive care, or the poor patient could be faced with a big bill and lots of unnecessary treatment for what turned out to be a bit of indigestion. . . .Can you hear me now?. . .Can you hear me now?
When I first got into this business many years ago now, my first boss gathered all us beginner programmer/analysts together and told us that before we put ANYTHING into production we must be able to tell him how it WILL perform meaning we had to test and prove. Coming to him with words like should, expect, hope, and etc. would likely get us fired. After all, many people depend on these systems for their livelihood. . . .Can you hear me now?. . .Can you hear me now?
As is true for pretty much all people in this business, all the systems I support are depended upon by people in offices of factories for their livelihood (and sometimes for their lives). I WILL NOT do anything that could prevent them from making a living because the systems I support do not work as best they possibly can. That means test and prove, test and prove. That is the life of an IT professional. It does not matter if he is a DBA, Developer, system admin, consultant, (and I have had to clean up after some pretty shoddy consultancy work) or anything else in this business. test and prove, test and prove, test and prove. . . that is you job. . . .Can you hear me now?. . .Can you hear me now?
If there were any Silver Bullets that would fix any ill-performing database, they would have already been built into the system Hello! (as my 13 YO daughter would say) . . . .Can you hear me now?. . .Can you hear me now?
P.S.: It now appears that there is a new honor that Oracle professionals can earn being banned from Don Burlsons site. And you are doubly honored if you can get you whole domain banned.
. . . .Can you hear me now?. . .Can you hear me now? . . . .Can you hear me now?. . .Can you hear me now?
Still testing and proving after all these years.
P.S.
AJ Allen, April 08, 2005 - 12:17 pm UTC
To "I wonder" by Invisible from UK on April 08, 2005:
I have been at a local OUG conference where Tom was speaking. I saw him sitting in the back of the room, while waiting and after his turn to speak, typing away on a laptop with an antenna sticking out the side. I am sure that part of what he was doing was answering questions on this site. As 'Proof' I looked at his site after I got home and found some postings with time stamps during the time he was sitting in the back of the hall typing on his laptop.
So it looks like Tom will take any slack moment he has to catch up on helping the followers of this site.
Just came across my email
Thomas Kyte, April 08, 2005 - 12:59 pm UTC
Oracle source
Marcio Portes, April 08, 2005 - 1:43 pm UTC
According to Don
</code>
http://dba.ipbhost.com/index.php?showtopic=1396&st=15 <code>
<quote>
QUOTE
Maybe I am missing something (wouldn't be the first time biggrin.gif ) but the "proof" or script if you will has NOTHING to do with platform, right? I mean, an Oracle Database is an Oracle Database.
Yes, Sami, right on! Oracle-is-Oracle, but there ARE differences in the Oracle "C" source code on each platform.
One reason why Oracle is super-successful is because each of the 60+ "platform versions" of Oracle (for any Oracle release number) are each optimized for their specific platform, to leverage on the OS features (at the Operating System Interface (OSI) layer).
Again, I could be wrong. It's just a "theory"!
Now it's obvious hardware makes a huge difference when we are talking about Oracle performance proofs, and a simple "set timing on" tells the story.
</quote>
Is it true? Does Oracle have one C source for each OS?
Regards, Marcio
April 08, 2005 - 2:03 pm UTC
There are two bits to Oracle
The portable bit.
The OSD (Operating System Dependent)
The OSD layer is very small relative to the rest. Very very small.
But when you look at the things I talk about -- you would be hard pressed to find OS differences.
AND, since I generally give you something to use to test with, you could FIND ANY. (tell me, if you read a glowing paper like his "Oracle users RAVE over multiple block size tablespaces", do you see a single caveat about OSD? Anything to test with to see if they'll help you? When they might be
a bad thing?)
Once again, his comments just back up
why you need something more than the verbal advice of an expert. Not the other way around.
I think the bee that has gotten under his bonnet in this case was the "predictive" thread (you'll notice a flurry of "science bad", "scientists not equal", "be careful of what you prove", "sqlplus scripts are hooey" articles dating after:
</code>
http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:35336203098853 <code>
In there, you will find no "performance proofs", just *facts* about how Oracle works.
If you have an index as I described that is inserted into randomly, rebuilding it will
a) cause it to generate more redo for the next couple of days as it fattens up again.
b) use more cpu as we fatten it up
c) access the blocks in current mode (for the splits) more
in an OS independent fashion (the amount of CPU used might go up or down from the observation there, but -- the CPU used will go up pretty much across the board, we have to do more work to fatten this index back up)
How the index gets flushed to disk, that might vary by the OSD layer.
How the latches used to do the consistent/current mode gets are implemented by the OSD layer might vary.
How lgwr writes to the OS might vary by the OSD layer.
But the facts above will remain.
Facts like "in order to scale in Oracle, one needs to pay attention to latching" are true, regardless of OS.
Facts like "a consistent get does latching" are true, regardless of OS.
Facts like "hard parsing is a very expensive operation requireing many latches" are true, regardless of OS.
The core of the Oracle product runs pretty much the same from Windows to Unix to the Mainframe.
All I see being asked for on that thread over and over was "one example of these bad performance proofs"
All we ever heard was:
<quote>
...
Reader: So, which of the hundreds of test cases that Tom has -- either in one of his two books or on his site -- don't hold up in the 'real world'. I'm curious to know which ones you means specifically for I've seen lots of his examples and they seem to hold water with me. Can you 'scientifically' tell, show or otherwise relate which ones don't hold up?
.........
DKB: That's the whole point. I don't really care enough to find out.</quote>
If he doesn't care enought to find out, I'm left wondering why he started the thread in the first place. Maybe just trying the old "if I say it, and I keep saying it, someone will believe me" trick.
If his "DBA war story" in the predictive thread even made sense (hey, I ran a delete purge and it puts space back into dba_free_space -- that would be magic)...
If he could just show us an unbalanced freelist, that he sees all of the time...
He could spend less time trying to convince people that having evidence is bad.
David Aldridge, April 08, 2005 - 1:59 pm UTC
>> Now it's obvious hardware makes a huge difference when we are talking about Oracle performance proofs, and a simple "set timing on" tells the story. <<
You know, if that were such a widespread impact on performance and Oracle internals (which I don't believe it is) then you'd expect to see some qualification of general advice, such as here </code>
http://www.dba-oracle.com/oracle_tips_multiple_buffers.htm
There's no mention of platform at all though, except to imply that "small, artificial single-user experiments" are not valid.
Actually though, that seems like a realy minor criticism of that article, when there are so many other low-hanging fruit to choose from. If you read the whole of that thread referenced
http://dba.ipbhost.com/index.php?showtopic=1239&st=45 as well as the associated
http://dba.ipbhost.com/index.php?showtopic=1396&st=0 <code>then you'll be left without any idea of why the single user test is not valid -- no theory, no wild guesses, no practical demonstrations at all. Hey, don't say I didn't try -- I asked several times for even the vaguest theory about why it wasn't valid, but I got nothing back.
I personally suspect that there are benefits to storing indexes on larger block sizes in some circumstances, but I saw no attempt there to promote any discussion of what those circumstances might be. And at the end of the thread the basis of the "20% reduction in i/o" comment is looking decidedly shaky. Doesn't sound like Steve is exactly "raving" about it, though I'd be happy to discuss it further with him.
April 08, 2005 - 2:44 pm UTC
doh, you beat me to the punch David. I used the same example!
I'd still like to know how the 20% reduction was measured.
If they measured from within Oracle -- the 20% reduction could actually have implied an INCREASE IN PHYSICAL IO over all. If the count of block IO's performed goes down by 20% but you've DOUBLED or QUADRUPLED the size of the data you do IO on....
David Aldridge, April 08, 2005 - 3:33 pm UTC
>> If he doesn't care enought to find out, I'm left wondering why he started the thread in the first place. <<
... or, indeed, why the forum at all? I hesitate to run to the dictionary, but </code>
http://dictionary.reference.com/search?q=forum
...
2. A public meeting place for open discussion.
3. A medium of open discussion or voicing of ideas, such as a newspaper or a radio or television program.
...
"Open" is the key word for me ... someone wants to slam my opinions in public then that's fine by me, because as long as I have right of reply then I feel that "truth will out" as they say in The Old Country. If I'm wrong then I'll just "suck it up" (as they say in The New Country) and get on with life. Hence
http://oraclesponge.blogspot.com, <code>where anonymous posters are (currently) free to express their opinions.
Ah, the internet is a wonderful thing.
Manifesto
Alberto Dell'Era, April 08, 2005 - 5:14 pm UTC
What about a short and concise Manifesto ?
I mean - I enjoyed reading, but
1. I already share the same mindset
2. I already know and agree to everything discussed
3. I have a "fair" command of English (for a foreigner)
4. I was willing to invest some time ...
What about a casual passer-by: would S/He get the message, or be scared instead, and go away, to fall prey of the easier-to-read papers of the handwavers ?
Perhaps a, say, five-points Manifesto with short explanations, perhaps with links to detailed discussions, may be more effective. IMHO, of course.
April 08, 2005 - 5:37 pm UTC
But isn't a manifesto all about hand waving? Not sure, let me think about it.
Alberto Dell'Era, April 08, 2005 - 6:31 pm UTC
> But isn't a manifesto all about hand waving?
It is, but only when not providing explanation.
Actually I was thinking about something like (just a fuzzy idea in the back of the head, not a blueprint):
1. test cases enable "scrutiny of peers".
That
a) makes it possible to advance knowledge at light speed. For some examples, click here [link to examples like "Jonathan Lewis raised his hand and asked Tom Kyte if he knew about the commit optimization"]
b) lessens the number of errors [link to similar examples]
2. Rules of thumb without evidence, without ways to test them before implementing them, are dangerous, seriously dangerous. [link to examples of ROT gone mad]
etc
---
A way to get the "big picture"; something that can be read in 5 minutes, with links to details. People disagreeing with a point may follow the link to check out the "why", people agreeing may skip the link and perhaps get back later to read the details of the story ...
Obviously, the really really hard part is choosing the right bullet points (and perhaps sub-bullet-points), and calibrate the level of detail ... choosing whether or not to provide some explanations in-line ... etc.
It may foster discussion as well, perhaps the Manifesto (better to choose another name btw ;) may even evolve with the contribution of readers.
April 08, 2005 - 6:37 pm UTC
Yeah, Manifesto rings bad in the ears for sure :)
I could see the headlines produced by that one.....
Peter Kinsella, April 08, 2005 - 7:37 pm UTC
My personal favourite
Page 5 (Mar 29 2005, 02:19 AM ):
"... and the script had a bug. I tested it (using the test instance), and trusted the author, but it was ugly when the table became corrupt after the reorg, causing a major outage, and it was 100% my fault.
I didn't get fired, but I got yelled at, big time. . . . Again, I trusted the author and the O'Reilly QA team. . . . Big mistake in this case. . . ."
******************************
Page 10 (Apr 2 2005, 01:43 PM)
"Me, I trust folks who are out-there tuning Oracle every day, and I don't need proof from someone that I trust."
BTW; does Tom K = Sami B ;-)
April 09, 2005 - 7:29 am UTC
Yeah, I thought the whole "I used a script once and I didn't know what I was doing, darn script corrupted a table. Therefore scripts are evil" is another "false causality" thing.
Don observes "I ran a script". "Table got corrupted by bad script".
Don hypothesizes "Therefore scripts are bad"
When the real cause could have been "PEBKAC" (problem exists between keyboard and chair).
But amusingly enough, it does help bolster the concpet:
</code>
http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:35336203098853#35502654508980 <code>
It is my personal opinion that I could have added nothing to the thread.
I think it speaks for itself, Don's been doing the "heavy" lifting here. (well, it speaks less and less for itself as the thread has been put on a diet, bits of it disappear... guess he doesn't remember we all have "file save as". I've been re-reading the Dune Series (Frank Herbert). He starts each chapter with a little quote, chapter 1 of chapterhouse dune begins with something I think is appropriate:
Those who would repeat the past must control the teaching of history:)
You article is one of a kind, brutally honest
raul, April 08, 2005 - 9:10 pm UTC
I've been working with Oracle since V5, SQL*Forms 2.3, SQL*Reportwriter days...Trust me :)
Back in 1996, an Oracle Tech support person told me to do period reorg of the production database. I asked him how? He said 'do a export/import'. And so I did ....next day I went around asking the users if they noticed any improvement in performance of the Powerbuilder application ...well, they did not notice a thing :( ...so much for the 1 extent myth.
What I learnt something today, if it wasn't for your site, "I'd be wrong much more often than I already am." :)
Cheers!
Excellent Food for Thoughts
Robert, April 09, 2005 - 3:20 am UTC
heheh seriously, is this really Tom Kyte "asking" Tom Kyte ?
<scratch_head>
hehe if yes, is this the very first time ?
April 09, 2005 - 7:40 am UTC
Can we have a sensible debate?
SFT, April 09, 2005 - 5:54 am UTC
So back to square one, to JL question. So far there is no debate on this thread, because everybody voicing an opinion here agrees that hand-waving is bad. Is there much debate on DB thread? I do not follow it, but I doubt it is sensible if any.
Are the hand-wavers just so stupid that they cannot understand the value of scientific method? Substitute hand-wavers with salesmen and you will see the question has no sense.
Salesmen hate inconvenient questions, all these yeahbuts, and they are trained to fend them off, not to give substantial answers backed by research and such.
Scientists accept yeahbuts, sometimes even with thrill, like Tom.
Is it possible to have sensible debate between salesmen and scientists? I dont think so. Its not that I do not respect salesmen, they just have different goals and different means to achieve it (remember that Protruding nasal hair is prohibited on </code>
http://www.dba-oracle.com/dress_code.htm <code>?)
But most of people are neither pure salesmen nor devoted scientists. Therefore its worth trying to educate as much software folk as possible and convince them that for their jobs its better to be a scientist, and reserve salesmans instincts for special occasions.
So keep on Tom, educate them (and us:)
April 09, 2005 - 7:52 am UTC
Well, I hope I gave some evidence (not just opinion) that hand waving is bad in the above article.
And I didn't say hand wavers are stupid. There are not smart hand wavers and potentially smart hand wavers and really smart hand wavers out there. (the magic of three, when asked about a performance tip "what if I do 'x'" I reply
one of three things will happen, a) things will go faster, b) things will not change measurably at all, c) things will go really badly wrong. Now, tell me about your situation and we can start to figure out which of a, b, c will apply to you...)
I did say:
<quote>
I truly believe that people wanting to provide information should back it up. Give evidence. Yes, I know, we are all busy people and providing the evidence is hard, but if you want to be a published author, if you want people to accept your advice, you have to go that extra mile. The information you use should be real, not made up. Youll find you make far far fewer errors that way. You should be able to explain everything in your findings not most things and you cannot just ignore things that dont fit your hypothesis.
</quote>
You can hand wave all you want, just don't expect anyone to take you seriously. And expect others to say "yeahbut" ALOT
(oh crud, not that dress code thing again. for a guy who is proud of the material on his site, pointing to that really sets him off for some reason)...
Re: Just came across my email :-(
Jerome, April 09, 2005 - 7:13 am UTC
Talk the Talk against Walk the Talk
Tarry Singh, April 09, 2005 - 11:27 am UTC
Tom,
The last couple of weeks here at your forum and at other places have been exciting(not to me beleive me) for people who have given a lot of publicity to other oracle(World Reknown, they say themselves) guys.
So my questions are..
1. Why engage in such an trivial "I know better than you" kind of fight?
2. There are people who're willing to listen to cr*p and others who want to "see" what someone else has to sell, be it advice, product etc. They will all be there, so why refute and debate on someone else's argument when it already has enough holes to fill the universe with another blackhole(sorry for exxageration)?
I come to this site to seek answers because no matter who says what, be it you or others, so I can work that piece of advice on my own system and see it really does what it should be doing.
All the oracle users who come here have one thing in common, they're not here for mushy stories, oracle bugs, check educational qualifications, cheap gossip but pure oracle related issues.
I have put up a simple poll on an independent oracle forum to see who people would like to listen, should they feel the need to </code>
http://www.dbasupport.com/forums/showthread.php?s=&threadid=46966 <code>
Cheers
Tarry
April 09, 2005 - 12:17 pm UTC
Not trying to "I know better than you", but rather explaining my point of view.
Because people actually do read their stuff, believe their stuff, I feel rather compelled to respond with something logical -- to explain my point of view.
nothing more, nothing less.
This article really explains the way I write, and why I write that way.
I truly believe you need to back things up.
I truly believe what you write should be factual, not made up.
I truly believe rules of thumb, without any caveats, are dangerous. Especially to beginners.
I truly believe that inaccurate material should be pointed out (and have actually tried in the past to do it privately, but that is another story). I welcome ANY and ALL corrections to material.
I truly believe this is a technology related topic, this very thread. It goes to the heart of it.
And I'm going to say it. (do you think that misinformation should be allowed to go unchallenged when people ask about it?)
Not sure if I agree with the concept of the poll, especially with the way the names are listed (not everyone will understand the "Some Grandpa DBA 30yrs exp." reference for example)...
Grandpa Reference is from Jonathan Lewis's comment in his article
Tarry Singh, April 09, 2005 - 12:38 pm UTC
On Don's interview with some one who had 30 yrs experience as a DBA. But anyways I've just asked the moderator to remove that reference.
BTW a poll is anyways a good way to ask people what kind of material they like to read. When I go looking for a book at amazon or a local shopping store, I do my bit of homework and I SURE do wanna know whos wrote the book and what good is he/she in that subject.
April 09, 2005 - 3:32 pm UTC
I sort of figured that was what it was but (at the risk of being pegged a scientist again :) the poll probably will be skewed
a) the grandpa thing would have made it "obvious" which way the poll taker was leaning (or at least it give the impression of leaning that way, that is how I read it). sort of like labeling something a "false proof", what more need be said after that ;)
b) unless you also advertise it on a site that draws the opposite perspective, the results will be skewed in favor one way.
A reader, April 09, 2005 - 8:52 pm UTC
Tom,
I am follwing a tug of war between you and Don Burlson & Co. Don's comment about his so called MIT degree etc...
I am not an Oracle expert but I am here to support you and your knowledge which is helping me and many others from last 5 years. As a person you look (haven'e meet you so far) gentelman and have very high respect for you.
Keep up the good work.
Thanks
April 09, 2005 - 9:35 pm UTC
I don't think Don ever claimed to have a degree from MIT.
Why wouldn't I respond?
Mike (No alias) Ault, April 09, 2005 - 9:43 pm UTC
To the gentleman who said "Good luck getting Mike to respond" why in the world would that be said? As Tom knows I readily respond, both to email and when I am made aware, on forums. Unfortunately I don't track daily in all forums everywhere to see who is saying what about me. Life is too short to get flustered over childish behavior in others.
I agree with the scientific method. However, one of the foundation stones of that method is that the test must as close as possible mirror the system that is being tested. That is the number one issue I have with single user, small database tests. A single user, small database test cannot be used as proof on a large multiuser database.
I tend to use the TPCH database when I perform testing (for example, does RAC perform better than multi-CPU systems, does SSD out perform SCSI arrays, what do configuraiton changes in SSD/SCSI do to performance?) I also perform testing at the component level (How does hdparm affect disk performance in Linux?) All of these tests, the scripts used, the results and timing data are readily available in the various books I have recently published.
In my books, I provide scripts, results and explanations. Anyone can repeat any of my tests on their systems. I was one of the first to document the issues, using scripts, that befall the shared pool and how to correct them. I also was one of the first to show how to look into your buffer cache to see how the blocks were actually used and not depend on such hand waving as hit ratios.
Before you paint everyone with that brush in your hand, better review the facts. Like Tom, I have given bad advice, been called on it and have apologized and retracted as required. That is what happens when you put yourself out in the public domain. No one is right 100% of the time.
When I suggest something, I provide the background data on why. It may be a reference at the end of the article or a link. In this world, you take good advice wherever it can be found and apply it. I even take occasional tips from Rogers and Foote and use them.
Yes, experts will disagree, on tips, on methods, on usage. I try not to make it personal if I can. However, I am human and at times may do things contrary to what I should, don't we all.
At the same time Tom is doing his answers while watching the presentations, I am proofing the latest books, answering client calls, helping other DBAs and traveling nearly 80%. Afraid it doesn't leave much time for generating pages of proofs except when they are germane to my current tasks. I don't get paid a salary, only hours worked as a 1099 consultant, each hour I spend answering questions from non-clients, writing presentations for users groups, articles for online and paper publications and trying to deal with online attacks takes away from my family and job and incidently, from time that could be spent testing and learning against the Oracle database.
One thing to remember, the real questions (other than those from folks that don't know where to look for answers, or who don't want to look for answers) usually don't make it to the forums, to TARs to asktom until they have been beaten to death by (or beat to death) guys like me attempting to solve the issues often against a deadline.
Anyway, enough said.
April 10, 2005 - 10:18 am UTC
Mike,
While I got your ear.
Regarding your recent "IMHO" article. (oh cool, I see it is now "a warning" to all that enter:
<quote src=burleson </code>
http://dba.ipbhost.com/index.php?showtopic=1473#entry5574 >
Good answer. Mike also has important warnings about the askom site:
</quote>
) watch out all, you might get hurt here now.
In that "warning" you start with a statement about "credible"
<quote>
It's can be very difficult to get credible information from some of the Oracle-owned forums such as asktom.oracle.com.
</quote>
would you care to elaborate on that at all? Seems the article has nothing to do with credibility at all?
It does however seem to be a reprint of a bunch of email Don sent us here at Oracle (pretty much point for point). Now, I know that Don would never "bash" anyone in public, rather -- he wants others to do it for him (picked that up from the proof thread where he used an anonymous 3rd party DBA with 30 years of experience) so I'm sure he is thankful you provided him the outlet to do that.
The whole "extra cost Oracle book" part in there was funny. (what, you don't write and expect to get compensated for it? Yeah, I'll get right on that free book deal...). (Oh, but I did point the reader to a slew of free books, over 10,000 pages of free books, but ship them one? I don't think so, no)
You see, this forum here is not moderated. I do not remove posts that just express opinions. I do not review material prior to its release (I do not review or even see all of the posts). When asked in the past, I've actually removed from view posts at Don's request.
As for
<quote>
the asktom web site is written in HTML-DB and it would be a simple code change to require positive identification of respondents
</quote>
<b>you were only kidding that it would be a simple code change to require positive idenficiation of respondents right?</b> Come on, as a technologist yourself you <b>must</b> know how hard that would be. Even Don alludes to it:
http://dba.ipbhost.com/index.php?showtopic=1396&st=30#entry5113 <code>
when one of your own users demonstrated how trivial it is to become anyone they want on
your forums. If you know of a simple code change that could "positively identify people", you would rich many times over. (it was also very curious to me that both you and Don posted many times after the alleged "identity theft", many times... 8 times in fact). So no, I didn't expect you to constantly monitor the forum,
I already knew you were doing so...and thanks for not using IM speak in your followup, much appreciated. (funny how the people using IM speak do in fact appear to otherwise have a full command of the english language, and the ones that do not have the command of the english language don't seem to use it -- as they haven't learned the language enough yet to figure out how. Those (without the command of the language) people are typically the ones most grateful that I make fun of IM speak since it makes reading really really hard for them)
I know Don had the list ready for you to go based on his email from Feb this year <quote>"There are people who feel that the tawdry c**p that you allow to be published on "asktom" makes all of Oracle Corporation look bad (e.g. "I think your vowel keys are stuck", etc., I have compiled an 8-page list)."</quote>
Evidence, any day of the week!
Kashif, April 10, 2005 - 1:29 am UTC
Hi Tom,
I really don't think you need our opinion/input on what is a universal truth, that the only way you can make any assertions about anything is by having hard facts to support those assertions. I'm surprised we even need to have this conversation and that people even need convincing, especially considering that we're all IT folks here and logic and reason rule above anything else, but anyway.
My boss once said that if I create a view using the "create view v as select * from t" syntax, I would never need to recreate such a view to include any new columns added to the table T. I thought to myself, that doesn't sound right, but I wasn't certain, so I ran a quick example, and of course he was wrong. The beauty of this exchange was that it was the simplest of things to disprove, and yet he made that assertion and fell into the trap of "I know it so I don't need to verify". And he had 10 years more experience than I had. Which goes to show that years or education don't count for much if you can't get your facts straight.
You always have and I expect always will keep it "real", no BS or irrelevant rants, just the cold hard facts, like a slap in the face first thing in the morning (at least for me when I get on to AskTom as that first cup of coffee enters my system)! I think most people know who the pretenders are, and who the real deal is.
So rock on!
Kashif
What's wrong with small scale testing?
Mark J. Bobak, April 10, 2005 - 2:16 am UTC
Mike,
I don't think there's anything fundamentally wrong w/
testing (particularly testing concepts and testing
"how the database works") using small scale testing,
even laptop based testing.
For example, things such as:
- Indexes do not become unbalanced over time
- INSERTs will not wait due to ITL shortages
- What's the largest b-level a B-tree index in Oracle can support?
(Definition of "balanced" above being: an index in which
traversal from the root node to any one leaf node is always
the same number of hops.)
These types of questions can (and have been) answered
using small test cases. These types of things don't
magically change under load. However, I would advocate
re-running these test scripts on each version of Oracle
you enocunter, because, as we all know, it's a moving
target. Oracle developers are always tweaking and improving
and trying to make the database better. The perfect
recent example is the bulk collect demo that Tom mentioned
previously. Suddenly, in 10g, the optimization happens
automatically, under the covers. By having a simple
demo/test script available, anyone can run it and try it
themselves. The result will either be confirming what
the author of the test claimed, or, perhaps it will
re-open debate or questioning. (That's a *good* thing.)
Mike blunders on...
Howard J. Rogers, April 10, 2005 - 3:09 am UTC
"I agree with the scientific method. However, one of the foundation stones of that method is that the test must as close as possible mirror the system that is being tested."
Right.
Last time I looked, I wasn't cute or furry with a long tail and a twitching nose, nor do I have pink eyes and a tendency to urinate everywhere to establish my territory. So all those tests pharmaceutical companies do on mice to establish the safety of their drugs before they begin randomised double-blind tests on real human beings can't be "scientific". They don't even get the number of legs or whiskers right! Not exactly a "close as possible mirror", I think even Mike might agree.
Point is, all the scientific method asks is that you falsify a hypothesis. If 1000 mice drop dead, it's a safe-ish sort of bet that double-blind clinic trials on real people are likely to be a bit "ambitious". If you rebuild an index into 32K blocksize on a laptop setup, and suffer performance problems immediately afterwards that weren't there before, it is similarly a fairly safe bet that sweeping advice that 'all indexes should be in 32K blocks'" must be wrong -that some qualifying factors need to be at least investigated, assessed and documented. The 'non-realistic' nature of the test platform is irrelevant for the purposes of establishing the need for caveats and qualifiers.
"Like Tom, I have given bad advice, been called on it and have apologized and retracted as required." Uh huh.
So that post you made on 10th May 2004 in the "Index Management" thread at comp.databases.oracle.server, where you wrote, "The problem lays with the fact that this was not a complex environment involving millions of rows and many users, but for the time being I will concede the point that a simple index rebuild, in a simple environment, will not result in a change to clustering factor for a standard b-tree type index."... that counts as an "apology and retraction", does it?
Allow me, please, to tell Mike what *does* count as an apology and a retraction: "I didn't research the matter properly; I didn't understand the clustering factor properly; I gave bad advice; I was wrong to do so, and I apologise for having done so".
Either that, or a year on, I think we're entitled to know the results of Mike's promise in the same post to "endeavor to check this in a more complex IUD situation than I can simulate on an XP laptop and if anomalous behavior (i.e. reduction in CF) does occur, I will report back to the list with the scenario and the statistics." Don't you?
Of course, for him to do so would be a bit awkward, since it would mean that a quick 3 minute test on a crummy laptop provided as much accurate information in this regard as nearly a year's constant work on big, realistic, complex systems. Which would in turn mean admitting that test-beds can indeed be useful... so no wonder Mike has been silent on the topic all this time. He has the consistency of his position to defend... and never mind about hob-goblins and foolishness.
"At the same time Tom is doing his answers while watching the presentations, I am proofing the latest books, answering client calls, helping other DBAs and traveling nearly 80%."
I think that probably explains a lot. If Mike would just stop being so busy, and spend some time doing basic research about how Oracle works, he wouldn't have made the sort of howlers he has (and the clustering factor is just one of them), and he'd be much more able to retract them properly in a timely and unstinting way. I think they call it "More Haste, Less Speed".
Less books, Mike... more technical knowledge. Please.
Once again Howard is stuck in a time warp
Mike Ault, April 10, 2005 - 6:19 am UTC
What part of "and if anomalous behavior (i.e.
reduction in CF) does occur, I will report back to the list with the scenario and the statistics." is not clear? Obviously since I have not reported back, it has not occured...duh.
You are beginning to act like a one trick pony in this regard Howard. Get over it.
Poll modified(Thanks Jurij Modic) at dbasupport.com
Tarry Singh, April 10, 2005 - 8:56 am UTC
Mike or Don will never have the dignity to trully admit they are (were) incopetent
Jurij Modic, April 10, 2005 - 10:11 am UTC
<quote from Mike Ault>
At the same time Tom is doing his answers while watching the presentations, I am proofing the latest books, answering client calls, helping other DBAs and traveling nearly 80%.
</quote From Mike Ault>
Mike, that's the most pathetic excuse you could ever made. I believe Tom is indeed paid by Oracle Corp., but not for answering questions on this forum - I believe his engagement in this forum is primarily his own free will, taking his own free time. So to speak, he is doing it "pro bono". While on the other hand, you are doing most of your (not so frequent) contributions to various public forums more or less with the sole intent to promote yourself and your employer.
It is not very surprising that lately you and Don became very sensitive about any unwelcoming comment about your publications. You realized that Internet is becoming a driving force of the knowledge spreading and that Internet is now enabling the whole Oracle community to notice when the emperor is actually naked in his new clothes. You must have realized this might lead to a devastating consequences regarding your consulting and publication business. So it is no wonder that you use such cheap and dirty tricks as you and Don are practicing lately on your websites. Pathetic!
<quote from Mike Ault>
One thing to remember, the real questions (other than those from folks that don't know where to look for answers, or who don't want to look for answers) usually don't make it to the forums, to TARs to asktom until they have been beaten to death by (or beat to death) guys like me attempting to solve the issues often against a deadline.
</quote from Mike Ault>
You are not serious, are you? You really think that nowadays, anyone with at least a little knowledge about what is available on the Internet *for free*, will not at first try to get some information from forums, TARs or asktoms? You really think they will first run to seek the answers from your's and Don's "modest-priced" consultancy? Well, less and less, that's for certain. Thanks to Tom, Howard, Jonathan and alike....
April 10, 2005 - 11:18 am UTC
by the way: unless it is clearly an Oracle presenter giving an Oracle presentation I could myself give, I try to watch and listen to the presentation. Unless I'm fixing my slides for my talk that is -- I tend to procrastinate sometimes.... I've learned as much as I've taught at user group sessions (in Indiana I learned a cool use for the resource manager I might never have thought of, as a resource usage "tracking" tool -- not as a thing to implement a policy, just to look at who is doing what)
David Aldridge, April 10, 2005 - 11:48 am UTC
>> You really think that nowadays, anyone with at
least a little knowledge about what is available on the Internet *for free*, will not at first try to get some information from forums, TARs or asktoms? <<
Back in '98 I worked with a company in which the Oracle support team had a surreptitious AOL dial-up account for downloading essential patches etc. -- corporate IT had banned all internet access company-wide (except for e-mail for some). So there were these small enclaves of backwardness then, and I bet there still are.
Even when the technology is there, there are plenty of people who have not heard of TK, JL etc., and who will not even buy their own copy of an Oracle book unless the company pays for it. Depressing perhaps, but also fertile ground for consultancy.
There's a point that Mike A. makes fairly often that I'd like to give out a shout of support to -- there are companies with severe problems out there, who do need an urgent fix but who don't have source-code access. Some of those have more money than time, and staff without any new knowledge since 1992, and a consultant's advice to rebuild some indexes and slap an order in for another 8Gb of memory may be appropriate. It's not a long term (or even a medium term) fix, but if it works well enough that their e-commerce site stays up then that's all the managers care about.
Ethics may demand that the staff is then educated on how to get access to the resources they need to do their job properly (for when the next crisis shows up) but even then education may not always be welcome. Not every DBA or developer is the sharpest tool in the box, and for some reason they tend to congregate in particular organizations. Am I right, men?
So, long story short there are dysfunctional organizations out there who need external help. The staff in those organizations are _not_ reading this thread.
April 10, 2005 - 12:06 pm UTC
Yes, I get direct email from people that have heard of me, but are not permitted internet access at all from work (i'll generally respond to them, but almost all others get a canned response saying "please use these freely available resources" -- just saying that so people don't start emailing me directly).
I agree that sometimes a hardware shot in the arm can fix things. Definitely. But you need to know the type of hardware you need (reference: Don's example of the DBA mislead by a proof in his thread. You remember the guy who was tricked by the evil proof into thinking that all problems are disk related). You still need to take a studied approach to the problem in order to fix it. there are no silver bullets. If a system is say 100% utilized CPU wise, is that a call to add more CPU?
Given that much information, the only answer is "I have no idea", because no matter what you say -- I'll give you an example where the opposite is true (adding more CPU can do one of three things
a) make it better
b) make it worse
c) have no effect whatsoever
So, to that end, as long as the consultant knows what to look for -- sure.
Getting Mike to respond.
Ajay, April 10, 2005 - 12:00 pm UTC
Mike,
I'm the "gentleman who said "Good luck getting Mike to respond"". Why in the world whould that be said, you asked. The reason is this: You're claiming that single user tests don't provide results that apply to "real-world" systems. The follow-up questions that have been asked are 1. Why don't the results apply? and 2. How do you know this? What do you look at to determine that the results of a single user script have crossed the "scalablity barrier" and no longer apply?
These questions haven't been answered so far, as far as I know. If I am wrong, please 1. accept my apologies and 2. point me to where these questions have been answered.
Ajay.
Hmmm dysfunctional: Every dysfunctional patients needs a functional surgeon
Tarry Singh, April 10, 2005 - 12:10 pm UTC
David,
I hate going off tangents but...
A company policy that prohibits employees to
go on internet because it's a time-sink is one issue but a company that does not realize the importance of core systems running in their shops are myopic. OK facts stated.
But NO company whatsoever deserves the 'beat me to death, Dear Consultant' because we're dysfunctional. no matter what, they'll soon find out that 'hey, darn the rebuild didn't work' or if there's some low level sys admin 'hey, the system is again slow'.
Bottom line: Truth will come out eventually and the consultant shop won't be called anymore.
True : People like TK,JL are not known in mid sector but every customer knows that if you have a problem
--file a problem/issue with the vendor
--ask your IT shop to figure it out
and NOT start calling consultancy shops.
And besides the intenet explosion and the knowledge level of customers is on the rise. Every one (almost everyone) knows what google is. people download software ,music, and do practically search everything on the web. It's almost become a de-facto term for 'Did you google?".
What ever said and done, bottom line:
Dysfucntional or not Dysfunctional, NO company deserves some lame tip/fix/script from some cheap(and I mean "we don't cost much" cheap) consultants who are out there to make a fast buck.
PERIOD.
Worth your Time?
GG, April 10, 2005 - 12:34 pm UTC
Tom,
Is continuing on this subject worth your time?
I have made my own decisions long time ago on which sessions/conferences to attend, whose books to buy,
which oracle websites to check for references/questions.
Just a personal openion.
April 10, 2005 - 12:47 pm UTC
I think discussion is good.
Not just discussion.
Ajay, April 10, 2005 - 1:13 pm UTC
And its not just the discussion that's good. It's the back and forth with actual, runnable scripts that provides real value.
In "The Cathedral and the Bazaar" Eric Raymond wrote:
"It's fairly clear that one cannot code from the ground up in bazaar style. One can test, debug and improve in bazaar style, but it would be very hard to *originate* a project in bazaar mode. Linus didn't try it. I didn't either. Your nascent developer community needs to have something runnable and testable to play with."
I think we've gotten some great evidence of how this applies to discussion boards. Some boards can't get beyond the vigorous waving of flags. No value, beyond real entertainment value, is added. On this site, people can get the SQL and statistics to do something useful on the "real world" systems they work with.
And still the blundering persists...
Howard J. Rogers, April 10, 2005 - 3:43 pm UTC
"Obviously since I have not reported back, it has not occured...duh."
Duh indeed, Mike. So when you told Tom in this very thread that when you're wrong, you admit it and apologise, you *meant* to say that when you're wrong, you just go quiet on the matter, and we're all supposed to work out that if Mike's gone quiet, it's his way of admitting he's wrong?
It's certainly a novel approach: "Silence equals admission". Reasonably people might simply prefer the less ambiguous "I was wrong. Sorry" approach.
What you don't get Mike is that if you would only adopt one tenth of the approach advocated by the likes of Tom, to actually test things before pontificating on them, you would never, ever have made such a stupid claim in the first place. That's the point here, I think. Not that you made a mistake. Not even that you don't gracefully and unambiguously acknowledge it when you do. But that this particular mistake wasn't a slip of the tongue, or a half-remembered init.ora parameter, but a full-blown, reasoned, rational argument about the way Oracle worked... that three minutes of testing would have proved totally incorrect.
It's your methodology, Mike, that's at question. It caused you to screw up big time less than a year ago. Are we to believe/assume that it was a peculiar anomaly that made that happen? Or is it perhaps your way of working that makes it rather more likely that you might be wrong on many other topics? Like index rebuilding, or large blocksize tablespaces, perhaps?
And that's a concern about the future, Mike. A future where someone follows your advice on some Oracle-related matter. It's not about being stuck in the past at all, and it's a shame you can't see that (or that your Boss won't let you acknowledge it).
It's a funny old game!
Tim..., April 10, 2005 - 6:12 pm UTC
I like to think I know a fair bit about Oracle. I've written loads of articles over the last few years and I'm now on my second book. I can theorize alot of people under the table, but I'm regularly made to feel dumb by questions on my forum. Someone asks the most simple sounding question and about 2 hours later I reply with a casual 2 line answer, or simple query. It's like a swan, serene on the surface and kicking like mad underneath.
When I get positive responses I glow. When I get negative responses I squirm, but it's all good.
Back away from the keyboard and do something unrelated to IT for a few hours and come back with a clear head and a bit of perspective.
Namaste!
Tim...
Forgot to say!
Tim..., April 10, 2005 - 6:18 pm UTC
Whoops, forgot to say...(in alphabetic order so as not to show a preference)
Don, I'm a big fan of your work.
Howard, I'm a big fan of your work.
Mike, I'm a big fan of your work.
Tom, I'm a big fan of your work.
Cheers
Tim...
this is Tom asking Tom, so -- what about evidence (aka proof)? good, bad or
Athi, April 10, 2005 - 7:14 pm UTC
Tom,
I remember the following quote from 'The Merchant Of Venice' in the Server Concepts manual (Old Versions)
<Quote>
I am Sir Oracle,
And when I ope my lips, let no dog bark!
</Quote>
I guess the No Proof people took it literally ...
I don't need this discussion ...
Vinayak, April 10, 2005 - 10:03 pm UTC
I don't need this discussion anymore because I'm already a great fan of you because of the approach you take to tackle the problems. I know which authors to read and believe.
BTW, I am eagerly waiting for your next book "Oracle 10g Expert Edition". When can I expect it?
Thanks.
April 11, 2005 - 8:35 am UTC
targeted for sept 2005
To Tim...
Howard J. Rogers, April 11, 2005 - 1:32 am UTC
You seem to suggest with your two posts that the whole discussion is out of hand, and that Tom and Mike (and Me and Jonathan, let's say) are all equally meritorious, and we should all take a day off, calm down and then we'd get over it. Apologies in advance if that was not what you were getting at.
Speaking entirely for myself, in any case, I am not in the least bit 'worked up', and this isn't in any case personal. There are important matters of technical advice at stake here.
Back in May, 2004, Mike Ault wrote on comp.databases.oracle.server: "Since clustering factor is the major base for index cost, a high clustering factor will eliminate a good index from consideration in the CBO, therefore reduction of the CF is desirable in most cases,
either by rebuild in single column index or rearrangement of the columns in concatenated index." (From a thread entitled "Separate data/inidex" [sic].
Four days later, in a thread entitled 'Index management' on the same group, Mike wrote: "To distill my advice on index rebuilds, I say to only rebuild indexes which show, through proper analysis, to have problems. Examples of possible probelms are: a large clustering factor to dirty base table
block ratio, a large number of levels (of course large is relative, some folks say 4 other 2 is where "large" starts), or a large amount of white space (I call it browned nodes, following with the leaf and tree metaphore.) I do not advocate rebuilding all indexes."
What we have here, in other words, is someone looking for measures or statistics that indicate when to rebuild an index -one of which (and I stress only one of which) is the clustering factor. He also looks to the height of an index. And the weird reference to 'browned nodes' is simply our old friend LF_ROWS versus DEL_LF_ROWS, as best I can tell.
Now, I have to ask myself: how in God's name did anyone ever come up with the idea of clustering factor as an indicator of the need to rebuild an index? Because once you grasp what it actually tells you, it's abundantly clear that it can indicate nothing of the kind. This isn't like me posting advice to a user to set TIMED_NUMBERS, when what I meant was TIMED_STATISTICS. That's a simple slip-up. But here, we have a frequently-published author using a statistic as one of the foundation stones for his advice on index maintenance. We are presumably entitled to expect that something so fundamental to such an important area of database day-to-day management would have been properly researched? I would have thought so anyway.
In fact, the only thing I can think of to explain Mike's bizarre belief that a poor clustering factor would indicate a rebuild is that we all know that the clustering factor can range from BLOCKS (good) to NUM_ROWS (bad); that there is therefore a spectrum of possible clustering factors; and that it therefore 'stands to reason' that there must be some point along that spectrum -say, 50% of the way towards rows- where you could exclaim "Enough! Time for an index rebuild".
Of course, whatever made Mike adopt this measure in the first place, he has (we are apparently now to understand) finally twigged that it can *never* indicate the need for a rebuild. Which leaves the question: if one of his index rebuilding indicators is a busted flush, what's he using these days to determine when a rebuild is appropriate?
Height and LF_ROWS/DEL_LF_ROWS, one presumes, given what he was writing a year ago... though I note from his most recent Blog posting that some sort of waits have something to do with it! He was recently on record in Don's forum (from which I am banned) along the lines of "Call me old-fashioned, but I still like LF_ROWS/DEL_LF_ROWS as an indicator", for example.
But then I think we are entitled to be a little gobsmacked that "one of the leading names in Oracle technology" (quoted from Don's Rampant Books blurb) can do no better by way of index rebuilding advice than the freebie official Oracle documentation, which has been pushing the LF_ROWS/DEL_LF_ROWS ratio for ages. (And never mind that that advice is itself deeply suspect, since that ratio will almost always sort itself out if you just leave the index alone.)
But whatever: at what height or at what LF_ROWS/DEL_LF_ROWS would Mike rebuild an index? I've tried asking, but Mike's latest post on his blog was simply to tell me to go and ask Tom instead! Either Mike now agrees with Tom (who once told me that my suggestion to 'possibly, maybe, perhaps, occasionally rebuild the odd index, maybe' was 'putting it too strongly'!), or we are none the wiser: Mike has 'measures' for when to rebuild an index, but can't or won't enlighten us on what they are, precisely.
As recently as April 9th, Mike wrote in his Blog: "There are several situations where rebuilding or coalescing of indexes is beneficial and when I see a need for it, I will recommend it." Which presumably means that Mike could describe HOW he "sees the need for it"! And after all, it was Mike himself who posted in this very thread, "When I suggest something, I provide the background data on why." So give us the data, and let us evaluate the usefulness of the advice to rebuild for ourselves! In a thread entitled "What about proof (aka evidence)", it is surely not too big an ask.
Otherwise, we are just left with Mike saying "I know one when I see one, but I can't tell you how I do it". We're into the land of magicians once more -and I dare say Tom would like to be able to spot such indexes as much as he'd like to be able to spot unbalanced freelists!
So, no: there is not equal technical merit here. Evidence, tests, verifiable measures mean advice can be evaluated rationally. Without those things, however, we have pure magic or mumbo-jumbo.
That is why I, at least, won't "get over it" as Mike so wishes I would. I don't subscribe to the "Trust Me" school of tuning.
HJR
All methods lead to Rome
Kevin Lidh, April 11, 2005 - 2:46 am UTC
I've been fortunate in my life to have had a breadth of work experience (and experiences). What I've postulated for years is that "everything is related." I saw this as a student while taking Art Appreciation covering two-point perspective and learning about graphics programming. As a software engineering instructor for the Air Force, I saw it when I was teaching object-oriented analysis and design and talking to another instructor who was teaching organization development regarding wings/groups/squadrons/flights. The principals were so similar and broadly applicable (and worthy of a dissertation one day). In regards to this thread as well as a good deal of the DKB thread, I think it's shortsighted to think that PL/SQL scripts can't be used to prove/disprove performance even on a single-CPU laptop. I've been reading a paper written years ago by a man named Taylor (I'll find the first name and report back if there's interest) that revolves around atomics and design. He breaks systems down to transactions and with the help of formulas and an understanding of how the system will scale, you can predict what resources (CPU, memory, etc) will be required. Once you have constructed the atomic model of transactions, you can introduce new customers, corporate takeovers, churn, or whatever and reasonably predict what the impact will be. I'm sure I'm preaching to the choir in this forum but I think its foolhardy to arbitrarily discount any methodology just because it isn't your own (e.g. proofs). What we seek is understanding of our problem space so we can effeciently and effectively construct and implement the solution. Some people learn from example and some people learn by direction.
Clarification
Tim..., April 11, 2005 - 5:00 am UTC
Howard, I didn't mean to suggest everyone was foaming at the mouth, but it's easy for someone from the outside to read it that way.
Also, there are alot of me-too posts that are getting a little nasty, which doesn't help matters. I think the content of this thread is really educational, but it's being diluted with posts that read like "My dad is bigger than your dad!".
This is by no means a criticism of the movers-and-shakers of the thread, like yourself.
I know I've read some really good stuff written by yourself, Tom, Mike and Don and I have alot of time for you all, but I think alot of people reading this thread could start making some very invalid conclusions about individuals as a result of it.
Let's not throw the baby out with the bath water.
Cheers
Tim...
something completely off-topic and insulting ...
A reader, April 11, 2005 - 6:08 am UTC
my guru is smoking something more funny than yours :-)
I just find it funny ....
Greg, April 11, 2005 - 9:01 am UTC
I followed most of the links and scanned most of the "discussion" on this thread, and the thread's on the other forums with Don/Mike ... Also read the "warning re: AskTom" ..
I only have 1 observation after reading other everything:
1) I think Don/Mike should be politicians, they are very good at "discussing" the credibility of a person ...
2) they are very good at dodging the questions at hand directed to them.
3) they talk alot ... but say very little ..
Ok, I'll admit, those are just observations ... but then it's kind of hard to "prove" conclusively those particular types of observations ...
Seems funny, though ... when I first found AskTom, I was highly skeptical of this strange: "Tom"-fellow ... I was thinking: "Hey, what does *this* guy know ... *hrmph*" ... but as I read the threads, and saw his answers, and tried them out myself, and challenged him (unsuccesfully, I might add .. heh) ... I learned really quick that this strange "Tom"-fellow knew his stuff ... period.
Regardless of how sarcastic he might be to certain posters (justified or not) ... the facts are the facts, and the questions got answered, and proof was there ... and the occassional time he did actually get something wrong .. he corrected himself ... publically ...
Reading the threads over on Don/Mike's forum, however .. I have found absolutely no proofs, no talk on *real* technical issues .. just speculation and personal claims and observations (just like my 3 points above about Don/Mike)
hmmm ... go figure ...
I guess the only real difference between myself and Don/Mike is that they get paid more than I do to make "observations" ... :)
I'm just glad Tom gets paid more to show "proofs"
;)
Tim's remark
Tarry Singh, April 11, 2005 - 9:26 am UTC
Tim,
In the last couple of days both "these" gentleman(?) have shown enough of their gentlemanly behavior. Ault's picking up snippents out Tom's site and reacting to
"This is how Tom reacts to his users". I can understand in your position playing neutral, since you work together with one of these gentlemen for writing your book.
Opinions on this article ...
Greg, April 11, 2005 - 10:20 am UTC
I couldn't think of a better place to ask this .. so I'll ask here .. if:
a) it's not the right place, just let me know where you'd rather me post it ... if at all .. ;)
or
b) if you haven't time to follow up on it so much and want to leave it as an "excercise for the reader" .. ;)
</code>
http://www.oreview.com/9905feat1.html <code>
I read over it, the first paragraph caught my attention as very good .. but as I read the rest of it .. my relative inexperience caught up pretty quick .. ;p
If I had to say so, I'd think it was a well written article .. but I was just curious ... as it seems to lack any supporting proof/evidence ..
Is there anything in that article which you (or anyone else) might disagree with .. and better yet - have anything that shows it (or suggests it) as not entirely accurate?
(or heck - even better - the converse: anything that supports it?)
Some example of things I'm a bit skeptical of and would love to see any "proof" either for or against ... (I'm not that sure of how to test some of these myself .. hence the post here ..) ;)
[quote]
In fact, for three times as many rows returned, we may expect the sort to take between 3.3 and 4 times as long
[/quote]
[quote]
Cleverly used, pin-pointed queries and intelligent programming usually achieve much better results
[/quote]
?? what the heck does that really mean, anyway? :)
[quote]
If you have any doubts, learn from people who know what relational databases are about
[/quote]
Ok .. I'm here .. ;p
[quote]
I advocate the computation of a size sensitivity index.
[/quote]
?huh? Is this something the author just created, or is it something I've just never heard of? ;)
Thanks for your time, Tom .. as always!!
April 11, 2005 - 10:38 am UTC
In general, it was good -- overall, I agreed with much of it, however:
<quote>
Keep your algorithms simple. Theres a strong tendency among programmers -- undoubtedly under the spell of the SQL language -- to write the SQL query that has everything including the kitchen sink, bringing the server to its knees. Cleverly used, pin-pointed queries and intelligent programming usually achieve much better results. If you have any doubts, learn from people who know what relational databases are about and just trace the execution of a DDL statement (a CREATE, ALTER or DROP) that does a number of updates to the data dictionary. If theres something critical to Oracle performance, its data dictionary access -- simple, basic, and efficient statements. Its not by chance that most data dictionary information is cached in the SGA. Many critical processes can have most of the work done by the client rather than server side. Oh, and Im still waiting for integrity constraints on the Oracle data dictionary.
</quote>
goes against my grain. I want to do as much in a single sql statement as possible (comparing our applications to DDL is not "fair", we generally don't have such simple 'store this metadata' requirements and we are not building database servers, we build applications)
It would have been nice to give a quick and dirty demonstration of a query that doesn't "scale" linear. This is a confusing topic for many people. An example would have made the point nicely. It is also why hinting queries to death is not a good idea (talked about this in </code>
http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:36496737148112 <code>with Dominic's presentation)
I did not fully grasp "Planning Ahead: Size Sensitivity Index", the table didn't make sense to me, I didn't get it.
The quote you have above, about sorting, would have been easy to demonstrate (and then we could have seen the assumptions). As it is, we have to take it on "faith". I would not agree with it in general. It could take lots more than 3.3 and 4 times as long (as we exhaust the ability to sort in ram), it could take lots less than 3.3 and 4 times as long. An example would help frame it.
I won't address the "pin pointed" thing totally, because it would probably come across "snarky". I like to have as little procedural code in general. Pin pointed queries sounds like "vsam keyed reads" to me.
Good read!
Arun Mathur, April 11, 2005 - 10:26 am UTC
Tom,
A nice side effect to how you conduct yourself is that it easily counters those who exhibit arrogance. Arrogance being defined as "overbearing pride evidenced by a superior manner toward inferiors". As stated in your article, you state your case, back it up with information that someone else can reproduce, and come to a conclusion. I for one have been in situations where I report to a lead. The problem being that the lead doesn't work as a "lead by example", but as someone to just say "Do this because I said so", and indirectly, "If you question me, you're part of the problem". Needless to say, those types of people are always frustrated when trying to push their ideals on the internet, mainly because we all have the ability to do something they're not used to : reply. It's no coincidence that people respect your name when you participate in discussions.
Thanks Tom for the nice side effect.
Regards,
Arun
The Evidence speaks for itself
Bill Schwartz, April 11, 2005 - 11:27 am UTC
Having opportunity to frequent many forums, and seeing the myriad discussions related to Oracle myth perpetuation, I have become more and more grateful for those Oracle "proponents" who continue to stress the "show me" school of philosophy. I use the term "proponents" as it can only be construed one way, unlike "scientist" and "expert" (OK - expert should ONLY be construed one way, but we all know that it is a meaningless label today). Discussion is vital to our industry - it keeps us innovative and promotes ideas and free thinking. Do opinions abound? Sure. Everbody got one? Absolutely. Are they valid? Ahhh, that is where discussion (and EVIDENCE) come into play. As a young man (19 or 20, can't recall the exact age I was) I, along with several others, witnessed an interesting phenomenon in the summer night sky over New York. The object we saw moved strangely to us, and did not behave in the manner of normally seen aircraft. Did we witness something strange? You bet. Was it a UFO? Maybe (following the strict definition, it was a flying object that we could not identify). Was it extra-terrestrial. Well now, there was certainly no evidence of THAT. Could have been, could have been an experimental aircraft as well. Who knows. So, factually we can state with utter certainty that four people (no drug use, no alcohol use at this time) witnessed something strange in the night sky. And that is as far as evidence takes us. Opinion is fine. Everyone is entitled to theirs. But to present your "opinion" as fact, and then refuse to provide the supporting evidence, is tantamount to wasting our time.
To Bill Schwartz
Kevin, April 11, 2005 - 11:37 am UTC
Aren't all airborne objects "extra-terrestrial"? ;0)
Kevin - how true!
Bill Schwartz, April 11, 2005 - 11:42 am UTC
But I was afraid to say "alien" for fear that it might offend people from other countries. Just too vague, if you ask me ;-D
Excellent work Tom
Russ, April 11, 2005 - 11:44 am UTC
As a consultant myself, the most frustrating part of my job would be the unrealistic expectations of some customers.
They frequently expect that I can tell them init.ora parameters to change (with no other knowledge of their environment). They think this will solve things. They are against testing anything. (We are paying YOU, the EXPERT, so why should we test it?) I spent two hours having such a discussion last week.
Tom, I agree with your methods 100%.
Thanks!
Thank-you, Tom!!
Greg, April 11, 2005 - 12:53 pm UTC
Thanks for the comment about the article ... clear and concise, as always .. ;)
Helps me out as I keep trying to learn these things ...
David Aldridge, April 11, 2005 - 1:08 pm UTC
>> But I was afraid to say "alien" for fear that it might offend people from other countries. <<
Personally, I'm used to it. I even have a card telling me that I'm one!
Alberto Dell'Era, April 11, 2005 - 1:36 pm UTC
> Having opportunity to frequent many forums, and seeing the
> myriad discussions related to Oracle myth perpetuation,
> I have become more and more grateful for those Oracle
> "proponents" who continue to stress the "show me" school
> of philosophy.
I couldn't agree more ... and that applies to any kind of technical discussion, and any kind of technology or product.
Peter Kinsella From Dublin - You didn't get a complete answer
Just Another Reader, April 11, 2005 - 3:09 pm UTC
Peter,
To answer your LAST question, no TK and Sami B. are not the same. And I should know ;-D
True Science vs. Shell-game, Smoke and Mirrors, and B.S.
Robert, April 11, 2005 - 5:52 pm UTC
I realize that Don B. and many of his ilk probably know a lot more about Oracle than I do.
And there is more than one way to skin a cat...most clients are looking for results, not 'science', and if someone can deliver true value to a client, regardless of their method, then more power to them.
But I have been a part of too many situations where consultants use intimidating tactics, false bravado, equivocation, shell-game diversions, and just plain B.S. to sell themselves and their point of view.
Trying to give the other guy the benefit of the doubt, I spent much time thinking maybe they were sincerely mistaken. But having seen this type of behavior for 15 or so years in IT, I have begun to suspect there is some calculated dishonesty going on to promote themselves and/or to cover their back sides.
Tom, however, is completely open and above-board; all cards on the table.
Many who would oppose Tom's method, seem too often to deal in the type of smoke and mirrors mentioned, above.
The knowledge, skill, and intuition gained from experience are not to be discounted and I'm sure Mr. B. and many of his followers bear the scars of many battles whereby they have nobly earned impressive amounts of the above.
BUT I TRUST TOM'S METHODS!
Yes the "Scientific Method" if you will.
Tom brings his methods and proofs into the blazing noonday sun for all to see.
Whereas many of those scoffing at this "Scientific" approach would scatter as roaches from under a rotten log if this kind of openness and candor were applied to their methods.
Tom teaches me to fish, whereas Mr. B. and his ilk only teach me to follow them.
I fear many (excluding Mr. Burleson, or Mr. Ault) who would oppose Tom's 'Scientific' method prefer the dishonest methods described, above, as a cover just to pad their bank account and get out the door so they can get on to their next victim.
Maybe this is just an argument between Truth and pragmatism... and pragmatism doesn't work. Pragmatism deals in short-sighted solutions based on the immediate situation without regard for the long term effects or ultimate Truth. True science welcomes the light of day. True science welcomes the criticism which reveals error and falsehood whereby it can improve itself.
Re: True Science vs. Shell-game, Smoke and Mirrors, and B.S.
Dan, April 11, 2005 - 9:30 pm UTC
Well said Robert!
Banned By Burleson!
David Aldridge, April 12, 2005 - 2:58 am UTC
April 12, 2005 - 7:25 am UTC
Very nice writeup
cool David
A reader, April 12, 2005 - 3:48 am UTC
David Aldridge - Banned By Burleson
Diogenes, April 12, 2005 - 8:32 am UTC
Welcome to the club (we have jackets; or at least, we should!). You just didn't learn the lesson quickly enough. Challenge authority, and you get removed. So now that you have free time, what are you going to do? ;-D
Alberto Dell'Era, April 12, 2005 - 8:56 am UTC
> Banned By Burleson
BBB - A "Triple-B" badge ;)
David Aldridge, April 12, 2005 - 9:07 am UTC
>> Challenge authority, and you get removed. So now that you have free time, what are you going to do? ;-D <<
Yup -- thought it best to go out with a splash, when so many posts went unanswered.
Free time, pah! I got work to do. And of course I have to blog every dumb thought that flits through my consciousness. Including any concerning obvious and elementary errors in the scripts of others.
A reader, April 12, 2005 - 10:31 am UTC
David, can't see how you could possibly have been banned from the site, particularly when they uphold the highest levels of 'dignity' and 'respect' at all times. Just take the following as an example (most notably the last two paragraphs):
</code>
http://dba.ipbhost.com/index.php?showtopic=1496 <code>
If I wanted to put myself out of business, treating my customers like this would be my first step !!
Reproducing here just in case the solar flares strike the forum again ;-)
<quote>
All of this is documented in the Oracle documentation and on metalink. If this is not clear to you maybe you are in the wrong business.
Or, just maybe, it has nothing to do with the real issue at all. Perhaps the real issue is you want to get someone riled up, get someone backed into a corner or just show that your...database...is bigger. If that is the case I say grow up folks.
</quote>
A reader, April 12, 2005 - 12:22 pm UTC
Only after getting this now:
"just show that your...database...is bigger"
Classic stuff !!!
Excellent response!
Prince Menon, April 12, 2005 - 12:44 pm UTC
Excellent response Tom! Don always had tall stories with corporate attitude. If you look in career section of his site ...one of the essential qualification is to wear an expensive suit for interview (or he says he will know it a mile away!!!)!
Your site has been always been good source of Oracle information and your examples are simple with style.
Another BBB Link
Warren, April 12, 2005 - 1:19 pm UTC
Here is another posting on this same line of discussion:
</code>
http://dba.ipbhost.com/index.php?showtopic=1476&st=0&#entry5671 <code>
As I was banned from Burleson's site previously, I had to create a new, BBB, account. BBB, another badge of honour!
I'm wondering how long it will be before this post is deleted as well.
April 12, 2005 - 9:04 pm UTC
well, most of the free world cannot read the site anymore so it probably isn't even relevant anymore.
David Aldridge, April 12, 2005 - 1:44 pm UTC
>> Reproducing here just in case the solar flares strike the forum again <<
Much appreciated!
A Reader sure gets around...
Mike Ault, April 12, 2005 - 1:50 pm UTC
A Reader sure gets around. I always found anonymous posting rude and usually meant someone had something to hide.
If you can't put your name on what you say, maybe you shouldn't be saying it.
The quote was from a posting of mine on the Burleson Froum to a poster who seemed intent more on rubbing peoples noses in possible mistakes rather than learning. If I was incorrect in my feeling about this, then I apologize.
April 12, 2005 - 9:15 pm UTC
I find anonymous posting just dandy and really cool. ALL of your posters on YOUR forum are 100% anonymous, 100% (and don't kid yourself any other way). Who are all of those names?
So you have an opinion on anonymous, no biggy. I have a DEFINITE opinion on IM speak in a forum, you seem to have a different take on it. so what. free place and all.
The poster on your forum just seemed to be trying to get an answer from my read on it. All I saw was smoke screen and "we googled harvard and found a presentation", quote from the guy who said "the internet is a dangerous place"
<quote src=DKB>
Here is another from Harvard University (a good school, I hear):
</code>
http://courses.dce.harvard.edu/~cscie256/docs/session2_2005.ppt
If the redo logs are mirrored sets, then both are written to simultaneously. There can be multiple LGWR processes.
</quote>
of course, this got hit in a solar flare or something, so it is gone (pull that presentation Mike and see if you can find any other errors. Maybe stamping harvard on things, resumes included, isn't as relevant as thought? The internet is a dangerous place remember
http://dba-oracle.com/oracle_tips_credence_qualifications.htm <code>
)
The question was asked very simply, by more than one person "can you have more than one lgwr". smokescreen about undocumented parameters, async io, smoking guns from Oracle none the less, the answer is
"NO"
Hope now that the article by Don is recognized to be "wrong", it is retracted. That would be a professional thing to do, rather than let the myths continue right?
But there is nothing credible on this site, is there?
My judgement of credibility is on the content of a site, is the technical content
o correct
o accurate
o useful
o open for debate/questioning
to me, if you hit 4 of 4 on that (so that anything not meeting the first 3 can become correct, accurate and useful...) you have a credible site.
But that is just my opinion.
David Aldridge, April 12, 2005 - 2:40 pm UTC
Special printing for Alberto - "Vietato da Burleson!"
Scientist
Richard, April 12, 2005 - 3:21 pm UTC
I have a problem with the phrase "Oracle Scientist": it dilutes the term scientist.
One could be a brilliant dustcart operative* with years of experience; have a masterful insight into the best ways of emptying the trash; have written wonderful papers on dustcart operations, with beautiful mathematics and examples underpinning one's theories; and, generally, be regarded as a no BS professional who adopts the most stringent standards and methodologies etc, etc, ad nauseum.
In this case, one would be a brilliant user; but not a scientist. The same goes for RDBMS users.
Yes, Oracle is technical, and has many aspects which rely heavily on science. Yes, elements of scientific technique can be applied to the product. But, no, that doesn't make a scientist of the user.
What's wrong with expert? It's meaningful, but doesn't take that extra step towards hubris.
* or anything that's not a true science.
April 13, 2005 - 8:42 am UTC
Not that I've been calling myself a scientist (seems it has mostly been Mike and Don saying "oh oh oh, look at the scientists".
Computer Science / Scientist
Database Engineer
Database Architect
Data Architect
System Analyst
and so on... they are all just labels assigned to people by other people.
Anyway, the term I prefer
Database Technologist.
What's wrong with the "Science of Dustcarting" ?
Robert, April 12, 2005 - 3:36 pm UTC
Hello Richard,
Surley you have your tongue planted firmly in your cheek! :)
Robert.
David Aldridge, April 12, 2005 - 4:24 pm UTC
You might like to regard the term "scientist" as a relative concept. In comparison to the bad old days where fixing a performance problem was almost a stab-in-the-dark, the use of real measurements and the understanding of Oracle's fundamental concepts as the basis for tuning almost rates as theoretical physics. Well, I exaggerate a little, I guess. In fact there is a strongly "applied" feel to it which is why I was rambling elsewhere about the profession being more akin to engineering than to science -- apply theory, measure results, explain difference, modify theory, measure results blah blah, and take a systemic view of the problem.
Contingency Theory
Mac, April 12, 2005 - 5:10 pm UTC
In solving a problem, whether one portrays herself as an expert, scientist, doctor, or whatever is besides the point. What matters is:
o Whether her advice solves the problem reliably and
consistently,
o The solution works under similar situations, and
o She details the limits of the solution.
Advice giver and receiver must always remember the contingency theory -- with a change in situation, things change.
April 13, 2005 - 8:51 am UTC
I don't think anyone here is disagreeing with that...
Accurate information
Useful information
use cases for when to use it and most importantly perhaps
when not to.
Excellent Points Made here
Steve G., April 12, 2005 - 7:20 pm UTC
I am somewhat new to Oracle (around 3 years of progressive experience) and too often I have had to go up against people with ancient notions of performance, proper techniques, and cool tricks they "read" from some book. Things like "foreign keys make the app too slow", "you need to seperate indexes and tables into seperate tablespaces to make disk access faster", and so on.
One of the main problems with oracle is the explosion of totally invalid information. Information from industry recognized experts, some even recognized by oracle corporation, that is complete nonsense. This explosion of "it worked here so it must work here", "i can make the database faster with my own neet tricks" are a severe detriment to the oracle community.
I truly believe that it is most important for any person to realize the fundamental principles of the SQL language. Process information in Bulk through SQL(understand relational algebra/calculus, set theroy...), normal form, and referential integrity. Let's start worrying about those fundamentals first, start from a solid platform. Once you have a strong base then worry weather or not bulk collect is faster, or what hints you might use. However do so through verfication and not because some industry expert said so..
Who would have thought....
yet another 3B member., April 12, 2005 - 10:30 pm UTC
Who would have thought that the database world could be so entertaining? I want to stop reading all this but it's like a car wreck, you just have to slow down and look. And Don and Mike are sure a BIG pair of wrecks in this one. Truely sad, yet entertaining. :/
Say what you like about Don, but at least he can laugh at himself...
Howard J. Rogers, April 13, 2005 - 1:42 am UTC
Who'd have thunk it?! Me leaping to Don's defence?!!
But this indicates, perhaps, that Don can indeed recognise humour, even when it's rather at his expense:
</code>
http://www.dizwell.com/blogindex.html <code>
(Check the comments for the April 12th entry)
I think my point is: target the man's manifest technical failings, and his heavy-handed approach to censorship, for sure. There's no need to have a go at him as a person, though. (And satire doesn't count, by the way!)
Regards
HJR
"I find anonymous posting just dandy and really cool. "
A reader, April 13, 2005 - 3:11 am UTC
*****
It's time for a new book
Georg, April 13, 2005 - 6:24 am UTC
Tom,
it's really time to write a new book. The title should be "Killing Oracle Myths". With proofs by Thomas Kyte.
You're just saying "I did it several times already"? You're right. But it's distributed in your books and these pages.
I really believe there is a market for such a book. I swear I would be the first who would buy it - even I know most of them.
Maybe I would even spent some more money and buy a copy for Don.
Half Knowledge can be dangerous...proved by Tom...time & time again
San_mat, April 13, 2005 - 6:39 am UTC
To all those Self proclaimed Oracle Gurus,
This is a real test... Tom proved it with simple questions/queries which we guys like always.
Thanks Tom for the good work.
BTW : I think i need to check my book shelf for all those books written by great authors ( having so...& so..experiences...and...great academic backgrounds..) and immediately remove those books to trash & again go through 'Expert one on one Oracle' to clear my thoughts on Oracle basics again.
A must-read for anyone in this thread!
Bill Schwartz, April 13, 2005 - 9:05 am UTC
Apparently, Jonathan Lewis has something to say about scripts and Oracle Performance. Very enlightening! And, as always, succinct and clear. Thanks Jonathan!
</code>
http://www.jlcomp.demon.co.uk/proof.html <code>
April 13, 2005 - 9:24 am UTC
Oh and I love the subject he picked, one of my all time favorite topics!
One more addition - just found this blog entry...
Bill Schwartz, April 13, 2005 - 11:18 am UTC
from Niall Litchfield. Niall has some very interesting things to note as well. Another must read for perusers of this thread, and generally anyone who is interested in Oracle. Also very enlightening!
</code>
http://www.niall.litchfield.dial.pipex.com/ <code>
April 13, 2005 - 11:27 am UTC
I liked what Mark had to say as well (found that from Niall's page, enjoyed Niall's talk as well)... Mark's comment mirror what I said
</code>
http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:6643159615303#38549440183885 <code>
that it is nice and all (mikes story, where once again he says "showing test cases are bad" but without actually having used a single test case anywhere), but provides nothing for the reader to act on or be able to use in the future.
other than "someone did this once and something happened and things were better but we don't know why".
Consultants vs Scientists (somewhat off topic)
Old Ada Guy, April 13, 2005 - 4:13 pm UTC
Just finished reading this (and related) threads and found it immensely educational and entertaining. What stands out most to me is that we are witnessing a protracted battle between two rival groups in the IT industry, and the "scientists" are wiping the floor with the "snake-oil salesmen" (sorry, they needed a descriptive term too); the former are wiping the floor with the latter. Speaking as someone who has spent most of his career cleaning up "magic solutions", I can attest to how attractive fast-talking consultants can be to large companies, and to how quickly they can hoover up T&M budgets. To get two of these guys to stand their ground long enough to reveal themselves for what they are is an impressive achievement.
Why, you might ask, would the snake-oil salesmen expose themselves to this humiliation? I think it is because our industry is maturing to the point where paying customers can track results empirically; to maintain market share, our two friends need to discredit this scientific method fad. It is gratifying that they did not succeed, because many influential members of large companies read this forum, and while they may not know the difference between a freelist and a freelist group, they *can* tell who is being honest and who is not. To put it another way, discussions like these put money in the pockets of those who will spend it well.
p.s. How do I become a Tom Kyte Minion?
Proof on large servers/instances.
Kerry Scott, April 13, 2005 - 7:23 pm UTC
I've noticed in this thread (and others) the criticism that Tom's proofs don't necessarily hold water when dealing with large or very large instances on large multi-cpu servers. Perhaps, but having worked for more than one of the "notable" Oracle consultant firms, I find this criticism laughable. Do you really think that these firms and their "gurus" have the money to own any large servers and instances on which to practice their tuning expertise? I wonder? They are probably lucky if they provide their consultants or themselves anything more than a commodity PC or laptop, running Personal Oracle. So where does their tuning expertise come from? Well, they learn on client's servers while on engagements. Thus, much of what is learned is based on a specific client environment/application set under circumstances not lending to tuning "proofs" but rather solving the clients specific problem. You can often tell this when reading their books, as you will see tuning recommendations that either work in very specific cases (the ones for which they had an engagement), or are also based on "small" environments. At one firm, for instance, I was told to help the customer lay out a 4 terabyte database, and use standard datafile sizes of 128MB because "it says that's a good size in our book". When looking for good advice in these books, we rarely if ever see "my credentials include work on 16+ processor Suns or HPs or NUMA-Qs and 4 Terabytes of EMC using triple mirroring for such and such a fortune 500 company" (your place of higher education seems to be more important). So, take the advice with a grain of salt and "prove" it.
April 13, 2005 - 8:12 pm UTC
(i actually don't do the work on a single cpu laptop much of the time, but don't tell anyone)
The fact is, these things show how the software actually works, so you can take wide open statements like "indexes like large blocksize tablespaces" and show why
a) sometimes they do
b) sometimes they do not
c) sometimes they don't really care at all
(the magic of three). The answer is "it depends" and all I want to know is "what are the decision points, what do I need to know in order to use this advice".
That's all.
Single-CPU laptops
Paul, April 13, 2005 - 8:59 pm UTC
For those of us that have been working with Oracle SW for awhile...
My early days were on 7.0.1.x.x on intel based SCO unix machines. 486's with extremely slow disks. The big server was a DX/2 50.
It does not mean that everything I learned about Oracle back then is irrelevant to what I do now. I just have to put it in context....7 is not 10g, a 486 is not an HP t520, a SAN is not a 4GB 5400rpm SCSI disk...
That's why this site is so valuable, as is the Oracle documentation, new concepts guide etc. (beats using strings on the oracle binary to look for hints </code>
http://searchoracle.techtarget.com/tip/0,,sid41_gci928422,00.html <code>
They SHOW you how things have changed, Tom shows us how to use the new features...analytics, pipe-line functions and lets us know that YMMV and directs us to learn for ourselves.
At any rate, my laptop runs circles around those 486s so I certainly think that anything I learn on it is relevant to any problem I may face today.
For "Old Ada Guy"
Bill Schwartz, April 13, 2005 - 8:59 pm UTC
How do you become a "Tom Kyte minion"? You don't, not really (Tom may have a following, but MINIONS?) As far as being a "show me" type of guy (basically what AskTom is all about), you already are ;-D. Take it from an old COBOL guy. There are TONS of snake-oil salesmen out there, our industry is chock-a-block full of them. Why? Check out their billing rates, and the fact that many times their clients are over a barrel and cannot wait for a "real-world" fix to actually resolve their problem. Or they can't afford the "real-world" fix (totally gut the offending app and design it right). So they go for the Magic of Merlin, Penn and Teller catching silver bullets in their teeth. You and I (and many of the folks here) recognize it for what it is - basically using FIX-A-FLAT on your just-got-a-nail-in-it tire. Unfortunately, most then continue rolling down the highway blithely (or perhaps not so blithely) unaware that that tire isn't going to hold out for long. Maybe just long enough for the manager to get promoted; then it becomes someone else's headache.
Lots of folks out there like sizzle - me, I like steak. Which is why I like to read Tom Kyte, Jonathan Lewis, Niall Litchfield, Howard Rogers, and all the rest of the "prove it" folks. It's more filling, and it tastes great.
An analogy for you...
Jer Smith, April 14, 2005 - 4:32 am UTC
Donald Burleson is attempting to compare Tom's and Jonathan's approach to working on the database to Quantum Mechanics and his approach to General Relativity.
In fact, the proper comparison is Newtonian Physics to Astrology.
I think Tom would concede that there are situations that happen in certain large or high-volume databases, on particular architectures, because of bugs, or because of outer limits of scalability in a particular area, that present as counter-intuitive based on extrapolation of basic knowledge of Oracle technologies.
Given one of these situation, you have two options: you can rely on what you have learned in the general sense and seek out additional causes or factors not considered, conduct several experiments, and attempt to incorporate those into your model and knowledge base. Alternatively, you can develop a different model based on a new hypothesis (throwing out the old) and as long as that model satisfies your single observation, you propose a new set of rules that match your results. It's up to you, but I believe in the progression of science, as do scientists. Einstein would be nowhere if he claimed Newton was full of crap.
I would love a book or article that spent its time discussing the behavior of Oracle as you reached the outer limits of OS process address space, network connections, I/O performance, etc. However, I would assume that such a book would either:
1) incorporate experimental evidence from more "normal" systems
2) disprove the assertions about Oracle behavior shown experimentally with _other_, _repeatable_, experiments
3) show how that becomes superseded by another behavior not relevant in other situations
If not, well...
I wonder if we can have some actual evidence from Donald B. about cases where any of Tom's rules actually break down--where running one of the experiments in a production OLTP system displayed results that disproved the assertion. That's my challenge. But it's a trick: to accept it is to already have lost. The reason is that in doing so, you'd only be further showing the value of the experiment by using it to better understand the system. Oh well.
April 14, 2005 - 8:02 am UTC
Jonathan Lewis named your book "Testing to destruction", take the test system beyond your requirements to observe what it'll look like right before failing totally. Not always doable, but fun if you have the time.
The funny thing in all of this, I don't think I have any of these so called "performance proofs". I haven't seen any, I don't remember calling them as such. The only one that was ever pointed to by Don was the one where I said (this comes from a now deleted posting on his forums, sent to me over the weekend):
....
<QUOTE (src=asktom)>
The author of that book was smoking something funny. You have my book -- you see when I say something,
I show all factors involved, I show the proof, I show the hard numbers and I make it so you can reproduce it
</QUOTE>
DKB: I'm suprised that the author did not sue Oracle for calling him a dope smoker.
DKB: Very unprofessional, IMHO, and it makes me think about selling my Oracle stock.
...........
(you can read the original quote in full at
</code>
http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:3512484632553#4227862321222 <code>
and sorry, but if someone says SQLServer 2000 is 200 times faster than Oracle, well, something is happening in the background there.... but unfortunately, we have nothing from which to see what was happening. Anyway.)
Anyway, does anyone see a "performance proof" in there?
Funny thing about that page, there are pretty no sqlplus scripts regarding anything performance related at all, one snippet on locking/blocking detection, but the page isn't about performance.
(oh, just for everyone else, most brokers suggest you never buy and sell stock based on emotional reactions to events. Ride the market, not the emotion)
Jonathan's Example
KP, April 14, 2005 - 7:12 am UTC
The example from Jonathan is quite simple, easy to understand, tests a concept, shows how Oracle works AND also goes to show that we can see the performance impact. But, my question is, in the real production environment, will we be having a loop like this:
for i in 1..10000 loop
execute immediate 'select small_vc from t1 where id = ' || trunc(dbms_random.value(1,10000)) into v1;
end loop;
I guess not. So, the question is, Is this loop same as the statement:
execute immediate 'select small_vc from t1 where id = ' || trunc(dbms_random.value(1,10000)) into v1;
being executed by 10,000 users concurently through 10,000 different sessions. The above example shows it in a single-session scenario.
Iam also trying to think of an application which will be having 10,000 "CONCURRENT" users. So, assuming that 10,000 'concurrent users' seems to be a higher number, does reducing it to 500-1000 have such a significant performance impact ?
Yes, it does hard parsing and the database spends a fraction of a second more to do this work (for 1 user), but if we don't use a loop, for ANY GIVEN USER at any point of time, will it have a performance impact for that user.
I mean if I run the loop 1 time, can I see any difference ? If not, how is any user connecting through a session feel any performance problem ?.
So, does this example (script) REALLY PROVE that bind variables are better in 'high-precision SQL statements in a highly "concurrent" fashion' ?
The way I read it, this example shows a batch job being executed and proves that bind variables should be used in batch jobs (which anyway is a single-user, single-session scenario). I can't read it as proving for 'concurrent' users. <\b>
Ok, I re-read the post of Jonathan where he mentions
>> QUOTE
If you have trouble realising the implications of the increased latching, it is very easy to modify the tests slightly and run multiple copies on a small server with a couple of CPUs (you can even get laptops with 2 CPUs these days). Use dbms_random (after seeding it) to vary the querys input parameter; make sure that each test restricts its queries to a different range in the table; put a small sleep into the loop so that you dont simply demonstrate that a CPU cant run at more than 100%;
<< \QUOTE
Which I think was what Mike Ault / DB were mentioning in their initial forum discussion ? That we cannot PROVE this through single-session scenario ?.
April 14, 2005 - 8:17 am UTC
he is measuring exactly how much latching goes on. how much cpu is used. that it takes longer.
It is known (provably so, you can scale it up and see, we have, those are published as well, for exmaple in effective oracle by design, scaled it up. In my "all about binds" seminar I give, scaled it up -- using different multi-cpu machines, this whole laptop thing is sort of funny, Oracle has a couple of computers here and there and some of them have more than one cpu) that the more latching you do, for the same latches, the less scalable you will be.
You see that hard parsing takes measurably longer by the wall clock. (if you call 15 times a fraction of a second, so be it. I call it 15 times)
You see that hard parsing consumes more cpu. ( 0.31 cpu seconds vs 12.38 cpu seconds. not sure that is a fractional increase either. The machine spends MORE TIME PARSING SQL than it does EXECUTING SQL in this case. that should be a red flag, unless you are a hardware vendor)
You see that hard parsing consumes more latches. (and latches are a type of lock, locks are serialization devices, serialization devices inhibit scalability)
As you scale up with something
a) latching like mad
b) that consumes N times the cpu
will you scale as you want? Probably not. Almost *certainly* not. Can you even imagine such a case?
We don't have to, it's been done many times.
He shows the effect of 10,000 parses -- hard and soft -- and the work done. That is generally a couple of seconds on many systems. With just a handful of users. Asktom for example:
ask_tom@ASKUS> select sysdate, name, value from v$sysstat where name like '%parse%';
SYSDATE NAME VALUE
-------------------- ------------------------------ ----------
14-apr-2005 08:20:11 parse time cpu 8765225
14-apr-2005 08:20:11 parse time elapsed 21360147
14-apr-2005 08:20:11 parse count (total) 562038184
14-apr-2005 08:20:11 parse count (hard) 3305453
14-apr-2005 08:20:11 parse count (failures) 130487
...
ask_tom@ASKUS> select sysdate, name, value from v$sysstat where name like '%parse%';
SYSDATE NAME VALUE
-------------------- ------------------------------ ----------
14-apr-2005 08:21:26 parse time cpu 8765392
14-apr-2005 08:21:26 parse time elapsed 21360410
14-apr-2005 08:21:26 parse count (total) 562049020
14-apr-2005 08:21:26 parse count (hard) 3305516
14-apr-2005 08:21:26 parse count (failures) 130493
took about 1 minute 15 seconds to get there. Now, I could have used 40 times the cpu during that period if had hard parses 10,000 statements, but since I only hard parsed 63 in that window, I didn't.
Actually, I would have used alot more than 40 times the CPU since this is a SMP box and while waiting for a latch, we would have spun -- burning even more cpu.
and the facts are, we do these tests too.
When you do something using
a) method 1
b) method 2
and measure what sort of work method 1 does and what sort of work method 2 does, you can certainly use that information.
No one said these are mathematical proofs to prove the square root of 2 is not a rational number.
Don and Mike have their unique, myoptic definition of 'proof'.
What we have from that simulation, that test is evidence that
a) hard parsing takes longer by the wallclock.
b) hard parsing consumes much more cpu, measurably more
c) hard parsing eats latches for lunch, latches that dba see as a point of
contention frequently
d) latches are locks of a type, locks are serialization devices, serialization
devices inhibit scalability. (this is a fact, not derived from the test
but a saying I want to have imprinted on brains, so I'll keep saying it)
So, I'd still love for someone to point to a "performance proof" (whatever they are) and say "ah hah, here is one that doesn't hold water", because we've been there with this hard/soft/no parse (on this site, in books, in real life over and over -- trust me on this one, don't even point at hard vs soft or no parse, that one holds water like you won't even believe)
First, they'll have a hard time finding such a "performance proof"
well, maybe that is last too? Not sure, I'll let you know when I see one.
For KP
Connor, April 14, 2005 - 9:38 am UTC
You don't need 10,000 users to get to 10,000 queries in quick succession. I traced one of apps here and picked up over 100 queries just to get to the main screen (security settings, which fields are accesible to this user, etc etc - lots of metadata = lots of queries).
So all you need is 100 users all starting up the app on Monday morning and voila! you've got 10,000 queries in quick succession.
Similarly, a batch program that trickle feeds changes could easily produce 10,000 queries that only differ by a literal in a small amount of time.
Don't get me wrong - there's a lot of successful systems our there that do everything *wrong* in Oracle. They're small enough to not notice the performance and scalability problems (although they often get nailed because they didn't understand read consistency, but thats another matter...). The problem here is that if a system is 'successful', then sometimes a whole lot more users want to use it...leading to the scalbility problem that was "not a problem" before.
Little Java apps against Oracle databases have a *lot* to answer for in this regard!
Just to note, Jonathan was actually being *generous* in his proof. Instead of a simple table, insert a very complex view - you might get a query that takes *seconds* for a *single* parse. Even in a one-user system, he/she ain't gonna be happy when that happens!
Cheers
Connor
April 14, 2005 - 9:58 am UTC
I should have mentioned, in that 1 minute 15 seconds that it took to get over 10,000 statements parsed on asktom, the max number of sessions was 40. And most of those are inactive.
This page you are looking at parsed 204 sql statements! But runs in .2 or .3 seconds (gets a little longer and longer each time as the page grows and grows, but the number of SQL statements remains the same)
Now, imagine if the 204 sql statments were all hard parsed, using many times the CPU, each and every time (even in single user mode).
Add to that the latching and consider what happens when:
1* select status, count(*) from v$session group by status
ask_tom@ASKUS> /
STATUS COUNT(*)
-------- ----------
ACTIVE 24
INACTIVE 30
ask_tom@ASKUS> /
STATUS COUNT(*)
-------- ----------
ACTIVE 26
INACTIVE 28
ask_tom@ASKUS> /
STATUS COUNT(*)
-------- ----------
ACTIVE 25
INACTIVE 29
ask_tom@ASKUS> /
STATUS COUNT(*)
-------- ----------
ACTIVE 24
INACTIVE 30
all 25 people go for the same latch at the same time.
I think the test, the evidence, in its simplicity speaks volumes. (and the larger multi-user tests to bear it out exist, have been printed, have been reproduced. But if you want to show someone "this is why hard parsing is bad for your application that parses hundreds of statements per second" as most apps are doing -- that is the compelling evidence that just drives it home.
Database Technologist, it's NOT science
Chuck Jolley, April 14, 2005 - 4:08 pm UTC
Database Technologist is a much better term than scientist
for what you do Tom.
Despite what one might read in a general science text book,
science throughout history has worked like this:
1, Formulate a theory
2, See evidence for that theory (whether it exists or not)
3, Have someone demonstrate later that you are full of it..
3b, and give their oun theory...
4, goto 2
If you don't believe me, take a couple of classes at a
school with a good History of Science Department.
Tom is not inventing Algorithms.
He is giving practical advice.
And the demonstrations (they are not proofs) on this
site and in his books are given with practical explanations of what makes them come out the way they do.
That makes the information that can be gained by the reader
much richer and more practically usefull than it would be
otherwise.
Thanks Tom.
Proof vs. magic
Todor Botev, April 15, 2005 - 1:42 am UTC
Hi Tom,
I think at the end it all comes down to:
1. You revealed the magic from Oracle. What we all can see in your examples - Oracle is just a program that works pretty predictably - it's all numbers and how to get them.
I think this is a new situation compared to 5 years before.
2. It seems to me Ault/Burleson - they don't like this new situation. They preffer that Oracle still remains a magic that only a tight circle of "wizards" have control on. The wizards throw words like "all in one extent" (before) and "multiple block size" (now) and the others follow.
Knowledge without proof - I think "magic" is one of the names for it. Sort of "the stars told it to me".
Pontiacs don't like vanilla ice cream
David Woodard, April 15, 2005 - 2:19 am UTC
"Correlation is not Causation"
What a perfect subtitle for your article. You need to go read:
</code>
http://www.thehumorarchives.com/humor/0000730.html <code>
to understand the title of my post. The story is true, I first read it almost 20 years ago in a trade magazine when I worked as an auto mechanic. Now that's a job where if you don't find the root cause, people stop coming to your shop.
I can show that two events are correlated, heck, I can even mathematically prove it. But that does not mean that one event is the cause of the other.
Those that merely correlate events and go no further, are, as you say, hand-waving. I've always found that the best technical people, like Tom, have the ability to question their own conclusions and find the true cause for a problem.
Of course I could be wrong, and it could just be that Pontiacs really don't like vanilla ice cream (wave hands now ;-) )
April 15, 2005 - 8:53 am UTC
sounds like a story I made up on another page (i loved the ice story, perfect example)
...
I have a hunch, call it intuition, that being in my car makes my cell phone
ring. Really, I'm not making this up. You see, almost every day when driving my car, it rings. When I'm not driving my car, it doesn't ring nearly as much. No fooling. I see it all of the time, very repeatable. I don't need to understand why it is true, it just is. So, in order to get more calls, I'll go sit in my car. It has to be my car, because when I'm traveling on business and don't have my car (using a rental car), it doesn't happen. It just has to be my car.
Actually, I basically drive to and from work and that is about it in my car. My wife calls me on the way home to stop somewhere to do something like get milk
frequently. So, maybe it has less to do with me being in my car than the fact
that I tend to leave at the same time each day to go home and she doesn't like
to bother me at work. Could be that -- or it could be that my car actually
induces the phone to ring.
To KP from Singapore
SFT, April 15, 2005 - 5:35 am UTC
<Quote>
But, my question is, in the real production environment, will we be
having a loop like this:
for i in 1..10000 loop
execute immediate 'select small_vc from t1 where id = ' ||
trunc(dbms_random.value(1,10000)) into v1;
end loop;
I guess not.
</Quote>
I have a sin to confess. Some years ago I implemented client application that was supposed to show the content of message table. Each message in the table had several attributes dependent on message type. Attributes were stored in two child tables and it was not evident how to join the tables in one SELECT. I had a C++ background with no experience in SQL whatsoever, so I boldly decided to fetch each attribute by separate SELECT statement. So with say 1000 rows in the main table I performed 5000 to 10000 SELECTs from the child tables, and guess what? All these SELECTs did not use binds... I (and my company) was lucky that the application was for corporate users on relatively small DBs with at most 10-15 concurrent sessions.
Slowly and painfully I came to quite different mind set. The last blow to my old attitude toward SQL, Oracle and programming at large was Tom's "Effective Oracle....
And one side effect of the "prove it to me" approach that was not explicitly mentioned AFAIK: in the last couple of years I learned much more about Oracle and SQL then in all my previous (pretty long) career
when will Oracle stock go up again ?
A reader, April 15, 2005 - 8:20 am UTC
hallo tom,
it seemed to me you suggested to keep Oracle stocks.
(sorry, my English is not so good, so I don't understand
everything written on this page)
As I also possess some of them and as I notice that they
are going down since 13/12/04 nearly monotonously at Nasdaq,
I wonder what to do.
Since you state "Correlation is not Causation", can I
deduce from that that this is not because of Peoplesoft and
Retek takeovers ?
I personally still tend to keep them,
since I still have enough money to buy me anything funny
I want to smoke, I am not reliant in selling them at the
moment.
And I bought them because I thought they make really good
software and long-term quality will be paid off
Do you have any opinions on that ?
Thank you in advance
April 15, 2005 - 9:34 am UTC
no, I was not suggesting buying nor selling of stock.
I was suggesting that to buy or sell based on emotion is not a financially sound thing. You buy/sell based on the company performance.
I'm not a stock market analyst, asking me to analyze or explain why a stock is up or down wouldn't be a very good idea either. However, I have observed that recently the entire technology sector is sort of down (IBM seems to have something to do with that this morning).
Not sure what the smoking reference is to...
I personally am in a long term accumulate position, but I do not give market advice. That is so far away from my core competency....
David Aldridge, April 15, 2005 - 9:42 am UTC
>> I was suggesting that to buy or sell based on emotion is not a financially sound thing. You buy/sell based on the company performance. <<
Wellllll, maybe. I might say that you buy and sell stock based on what you believe others will value the stock at in the future -- whether this has any relationship to company performance or not, that's an interesting point. Not a technical one, of course.
April 15, 2005 - 10:36 am UTC
right, that would be more accurate.
emotion -- no.
future percieved value -- yes.
Facts and proofs
Richard Evans, April 15, 2005 - 11:09 am UTC
Stirring the pot again...
I, as most who visit this site, have been following the commentary of Tom, Don, Mike, Jonathan, Howard, and others intently and I think the entire debate hinges on the differences between Tom and Don's starting points.
Tom constantly says "What are you going to use X for? Write it for X! Tune it for X! Your goal should drive every aspect of the process." Absolutely.
What Don says is "It's broke and I have to fix it." Absolutely.
Both are true. Tom has the luxury, if you will, of advising people who can (and do) implement changes into the design and development process. Tom can tell you it's not appropriate to start a campfire, in South Texas, in August, in the dry grass. Don is asked to put out the fire after it's already burned 1000 acres. It's past the point of 'Hey, You shouldn't light that match'....
That established, Don is still professionally obligated to provide evidence of what he is doing, why he is doing it, what it helped and why it helped. If I were a customer of Burleson Consulting I would want to *SEE THE PROOF.* I would want to know that this is not a Band-Aid approach and I'll just have to call you back in 6 months to redress the wound.
Don and Mike, I have several of your books and in a side-by-side comparison I appreciate and expect the proofs that Tom provides in his. Both of you are obviously talented and experienced individuals; please bring that knowledge down to my level and explain step-by-step what about your process is better, faster, or more efficient. Then explain the WHY part of that answer.
Examples that include Ms. Yutz or Ms. Dumass are not facts, not proofs, nor are they professional.
Thanks.
For Jer Smith, windering about DB providing proof
Bill Schwartz, April 15, 2005 - 11:27 am UTC
Jer,
I spent less than two months posting in the Oracle DBA forum "over there". The main thread that I posted in was the "Do Single-User Scripts 'prove' Oracle Performance" thread, as I just HAD to see Don and Mike 'prove' that they (scripts) don't. Not only did I ask them repeatedly to show us
a) some scripts that purported to show Oracle Performance
b) why they were bad or incorrect
BUT I was even kind enough to provide them not once, but twice , with the dictionary definition of the term "proof" as I meant it. One of my final comments in that was a feeling akin to being a one-armed man in a rowboat: I can feel motion, I get a sensation of progress, yet I keep seeing the same scenery over and over again.
Don't hold your breath waiting for proof. David Aldridge, who is vastly more technical than I, continuously called them on points that they raised, and he NEVER got anything but obfuscation back. ora_dba_guy as well not only called them on the technical issues, but also managed to systematically destroy their faulty arguments. But through the whole thread, there was point that was glaringly obvious: nothing would be proven except the fact that from the self-proclaimed "Oracle Experts" nothing would be proven. So don't hold your breath waiting for proof. In the Oracle DBA Forum, "proof" is a four-letter word.
Slightly different thought
David Aldridge, April 15, 2005 - 12:10 pm UTC
I'm wondering why 2kb block sizes seem to be less used nowadays.
Is this something hardware/opsys related, whereby a 2kb read is implemented as a 4kb read regardless? Does it reflect changes in application design, with longer rows being common in COTS packages, and COTS packages being more common adopted? Is hash clustering more widely adopted, and that being more suited for a larger size (as in the TPC-C tests where clusters are commonly placed on 16kb blocks)?
Or is it just that they are not mentioned so much in examples and documentatation now?
April 15, 2005 - 1:14 pm UTC
represents a change in the OS layer mostly I think. the OS does the IO if the OS io size is 4k, a 2k read is 'wasting' half the work.
Howard J Rogers has written about this in the past, goes beyond my level of knowledge however (the hardware/IO size related stuff). Not my area of expertise.
some of our tpc-c's have used 2k blocks (even fairly recently, not just way back in the v7.3.4 days).
but 2k blocks have pros and cons. They are fairly rare because of limits like sorting and grouping based on block sizes, and the fact that row chaining (truly chained rows) and row migration would be frequently found.
advocacy
Alberto Dell'Era, April 15, 2005 - 3:44 pm UTC
It seems that we all agree now with the "proof method", so how to take the next step - advocating it with fellow humans living in the Oracle ecosystem ?
Ie what is the best way, or the worst way, to convince people that this method, at a first glance "hard and difficult", is the best one in order to learn/design/etc ?
I've had mixed results in the past when playing the advocator, and i would like to hear from someone that "has been there" (Tom may have a thing or two to say I bet).
I'm happy to share my experience
Giovanni Cuccu, April 18, 2005 - 6:02 am UTC
Hi Alberto,
I share my experience. I found the main difficulties while explaining this working method to managers who have little technical skill because it seems they usually think only in term of money; this could be right for a manager, but when someone asks for the cost of a solution without giving you the ability to investigate on the problem, I know I'm going into troubles. I think this is a perfect fit for consultants with crystal ball, the came and they say "do you have a perfomance problem? ok let's do ......." without investigating where the problem is.
My humble opionion is that since these managers hardly change their mindset we can only hope they got promoted to an higher position where they delegate these problems to someone else.
Just my 2 cent,
Giovanni
Begging the question
Jim, April 21, 2005 - 12:49 pm UTC
I often run into the whole problem that the solution has nothing to do with the real problem. Instead it is the old hammer and nail problem. Someone has a hammer (solution) and everything (problem) looks like a nail. (eg XML) Also you run into the tyranny of the requirement problem. This situation is where the requirements are written in such a way, could be a very conveluted way, that the answer they are promolgating is the only possible one. Any future problems mean "new unheard of requirements" are invented all to bolster the solution.
It is amazing how political technical solutions can be!
Should be an Oracle Expert
A reader, April 22, 2005 - 4:51 pm UTC
"Database Technologist is a much better term than scientist
for what you do Tom.
Despite what one might read in a general science text book,
science throughout history has worked like this:
1, Formulate a theory
2, See evidence for that theory (whether it exists or not)
3, Have someone demonstrate later that you are full of it..
3b, and give their oun theory...
4, goto 2 "
I think the best titile for Tom is, like he used in his books, Oracle Expert. In my mind, Technologist is a lower rank. I agree with you on that he is not a "scientist" because he did not invent Oracle (or other databases and database theory), he knows every coner of Oracle and certainly an expert.
I appreaciate all the help Tom had provided.
Farewell
DaPi, April 27, 2005 - 6:03 am UTC
They didn't even say goodbye: </code>
http://www.dba-oracle.com/consultant_forum_guidelines.htm <code>"Effective April 25th 2005, consultants working for Burleson Consulting MAY NOT participate in any forum, message board or blog where participants are not positively identifiable via an affirmative check . . . . . "
April 27, 2005 - 8:12 am UTC
Funny thing is, his forums offer the same amount of positive identification as all of the forums he labels "evil".
This morning, "OraKO", "Jester", "weekend79", "sweetness_and_light" has postings on the first 3 pages of his forum. I'm sure they were postively identified.
As I said above, they are misleading people by pretending otherwise:
...
you were only kidding that it would be a simple code change to require
positive idenficiation of respondents right? Come on, as a technologist
yourself you must know how hard that would be. Even Don alludes to it:
</code>
http://dba.ipbhost.com/index.php?showtopic=1396&st=30#entry5113 <code>
when one of your own users demonstrated how trivial it is to become anyone they
want on your forums. If you know of a simple code change that could
"positively identify people", you would rich many times over. (it was also very
curious to me that both you and Don posted many times after the alleged
"identity theft", many times... 8 times in fact). So no, I didn't expect you to
constantly monitor the forum, I already knew you were doing so...
................
Is this Ironic?
A Reader, April 27, 2005 - 11:50 am UTC
For someone who seems (from what I have read) to hold "simple scripts" as not being able to prove Oracle performance, you can now buy hunderds of them...
</code>
http://www.oracle-script.com/ <code>
Perhaps you need to have "lots" of them, or maybe they turn into a gestalt script when you run them!?!
Re: Is this Ironic?
Shawn Brockway, April 28, 2005 - 11:53 am UTC
My personal favorite part of the website that was posted above is the warning that is listed at the bottom of the page:
Caution - These are extremely powerful undocumented Oracle data dictionary scripts. They should only be used by a certified Oracle DBA, and after a careful review of the Oracle data dictionary script functionality. Thses Oracle scripts are not for beginners and knowledge of the Oracle data dictionary is required to fully utilize these scripts. USE AT YOUR OWN RISK.
Hmmm...Since you MUST be a certified Oracle DBA to use these scripts, does that mean that my co-worker, a DBA of 15 or so years, can not use these since he is not an Oracle Certified Professional? Of course, I may be a prime candidate for these scripts since I am Oracle Certified (though only have 5 years experience). :) Definitely brightened my morning.
who likes deadlines?
Gabe, April 28, 2005 - 2:59 pm UTC
<quote>Let's call it a trilogy</quote>
Still planning for a trilogy? Just curious.
April 28, 2005 - 3:09 pm UTC
Are you wrong to schedule table and index reorganizations?
Dan Clamage, May 03, 2005 - 6:18 pm UTC
Oracle Myths !Debunked
Old Ada Guy, May 04, 2005 - 3:38 pm UTC
May 04, 2005 - 5:59 pm UTC
And it is all Don's material to boot :)
What amazes me, truly, is a "debunk" article that says things like:
"Today, it is well understood that tables and indexes ..."
Well understood? Why? Where? That is like saying "it is true because I said it is true" -- but wait, that is their corporate policy now:
</code>
http://www.dba-oracle.com/consultant_data_obfuscation_privacy_falsification.htm <code>
<quote>
We are under no obligation whatsoever to provide reproducible evidence or test-cases to beginners and it is our readers trust in our judgment and real-world experience that counts. Resist the temptation to fall-in to the prove it trap.
</quote>
wow. That is all I can really say -- wow.
But unfortunately, Dons -- wait I mean Mikes paper of course doesn't get a single fact correct.
Take a pargraph like:
...
Most people agree that many of today's Oracle myths were perfectly valid during their day (e.g., "disk load balancing is critical to performance") but they became mythological as hardware and Oracle software improved.
.....
I did not know that disk load balancing was no longer critical to performance. Did you?
Maybe I'll make that paper the subject of the part III -- I was just going to do the "indexes *love* large blocksizes" theory, this would be easier.
then again, it is almost just not worth the bits.
Sweet Odin's Raven
Alex, May 04, 2005 - 7:24 pm UTC
I seriously had to read that policy like 3-4 times to believe someone actually wrote that....about their own company!?
I also like how Don plays the condescending card every time too as an excuse not to prove anything. If you are a beginner, than you do not have the mental capacity to possibly comprehend awesomely deep explanation that is reserved only for experts. Oh wait, no, only experts who went to ivy league schools....and wear suits etc.....etc.
May 04, 2005 - 7:58 pm UTC
I utterly and completely fail to see how a top 5 timed events from a real statspack would identify any individual -- EVER.
That is what this comes down to. It is not obscuring data, there is a stronger word that can be applied.
</code>
http://www.jlcomp.demon.co.uk/review_01.html <code>
(and you can see their resumes, there are no ivy leagues amongst them)
Condescension - an example
Doug, May 04, 2005 - 7:40 pm UTC
Yes, that reminds me of conference presentations with titles like this (not one of BC's, but the same principle)
"Tuning for the Advanced DBA; Others will Require Oxygen"
Others might require a bit more technical detail and a lot less arrogance, actually
May 04, 2005 - 8:08 pm UTC
I always want a category of "all comers" for most sessions (they never have that one)
I almost always target beginner and intermediate and advanced alike. I've been fairly succesful doing so (i've been told). It doesn't have to be complex.
Ajay, May 04, 2005 - 7:43 pm UTC
The whole fiasco of the last few months boils down to this:
1. Don "sponsors" a predictive re-org challenge.
2. The precepts of the challenge are challenged.
3. Instead of responding to the challenge, Don and Mike claim their "real world systems" are significantly different beasts from the systems us Oracle hobbyists are used to. No scripts (or maybe an incorrect script or two, can't remember) are offered. That doesn't fly.
4. Don and Mike then cite client confidentiality concerns as the reason for not being able to present test-cases to back up what they claim. At this point, they leave the debate without having said much for themselves.
5. Don forbids his employees from using the "prove-it" approach. Burleson Consulting becomes the world's first faith based Oracle organization.
May 04, 2005 - 8:08 pm UTC
#5... nice.
but yes, it would be neat to be taken on technically, instead of personally.
If they could point to a SINGLE example, just one... That'd be really neat.
A reader, May 04, 2005 - 9:20 pm UTC
Loved the article of JL, i was laughing when i was going through it.
Thanks Tom & JL if you are reading this thanks for a such a wonderful & hilarious article.
Jay.
David Aldridge, May 04, 2005 - 10:25 pm UTC
Hey Ajay,
You missed out a critical stage -- where scripts supporting one theory are hailed as "proof", while scripts questioning it are derided as the work of neophytical "script kiddies". Don't forget that one. Verrry important.
Schrodinger's Cat, Oracle Kung Fu and The Deaf Fly
Billy, May 05, 2005 - 7:46 am UTC
I met Schrodinger's Cat as a lad in school delving into the very basics of quantum physics with ideas of becoming an astronomer some day (before the computer bug bit me hard and I got terribly infected).
As reality and truth are closely related, I got in the habit of writing the following phrase in all my text books (at school and university). "Everything in this book may be wrong". And tried to make it less threatening sounding by adding (in large friendly letters of course), "Don't Panic".
There was rumoured to be this "Empirical Scientist". He placed a fly on a table. He clapped his hands. The fly flew away. He dutifully recorded on his clipboard, "Fly's hearing is excellent". Caught the fly again, broke off its wings and repeated the experiment. The fly this time around remained on the table. He then wrote on his clipboard "Fly exhibits severe hearing loss without wings".
We must remember that we do not have the foggiest of notions whether or not the Cat is alive or dead until we question, seek proof and open the steel box and see what the Cat is up to.
And hell yeah.. nor are the ears of a Fly always located in its wings.
But what does this have to do wih me practising my Oracle Kung Fu?
I always put SQL*Plus scripts together (often with core stuff borrowed from Tom, Jon and others) to work out the proof and the "reality" and "truth" for myself. Using the manuals and "expert opinions" with as much common sense as possible. Side note: Commen Sense. Often the ingredient we fail to use.
Show me a better way of finding proof, and I will give it a bash. Wingless Deaf Flies ain't it. So until then, SQL*Plus scripts it is.
But at the same time, I know that the proof of that "reality" and "truth" is only applicable to what was "observed" and "measured" at that point in time. Know that there are no single or absolute truths to performance tuning, and no magic wands that can be waved.
To every rule, there is an exception. Heck, there are even exceptions to the exception. And exceptions to that... (and then there is a really weird Feynmann diagram with funky photons moving into the past to create that which creates them).
Schrodinger's cat is.. well neither alive or dead until you actually peek.. but is doing pretty well otherwise.
Which pretty much sums up this whole debate for me.
Faith healing ?
Greg, May 05, 2005 - 8:09 am UTC
[quote]Burleson Consulting becomes the world's first faith based Oracle organization.
[/quote]
What's even scarier is the apparent popularity that Faith Healing has in some areas, and how many people are "believers" ... and yet, when anybody actually does some digging "under the covers" (I recall watching a show by W5? or CNN? or something trying to ask Benny Hinn a few questions ... which never got answered) you find that there's a lot of deceipt and deception .. (aka "smoke and mirrors").
I guess P.T.Barnum (or David Hannum - depending on which side of History you want to read) ;) was right:
"There's a sucker born every minute."
That kind of makes the fields "ripe for the picking" if you have something that sounds great .. easy to pull in the ones who don't ask questions and just don't have the experience.
I have my own faith, however, in the basic fact that no matter what .. (no matter how much cover-up, smoke and mirrors, etc.,) that basic education, investigation and research ... the fundmental ability to ask "Why?" .. or even "How?" .. or whatever is appropriate ... will bring to light the truth ... the whole truth .. and nothing but the truth ..
... eventually ... ;)
Just a thought for Don and Mike and any others who prefer this "faith healing" style of work, as opposed to a methodical approach in understanding of the basic fundamentals ..
How was it that we managed to put a man on the moon, without the ability to "test" the ability to put a man on the moon?
That is, if testing in a "smaller" environment, in a smaller scale is "nonsense" ... then why did NASA waste so much time with their astronauts in "UNDERWATER TANKS" practicing for the "real thing" .. ???
According to the Big "B", that's impossible, and was doomed to failed from the start ...
.. and yet ... it didn't fail ... it worked ... several times ...
How did we launch the Space Shuttle without "testing it" ? According to the "faith healing" approach, you just slap it together and "hope" it works the first time ...
Personaly, I prefer Tom's methods (and Nasa's, btw) ... of testing the individual components and making sure they all work independantly of each other first ...
Then - and only then - do you piece things together in a bigger picture *using the knowledge you have* of the smaller pieces. That allows you to better understand the entire approach.
Are mistakes made? of course, everyone makes mistakes ..
(2 shuttle disasters, ok .. but how many successes? not a bad track record, IMHO) ... but even then, the track record of somebody who thoroughly tests the fundamental components before diving into a "large environment" is laps ahead of the person who skips the basics ...
Faith based? Sounds familiar
Bill Schwartz, May 05, 2005 - 9:20 am UTC
Indeed, there may well come a time where every neophyte DBA wannabe who subscribes to this stuff will ONLY be able to get a job in one place (guess where?), leaving the rest of us to do our jobs in relative peace. While they cry about being derided and discriminated against. But hey, at least they won't have to "prove" that they are right. Compelling argument, when you don't have to prove anything you say. Like I said originally in the performance challenge thread, when I want myth, I know where to find it. When I want the truth, I know where to find that too. Thanks to all the truthsayers - we are grateful for your willingness to share the hows, whys, and wherefores.
May 05, 2005 - 9:40 am UTC
This is to me the crux of the paper way above, way up there.
If you want to publish, if you want to have articles, if you want to give information in forums -- provide evidence that what you say is true. Period.
Otherwise you are not credible at all.
It happened to me the other day. I was in a rush.
</code>
http://tinyurl.com/bo7hl <code>
I was asked a question and simply said "yes".
But didn't give an example. Led to more confusion than anything else on the posters part. So I came back and said "yes and here is how......."
Anyone that says "trust me, because I'm experienced" should be ignored. totally.
I have 17 years experience with Oracle -- that makes me wrong more often than not, until I test it.
Don't need to prove to "beginners" ???
RBARAER, May 05, 2005 - 10:24 am UTC
About Burleson Consulting "Data Obfuscation and Client Privacy" :
"We are under no obligation whatsoever to provide reproducible evidence or test-cases to beginners and it is our readers trust in our judgment and real-world experience that counts."
Tom, it seems that you fall into DKB's "beginners" category. Well, since he seems to consider himself as the world's top Oracle expert, it must be an inverted scale ;), so IMO it's a great honour ! Just kidding. Really pathetic.
Well, anyway, it seems like DKB once again broke his own etiquette :D.
Off topic, since it's the first time I write here on asktom, I just wanted to thank you for all you bring to the Oracle Community. It's been several months now that I've been learning things from this site, and it was more than useful. I also bought your two books and already read "Effective Oracle By Design", which is EXCELLENT. I fully agree with your "prove-it" philosophy, and I really hope the crusade against self-proclaimed "trust me" Oracle experts will succeed someday...
Best Regards,
RBARAER
No silver bullets
Jim, May 05, 2005 - 11:23 am UTC
Don now is hawking a book on silver bullets for Oracle. (literally Silver bullets is in the title)
I don't know what is required reading in computer courses these days, but I hope Fred Brook's The Mythical Man Month is. Fred also wrote an article "No Silver Bullet Essence and Accidents of Software Engineering" back in 1987. (</code>
http://www.virtualschool.edu/mon/SoftwareEngineering/BrooksNoSilverBullet.html <code> To me both are still classics and relevant.
Tantric Oracle
Ajay, May 05, 2005 - 11:26 am UTC
"We are under no obligation whatsoever to provide reproducible evidence or
test-cases to beginners and it is our readers trust in our judgment and
real-world experience that counts."
Lots of chanting, and practices based on firmly stated, unverifiable beliefs. Just like Tantra.
David Aldridge, May 05, 2005 - 12:39 pm UTC
I think that we have to acknowledge that none of us are under any obligation whatsoever to provide scripts, evidence, or even a theoretical background for our beliefs and advice. We do it because we want to be technically correct and we want to have professional pride in our work. We do this through the mechanism of peer-review. And by "peer" we don't mean "our friends and co-workers", we mean external experts, and we permit review in an open and non-censored public forum.
I think that abandoning this principle is a serious impediment to the continued credibility of any author.
May 05, 2005 - 1:12 pm UTC
I do it so you can see what I did, what I meant, what the context was.
And so you can see any holes that might be in the work. What mistakes I might have made. What assumptions are implied.
besides, i'm usually wrong when I don't do it, and I think those that don't do it are too.
True David, but we are obliged to back up what we say
Bill Schwartz, May 05, 2005 - 2:33 pm UTC
While there is certainly no "obligation" to provide scripts or evidence, there is the obligation of backing up what you say with some tangibles. Otherwise we are back to the shaman using arcane procedures to protect us from evil. One of the reasons why I like reading the things folks like you write, is that you don't try to keep this "evidence" if you will to yourself. You let the rest of us see the fruits of your labor, which in turn helps us to better ourselves. In my mind, a win-win situation (you get the gratification of having helped someone, I get the gratification of learning something I didn't know). So, I guess my position would be - if you don't want to back it up, then you shouldn't say it. At LEAST give everyone a plausible reason for what you say (or why you say it). That much we OWE to the folks reading what we write.
David Aldridge, May 05, 2005 - 4:37 pm UTC
>> besides, i'm usually wrong when I don't do it, and I think those that don't do it are too. <<
Well here I'm trying to distinguish, and not very well I guess :( , between doing the background investigative/research work on one hand, and publishing that work with your conclusions on the other -- so my comment was aimed at the publishing part more than making sure that the work is correct to your own satisfaction (which I rather take for granted).
I mean to say, what sort of idiot wouldn't actually go and test their hypotheses in private at least before publishing them?
May 05, 2005 - 6:10 pm UTC
Publishing part and advising part. That's my "bug" too. You want to publish, you want to write, you want to hand out advice. Then show me. Give me some compelling evidence. Else you probably have correlation without causation.
I observed that my cell phone rings most often in my car.
I don't know why... it just does.
My car must have some influence over that....
(or my wife knows when I drive home and always calls me to tell me to stop somewhere to pick up something, that couldn't be it could it -- nah)
"What kind of ..... hmmm....."
David Aldridge, May 05, 2005 - 4:42 pm UTC
>> While there is certainly no "obligation" to provide scripts or evidence, there is the obligation of backing up what you say with some tangibles. Otherwise we are back to the shaman using arcane procedures to protect us from evil. <<
Ah ha, shamen are an interesting case. They weren't obliged to backup their work with anything, which is why you don't see many of them around today I suppose. So the tale is a cautionary one for those who believe in *ahem* more faith-based methodologies.
"what sort of idiot"
Doug, May 05, 2005 - 7:02 pm UTC
Well, having slipped up in a *big* way recently due to my lack of self-checking (and being corrected on it by Tom and others), I must say I feel like an idiot, but then I know I'm not (at least not always). It's easily done, actually, if you let your standards slip and you're convinced you 'know' something which isn't true. Really it's just laziness and I think sometimes the more you *think* you know, the more lazy you become.
As Tom said above "besides, i'm usually wrong when I don't do it" - so there have been times when Tom hasn't done it?. Even just the one time? Does that make Tom an idiot (I don't think so). It's so easy to take shortcuts ...
If you publish your ideas, though, I think you're implying that they have some value and should make sure that you can back them up. (and trust me, I'll be paying much more attention to that in future)
I'd like to think I wasn't a complete idiot though - maybe what makes you an idiot is the inability to accept your mistakes, correct them and move on?
May 05, 2005 - 7:13 pm UTC
... *think* you know ...
therein is the issue I think. I talk about that alot in the "things you <think you> know" talk I give.
It ain't the things you don't know that get you in trouble, it's the things you know that just ain't so (or ain't so anymore)... artemus ward.
... maybe what makes you an
idiot is the inability to accept your mistakes, correct them and move on?...
I like that.
Ajay, May 05, 2005 - 7:10 pm UTC
Good Old Story...
Rajesh, May 06, 2005 - 5:58 am UTC
<Quote>
I observed that my cell phone rings most often in my car.
I don't know why... it just does.
My car must have some influence over that....
</Quote>
When reading this, I got reminded by the good old story of a GM Car Owner. Being a person with email account(s), you should have come across this story many times...Just incase if you need the link here is one </code>
http://www.rotaryniagara.org/thoughts/impossible.shtm <code>
May 06, 2005 - 8:03 am UTC
ctl-f for
Pontiacs don't like vanilla ice cream
above :)
War Story
DaPi, May 06, 2005 - 8:16 am UTC
Sorry - I can't resist.
Coffee is better than strawberry ice-cream - true story:
I drove a vintage car from UK to Switzerland in January (that proves I'm crazy). It had performance problems that were solved by drinking coffee.
.
.
.
.
.
.
.
.
The carburators were icing up and, in the time it took to drink a coffee, the heat from the engine melted the ice.
Truth
David, May 06, 2005 - 9:29 am UTC
May 06, 2005 - 9:44 am UTC
I just ordered Crimes Against Logic, the US version of the same :)
Youre a better man than I
Christopher Jung, May 11, 2005 - 9:04 pm UTC
... to keep your response to other alleged experts this professional.
Thanks for all youve contributed to the community!!!
The "cut-and-paste" generation (sigh)...
A Woodward, March 29, 2006 - 8:38 am UTC
Hi Tom
I've followed the ongoing debate about Don Burleson with some interest, and have come across various comments about Rampant books along the way. With no experience of Rampant books, I can't add anything to that debate.
Not that I'm wanting to pour petrol on this particular fire, but imagine my surprise when having Googled 'PLSQL_OPTIMIZE_LEVEL', I came across this link to an 'Oracle Tip by Don Burleson':
</code>
http://www.rampant-books.com/10g_193.htm
"...The above text is an excerpt from "Oracle Database 10g New Features: Oracle10g Reference for Advanced Tuning and Administration", by Rampant TechPress. Written by top Oracle experts, this book has a complete online code deport with ready to use scripts. "
How strange that looking in the Oracle documentation for this feature:
http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14237/initparams164.htm#sthref685 <code>
...we get EXACTLY the same text, except that it lists the possible values for the parameter as 0/1/2, whereas DB's "tip" just lists 1/2. Written by top Oracle experts indeed - in fact written by Oracle!! Mind you, Don's no mug - cutting and pasting from the (free) documentation and selling it (for real money) as his own work. Pretty smart, huh? :-)
Anyway, many thanks for all your work on this site - very much appreciated by anyone who works with the Oracle database.
with best wishes
Andrew
Scripts for Certified DBAs
Marco, January 02, 2007 - 10:36 am UTC
err... politics aside - I have a question about the original article
Scott Wesley, February 13, 2008 - 9:05 pm UTC
G'day Tom,
In regard to the original article "In search of the truth", (which I think should be standard reading for all university students) you made a comment about 10g functionality.
"The reason - in Oracle 10g, PLSQL is silently array fetching 100 rows at a time for us"
I've been looking hard in the Oracle documentation for a reference to this feature to no avail. Any pointers?
Cheers
February 13, 2008 - 11:13 pm UTC
not everything can be, should be, or will be documented. Every nuance of Oracle - not documented.
this is one of the hundreds of "plsql compile time optimizations introduced in 10gr1"
they rewrote the plsql compiler to be an optimizing compiler for the first time with that release, with various optimization levels. At the default level, one of the many things it does is this array fetching, which we can observe with tkprof easily.
... from the doc ....
Release 1 (10.1)
Improved Performance
PL/SQL performance is improved across the board. Most improvements are automatic, with no action required from you.
Global optimization of PL/SQL code is controlled by the PLSQL_OPTIMIZE_LEVEL initialization parameter. The default optimization level improves performance for a broad range of PL/SQL operations. Most users should never need to change the default optimization level.
Performance improvements include better integer performance, reuse of expression values, simplification of branching code, better performance for some library calls, and elimination of dead code.
.................................................
consider this in the "better performance for some library calls" category.
'Advanced' Scripts
SeánMacGC, February 14, 2008 - 6:34 am UTC
@ www.oracle-script.com
USE THESE ORACLE SCRIPTS AT YOUR OWN RISK
You don't say! ;o)
Forgot to mention
SeánMacGC, February 15, 2008 - 8:27 am UTC
What an exquisite gem of a question in the first instance!
all lies need proofs
Adrian Angelov, April 01, 2012 - 3:38 am UTC
Hi Tom,
it's 1st of April, so please, don't lie :)
What is the biggest lie you've ever heard about Oracle database from technical point of view?
What is your personal best ever answer that proofed that such a lie is wrong(I'm sure that you can think of thousands of these, but which one pops up in your mind first, something superior, something that makes you proud of)?
and the easiest one, and I don't lie.
What is the biggest lie you've ever wrote about Oracle database from technical point of view and needed some time to realize that you're mistaken?
Thank You
April 03, 2012 - 6:11 am UTC
I'm not sure the word "lie" is the best word.
Misinformation, myth, wrong conclusion, bad ideas - etc. Most of the time I don't think people are lying inasmuch as they are just "wrong"
There are too many out there to have a "best"
Maybe this one though:
http://asktom.oracle.com/Misc/birth-of-asktom.html Not so much a lie as really horrible advice.