Alexander, August 05, 2009 - 2:56 pm UTC
Tom,
I'm curious why you never bring up the fact that RAC reboots servers if it feels it needs to when people ask this question. I would think that would be the quickest, easiest way to end this potential bad decision.
So if rac node 1 detects a problem on rac node 2 and decides it's time for a reboot, and there's random instance xyz on node 2, it's going down as well.
August 05, 2009 - 4:19 pm UTC
there are so many reasons - but yes, an eviction of an instance could take that host with it.
All about that shared resource (the HOST) and the fact that if instance 1 wants to completely obliterate it - there is nothing instance 2 on that HOST can do about it.
Deja Vu
Charlie, August 05, 2009 - 3:10 pm UTC
This is exactly what we have now - no you don't want to be like us - a single bad sql will destory nodes and make them reboot one by one like chain reaction.
Low usage nodes
Curtis Ruck, August 05, 2009 - 4:01 pm UTC
Tom,
What about low usage instances? For example we have two databases that are on a single RAC cluster. So each node has two instances. Our instances barely do work and we only have them RACed for redundancy, and if it wasn't for one of the applications using one database requiring (some days i truely hate non-db smart friendly application developers) a few too many public grants we would of had them in one database.
August 05, 2009 - 4:28 pm UTC
we are still talking about taking that single HOST and making it TWO HOSTS
why bother with redundancy for the low impact node - it cannot be sure it will be able to get any resources, the high usage node "owns" that node if it want's to - starving the low use out out of existence.
You are using RAC for one of two reasons:
a) horizontal scale out
b) reduced unplanned downtime
(a) cannot be the case here - not if you are going to run two instances, if you were running out of capacity and needed to scale out horizontally - you would NOT have two instances, that would be beyond crazy
so it must be (b), but, well, you won't get (b), you'll get the OPPOSITE of (b)
In your case, I would be consolidating the low usage node application into the other one - one database, not two.
two or more DB on RAC,
A reader, August 05, 2009 - 4:50 pm UTC
Tom,
Thanks for reviewing my original question. The team is thinking of having RAC DB for both the databases and reduce the hardware cost in doing so. This option at a very high level looked feasible (not thinking too technically). However, there are lots of disadvantages and the main is node failure impacts both the database and performance bottleneck on one of the DB impacts others.
Which option to choose? Spend more money on hardware and licenses or take this risk? Hard to debate!
August 05, 2009 - 5:12 pm UTC
better choice 3) LOGICAL PARTITIONS - VIRTUALIZE - make a single host into TWO hosts.
better better choice 4) have a *single database* - consolidate - spend less money altogether maintaining stuff.
Excellent options,
A reader, August 06, 2009 - 10:26 am UTC
We are planning for option 4 (consolidating database) perhaps next year.
Do you have any white papers or documents or URL about option 3 (virtualizing host)? We are running on HP blade server using RHEL 5 version.
I appreciate your help (as always).
Thanks,
August 06, 2009 - 11:00 am UTC
Oracle VM,
A reader, August 10, 2009 - 2:45 pm UTC
I have few questions about Logical partions/virtualization of servers.
1. In our existing 3 server/blade environment, we can make it 6 using virtualization. Does Oracle's VM does that or do we need to use VMware?
2. If we go for virtualization, then we end up having 2 Oracle RAC databases (each running on 3 virtual servers). Do we need buy extra licenses for the 2nd Oracle RAC database?
3. What additional hardware components needed for setting up virtualization? I believe, the fibre channels (HBAs) to connect to common SAN strorge remain same, the etheret cards for public and private network remain same. Anything other components you think is required to support virtualization?
4. How easy(difficult) is for the OS team to support and monitor the virtual servers? Does Oracle VM or VMware provide good tools to monitor the servers?
Thanks a lot,
August 13, 2009 - 8:52 am UTC
1) we can do that.
2) you license to the hardware, you license to the cpus
3) you'd want each VM to be as hardware independent from the others as possible. You need to duplicate nothing, you want to duplicate as much as you can.
4) yes we provide tools, it is just the same as monitoring any OS would be.
Excellent inputs from TOM
Adarsh Kumar, August 13, 2009 - 1:00 pm UTC
my two cents having two database on one machine...
I agree 2nd database on single instance may cause unplanned outage but you will save licencing cost by not having to install other Database on seperate cluster machine.
I think it's a trade off and a business decision to make
Thanks
Adarsh Kumar
August 24, 2009 - 7:07 am UTC
there are no cost savings here - you lose the ability to control everything - you gain nothing.
If all you have is a single physical machine, virtualize it so you have many HOSTS on it. That was said above:
Unless you logically partition these machines - so that instead of 3 hosts you have 6 - do not go down this path.
You have added zero cpu's to the configuration, you have doubled the number of hosts, you have the ability to assign specific resources to specific hosts and regain control once again.
Oracle VM
Kyle Brown, August 18, 2009 - 8:34 pm UTC
I'm like most people reading this wondering how can I modify my existing environment to conform to the Ask Tom best practice of having one database per host. Oracle VM to the rescue right? But I'm not sure I understand. You said that one reason was tunability. Which is easier to tune, multiple databases on one server where all the proceesses are out in the open or processes that are hidden by VMs? Either way there are some very good reasons for having multiple databases rather than one large one, I'm just not sure I understand why VM would be better.
August 24, 2009 - 4:46 pm UTC
processes hidden by VM's that are DEDICATED, ALLOCATED a set of resources
definitely.
With virtualization you get to say "this VM gets X% of the CPU and has Ygb of ram"
without it, you get to say nothing.
Uh oh, should I rethink my architecture?
M. Tobin, August 31, 2009 - 3:34 am UTC
I was planning to run my SOA Suite DB on the same 2 node RAC cluster as my Oracle EBS r12 DB - 10.2.0.4.
My main reasons were:
provide high availability for both EBS and SOA;
reduce server costs;
reduce Oracle license costs;
eliminate DB version lock between SOA and EBS;
and reduce hosting provider costs.
My hosting provider will not support virtualization on the DB tier.
Do you think it's time for me to get two more servers for SOA DB?
August 31, 2009 - 7:33 am UTC
... provide high availability for both EBS and SOA; ...
makes it less reliable - one node does a bit more work than the other - consumes all of the CPU's on the machine and that one node will cause the other node to evict it due to poor response times. Or, you just always experience unexplained "performance issues - but nothing that any tools shows you - since they look at the instance and the performance issue is the other UNRELATED instance"
... reduce server costs;
reduce Oracle license costs; ...
virtualize - same amount of hardware, same number of cpus, but you have a firewall between them - you will want separate NIC's and networks for each - so you'll need a bit more hardware, but not very much more.
.. and reduce hosting provider costs. ..
you are increasing it by having two separate databases...
... My hosting provider will not support virtualization on the DB tier. ..
new hosting provider time?
instance Caging
A reader, September 23, 2009 - 10:11 am UTC
Would be the new advertised feature - Instance Caging - a suitable solution for this case (instead of virtual machines or LPAR)?
September 28, 2009 - 2:24 pm UTC
you still have memory issues
you still have IO issues
you get BETTER control over the amount of CPU, but you will not be able to really truly limit it to N-cpus used by instance X. The caging only affects FOREGROUND processes, not the backgrounds at all.
so, does it help? a bit.
does it solve the problem? no, not entirely.
what would it be most useful for (instance caging)? On a SINGLE INSTANCE - if you have say a 16 cpu machine, you might reduce the foregrounds to 14 cpus to ensure the backgrounds always have access to some cpus and that the machine doesn't grind to a halt when you hit 100% utilization.
We could achieve the same effect today by using the resource manager and using a policy that limited the number of concurrent sessions to 14 (on that 16 cpu machine) and putting everyone into the group with that policy. This (instance caging) makes it easier to implement this rule.
Services and Resource Manager
S. Moon, February 11, 2010 - 2:24 pm UTC
Well I am very much a supporter of VMs it seems to me that it has been touted as the fix-all solution for all datacenter problems. Concerning RAC and multiple instances, what about implementing services and resource management through Oracle instead of using virtual machines?
February 16, 2010 - 11:04 am UTC
... it seems to me that it has been touted
as the fix-all solution for all datacenter problems. ...
It seems to me to be one of many suggestions - I cannot think of a data center that would not benefit from it, but I don't think anyone is saying "if you use it, you will be better than anyone else" either.
It is a tool - one that should probably be used by most everyone.
I will say and continue to say for my entire life:
The maximum (and to me minimum :) ) number of instances per host are.......... ONE and only ONE.
If you are thinking of doing more than one instance on a host, use VM's to make more hosts.
If you can consolidate N databases into 1 - that is even better, but if you cannot - 1 host = 1 instance.
host
A reader, March 15, 2010 - 12:08 pm UTC
Tom:
<<<The only correct number of instances per host is - one. If you have more than one instance per host, you cannot prevent one instance from consuming all of a given resource - causing a virtual outage of the other instance.>>>
Are you serious? The big unix server/machine that supports our applications has 100 instances (DEV, TEST, PROD) running on it that supports different groups and databases.
There is one instance/database.
Let me get clear on this.
Are you saying that the architects should have one huge instance for everyone and you consolidate 100 databases into one. It is not practical.
can you explain.
March 15, 2010 - 12:17 pm UTC
... Are you serious? ....
deadly serious.
... Are you saying that the architects should have one huge instance for everyone
and you consolidate 100 databases into one. It is not practical.
...
Sam also known as SMK - I truly believe once again that you did not read what I wrote, or you did not understand it. I'll copy the relevant bits here again:
"Unless you logically partition these machines"
"better choice 3) LOGICAL PARTITIONS - VIRTUALIZE - make a single host into TWO hosts. "
"If all you have is a single physical machine, virtualize it so you have many HOSTS on it. That was said above: ..."
"virtualize - same amount of hardware, same number of cpus, but you have a firewall between them - you will want separate NIC's and networks for each - so you'll need a bit more hardware, but not very much more. "
and to sum it all up
"The maximum (and to me minimum :) ) number of instances per host are.......... ONE and only ONE.
If you are thinking of doing more than one instance on a host, use VM's to make more hosts.
If you can consolidate N databases into 1 - that is even better, but if you cannot - 1 host = 1 instance. "
If you run dev, test and prod on a single host - well, that doesn't sound too brilliant. We all have stories of the test instance going bonkers, the development instance bringing the machine to it's knees.
If you truly have 100 instances on a single machine either
a) your computing needs are trivial
b) you have a machine much larger than I can fathom or have seen
... Are you saying that the architects should have one huge instance for everyone
and you consolidate 100 databases into one. It is not practical.
.....
Never said that, if you think I did, please point out where. I will say, have said, will continue to say:
the minimum and maximum number of instances per HOST is one. (Host is very specifically chosen as the word there - not server, not machine, not piece of hardware - but HOST, operating system image, a thing with certain cpu, memory and IO resources, a container, a virtualized environment)
instance
A reader, March 15, 2010 - 12:33 pm UTC
Tom:
I think the server runs a few hundred thousands dollars. It is 100 gig RAM with 16 dual-core CPUs, etc..
To me HOST = SERVER = MACHINE.
How did you defne the HOST? I did not understand your defintion.
Virtualization is a new thing and might take time to implement and requries some expertise i guess.
Would there be serious consequences if you have one instance per database and 100 of them run on same machine. It seems to run OK for the x number of years. I have not major problems.
March 15, 2010 - 12:43 pm UTC
even with 16 dual core cpu's, you don't have enough for 100 instances unless...
Your computing needs in each instance is trivial.
"To me..."
"Virtualization is a new thing..." bzzzzt so wrong
Well, get over that (the to me part), it isn't true, hasn't been true, will not be true. On virtualization, what you said - it never was true; from the early days of computing (think big bad mainframes, they were 100% virtualized with many LPARS - logical partitions - from the beginning). Isn't true today - Solaris comes with virtualization, Vmware, Zen, etc - they are run of the mill - dozens of the. Won't be true tomorrow - you'll see even more of it.
Sam, SMK - If you have a big bad machine - I'll betcha your data center people are doing it and you just don't even know it. Do you really mean to say that you can do a ps -aef (or whatever you prefer to type to get all processes) and in the output you see 100 pmons?? Really?
host
A reader, March 15, 2010 - 12:52 pm UTC
Tom:
Thanks for the info. I will check if they do any virtualization.
But what is your definition of HOST??
is it virtual hosts on same machine.
March 15, 2010 - 3:32 pm UTC
a host is an OS image, the feeling that you are all alone - that no one else is on the machine (virtually or otherwise) with you, that you own the resources you have been granted.
read the original answer where I wrote:
... Unless you logically partition these machines - so that instead of 3 hosts you have 6 - do not go down this path. ...
they have 3 nodes, 3 machines, they want two databases - on three nodes, they should create six host "operating systems" - two on each physical machine - to do that. (but even that would be not as good as just have a single database)
virtualization
A reader, April 26, 2010 - 10:24 am UTC
Tom:
<< If you have a big bad machine - I'll betcha your data center people are doing it and you just don't even know it. Do you really mean to say that you can do a ps -aef (or whatever you prefer to type to get all processes) and in the output you see 100 pmons?? Really? >>
yes, i found what i said is true. They have 100 instances running on one host/partition and they see 100 pmons.
what do you think of that?? They do have 5 oracle homes though which they seem to think is a different server.
Now, that may explain many of the issues we had with one production instance because one *hosed* instance can affect all others.
When you do the MEMORY allocation for each instance, cant oracle only use the memory specified.
April 26, 2010 - 10:35 am UTC
... what do you think of that??...
guess what I think of that - it won't be hard, just read the original entry.
... They do have 5 oracle homes though which they seem
to think is a different server.
...
either you are confused by their information or they are severely confused about how things work.
... When you do the MEMORY allocation for each instance, cant oracle only use the
memory specified.
...
you can limit SGA memory - but what about pga memory - instance A can (and probably will some day) allocate all memory on the machine - paging/swapping out the other guys SGA and PGA allocations, killing the machine.
And your Admins have to know how to allocate memory to the SGA's in a way to support 100 instances on that single machine.
memory
A reader, April 26, 2010 - 1:13 pm UTC
Tom:
it sounds like having 100 people riding one bus. if one has a FLU , then everyone else may catch it
Can you clarify this.
<<you can limit SGA memory - but what about pga memory - instance A can (and probably will some day) allocate all memory on the machine - paging/swapping out the other guys SGA and PGA allocations, killing the machine.>>
If you configure oracle instance A with 200 M SGA and 50 M PGA, can it still exceed that and grab more RAM from other intances if it needs more than that.
if you create 100 LPARS, is not that really hard/difficult to manage?
April 26, 2010 - 1:35 pm UTC
... it sounds like having 100 people riding one bus. if one has a FLU , then
everyone else may catch it
...
and when the bus crashes... I like it, thank you - I'll be using that.
see
http://asktom.oracle.com/pls/asktom/asktom.search?p_string=untunable+pga+memory ... if you create 100 LPARS, is not that really hard/difficult to manage?...
If you create 100 instances all sharing the same infrastructure, is not that really hard/difficult to manage?
In for a penny, in for a pound, if you have the actual need to run 100 instances (I'm skeptical), then you need to have the setup to do it. There are tools to manage these things.
Alexander, April 26, 2010 - 1:42 pm UTC
"yes, i found what i said is true. They have 100 instances running on one host/partition and they see 100 pmons."
He/she just made me feel a whole lot better about my job. That's unholy.
Database/Instance/Schema
David Cryderman, April 27, 2010 - 3:40 pm UTC
I can not wrap my mind around 100 instances on a single host, I do find the terms Database/Instance/Schema often used in the same context. My hunch is that this is yet another miss use of terms...
April 28, 2010 - 7:32 am UTC
No, there are 100 instances (100 pmons) for 100 databases on this single machine.
No misuse of terms - I asked specifically 'what is ps | grep pmon' - answer - 100 of them...
About Licencing
Rajesh Ranjan, April 07, 2011 - 2:05 am UTC
Hi Tom,
I am new to oracle.
I have oracle database 11g R2 enterprize edition.
Questions:-
1. What are the featurs oracle provide by default?
2. What are the featurs for which I wiil haveto take licence?
Thanks
April 12, 2011 - 1:19 pm UTC
A reader, April 12, 2011 - 2:50 pm UTC
Not that excellent. Bad answer.
April 13, 2011 - 9:53 am UTC
bad review. What do you need to have it SPOON FED to you. I pointed you directly to the document that completely, utterly and totally completely, unambiguously answers your question.
If I were to answer your question myself, you know what I would have to do? I would have to write that document - that would be stupid wouldn't it.
Thanks for clarification
Jessica, June 09, 2023 - 5:33 pm UTC
Thanks Tom for clarification.
"The only correct number of instances per host is - one" this best implementation also apply for 19c?
Or 19c can accommodate two DB instances or more well?
June 12, 2023 - 4:34 am UTC
Well....dont forget this question was answered in 2009...so times change.
Having said that, given our multitenant architecture, many scenarios where in the past you might have had multiple database instances on one host can probably now be done with a single database (and multiple pluggable databases).
And dont forget here that Tom is referring to a *specific* scenario, namely:
If I want an 'n' instance RAC cluster, can I do it on less than than 'n' nodes
and the answer to that is no. This was not a general "Hey, can I run multiple dbs/instances on a single box"