Skip to Main Content
  • Questions
  • multiple databases in a single RAC cluster

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question.

Asked: November 05, 2007 - 12:13 pm UTC

Last updated: March 12, 2012 - 7:43 am UTC

Version: 10.2.0.3

Viewed 10K+ times! This question is

You Asked

Hi Tom,

we are planning to create our development database tier as 3-Node Oracle RAC cluster with 4 databases
like

Four databases
--------------

- DB1
- DB2
- DB3
- DB4

Node1: (DB1 with instance DB11)+(DB2 with instance DB21)+(DB3 with instance DB31)+(DB4 with
instance DB41)
Node2: (DB1 with instance DB12)+(DB2 with instance DB22)+(DB3 with instance DB32)+(DB4 with
instance DB42)
Node3: (DB1 with instance DB13)+(DB2 with instance DB23)+(DB3 with instance DB33)+(DB4 with
instance
DB43)
And Planning to use ASM

- Do you support this type of implementation?
- If so ,will there be any performance degradation ?
- Can I use single oracle home for multiple instance's in a node or seperately?
- Can you explain how load balancing feature help in this kind of environment

thank you


and Tom said...

o do I support this

nope, not at all. One machine, one host - ONE INSTANCE

o any performance issues?

all of the time, every day, in every way. Think about it - how could it not be? instance db11 takes 100% of the cpu on node1, queries against db2 are reported as slow. db2 reports "all is well in the world". Now what?

one machine/host, one instance

o sure you can, but there are LOTS of things you "can" do. There are fewer things that you "should do"

and having more than an instance per host (especially with RAC, you'd be defeating the purpose entirely) is not one of them..

o nothing could help this, short of a decision to go with one instance per machine.

Rating

  (14 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, November 05, 2007 - 10:25 pm UTC

Don't understand why people keep on asking the same questions over and over again rather than performing a search and saving everybody's time.

RAC for development ?

Gary, November 06, 2007 - 6:43 am UTC

"we are planning to create our development database tier..."
If the eventual production version will be RAC, it is obvious that any load/performance testing environment should be RAC too.
For Development/unit testing/integration testing/business testing, is there a compelling reason for those environments to be RAC ? [I can think of specific cases, eg UTL_FILE may need access to directories on the clustered file system, but nothing saying "MUST" rather than "NICE"]
If so, is it important to have the same number of nodes ?

Tom Kyte
November 06, 2007 - 9:36 am UTC

yes, you would use rac in development if rac is the deployed platform.

having same number of nodes, I'd want to have 1, 2 or 3 nodes - 1 if I was not using rac in production, 2 if that is what I was going to use in production, 3 if nodes in production where 3 and above...

the algorithms for certain operations change from 1 to 2, and 2 to 3 - but beyond three not so much.

Except..

chris_c, November 07, 2007 - 9:31 am UTC

When dealing with third party applications you may have to run more than instance per server, Its still better to partiton your server into multiple virtual machines and give each dataabse its own seperate OS.

Multiple databases on a single cluster turns into a real headache as there are multiple resources that are shared between all the databases and you can't realistically control which instance gets what. (CPU and the global interconnect are the two I can think of if an instance want 100% cpu it will just take it same with the interconnect bandwidth).

for Rac the better solutuion is one database (one instance per node) and use services and resource manager to control the applications resource uasage. services can be used to restrict an application to certain nodes, resource managaer can control the applications resource usage on each node.
Tom Kyte
November 07, 2007 - 5:59 pm UTC

you should virtualize that machine - solaris containers, domains, vm's - whatever.

That is why I said "server/host" - I'll be much more specific in the future. That was my intent, I missed the boat by not being much more specific.

algorithm

David, November 07, 2007 - 12:26 pm UTC

Can you please clarify "the algorithms for certain operations change from 1 to 2, and 2 to 3 - but beyond three not so much. "
Thanks,
Tom Kyte
November 07, 2007 - 6:11 pm UTC

just like our algorithms change when cpu_count goes from 1 to 2 (but not so from 2 to 3, 3 to 4 and so on) - some internal things change from 1 to 2 nodes (obviously, the interconnect is introduced...) and from 2 to 3 nodes (2 nodes coordinating a buffer cache is 'easier' - some shortcuts maybe available - than 3 or more).

that is all - nothing specific I want to mention, and remember "things change"

so, 1, 2, or 3 nodes... depending on whether you will be 1, 2, or 3 and above...

Multiple databases in single RAC cluster

Paru, November 29, 2007 - 11:04 pm UTC

Tom,

We are also planning on moving our existing single instance database into a 2 node (to begin with) cluster running 11g on Netapp NFS technology. But, we also have couple of 3rd party databases and some more home grown databases which we want to move into RAC for scalability reasons and some single instance databases into the same NetApp array.

Do you still recommend one instance per node? Can we virtualization be done on Linux side?

Thanks
Tom Kyte
November 30, 2007 - 12:56 pm UTC

ONE INSTANCE PER HOST.

be the host a virtual machine, or real machine.

yes, Linux does virtualization. You can either use something like our current announced offering of Xen (no OS, just a bit of code to boot up VM's) or VMWare - which require a full blown OS.

Sun Containers and RAC

Bob, November 04, 2008 - 2:42 pm UTC

Thanks Tom for the hundreds of pages I have reviewed and learned from over the years.

The following is to hopefully save other people some time (and possibly buying the wrong hardware for their purposes).

At this point (Nov 4 2008), RAC is not supported for instances in anything but the global zone of a Sun Solaris machine (of which there is only one per machine).

Ref: Metalink white paper 317257.1

However, using Oracle VM (x86, x86_64 only (Nov 4 2008)), all zones can contain instances that are part of RAC cluster(s). I think it is also possible for some other VMs too.

Raise service request actually states this IS supported.

theslothuk, November 23, 2009 - 6:24 am UTC

Hi Tom,

this is a valuable forum for oracle related queries and has often helped me in the past. Here though, I was confused by conflicting information on myoraclesupport and Ask Tom so I raised an SR to clarify. (Allow me to say first though that I am fully aware as I'm sure is everyone what Ask Tom is here to do) Here is an excerpt:

"...informed customer that ASK TOM is not a portal that you can ask it for the support status or the certification for the Oracle products, this portal is to provide advises and help in resolving issue unofficially.

The official support portal for Oracle products is the MyOracleSupport, for this reason we have provided him with the formal answer for his question which is "Yes, it is supported to have more than one RAC DB running on the same set of cluster nodes" and the reference is Note.220970.1"

Is this something that has changed with time?... or when you say "Nope, not at all one machine one host one instance" is this just your view on best practice? Please don't think I'm attempting to play one set of advisors off against the other, I'm just keen to find out on behalf of myself and others considering future architecture what is correct.

I'd be interested to hear your views.

Thanks in advance.

Sloth
Tom Kyte
November 23, 2009 - 4:28 pm UTC

where did I say it was not technically supported?

I'm telling you "very bad idea, don't do it"

You can run 1,000 instances on a single machine too - but so what, you shouldn't.




Please note what my answer started with:

do *I* support this

*I* being the very operative word.


Yes, this is a supported configuration, it is just a bad idea.


We support OLTP applications that do not use bind variables
We support triggers
We support when others then null
We support autonomous transactions
We support dictionary managed tablespaces
We support lots of bad ideas


Because it is supported does not mean you should do it.


And, I never said "Oracle corporation does not support that configuration", I said "I do not"

clarification

A reader, March 07, 2012 - 11:19 pm UTC

Hi Tom,

Currently our development database is in single instance. We are planning to use RAC in UAT/production environment. Is this good idea? Will there any performance issue/overhead?

Is there any compelling reason for development environment to be RAC or just for best practices standard?

Our current database version oracle 11g Release 1.

Regards,
Richard

Tom Kyte
March 08, 2012 - 5:43 am UTC

Is this good idea?

It would be as good an idea as developing on DB2 and deploying to Oracle.

Will there any
performance issue/overhead?


probably.

Is there any compelling reason for development environment to be RAC or just
for best practices standard


would you develop on windows and deploy to unix?
would you develop in C and deploy in Java?
would you ........

You will not have any ability to reproduce any production issues in a non-production environment.

You will not know what product issues you might hit, until you get into production.

You will not understand the performance characteristics of your application, until it is too late.

You will not know if something works a little differently in RAC than in non-RAC. Eg: In non-rac, sequence numbers might be observed to be monotonically increasing. Your code might assume they always will be and might rely on that. Then you go production to discover they are certainly NOT monotonically increasing.

etc
etc
etc

clarification

A reader, March 09, 2012 - 5:16 am UTC

Thanks Tom for your response. Your answers are awesome as usual. However, I am finding it difficult to convince my Manager. The reason is that the database setup (dev-single instance; production - RAC) was proposed by one of the Senior DBA here and he has very good rapport with my manager. Are there any detailed explanations on your book or from the oracle documentation?

Thanks in advance.
Regards,
Richard


Tom Kyte
March 09, 2012 - 9:25 am UTC

sorry, when something is so obvious, we tend to not "document the need for it". I don't know what to say.

Ask the DBA if they have any successful large scale examples of doing this in their resume I guess.


good luck and keep your fingers crossed. If the system is lightweight and small, you might get lucky.

What about ODA and one/many dbs and RAC?

Rob, March 09, 2012 - 8:12 am UTC

How do you feel about the one database per server/host
in an environment like the Oracle Database Appliance?

It seems this environment is more supportable, stack wise by Oracle, with pay as you grow licensing, grid control configuration, ASM,minimal server/file system administration, etc, is fairly easy to install and manage,
and having more than one database per server would at least be easier to manage vs a home grown solution?
I am not proposing millions of tiny db's, but at least
there could be a few larger, consolidated db's, in a RAC, or RAC One Node configuration that would be easy to manage, and provide a little more flexibility for upgrade issues.
They can even be data guarded to another ODA.

With the 11.2.0.3 release for ODA, multiple homes are now supported, so migrations/upgrades could occur based on what applications can or can't be upgraded.

For smaller organizations that can't afford Exadata or to have many servers licensed based on the cost of Oracle licensing, it seems that the only way to go is many databases on one host/server.

I believe the problem of many db's on one server/host will continue to be a problem in terms of administration/troubleshooting until the licensing structure for Oracle changes. Not all shops can support Oracle's VM and VMWare Oracle licensing is expensive. Alternatives are lpar's on UNIX variants where system admin support is expensive also.
Tom Kyte
March 09, 2012 - 9:31 am UTC

In light of the fact that "things change", I do have to update this.

Now that we have cpu instance caging (across instances)
Now that we have an IO resource manager (across instances)
Now that we have memory targets

On these engineered solutions - it is as if we have virtualization on them. We can wall them off from each other. I always said "you need to wall them off - use separate machines, use virtualization". With the ability to limit cpu, IO and memory pretty much - we have what we need without true virtualization.


I/O resource manager?

Stew Ashton, March 09, 2012 - 12:29 pm UTC


Hi Tom,

Your update just above is most interesting and I applaud your willingness to adjust as things change.

I have a few questions:

- in what configurations is I/O resource management across instances available to us? I was told by an Oracle France rep that it only exists for Exadata.

- Also, is it still true that instance caging only works on background processes? Oracle France didn't like it when I said that.

If I back off my "one host, one instance" stance at work, I want to provide all the caveats I can.
Tom Kyte
March 12, 2012 - 7:17 am UTC

it exists for exadata, zfs and pillar storage.


instance caging works on your dedicated servers. It doesn't (and shouldn't) affect dbwr, lgwr, and the like. Instance caging affects the scheduling of the client sessions in the database.



re:What about ODA and one/many dbs and RAC

Rob, March 09, 2012 - 3:18 pm UTC

Tom,

Thanks for following up, I appreciate your comments. We are migrating to ODA in the very near future, our loaner is coming in Thursday, can't wait! Wish we could afford Exadata, but in time!

Our goal is to consolidate as many db's as possible on the migration to ODA, and put apps/db's on different ODA's based on SLA's, application performance, data growth, data security issues, etc that we have with our customers.

Looking forward to using RAT and Orion next week to test hypothetical workloads and some real workloads.

Thanks
Rob
Tom Kyte
March 12, 2012 - 7:24 am UTC

Please explain Why ?

A reader, March 11, 2012 - 12:11 am UTC

For Non Rac databases :
is it still true
o do I support this

nope, not at all. One machine, one host - ONE INSTANCE


Usually on HPUX and SUN machines we have multiple Database instances . We have around 16 to 24 gb memory multiple CPUS to support this

Do you mean with respect to Linux x86 64 bit machines :One machine, one host - ONE INSTANCE ?

That would be a very expensive idea buying new hardware for each database system unless on VM
Tom Kyte
March 12, 2012 - 7:43 am UTC

That would be a very expensive idea buying new hardware for each database
system unless on VM


you have solaris zones which can do what is needed as well.

If you run more than one instance - you need to be able to wall them off from eachother

o memory wise
o CPU wise
o IO wise


Virtual machines can do that. Zones can do that. Exadata with IORM (IO resource manager) can do that. And so on.

e:What about ODA and one/many dbs and RAC

Rob, March 12, 2012 - 12:16 pm UTC

Sorry, yes ODA is Oracle Database Appliance. Been typing Oracle Database Appliance a lot lately for the purchasing department, got lazy..no broken keyboards..