In my experience, sharding works if the "partitionioning" of data is static enough, and clearly defined from start, because changes are much harder to implement and have deep impacts down to the hardware level.
Also, they usually require an application-level knowledge of where data are and how to integrate them - up to the risk of having client side joins and aggregations of large data.
What the article tells about maintenance is just half the truth: it is true managing a large database is complex, but managing data scattered among multiple systems has issues as well - consolidation was meant to avoid them because they exist.
Take the "issue", SAN, for example - one gets a large, fast, fault tolerant storage. Sharding and storing data locally just means to replicate fault-tolerance at every node level. A SAN can allocate space as needed, while local storage may have wasted space or lack of it, and outgrow the available hardware.
In some applications it could work well - Google indexes words in web pages - that "domain" is static enough, alphabets, words and languages don't change over time fast enough - and there are good statistical analisys about distribution, there could be good ways to "partition" those data across several machines, while merging results is simple enough - it returns web addresses in a simple display.
Other database may need to present data to different applications, and allow for more complex data manipulation - IMHO sharding in those cases just add a layer of complexity that has to be managed at the application level, while the "shared everything" approach is more flexible.

shared nothing vs shared everything is like a religion, Inmon vs Kimball, ... we will never find the truth simply because there is no "better".

Teradata database is shared-nothing and although the cost is very high because of this, you can scale better and can fit better is some situations. I've seen Teradata perform great in pretty fast changing active data warehouse environment and I disagree that shared-nothing adds big complexity for DBAs or Developers (it may be a bit more complex to design the indexes, however it may be easier to manage load balancing). Again, there is no better or worse, it's simply different.

You mentioned "at the storage level we can go shared nothing (transparently, with great flexibility and the ability to dynamic distribute and redistribute the data - unlike the static partitioning of a teradata)" I am very curious to know which feature you are refering to.

Shared everything RAC is only logically true.
There is a notorious thing called interconnect trafic which eats out most of the horizontal scalability gains.
I should not have said "notorious", in fact, interconnect is the reason RAC is shared everything.

In practice most RAC installations will centralize specific modules on specific node. The loose term for this is "Application Partitioning". This ensures one node rarely needs data handled by other node.

This is indeed a bias towards shared nothing.

RAC has got other benefits like Transparent Failover for some basic SQLs. They are good from availability standpoint.

And of course I disagree with Tom on share-nothing at storage level. This will be of very limited or no use, if there is heavy interconnect traffic. Global Cache series of waits in a RAC AWR is a very interesting study.

Tom,

I have heard claims from IBM that though Oracle RAC is shared disk there is a master node for each block in the database. They claim that because each block has a master node any modifications to a block have to first check with the master node and then with the node that is currently holding the latest block. They further claim that this communication between all the nodes is what inhibits RAC scalability. They have also claimed that when the master node goes down all the locks must be remastered during which the data is unavailable.

So I have some questions for you in this regard.

Is it true that there is a master node that keeps track of all the blocks?

Is it true that a single block has only one master node? meaning if that node fails that block must be propagated to another surviving node?

Thanks
Is it true

Thanks for answering the questions Tom. Just to clarify - when a block needs to be modified by instance A and the latest block is in instance C and the master instance is D what are the series of events that take place before that block is modified?

IBM has a new solution for Linux/unix/Windows called DB2 Purescale which is shared-disk architecture similar to the Sysplex solution. The difference according to them is that all the lock and cache management happens on a Central node (this is not a DB node but one that is dedicated to lock/cache management). And because of this they claim the inter-node communication is minimized.

Dear reader ,

Thought to shed some light on to this , since I am also a Ask Tom fan and a Oracle DB fan.

For couple years Oracle RAC was the only transparently Horizontally scalable database in the market.

The IBM Purescale doest have much technical documents out there yet , but I managed to figure out few things.

- they took some concepts our of the z-series mainframe
- Boxed it in a P series HW with AIX/Linux
- Of course there is always a Mainframe version

So as per my understanding IBM DB2 Purescale allows two (could be more )centralized locking node's to be present for HA reasons in their configurations. requires minimum one purescale node for the solution to work in Cluster mode.
The purescale node acts as a dedicated message box for all other nodes. It is constructed with low latency IB/RDMA interconnects.

What they have basically done is created a additional node in the middle dedicated to handle what Oracle RAC does in Cache Fusion. As per the currently available literature , It seems to be limited to IBM HW. The Clever point to recognize is that they centralized the Caching to a single dedicated node and made that node highly available , which dramatically reduce the inter node synchronization traffic. Where as in Oracle RAC everybody know everything (for great extent) ..but this cause's some traffic.

Where as Oracle RAC is generic solution , that allows us to create on many platforms. PureScale I believe will more or less fight at the level of Oracle Exadata not with Vanilla RAC.

On the HW side of the techniques, they have invested on infini band with RDMA and exposing the DB2 code to to the Cache sync via that. Couple of years back Oracle RAC also took steps towards improving interconnect speed by adopting RDA protocol over IB(UDP) on Linux. As per IBM they can do the scaling via this dedicated RDMA concepts by bypassing OS layer and other additional layers (interrupts ).

I am sure if there is a Market competition from IBM for Exadata , it will only be a temporary set back for Oracle. Since decades a go Oracle and Sun did some really good innovative feature like Solaris KAIO ..which made a difference when it came to IO load.

I am sure with up coming Solaris critical application API and some focus on RDMA based optimization will allow the Oracle to do the same much better ! or they might do the whole Cache - Fusion operation in HW / dedicated solely for cache coherency.

Another point is that most of these approaches were materialized some time back in Clustering world ..., looks like it is all coming back ..., instead of proprietory interfac ..looks like vendors are joining the IB Band wagon.

For your scenarios on what happens to a data block and how it gets transferred and how the cache fusion is keeping this information is explained very well in two very good books on the subject.

- Real Application cluster handbook - K Gopalakrishnan
- Pro Oracle Database 10g RAC on Linux: Installation, Administration, and Performance By Julian Dyke , Steve Shaw

Last but not least , there is no master instance or master node concept in RAC, some times one could get confused with terms like master instance ...which simply refers to a state of a RAC memory structure in a given time quantum under specific Cache coherence state.

If you are interested more on IBM DB2 PuresScale , you can also follow this blog on ,
http://it.toolbox.com/blogs/db2luw/db2-purescale-scalability-part-1-35173


Thanks and Good luck ! Nothing can beat the Oracle Innovation !

Hi Tom,

The primary reason behind my statement is that ,As per my understanding Cache fusion architecture consists for a distributed cache and its management. Oracle adheres to cache coherent protocol. So in this , in addition to the Data block sharing , synchronization information (info kept in GRD) is also shared between nodes, also concepts like resource master and who is master what resource is kept synchronized across instances. And the way oracle currently implements the interconnect is very generic , I believe it could be optimized further , but then people might start saying its proprietory and it is more costly.

IBM is trying a short term workaround , by no means I believe the current approach is as sophisticated/ or scales better compared to oracle ,e.g because there is no information on dynamic resource mastering and workload balancing. In IBM world , that distributed nature of the memory structure is missing for great extent and always single point of contact (everything is in purescale node). As you said this could be a negative point. I believe due to the fact that IBM model uses a very finite messaging mechanism and it is currently optimized for OLTP as well , so with current purescale vision works for some extent.

May be Mr.Cary Millisap could help us here with a small queuing model.

Hope this explains my initial thoughts. of course it is yet to be proven.

I believe I understand shared nothing/shared architecture. each have pros and cons . in Oracle if large ,highly concurrent tables are partitioned in different tablespace in different disk groups/grid disks will it not give the benefit of both the worlds. IN ASM I am always seeing only one disk group for +DATA and it is best practice in RAC. What abt Manual.