Elaborating on points in which the response needed some clarity
sasanka ghosh, April 24, 2017 - 5:52 am UTC
Hi ,
Thanks .
Elaborating on points which u asked some clarification.
I started my question with re-distribution. In MPP systems if Co-location not present or the query join criteria does not include the columns distribute keys then at run time all big tables get redistributed . Though with Columns projection etc. the cost has come down but it is a heavy duty process.
The second point regarding writes I was trying to point out that shared nothing MPPs one of the USP is very Fast data Loading. The shared disk SMPs will suffer little bit but as DW/Mart is mainly write once read multiple times unless and until it is significant then that it should not be much of an issue .
In short what i was trying validate is that for 100-200 Tb DW/ Data Mart SMP systems,With Full redundancy and RAC kind of structure can be better option than MPP systems.
I was thinking Core Analytics with limited period detail data, rest summary/aggregated data 100-200 TB should be enough for most organization .
Beyond that Historical data or other kind of analytics with latency we have the "BIG DATA stack".
April 24, 2017 - 6:44 am UTC
My thoughts on this style of discussion is that we're heading into a "polyglot" world, namely, people will pick the best solution for a *particular problem requirement*.
For OLTP transactional-systems, I'm picking rdbms for its acid, maturity, reliability etc. I dont want to be pioneer in this space, not when I'm dealing with people's money, their credit history, their personal details etc.
For unstructured data, I might choose a NoSQL solution.
For archival data, I might choose something sitting over Hadoop.
But what I also think is critical is when we define the word "best", that we dont *just* include the technology component. It's similar to when someone asked a question on AskTom recently - they had a lot of SQL Server on site, and a one Oracle db, and were asking for some advice in a particular requirement. Even though I'm obviously biased toward Oracle, I advised them to use SQL Server - because part of choosing the "best" of something, is having the people skills to manage it.