Charlie Weaver, The Grumpy Oracle DBA
Follow me on http://twitter.com/GrumpyOracleDBA
December 23, 2010
'There are three kinds of lies: lies, damned lies, and statistics.' Mark Twain If Mark Twain were still alive, he would add vendor benchmarks to the quote!
Oracle Announces Record TPCC Benchmark Performance
On the front page of today’s Wall Street Journal (print edition), Oracle proclaims setting the world record in performance using Oracle SPARC SuperCluster with T3-4 Servers with 30 million tpmC.
If you dig a little deeper, you’ll see that the system that Oracle it testing against won’t be available June 6, 2011. What’s up with that? Check out Oracle’s information at http://www.oracle.com/us/corporate/features/sun-oracle-faster-072797.html. By going to http://www.tpc.org/tpcc/results/tpcc_perf_results.asp, you’ll see IBM at number two with 10 Million tpmC. That’s well and good, since TPC.org tested using DB2 version 9.7 on an IMB Power 780 Server Model 9179-MHB system. Then Oracle as the nerve to dig HP by going to #5 on the list to compare the HP Integrity Superdome Itanium2/1.6GHz/24MB iL3 using Oracle Database 10g R2! What’s up with that? Why not run the tests again with 11g R2? Is Mark Hurd that bitter against his former employer?
A Greener Planet with HP and Microsoft SQL Server
I encourage anyone who looks at benchmarks to poke around http://www.tpc.org. For example, if you look at the Top 10 Watts/Performance Results, HP with SQL Server 2005 comes out on top.
One can only imagine how SQL Server 2008 R2 would perform in the “Green” test.
Oracle continues to simply ignore Microsoft SQL Server, most recently at Oracle OpenWorld this year. When I first attended Oracle OpenWorld in 1997, Larry Ellison would often poke fun at Microsoft with quotes like – would you want to run your enterprise software built my a company that ships Microsoft Dogs. At OpenWorld 2010, none of the keynote addresses mentioned Microsoft. – see http://www.oracle.com/us/openworld/keynotes-143370.html
Microsoft SQL Server Does Decent in Price Per Transaction Too
Here is an example of how well Microsoft does on TPC-C Price/Performance – You’ll notice that Sun hardware is not in the picture.
Is TPC-C Relevant Anymore?
What drives me nuts is that Oracle continues to insist on using the TPC-C benchmark which was established 18 years ago.
Today’s workloads are more complex that TPC-C. Oracle just wants to stick to TPC-C, while other vendors realize that TPC-C can be “gamed” and have promoted other workloads like TPC-E (simulates an OLTP workload based on a brokerage firm) and TPC-H (simulates an adhoc decision support environment). One has to wonder why Oracle doesn’t participate in these newer benchmarks. To Oracle’s credit, the acknowledge the benchmarks - http://wiki.oracle.com/page/Database+Benchmarking. They just don’t want TPC.org to test against them.
Oracle Real Application Testing
In practical terms, Oracle’s Real Application Testing provides the best way to assess your businesses actual workloads when comparing hardware and software configurations. This is because, Oracle RAT provides record and playback that includes an option to “turn up the amp to 11” – from Spinal Tap.
Microsoft SQL Server Profiler and Replay with Denali Release
One of my SQL buddies who attended SQL PASS 2010 in Seattle this year mentioned that Microsoft is answering Oracle RAT with SQL Server CAT (Change Assessment Toolkit) for the Denali release of SQL Server. They mentioned that Joe Yong in his session “Future SQL Server Upgrade Planning Tools” that Replay can be configured to use multiple workstations and threads per workstation to playback SQL trace files just like Oracle RAT. The chase is on!
Don’t Believe What You Read in an Ad
In my book, benchmarks – not matter who runs them – need to be taken with a “gain of salt”. Hardware and software are always changing and it’s almost impossible for mere mortals to reproduce the results without a team of experts from the database vendor to tweak every once of performance from the system.
In my world, it’s all about getting the application up and running as fast as possible. As long as it’s fast enough, my users don’t care and certainly can’t afford some of the Oracle/Sun high end machines. I’ve watched some of my DBA buddies working with SQL Server stand up a database and get a new application up and running for their users in days. With all of the audit processes that I need to worry about for our Oracle applications, sometimes I have to sigh with envy….
No comments:
Post a Comment
If you like what you see, please comment. If you disagree with something I say, feel free to comment as well. Please back up your comments with something substantial so that everyone can benefit.