Unixbench Roundup for EC2, Joyent, and my Brand Spankin’ new Macbook Pro
March 31st, 2008So, I felt like playing around tonight, so I decided to see If I could generate some data that some might find useful. Like anything on the internet, please take this with a grain of salt… skepticism is a good thing.
I decided I would run the unixbench suite of software across the various hardwares I have available to me.
The Setup
unixbench 4.1: http://www.tux.org/pub/tux/niemi/unixbench/unixbench-4.1.0.tgz
Contestant #1 ec2-small instance Centos 5 1.7 GB RAM 1 CPU Unit (roughly a 1.2 GHz Opeteron circa 2007, or 1 EC2 compute unit) Contestant #2 ec2-large instance Centos 5 64-bit 7.5 GB RAM 4 CPU Unit (2 virtual cores with 2 EC2 Compute Units each) Contestant #3 ec2-x-large instance Centos 5 64-bit 15 GB RAM 4 CPU Unit (4 virtual cores with 2 EC2 Compute Units each) Contestant #4 Joyent XL accelerator Solaris 10 2 GB RAM 1CPU Burstable up to 95%, Opteron of some flavor within 2.4-2.8GHz (based on the available offerings from sun in the X4100 series) Contestant #5 Macbook Pro Leopard 4 GB Ram Core 2 duo 2.6 GHz
Unixbench was not run under complete isolation: I have no control over what other users are sucking up resources on the virtual machines and so I can’t say how outside effects where affecting the results. Nor was my laptop completely idle, although I let it sit and didn’t use it while the benchmark was running.
I can’t comment on the use of concurrency within the benchmark. It appears that every test is single threaded with the exception of the shell script tests. Take what you will from the data, but keep that in mind.
The results: http://spreadsheets.google.com/ccc?key=pYJF9YZ-X2yCZnVWs8WmgGw&hl=en
Some interesting notes
The Macbook pro cleaned up when it came to pure arithmetic, winning an overwhelming majority of those tests. Some by very large margins. I’m curious how a modern Xeon would compare.
The Joyent Accelerator showed behavior similar to my personal experience with their service: poor I/O. Some of the filesystem tests approached 50% slower than the winner, whereas the largest block size test had it off by an order of magnitude! This result could be atypical, but it seems to lend credence to my perception that I/O is inconsistent on the box.
The ec2 small instance had surprisingly poor filesystem results, falling off more sharply as the block size went down than the other machines.
The MBP had the worst filesystem performance of the big block tests for Writes.
Given these tests, the sweet spot seems to be ec2 large instance: It is very close to the XL instance in all of its marks, but it’s half the price. Its monthly price is similar to the Joyent XL, but it has 4 times the memory and significantly faster I/O
March 31st, 2008 at 10:52 pm
Great stuff! And now for a bandwidth test…
April 1st, 2008 at 9:23 am
It would be interesting to see results from your MBP with CentOS 5 installed, to measure filesystem performance with the same kernel and filesystem type.
That said, I suspect that the real issue is that 2.5″ laptop hard disks are quite slow (5400 rpm, usually) compared to desktop or server class hard disks.
April 1st, 2008 at 9:51 am
I’ve been doing a lot of comparisons between a Joyent L accelerator and a 2 year old Toshiba laptop w/ a Pentium M (both with 1GB of RAM):
One initial benchmark I ran was a slow query. It took 8 seconds on my laptop and a whopping 6+ minutes on the Joyent Accelerator.
So I contacted Joyent support.
Turns out Joyent has two different hardware configurations, one that is “old” and one that is “new”. The “old” ones have dirt slow IO.
After Joyent switched me to a 1GB accelerator on the “new” hardware, the same query now executes in between .8 seconds and 1.2 seconds (probably depending on the overall load on the box).
So I suggest you contact support and verify that you’re on the “new” hardware configuration at Joyent. It took me a while to get this to happen… Joyent tried to blame the query, mysql, my app code, my my.cnf settings, etc.
Thanks to this benchmark I’ll probably give the query at try on a Large EC2 instance just to compare…
Note: I think Joyent should be proactively getting people off of the atrocious “old” hardware config, but that does not seem to be the official policy. a senior tech at Joyent told me that doing so is a “least desirable option” from his perspective. What surprised me most about this was that Joyent was not proactive in moving people off of the old hardware. Once the decision to move was made, everything went smoothly, but it was not fun trying to get anyone there to acknowledge that there are IO issues. The first line of response is “it must be your query, your my.cnf, or your app code”.
April 1st, 2008 at 10:41 am
I should add that I am very happy with Joyent and am satisfied with the resolution to the issue. I love the accelerator concept (easy vertical scaling) and now that I’m on the new hardware everything is great.