19 Feb 2009

Amazon EC2 benchmark – performance 

By - Linux 4 Comments

“Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers”, according to to Amazon.

All EC2 instances are sold based on hourly usage and instance type.  Each instances type consists of a certain number of EC2 Compute Units (ECU) and a set RAM size. One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.  Since the ECU is such a vague definition, I wanted to see some benchmarks (but couldn’t find any).

The information below is completly CPU based (using the Python Pystone test), and does not take into account any other factors.

The pricing model for Amazon EC2 looks attractive from the surface, but when you get down to it — monthly pricing for similar performing hardware can be much cheaper at a dedicated server provider like Softlayer.com. The use of EC2 is climbing, but I am concerned that many of the current uses of EC2 are better served using leased hardware.

Here are a few examples based on monthly costs, using the 28 days as the pricing point

Instance Type Monthly Pricing ECU’s Cores
Standard Small $67.20 1 1
Standard Large $268.80 4 2
Standard X-Large $537.60 8 4
High Medium $134.40 5 2
High X-Large $537.60 20 8
Server Specs Monthly Pricing Cores
1×2.0Ghz Xeon 5335 4GB $259.00   4
1×2.6Ghz Xeon 5430 4GB $319.00   4
4×1.8Ghz AMD 8346 16GB $899   16

Most importantly, compare the performance.
Amazon EC2 benchmark

The number on the left is the pystones per second, the number on the bottom is the number of concurrent processes.

The number of concurrent processes and pystones should scale rather linearly with the number of CPU cores — And does on the dedicated hardware.

The EC2 instances are sporadic and took a number of tests to produce these results (which were the best of all results).

If you are using Amazon EC2, you may not be getting what you think you are paying for — in terms of available CPU.

4 Responses to “Amazon EC2 benchmark – performance”

  1. Jackqueline Rhodus says:

    I’ve just come across your site regarding Cloud Computing and Virtualization. There is some good information and we may be interested in including you in our blog. Please feel free to contact. Cheers

  2. Diego says:

    Even when the EC2 benchmarks aren’t what they probably should be, there is a misconception on your analysis:
    The fact that the High-CPU-Medium has 5 EC2 it does not mean that it has 5 cores. The specification for the instance types states that that particular instance has 2 cores with the equivalent power of 2.5 EC2 each. So trying to push it with 4 concurrent processes seems wrong, it should stand only 2 of them since that is the amount of cores available,

  3. admin says:

    Diego, That is an interesting point. We will get back to benchmarking this again in the future, and will keep that in mind.

  4. D.Q. says:

    Can you share raw data please?

    I want to make my own plot (pystone/mhz) vs (processes/cores).

Leave a Reply