Thursday, August 18, 2011

Performance of Rackspace Cloud Files Upload Over Private Network




Does using the private network interface on a Rackspace Cloud Server instance to transfer data into Rackspace Cloud files save you anything?

I am certain it saves you money. You pay for outgoing bandwidth on your Cloud Server instance's public IP address, but not on the private one. But, I've also read that it is much faster to use the private IP, or "ServiceNet" as they call it. "Lightning fast" even. So, I tried make a quick-and-dirty benchmark to see for myself.

In short, I don't think it makes much difference, considering the margin of error. Here are the results of a typical test run that shows the timings of uploads of a 1MB file alternating over the public and private interfaces, with a cumulative time printed out at the bottom:

https://gist.github.com/1155335

That example is the most sane run of many test runs I did with this script. What is more disconcerting is that both public and private have wild outliers. Sometimes a transfer that normally takes a tenth of a second can take 30 seconds or even two minutes!

Check out this test run: https://gist.github.com/1155349

Whoa! The 31st upload took 132 seconds! In my experience over multiple runs, the private snet almost always comes out way ahead. It seems like it is just as likely to have 10-second outliers, but less likely to have 30+ second outliers.

If you are interested in reproducing my tests, you can get the code at: https://github.com/scottwb/cloudfiles-snet-bench

On caveat to this test, other than generally not being very scientific, is that I am not using large files (which is the use case I am actually interested in). It is possible that large transfers may get big speed gains over the private servicenet.

1 comment:

Anonymous said...

Did you find a solution to the odd upload which took > 10 seconds?

I'm encountering the same problem where I'm uploading a series of 20 image files, all about 16k each and I'll often get 1 or 2 of these taking 80+ seconds.