I've tried to have a discussion with Linode support but didn't get very far. I would write paragraphs where they would write one line responses. So I thought I would post here to see if anyone else has observed similar performance numbers. It's funny because I was writing a blog post extoling the new upgrades at Linode when I stumbled across this (pretty big) performance delta after being upgraded.

I wrote a blog post back in 2011 posting Unixbench performance numbers for a number of virtual and physical servers that I use in my day to day choirs.

http://blog.michaelfmcnamara.com/2011/1 ... s-hosting/

In short, Linode 512 came in at 495 beating RIMU at 290 and even my physical IBM x345 at 387.

I upgraded from a Linode 512 to a Linode 1024 back in October 2012.

On April 5 I ran Unixbench again only to discover that my performance had dropped to 272. I ran the test again on April 7 and received a score of 358.

I performed the migration from a Linode 1024 to a Linode 2048 and ran the benchmark again only to get back scores of 189. I contacted support and they migrated me to another host and there I got a score of 119. I ran the test one last time on April 12 and got 178. The migration from a Linode 1024 to a Linode 2048 appears to have a very different CPU, Intel(R) Xeon(R) CPU E5-2630L 0 @ 2.00GHz (4000.1 bogomips) as compared to the previous CPU of Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (4522.0 bogomips).

As I explained to Linode support, I certainly understand that benchmarking virtual guests (servers) is very tricky subject but while I expect there to be some variance I'm at a loss to explain the delta I'm observing here.

I've gone from extolling Linode to wondering if I need to move to another provider.

Cheers!

I too have had issues on the 2630's. I've been back and forth with support (always very nice) and I can't find numbers that would indicate that there's an issue, but it definitely feels slower.

I have 4 nodes on different E5-2670's in London and so far all feels more sluggish after the move. Other than one unlucky node its has been mostly I/O that has been hampering the application performance though. Even the tiniest trip to disk could take up to 500ms on the worst one, from typically beeing a <5ms affair.

I've been chalking it up to early upgraders beeing resource hogs (or omg, new shiny have to benchmark it! ), and planning to wait it out for a bit, especially with all the ruckus going on and all. But I'm really starting to wonder if something is botched on the newer hosts. Like the vm images not beeing aligned to the raid stripes or something to that effect.

For now I've just made the apps cope, the extra RAM helped quite a bit in that regard. Hopefully things will get sorted when things settle down a bit.

Following up - seems two of the nodes have improved significantly since last I checked on them. Knock on wood.

Which version did you use? Unixbench 4 or 5?
I want to test some stuff myself, and prefer to use same version.

I am testing a cpu intensive webapp but performance is 3x slower than on my old desktop Core 2 Duo E6500. And that while I am on a new E5 2670 host and it shows idle as load in the manager. I expected at least similar performance. (I am on 1024 plan)

Also, do I need to provide some command line option to run it on only 1 core or so? Or is it balanced by itself when I compare it with non-multi-core systems?

Update
Turns out to be a MONO issue. The same file leads to 3x the running time of the Windows .NET original framework. That's a huge disappointment. I guess I need to look for windows hoster for this project.

I tested a few systems and got wildly differing results with UnixBench5.1.3 without graphical tests.

Linode 1024 - Xeon(R) CPU E5-2630L 0 @ 2.00GHz
CentOS 5.9 - 32 bit
Result: 164.5 / 143.9

Linode 1024 - Xeon(R) CPU E5-2670 0 @ 2.60GHz
Ubuntu 12.04.2 - 64 bit
Result: 1105.6 / 1326.6

Linode 1024 - Xeon(R) CPU E5-2630L 0 @ 2.00GHz
Debian 6.0.7 - 32 bit
Result: 199.3 / 184.5

Home workstation - AMD Phenom(tm) II X4 955
Ubuntu 12.04.2 - 64 bit
Result: 1449.4 / 1442.4

AWS micro instance - Xeon(R) CPU E5-2650 0 @ 2.00GHz
Ubuntu 12.04.2 - 64 bit
Result: 260.1 / 99.0

I don't get how the E5-2670 Linode can get a score 5 times higher than the other two. Maybe it's on an underused host. Maybe they upgraded more than just the CPU.

The AWS micro instance is particularly impressive as Amazon make it clear that that type of instance isn't suitable for anything but very light usage. Its root device is a network block device, it only has 600 Meg of memory, and only 1 CPU core.

EDIT: I ran the tests again. The E5-2670 Linode got the same amazingly high result, the AWS micro instance scored a lot lower the second time around.

Those 2630 benchmarks look really low to me. The AWS one looks right, the 2670 looks lower than I'd hope. Here's one from a while back I did on a 1024


BYTE UNIX Benchmarks (Version 5.1.3)

System: poseidon: GNU/Linux
OS: GNU/Linux -- 2.6.32-344-ec2 -- #46-Ubuntu SMP Wed Mar 7 13:48:15 UTC 2012
Machine: i686 (unknown)
Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
CPU 0: Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (4536.0 bogomips)
Hyper-Threading, MMX, Physical Address Ext, SYSENTER/SYSEXIT
CPU 1: Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (4536.0 bogomips)
Hyper-Threading, MMX, Physical Address Ext, SYSENTER/SYSEXIT
CPU 2: Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (4536.0 bogomips)
Hyper-Threading, MMX, Physical Address Ext, SYSENTER/SYSEXIT
CPU 3: Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (4536.0 bogomips)
Hyper-Threading, MMX, Physical Address Ext, SYSENTER/SYSEXIT
00:43:31 up 19 days, 7:36, 2 users, load average: 1.98, 2.68, 2.00; runlevel 2

Single test:
System Benchmarks Index Score 682.6

Parallel tests:
System Benchmarks Index Score 1703.8

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

I agree with obs - it's not that the 2670 numbers are good, but that the 2630L is really bad. Since migrating to an E5-2630L host for the free double RAM my linode 1536 just feels incredibly sluggish. Maybe everyone is bogging down the new machines with benchmark runs???

   BYTE UNIX Benchmarks (Version 5.1.3)

   OS: GNU/Linux -- 3.7.10-x86_64-linode30 -- #1 SMP Wed Feb 27 14:29:31 EST 2013
   Machine: x86_64 (GenuineIntel)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2630L 0 @ 2.00GHz (4000.1 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
...
   CPU 7: Intel(R) Xeon(R) CPU E5-2630L 0 @ 2.00GHz (4000.1 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   16:36:38 up 8 days, 23 min,  2 users,  load average: 0.16, 0.08, 0.06; runlevel

------------------------------------------------------------------------
Benchmark Run: Sat Apr 20 2013 16:36:38 - 17:05:03
8 CPUs in system; running 1 parallel copy of tests
System Benchmarks Index Score                                          93.9

Benchmark Run: Sat Apr 20 2013 17:05:03 - 17:34:39
8 CPUs in system; running 8 parallel copies of tests
System Benchmarks Index Score                                         296.4

FYI I've upgraded several servers over the past week or so and I've landed on a right mix of hardware

2x Intel(R) Xeon(R) CPU E5-2650L 0 @ 1.80GHz
1x Intel(R) Xeon(R) CPU L5630 @ 2.13GHz
4x Intel(R) Xeon(R) CPU E5-2630L 0 @ 2.00GHz
1x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz

Of those 2 are old hardware (L5*). I've not landed on any 2670. I've had people question the performance of two of the 2630L but I've not had a chance to verify any performance changes yet. I also had to migrate some servers twice and one three times due to a Xen bug not being able to boot 32 bit guests. At this point I'm recommending people don't upgrade unless the ram can really help them. It's just too much potential hassle.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

I've measured huge single threaded performance loss after migration to a E5-2630L in London. Some data aggregation processes we run every few hours took up to 15 seconds before migration, now they take over a minute. It could be the migrations are not over yet and/or people are massively benchmarking the new servers, so I guess it'll stabilize...

Although I suspect Linode has lowered singlethreaded CPU allotment in favor of more CPUs...

I tested on a 1GB machine from digital ocean and got 1339.7

I was actually expecting them to get a rubbish score.

Digital Ocean's performance isn't bad it's their support that's flakey. I'm rather fond of ramnode (http://serverbear.com/9756/ramnode) at the moment, they're too new for me to consider using in production but they're good for backup/hobby servers.

I've contacted Linode support about the poor performing E5-2630L and got migrated to a E5-2670 0 @ 2.60GHz it still doesn't perform as well as the old L5520. I've asked Linode to look into it we'll see what they come up with.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

From a Linode 1024 on the new E5 2670:

Unixbench 5.1.3.
380 / 1808

 BYTE UNIX Benchmarks (Version 5.1.3)

   System: tsa: GNU/Linux
   OS: GNU/Linux -- 3.8.4-linode50 -- #1 SMP Mon Mar 25 15:50:29 EDT 2013
   Machine: i686 (i386)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 1: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 2: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 3: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 4: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 5: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 6: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   CPU 7: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (5200.1 bogomips)
          Hyper-Threading, MMX, Physical Address Ext
   16:52:24 up 6 days, 17:39,  2 users,  load average: 0.70, 2.59, 2.29; runlevel 2

------------------------------------------------------------------------
Benchmark Run: Sat Apr 20 2013 16:52:24 - 17:20:41
8 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       17501126.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     2672.2 MWIPS (10.0 s, 7 samples)
Execl Throughput                               1296.3 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks         99418.0 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           26767.1 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        769607.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                              123387.6 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  18226.4 lps   (10.0 s, 7 samples)
Process Creation                               2459.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   4279.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1665.1 lpm   (60.0 s, 2 samples)
System Call Overhead                         465484.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   17501126.2   1499.7
Double-Precision Whetstone                       55.0       2672.2    485.9
Execl Throughput                                 43.0       1296.3    301.5
File Copy 1024 bufsize 2000 maxblocks          3960.0      99418.0    251.1
File Copy 256 bufsize 500 maxblocks            1655.0      26767.1    161.7
File Copy 4096 bufsize 8000 maxblocks          5800.0     769607.5   1326.9
Pipe Throughput                               12440.0     123387.6     99.2
Pipe-based Context Switching                   4000.0      18226.4     45.6
Process Creation                                126.0       2459.9    195.2
Shell Scripts (1 concurrent)                     42.4       4279.6   1009.3
Shell Scripts (8 concurrent)                      6.0       1665.1   2775.1
System Call Overhead                          15000.0     465484.2    310.3
                                                                   ========
System Benchmarks Index Score                                         380.0

------------------------------------------------------------------------
Benchmark Run: Sat Apr 20 2013 17:20:41 - 17:49:07
8 CPUs in system; running 8 parallel copies of tests

Dhrystone 2 using register variables      100086132.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                    19927.2 MWIPS (10.0 s, 7 samples)
Execl Throughput                               7327.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        496250.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          108040.8 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1610986.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                              962477.2 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 227418.8 lps   (10.0 s, 7 samples)
Process Creation                              11781.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                  15480.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   2061.2 lpm   (60.2 s, 2 samples)
System Call Overhead                        2952615.5 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0  100086132.2   8576.4
Double-Precision Whetstone                       55.0      19927.2   3623.1
Execl Throughput                                 43.0       7327.2   1704.0
File Copy 1024 bufsize 2000 maxblocks          3960.0     496250.3   1253.2
File Copy 256 bufsize 500 maxblocks            1655.0     108040.8    652.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1610986.5   2777.6
Pipe Throughput                               12440.0     962477.2    773.7
Pipe-based Context Switching                   4000.0     227418.8    568.5
Process Creation                                126.0      11781.3    935.0
Shell Scripts (1 concurrent)                     42.4      15480.2   3651.0
Shell Scripts (8 concurrent)                      6.0       2061.2   3435.3
System Call Overhead                          15000.0    2952615.5   1968.4
                                                                   ========
System Benchmarks Index Score                                        1808.2

Maybe more telling than running benchmarks is some real-world data - time for clamd to reload it's database.



Can you guess when I migrated to an E5-2630L to get my free double RAM?

That clearly illustrates a serious problem here. I'm wondering what changed, has Linode started overselling RAM or something?

Anyone know if the problems happened with the RAM upgrade or the CPU upgrade? Maybe this is due to some flaw in xen that's killing CPU caching or something of that nature.