Author Topic: Benchmarks on real vs virtualised hardware  (Read 8521 times)

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Benchmarks on real vs virtualised hardware
« on: 2011.04.13, 00:30:24 »
A couple of weeks ago, I bought a new Intel Core i7-2600 processor, and a Gigabyte GA-P67A-UD3R-B3 motherboard, and 4gb of DDR3 ram to go with it, hoping to use it to continue to run eComStation 2.0.

Whilst the system does run eCS, and all 4 cores are recognised by acpi.psd, some other aspects leave a lot to be desired:
- SATA controller is recognised only in generic mode
- Panorama is unable to set MTRRs, so video speed sucks.

The above led me to explore running Ubuntu - something I've dabbled with in the past on my Thinkpad.  Over the last week since I had knee surgery, I've dabbled with Ubuntu some more on the Thinkpad due to intermittent wireless problems with eCS on the Thinkpad, and have grown to like it as a desktop OS - especially once I found I could run PMView for Windows using WINE directly from the desktop.

I've now installed Ubuntu 10.10 on my new hardware, and yesterday installed eCS under Virtualbox 4.0.4 as an experiment to see how quickly things compiled.

The results are as follows (not sure how cleanly this will display) (All times in minutes)

App                Intel Q9400 Native    Intel i7-2600 Native         Intel i7-2600 Virtual**

Bind 9.8.0               3:17                            4:04                                  1:47
Quassel 0.7.2        15:51                          15:00                                  8:45
Ghostscript 9.02      5:42                            5:18                                  2:43
MySQL 5.1.56        28:55                          22:22                                12:16

**Intel i7-2600 Virtual represents eCS 2.0 running under Virtualbox 4.0.4 under a 64-bit build of Ubuntu 10.10

As can be seem, the Intel i7-2600 Virtualised is nearly twice as fast as either of the native systems.

I'm continuing to work to get the SATA controller and system MTRR problems addressed for the i7-2600 using eCS - but at this stage it's looking like I'll be switching to Ubuntu for my main desktop OS real soon now.

Note that I do intend to continue to support my ports of OS/2 and eCS software at http://os2ports.smedley.info - porting software is something that I continue to enjoy to do!
« Last Edit: 2011.04.13, 01:22:17 by Paul Smedley »

David McKenna

  • Full Member
  • ***
  • Posts: 130
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #1 on: 2011.04.13, 01:50:06 »
Hi Paul,

  Those results are fantastic! I have often wondered if it would be possible to set up an absolutely minimalist Linux (I haven't found a distribution I really liked) that all it does is run VirtualBox and somehow you could have it boot to a VirtualBox menu where you then choose your OS... sort of like a Boot Manager or AirBoot ... and then run whatever OS's you like virtually. A 'VirtualOS' or BootOS' so to speak. Your results show it could be a viable solution.

  As you know, I too got a new motherboard and processor and am less than thrilled with how eCS runs on it, although it seems certain the issues will be taken care of eventually. I think I will spend some time looking at Linux again...

  Hope everything works out with your knee.

Regards,

Dave McKenna

Andreas Kohl

  • Newbie
  • *
  • Posts: 5
  • Operate at a higher level
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #2 on: 2011.04.13, 01:57:46 »
I'm curious about the configuration of your virtual machine for virtual box.
With some tuning some better results may be possible.

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #3 on: 2011.04.13, 03:04:37 »
Hiya Dave,

  Those results are fantastic! I have often wondered if it would be possible to set up an absolutely minimalist Linux (I haven't found a distribution I really liked) that all it does is run VirtualBox and somehow you could have it boot to a VirtualBox menu where you then choose your OS... sort of like a Boot Manager or AirBoot ... and then run whatever OS's you like virtually. A 'VirtualOS' or BootOS' so to speak. Your results show it could be a viable solution.

  As you know, I too got a new motherboard and processor and am less than thrilled with how eCS runs on it, although it seems certain the issues will be taken care of eventually. I think I will spend some time looking at Linux again...

  Hope everything works out with your knee.

Yeah I must say I was surprised - I was expecting to find a penalty in terms of compiler speed running under a virtual machine - I did the tests to determine how big the penalty would be, and could I live with it.

I was absolutely staggered to find that the virtual machine gives the best results.

I have a 2nd hard drive with eCS installed in the new machine, so I'll continue to test things out as they develop - ie fixed gradd.sys for mtrr settings, updated danis506.add, etc

However for me, my desktop usage at home is pretty much email and web and compiling code.

Email and web I can do from any OS, compiling code I need eCS - as I don't actually create code, I'm not a good enough programmer for that, I just port stuff to eCS - something that gives me a lot of satisfaction, and something I do as a 'release' from my day job.

Knees are coming along well - a 'clean out' of some arthritis from behind both knee caps.  Should be back at work next week.

Cheers,

Paul

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #4 on: 2011.04.13, 03:05:54 »
Hi Andreas,

I'm curious about the configuration of your virtual machine for virtual box.
With some tuning some better results may be possible.

So far, I've done no tuning.  It's basically a stock Virtualbox setup except I changed the NIC to an Intel gigabit adapter, and set the RAM for the virtual OS to about a gig.

I'd be interested in any tips to further improve performance!

Cheers,

Paul

Sigurd

  • Full Member
  • ***
  • Posts: 245
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #5 on: 2011.04.13, 07:48:31 »
Hi Paul,

while playing around for two years with my Lenovo X200T (Core2Duo SL9600 Centrino 2 Vpro) and various eCs and OS/2 versions I did figure out that (please note: on this hardware):

- OS/2 Warp 4.52 FP 5 does run more reliable and stable and faster on this hardware (even if, as allways, some do not want to read nor hear it)
- OS/2 Merlin does run in AHCI while eCS2 not, some times I need to reboot Merlin because Systemactivity rises with AHCI direct after being started
- SNAP in Vesa 3.0 mode does still outclass Panorama VESA - you might give the SNAP FULL version (not the "eCs" one) a try, it reduced the coloursheme faults etc. dramaticly, even though it has the Inbuild Intel 4500 GraficsHardware - I have no MTTR problems with SNAP - might be different on your machine, but I guess it is worth a try, and at least it is good to have less temperature
- I do not know if ACPI with Powermanagement is working on your machine. The "Reactor widget" of eCS2 sometimes did show me strange temperatures - with OS/2 Merlin (because of having no ACPI nor Powermanagement) I do use a small programm from Nickk to see the CPU temperature

-> Please do not missunderstand: I do not want to persuade you to use Merlin instead of eCS 2 as I did, this is just the result of my experiences.

And as I am interested in to compare something from my X200T with Merlin from SSD with your machine but do not know how to compile something: Is there a benchmark suite or program I could use to compare some results with you? That would be really interesting for me!

And - if allowed - another question: Does the BIOS of your PC does still support a "compatible mode" for SATA?

Thanks!

@ David McKenna:
I did try the same as you suggested, does have a USB Ubuntu stick running with OS/2 Warp 4 inside Virtualbox 4.04. However I did not configure it to "autostart". These links here might be interesting for you:

http://forums.virtualbox.org/viewtopic.php?t=12426&postdays=0&postorder=asc&start=15
(How to autostart a VM with Virtualbox)

http://www.pendrivelinux.com/yumi-multiboot-usb-creator/
(How to boot Multi OS with a small Linux)

Hope this is of some help.

Cheers

Sigurd

rudi

  • Full Member
  • ***
  • Posts: 116
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #6 on: 2011.04.13, 08:30:28 »
**Intel i7-2600 Virtual represents eCS 2.0 running under Virtualbox 4.0.4 under a 64-bit build of Ubuntu 10.10

As can be seem, the Intel i7-2600 Virtualised is nearly twice as fast as either of the native systems.

At the company I have a Gigabyte GA-X58-USB3 (Core i7-950, 8GB RAM) running Windows7-x64. I noticed that compiling Qt in a "VirtualBoxed" eCS is pretty quick. However, since I never bothered to install any incarnation of OS/2 natively on that machine, I don't know how it would compare.

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #7 on: 2011.04.13, 08:53:42 »
Hiya Sigurd,

And as I am interested in to compare something from my X200T with Merlin from SSD with your machine but do not know how to compile something: Is there a benchmark suite or program I could use to compare some results with you? That would be really interesting for me!

And - if allowed - another question: Does the BIOS of your PC does still support a "compatible mode" for SATA?

I used the compilation as it was the most meaningful benchmark to me - as it's my primary use of eCS :)

If you were that keen to compare, I could send you a link to my build environment, and to some sample code in order to run the same test.  I'm not sure though what relevance the results would have though.  I was trying to emphasise the lack of improvement in compilation with native eCS, and the enormous improvement by using modern, virtualised hardware.  Really, I think the results and timings are relevant only to the examples I've posted.

Now if I were to do the same benchmarks with native eCS and an SSD - that may show something meaningful in terms of the impact SSD has compared with a standard hard drive.

Cheers,

Paul

Yes my BIOS supports a compatible mode for SATA - this is how I currently have the BIOS set so that I can still boot eCS as well as Ubuntu.

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #8 on: 2011.04.13, 08:55:15 »
Hiya Rudi,

**Intel i7-2600 Virtual represents eCS 2.0 running under Virtualbox 4.0.4 under a 64-bit build of Ubuntu 10.10

As can be seem, the Intel i7-2600 Virtualised is nearly twice as fast as either of the native systems.

At the company I have a Gigabyte GA-X58-USB3 (Core i7-950, 8GB RAM) running Windows7-x64. I noticed that compiling Qt in a "VirtualBoxed" eCS is pretty quick. However, since I never bothered to install any incarnation of OS/2 natively on that machine, I don't know how it would compare.


Based on my results, sadly, it's most likely the "Virtualboxed" eCS would provide better results than native eCS.

Sigurd

  • Full Member
  • ***
  • Posts: 245
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #9 on: 2011.04.13, 09:29:49 »
Hi Paul,


I used the compilation as it was the most meaningful benchmark to me - as it's my primary use of eCS :)

If you were that keen to compare, I could send you a link to my build environment, and to some sample code in order to run the same test.  I'm not sure though what relevance the results would have though.  I was trying to emphasise the lack of improvement in compilation with native eCS, and the enormous improvement by using modern, virtualised hardware.  Really, I think the results and timings are relevant only to the examples I've posted.

Now if I were to do the same benchmarks with native eCS and an SSD - that may show something meaningful in terms of the impact SSD has compared with a standard hard drive.

Cheers,

Paul

Yes my BIOS supports a compatible mode for SATA - this is how I currently have the BIOS set so that I can still boot eCS as well as Ubuntu.

With the benchmark I just wanted to see if I just "think or believe" that Merlin on SSD is that fast or if it is really so. It was not intended to be a shootout  ;) I will leave it as it is for now.

Thanks for the BIOS information, I may replace the X200T with the new X220T with an i7-2620 Sandy Bridge later this year, and I already asked for this in Thinkpad Forums - did not get an answer though because those versions are still not being delivered - even though Merlin does run form AHCI it seems to run more reliable in Compatibility Mode, so this is good to know that there is at least the possibility to switch. Have to wait if this is implemented in the Lenovo Bios though.

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #10 on: 2011.04.13, 09:33:40 »
Hiya Sigurd!

With the benchmark I just wanted to see if I just "think or believe" that Merlin on SSD is that fast or if it is really so. It was not intended to be a shootout  ;) I will leave it as it is for now.

Thanks for the BIOS information, I may replace the X200T with the new X220T with an i7-2620 Sandy Bridge later this year, and I already asked for this in Thinkpad Forums - did not get an answer though because those versions are still not being delivered - even though Merlin does run form AHCI it seems to run more reliable in Compatibility Mode, so this is good to know that there is at least the possibility to switch. Have to wait if this is implemented in the Lenovo Bios though.

It would be interesting if you could compare compilation speed on the X200T with SSD and with a normal hard drive.  _I'd_ be interested in that :)

Andreas Kohl

  • Newbie
  • *
  • Posts: 5
  • Operate at a higher level
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #11 on: 2011.04.13, 14:26:34 »
Virtual Hardware behaves similar to real hardware, so to improve processing and i/o you could use SCSI in favour of IDE/ATAPI for the emulated host bus adapter. VirtualBox supports emulated LSI Logic and BusLogic SCSI  adapters. By using VMDK you could even connect to physical disks or partitions.

Fahrvenugen

  • Full Member
  • ***
  • Posts: 203
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #12 on: 2011.04.13, 17:34:35 »

So far, I've done no tuning.  It's basically a stock Virtualbox setup except I changed the NIC to an Intel gigabit adapter, and set the RAM for the virtual OS to about a gig.

I'd be interested in any tips to further improve performance!

Cheers,

Paul

Hi,

I've got a few theories on this, but it would require additional testing.

First, from what I understand, modern 64 bit CPU's operate more efficiently when running a 64 bit OS then when run in 32 bit mode.  The fact that you're running the 64 bit build of Ubuntu and then eCS (which we know is 32 bit) virtualized, I'm wondering if some of the extra efficiencies that you get by having the CPU in native 64 bit mode is making a difference.  To test this, it would be interesting to run the same test but using the 32 bit version of Ubuntu.

The second thing I'm wondering - is Virtualbox set up to emulate a single CPU, or SMP?  Also, is the OS/2 build of GCC set up to take advantage of SMP, or does it only use 1 core when compiling?  I seem to recall way back when (in the OS/2 2.1 SMP days) that some apps that were only coded to use a single core actually saw minor performance drops (usually less then 5%) when used on the SMP kernel with multiple processors.  Of course this wouldn't account for the over 50% difference that you're seeing.  Just thinking of theories...


David McKenna

  • Full Member
  • ***
  • Posts: 130
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #13 on: 2011.04.13, 22:55:51 »

@ David McKenna:
I did try the same as you suggested, does have a USB Ubuntu stick running with OS/2 Warp 4 inside Virtualbox 4.04. However I did not configure it to "autostart". These links here might be interesting for you:

http://forums.virtualbox.org/viewtopic.php?t=12426&postdays=0&postorder=asc&start=15
(How to autostart a VM with Virtualbox)

http://www.pendrivelinux.com/yumi-multiboot-usb-creator/
(How to boot Multi OS with a small Linux)

Hope this is of some help.

Cheers

Sigurd

  Thanks for those links! I'll give this some consideration over the weekend...

Regards,

Dave McKenna

Paul Smedley

  • Sr. Member
  • ****
  • Posts: 322
    • View Profile
Re: Benchmarks on real vs virtualised hardware
« Reply #14 on: 2011.04.13, 23:38:12 »
Hi Andreas,

Virtual Hardware behaves similar to real hardware, so to improve processing and i/o you could use SCSI in favour of IDE/ATAPI for the emulated host bus adapter. VirtualBox supports emulated LSI Logic and BusLogic SCSI  adapters. By using VMDK you could even connect to physical disks or partitions.

Sounds interesting, will have to do some trials with this when I get some free time.

One thing I'm wondering if it's possible or not...

I'd like to somehow image my existing build drive and make it available to virtualbox.

I've got all the compiler tools in virtualbox already, but I'd like to move all my source code over, and xcopy ain't going to cut it :)