I've been using Linux/Unix for many years. I've always had a strong interest in technology in general and computing specifically.

These are my opinions. Opinions are like noses, everyone has one, and they all smell.

Enjoy your visit.
April 2011
« Jun   Dec »

Three Virtual Machines, one host

Just a word about this post:

If this seems disjointed, illogical with more than a few misspelled words, come back later. This started out as my notes on configuring a set of virtual servers inside a client’s network. So I started writing as I went along. Eventually I’ll have it edited and make it final, but for now it’s just my notes. When it’s finished, I’ll remove this note.

Just sayin.

I have been tasked with setting up three virtual machines on a single host. These hosts will each have only one basic function on the company intranet. They will be a cvs server, a ftp server and a samba server. This will be a place to keep my notes.

Since I’m working with x86-64 hardware and all guest OS’s will be Linux, KVM seems the best choice. What I’ve read says it imposes very low overhead, and I like the ability to use a logical volume directly. I planned to install each virtual host into a separate logical volume, with the intention of being able to adjust the size of the hard drive inside the virtual machine, as needed by a changing business. We chose CentOS 5.5 as it seems a good choice for the standard on all this company’s servers. Most of their servers are RH or clones thereof.


Dell PowerEdge R210 Intel® Core™ I3 540 3.06GHz, 4M Cache, 2C/4T 16 Gb Memory (4x4Gb) 2 x 2TB 7.2K RPM SATA 3.5in Cabled Hard Drive mirrored, DVD-ROM Drive and BMC.

On some Dell hardware, you also need to disable “Trusted Execution”, otherwise VT will not be enabled. That was not the case on this hardware. The CPU does have the VT extensions.

ftp server
200 Gb
4 Gb RAM

cvs server
4 Gb RAM

samba server
4 Gb RAM

The machine came with 2 drives so that we could mirror them. The system came with software raid controller, so I just chose to use Linux built in software raid. I configured most of the the drive space in RAID I. To install the system I needed to create a 100M partition outside the raid, because /boot cannot be within the software raid.

Raid Devices

/boot  100MB
/opt 100MB

After the install, I changed the options in fstab to ro,noauto,nouser,sync and then did a poor mans mirror

dd if=/dev/sda1 of=/dev/sdb1

The unexpected result was that this changed the label so the partition would no longer mount on /opt. I’ll have to relabel the partition and then add the entry back to the fstab file. I wonder what happens when you have two disks with the same disk label?

I found later one consequence, when I got in a state where I needed to try to upgrade to fix some things. The boot CD found two partitions /dev/sda1 and /dev/sdb1 with disklabel /boot. It refused to continue, telling me to fix that first. When I do this next time I will use tune2fs to relabel the partition. If we ever lose the drive with /boot, my hope is that it will allow us to continue running the system. This is a mirrored drive setup, and with a bootable partition, it can be recovered more simply.

The rest of the 2 terrabyte drive  became the Volume Group System. I’m not a fan of the default names used during the install. Which logical volume inside VOL_GROUP00 contains the /usr partiton? Is it VOL00 or VOL01 or VOL05? I override the names and give them names that will help me identify the data, when I have to boot from a rescue CD and start copying the data off a failing system, or make a change to the fstab to get the system to boot from the still good drive in a mirrored pair.

Inside that I created logical volumes

logical volume root mounted on /

logical volume swap

Since the installer always wants a mount point for each partition and logical volume, I wait until after the install of the host system to create:

logical volume ftp for the ftp server

lvcreate -L 200G -n ftp System

logical volume cvs for the cvs server

lvcreate -L 200G -n cvs System

logical volume samba for the samba server

lvcreate -L 200G -n samba System

Resizing logical volumes inside logical volumes

1. shutdown virtual machine
use kparx to add the logical volumes

kpartx -a /dev/System/ftp


LV            VG      Attr    LSize     Origin  Snap%  Move  Log Copy% Convert
samba      System    -wi-a-     1.00T
cvs           System    -wi-a-  198.00G
ftp           System     -wi-ao 248.00G
root         System     -wi-ao    3.91G
swap        System    -wi-ao    1.94G
LogVol00 VolGroup00 -wi— 192.22G
LogVol101 VolGroup00 -wi–    5.66G

2. extend HOST logical volume

lvextend --L+50G /dev/mapper/System-ftp
Extending logical volume ftp to 248.00 GB
Logical volume ftp successfully resized

vgdisplay -v

3. resize the physical volume on the virtual machine

pvresize --setphysicalvolumesize 248G /dev/vda2

lvextend -L+50G /dev/VolGroup00/LogVol00

This failed. I was able to resize the logical volumes on the host, but I kept getting errors similar to:

device-mapper: reload ioctl failed: Invalid argument Failed to suspend LogVol00

I found plenty of links to other people who’ve encountered this problem, but no solution. So we decided to fix the size of the logical volumes for each host and move on.

So I set fixed sizes for the logical volumes.

400G for the ftp server

557G for the cvs server

900G for the samba server

Then I installed CentOS 5.5 on each.


Getting the machines to connect to the LAN with addresses on that LAN was another challenge. The default CentOS install set up a virtual network between the hosts with an outbound NAT connection. I wanted each machine to have a discreet IP address and be as separated as possible. Same reason I had them running in their own logical volumes.

I found lots of descriptions of what I needed to do, but they all seemed to lacking a small piece of information. I discovered you must create the bridge device on the host first, before you install the virtual hosts. At least for me, I wasn’t able to install and then change the network configuration. I may not have understood the required modifications well enough.

I used the instructions from this site:

CentOS / Redhat: KVM Bridged Network Configuration

I first created the bridge device. I made backups of original files before I started. Remember to use prefix, not postfix notation. bak.ifcfg. The scripts look for any file that starts with ifcfg and then acts upon them. First a new file for the bridge device.

vi /etc/sysconfig/network-scripts/ifcfg-br0


Then edit the ethernet configuration file, after making a backup:

vi /etc/sysconfig/network-scripts/ifcfg-eth0


Other interesting links:

Libvirt overwrites the existing iptables rules

Redhat Hypervisor Deployment Guide

Xen Cloud Platform

Setting guest network

VLAN bridge config

How to Get Maximum Network Performance using paravirtual drivers and bridged networking

Using bridged networking with Virt-manager

A Quick Guide to Using KVM with CentOS-5.1

Virtual Networking

KVM is interesting again.. and how I setup my virtual network…

KVM Bridged Network – solutions and problems

CentOS / Redhat: KVM Bridged Network Configuration

How to setup Windows guest paravirtual network drivers

CentOS 5.x Samba Domain Controller With LDAP Backend


Comments are closed.