Three Virtual Machines, one host

pacneil — Wed, 06 Apr 2011 17:17:22 +0000

Just a word about this post:

If this seems disjointed, illogical with more than a few misspelled words, come back later. This started out as my notes on configuring a set of virtual servers inside a client’s network. So I started writing as I went along. Eventually I’ll have it edited and make it final, but for now it’s just my notes. When it’s finished, I’ll remove this note.

Just sayin.

I have been tasked with setting up three virtual machines on a single host. These hosts will each have only one basic function on the company intranet. They will be a cvs server, a ftp server and a samba server. This will be a place to keep my notes.

Since I’m working with x86-64 hardware and all guest OS’s will be Linux, KVM seems the best choice. What I’ve read says it imposes very low overhead, and I like the ability to use a logical volume directly. I planned to install each virtual host into a separate logical volume, with the intention of being able to adjust the size of the hard drive inside the virtual machine, as needed by a changing business. We chose CentOS 5.5 as it seems a good choice for the standard on all this company’s servers. Most of their servers are RH or clones thereof.

Hardware

Dell PowerEdge R210 Intel® Core™ I3 540 3.06GHz, 4M Cache, 2C/4T 16 Gb Memory (4x4Gb) 2 x 2TB 7.2K RPM SATA 3.5in Cabled Hard Drive mirrored, DVD-ROM Drive and BMC.

On some Dell hardware, you also need to disable “Trusted Execution”, otherwise VT will not be enabled. That was not the case on this hardware. The CPU does have the VT extensions.

ftp server
200 Gb
1 CPU
4 Gb RAM

cvs server
300Gb
1 CPU
4 Gb RAM

samba server
800Gb
1 CPU
4 Gb RAM

The machine came with 2 drives so that we could mirror them. The system came with software raid controller, so I just chose to use Linux built in software raid. I configured most of the the drive space in RAID I. To install the system I needed to create a 100M partition outside the raid, because /boot cannot be within the software raid.

Raid Devices

/boot 100MB
/opt 100MB

After the install, I changed the options in fstab to ro,noauto,nouser,sync and then did a poor mans mirror

dd if=/dev/sda1 of=/dev/sdb1

The unexpected result was that this changed the label so the partition would no longer mount on /opt. I’ll have to relabel the partition and then add the entry back to the fstab file. I wonder what happens when you have two disks with the same disk label?

I found later one consequence, when I got in a state where I needed to try to upgrade to fix some things. The boot CD found two partitions /dev/sda1 and /dev/sdb1 with disklabel /boot. It refused to continue, telling me to fix that first. When I do this next time I will use tune2fs to relabel the partition. If we ever lose the drive with /boot, my hope is that it will allow us to continue running the system. This is a mirrored drive setup, and with a bootable partition, it can be recovered more simply.

The rest of the 2 terrabyte drive became the Volume Group System. I’m not a fan of the default names used during the install. Which logical volume inside VOL_GROUP00 contains the /usr partiton? Is it VOL00 or VOL01 or VOL05? I override the names and give them names that will help me identify the data, when I have to boot from a rescue CD and start copying the data off a failing system, or make a change to the fstab to get the system to boot from the still good drive in a mirrored pair.

Inside that I created logical volumes

logical volume root mounted on /

logical volume swap

Since the installer always wants a mount point for each partition and logical volume, I wait until after the install of the host system to create:

logical volume ftp for the ftp server

lvcreate -L 200G -n ftp System

logical volume cvs for the cvs server

lvcreate -L 200G -n cvs System

logical volume samba for the samba server

lvcreate -L 200G -n samba System

Resizing logical volumes inside logical volumes

1. shutdown virtual machine
use kparx to add the logical volumes

kpartx -a /dev/System/ftp

lvs

LV            VG      Attr    LSize     Origin Snap% Move Log Copy% Convert
samba    System    -wi-a-     1.00T
cvs         System    -wi-a- 198.00G
ftp           System    -wi-ao 248.00G
root         System     -wi-ao    3.91G
swap        System    -wi-ao    1.94G
LogVol00 VolGroup00 -wi— 192.22G
LogVol101 VolGroup00 -wi–    5.66G

2. extend HOST logical volume

lvextend --L+50G /dev/mapper/System-ftp Extending logical volume ftp to 248.00 GB Logical volume ftp successfully resized

vgdisplay -v

3. resize the physical volume on the virtual machine

pvresize --setphysicalvolumesize 248G /dev/vda2

lvextend -L+50G /dev/VolGroup00/LogVol00

This failed. I was able to resize the logical volumes on the host, but I kept getting errors similar to:

device-mapper: reload ioctl failed: Invalid argument Failed to suspend LogVol00

I found plenty of links to other people who’ve encountered this problem, but no solution. So we decided to fix the size of the logical volumes for each host and move on.

So I set fixed sizes for the logical volumes.

400G for the ftp server

557G for the cvs server

900G for the samba server

Then I installed CentOS 5.5 on each.

Networking

Getting the machines to connect to the LAN with addresses on that LAN was another challenge. The default CentOS install set up a virtual network between the hosts with an outbound NAT connection. I wanted each machine to have a discreet IP address and be as separated as possible. Same reason I had them running in their own logical volumes.

I found lots of descriptions of what I needed to do, but they all seemed to lacking a small piece of information. I discovered you must create the bridge device on the host first, before you install the virtual hosts. At least for me, I wasn’t able to install and then change the network configuration. I may not have understood the required modifications well enough.

I used the instructions from this site:

CentOS / Redhat: KVM Bridged Network Configuration

I first created the bridge device. I made backups of original files before I started. Remember to use prefix, not postfix notation. bak.ifcfg. The scripts look for any file that starts with ifcfg and then acts upon them. First a new file for the bridge device.

vi /etc/sysconfig/network-scripts/ifcfg-br0

DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
IPADDR=192.168.1.5
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
ONBOOT=yes

Then edit the ethernet configuration file, after making a backup:

vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
HWADR=12:34:56:78:91:23
BRIDGE=br0
ONBOOT=yes

Redhat Enterprise 5.4 Released

pacneil — Thu, 03 Sep 2009 19:26:34 +0000

This is at it’s heart an update to RHEL 5. But Redhat is promoting it as “the first product to deliver commercial quality open source virtualization featuring Kernel-based Virtual Machine (KVM) hypervisor technology.” The kernel version is still 2.6.18, with backported patches. Redhat is promoting upcoming Red Hat Enterprise Virtualization products along with RHEL 5.4 on the announcement page .

The server release is touted as providing “the most cost-effective, flexible, and scalable environment”. There are two flavors, Red Hat Enterprise Linux and Red Hat Enterprise Linux Advanced Platform. The number of guests on Advanced Platform is unlimited while the standard release is limited to four guests.

Redhat Enterprise Linux is certified as a guest OS on the following platforms:

VMware ESX and VMware ESXi
BM POWER LPARs
IBM System z

Redhat Enterprise Linux supports three distributed system technologies:

Logical Volume Manager (LVM)
Global File System (GFS)
Distributed Lock Manager (DLM)

Redhat also has new managment tools for managing virtualized environments. And of course it provides tools like Samba for integration into Windows environments, Apache Web server, MySQL and Postgresql databases. And of course all the enterprise network services such as DNS, dhcp, and firewall capabilities are included.

Redhat is also promoting Redhat Enterprise Desktop as an alternative to “proprietary” desktop operating systems. They’re selling the “security” features and cost benefits of managing their system. And if you must run one of those “proprietary” systems, they offer virtualization to run it as a guest OS.

While not everything on RHEL 5.4 is the latest release, it does provide the kind of system and support that will make management comfortable.

Subscribers to RHEL 5 will get the updates automatically for free. New subscribers will pay about $349 for the server version and prices start at $80 for the desktop version.