Run your entire network on a single server (EN)
Aus Xen-Forum-Wiki
Article-Licens
© Copyright 2005 - Julien Danjou
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License.
Inhaltsverzeichnis |
Introduction
Since several years, I build my own network at home, running 2 to 7 machines at the same time (gateway, firewall, workstation, devstation, servers...). In my flat, it produce a lot of noise, take too many space and consume electricty. I decided to stop that, and to run my all network into a single machine, using virtual machine.
The technology that convinced me is Xen.
What I wanted to do: all-in-one
My network topology was this:
[PUB] [DMZ]
Internet --- gateway --- server
|
| [ADM]
devstation --+-- workstation
My server was on a network called dmz, and my {work,dev}station were on a network called adm.
What I wanted to do was the same, but with my server, my gateway and my devstation being 3 virtual machines, in a real machine.
Requirements
- A GNU/Linux Debian machine.
- Some coffee
- Some theorical knowledge about Xen: at least what is a dom0 and a domU.
The beginning
Install a standard Debian Sarge. Usage of LVM is not mandatory, but it is very very efficient. You will be able to create partitions (logical volumes) for your VMs. You can also prepare a big partition and store image files for your VMs root filesystems, or many partitions.
Add Xen repository for Sarge to your sources.list.
And now, let's begin.
apt-get install xen
Easy, isn't it?
You should also install others packages needed by some scripts:
apt-get install iproute bridge-utils
You will have to disable TLS, or your Xen machine will be slower.
% mv /lib/tls /lib/tls.disabled
Prepare Grub
Install grub and run update-grub if needed. Then, add this lines to your /boot/grub/menu.lst
title Xen 2.0 / XenLinu 2.6 kernel /boot/xen.gz module /boot/xen-linux-2.6.12-xen0 root=/dev/hda1 ro
Add them just before ### BEGIN AUTOMAGIC KERNELS LIST or after ### END DEBIAN AUTOMAGIC KERNELS LIST
Prepare your kernel
Using Xen provided kernels
Simply download the Xen binary tarball, grab linux-*-xen0 and linux-*-xenU, then put them in /boot.
Build your own kernel
For testing purpose, I used kernels provided by Xen, but since I needed more stuff so I decided to build my own kernels.
Download a Linux kernel source tarball from your local mirror, unpack it and patch it with the Xen patch.
Here is how I managed to compile my custom kernels:
% vi Makefile % make menuconfig ARCH=xen [configure like a normal machine] For xen0 kernel: [*] Privileged Guest (domain 0) [*] Block-device backend driver [*] Network-device backend driver [*] Scrub memory before freeing it to Xen For xenU kernel: [*] Block-device frontend driver [*] Network-device frontend driver % make-kpkg --append-to-versision -xen0 --arch xen kernel_image or % make-kpkg --append-to-versision -xen-u --arch xen kernel_image
Build your xen0 and xenU kernels, and then dpkg -i the new Debian files you got.
kernel-xen0-2.6.12-xen-u_10.00.Custom_i386.deb kernel-xen0-2.6.12-xen0_10.00.Custom_i386.deb
It will install the kernels as:
/boot/xen-linux-2.6.12-xen0 /boot/xen-linux-2.6.12-xen-u
Ok, now you can boot Xen using your xen0 kernel. You should already have edited your /boot/grub/menu.lst so just type reboot and pray!
Installing a new Debian virtual machine
Using partitions
In this example, I will use real disk partitions. Personnaly, I use LVM, which is a very more efficient way.
% mkfs.ext3 /dev/hdX % mount /dev/hdX /mnt
If you want to use more partitions (for /usr, /var, etc): Example for /usr:
% mkfs.ext3 /dev/hdY % mkdir /mnt/usr % mount /dev/hdY /mnt/usr
Maybe useful for swap for example.
If you need to create a swap file or partition, use the mkswap command:
mkswap /dev/hdX
Using files
Create a file for storing your partiton.
% dd if=/dev/zero of=/srv/dakara.img bs=1M count=<size_in_MB> % mkfs.ext3 /srv/dakara.img % mount -o loop /srv/dakara.img /mnt
If you want to use more partitions (for /usr, /var, etc): Example for /usr:
% dd if=/dev/zero of=/srv/dakara_usr.img bs=1M count=<size_in_MB> % mkfs.ext3 /srv/dakara_usr.img % mkdir /mnt/usr % mount -o loop /srv/dakara_usr.img /mnt/usr
Maybe useful for swap for example.
If you need to create a swap file or partition, use the mkswap command:
mkswap dakara_swap.img
Configure your new VM
Run debootstrap <distrib> /mnt <mirror>
% debootstrap sarge /mnt ftp://ftp.fr.debian.org/debian
If you have built modules in your xenU kernel, install it in your domU:
% cp kernel-xen0-2.6.11.12-xen-u_10.00.Custom_i386.deb /mnt/root
Now chroot to your new system:
% chroot /mnt /bin/bash
Install the DomU kernel if needed:
% dpkg -i /root/kernel-xen0-2.6.11.12-xen-u_10.00.Custom_i386.deb
Add a correct entry for /
% vi /etc/fstab /dev/hda1 / ext3 defaults,errors=remount-ro 0 1
If you use several partitions, think to add them
/dev/hda2 /usr ext3 defaults 0 2 /dev/hda3 /var ext3 defaults 0 2 etc..
Run base-config
% base-config new
Edit your network configuration:
% vi /etc/network/interfaces % vi /etc/resolv.conf % vi /etc/hosts
You will have to disable TLS, or your Xen machine will be slower.
% mv /lib/tls /lib/tls.disabled
You may have to disable useless getty. Edit /etc/inittab and delete or comment out the following lines, like this:
1:2345:respawn:/sbin/getty 38400 tty1 #2:23:respawn:/sbin/getty 38400 tty2 #3:23:respawn:/sbin/getty 38400 tty3 #4:23:respawn:/sbin/getty 38400 tty4 #5:23:respawn:/sbin/getty 38400 tty5 #6:23:respawn:/sbin/getty 38400 tty6
If you need to load modules, think about eventually install module-init-tools (needed for 2.6 domU kernels)
apt-get install module-init-tools
Finish.
% exit
If you have created several partitions, unmount then:
% umount /mnt/usr % umount /mnt/var etc...
Then umount root filesystem:
% umount /mnt
Configure your VM
Now, create the configuration file for your VM. Mine is named dakara. There are examples in /usr/share/doc/xen/examples, which are commented out.
% vi /etc/xen/dakara kernel = "/boot/xen-linux-2.6.11.12-xen-u" memory = 128 name = "dakara" root = "/dev/hda1 ro"
If you use a partition, add:
disk = [ 'phy:/dev/hdX,hda1,w' ]
If you use several paritions, add:
disk = [ 'phy:/dev/hdX,hda1,w', 'phy:/dev/hdY,hda2,w' ]
If you use one file, add:
disk = [ 'file:/srv/dakara.img,hda1,w' ]
If you use sevral files, add:
disk = [ 'file:/srv/dakara.img,hda1,w', 'file:/srv/dakara_usr.img,hda2,w' ]
Think that your Xen disk entries should match your domU fstab, obviously.
Let's go
Everything ok ? Here we are.
% xm create -c dakara
Will boot your new VM and launch a console (-c option).
You should be able to login via 'root' login. You can configure your eth0 interface.
You can exit the console by pressing Ctrl+].
You can return to the console by running:
% xm console dakara
You can shutdown your domain by running:
% xm shutdown -H dakara
You can destroy (won't do a clean shutdown!) your domain by running:
% xm destroy dakara
Typing xm help will give you a full list of command, like shutdown, destroy, etc.
If you want your domain to be booted when your dom0 boot, simply do:
% cd /etc/xen/auto % ln -s <configfile> .
Building 3 networks in one machine
By default, Xen will create a bridge called xen-br0. In my case, I have two NIC and 3 networks: pub, adm and dmz.
My DMZ is not connected physically to any NIC, so I use the dummy driver to emulate a third NIC.
% echo dummy >> /etc/modules % modprobe dummy
In /etc/network/interfaces:
auto dummy0 iface dummy0 inet static address 192.168.3.30 netmask 255.255.255.0 gateway 192.168.3.1 auto eth0 iface eth0 inet manual up /sbin/ifconfig eth0 up auto eth1 iface eth1 inet manual up /sbin/ifconfig eth1 up
My Xen domain0 host will be only connected to the dmz by dummy0. Real Ethernet interfaces will be used for bridging purpose only. My server (dakara) will be connected to dmz (its eth0 will be bound to bridge xen-br-dmz with dummy0). My gateway (chapai, running NetBSD 2.0) will be connected to all NIC (eth0, eth1 and dummy0).
By the way, my dmz and adm gateway (192.168.3.1 and 192.168.2.1) will be a Xen host (chapai, running NetBSD).
Tweaking the bridge
As I said, xend build only one bridge by default on eth0. Here I wanted to build 3 bridges:
xen-br-pub -> eth0 xen-br-adm -> eth1 xen-br-dmz -> dummy0
So I modified the network script this way:
% cd /etc/xen/scripts % mv network network.xen % editor network
My /etc/xen/scripts/network script:
#!/bin/sh
# Exit if anything goes wrong.
set -e
# First arg is the operation.
OP=$1
shift
script=/etc/xen/scripts/network.xen
case $OP in
start)
$script start bridge=xen-br-adm netdev=eth1
$script start bridge=xen-br-dmz netdev=dummy0
$script start bridge=xen-br-pub netdev=eth0
;;
stop)
$script stop bridge=xen-br-adm netdev=eth1
$script stop bridge=xen-br-dmz netdev=dummy0
$script stop bridge=xen-br-pub netdev=eth0
;;
*)
echo 'Unknown command: ' ${OP}
echo 'Valid commands are: start, stop'
exit 1
esac
Another solution is to set up your bridges in /etc/network/interfaces and to disable /etc/xen/scripts/network.
In each domain configuration file, you can specify the bridge with the vif option for each interface:
vif = [ 'bridge=xen-br-adm' ]
For my domU gateway, I will have 3 virtual NIC and 3 vif bound to 3 different bridges:
vif = [ 'bridge=xen-br-pub', 'bridge=xen-br-adm', 'bridge=xen-br-dmz' ]
Installing a NetBSD domU in a Linux dom0
First grab netbsd-INSTALL_XENU from your local NetBSD mirror. This is a NetBSD 3.0 beta kernel with an installer, so it will enable you to install NetBSD.
Create a new domain configuration file using your netbsd-INSTALL_XENU as kernel.
kernel = "/boot/netbsd-INSTALL_XENU"
During the installation, install NetBSD 2.0 tools, not the 3.0 ones, since we are setting up a NetBSD 2.0 host.
After installing NetBSD (this is pretty simple, read the installation doc if you need to), halt it.
Download the Xen 2.0 sources and untar it. Edit Makefile. Change line KERNELS into this:
KERNELS ?= netbsd-2.0-xenU
Type make.
Pick out your NetBSD kernel and move it as /boot/netbsd-2.0-xenU.
Now edit your configuration file and modify it with the new kernel:
kernel = "/boot/netbsd-2.0-xenU"
Simply boot your new NetBSD system with xm create.
Your xennet devices will be connected respectively to the bridges listed in the vif configuration directive.
If you have problems, try to read this.
