OS

Dragonfly BSD 4.0.x

Date

17.07.2015

Advanced Jails in Dragonfly BSD

In this example we install a Jail using a central read-only Base-Jail like ezjail does.

An advanced Jail consists of an read-only Master-Jail and an writable jail-specific part, which can be located in it's own PFS (Pseudo File System).

There it can be snapshotted or replicated at will.

Directory Structure

Jails Root Directory

/usr/jails/

Base Jail Root

/usr/jails/basejail/

Master (ro)

/usr/jails/basejail/master

Skelleton (rw)

/usr/jails/basejail/skel

Jails Pseudo File System

/usr/jails/pfs/

When building a Jail we copy the Skeleton to the Jail-PFS (/usr/jails/pfs/) and mount it. If we update the World we can create a new Master mounting it read-only to every Jail.

Building the Frame

We assume that jailing has been activated in /etc/rc.conf

#Jails
jail_enable="YES"               # Set to NO to disable starting of any jails
jail_sysvipc_allow="YES"        # needed for Apache - see documentation
jail_list="mail"                # Space separated list of names of jails

Building Master & Skeleton

The following script builds the master and skeleton Jails.

# define variables
workingdir="/usr/jails/basejail"

#####################################
#####Create Master & Skelleton ######
#####################################

mkdir -p $workingdir/master
cd /usr/src
make installworld DESTDIR=$workingdir/master
cd etc 
make distribution DESTDIR=$workingdir/master -DNO_MAKEDEV_RUN
cpdup  /usr/dports $workingdir/master/usr/dports 
cpdup /usr/src $workingdir/master/usr/src
cd $workingdir/master
mkdir $workingdir/skel $workingdir/skel/home $workingdir/skel/usr-X11R6 $workingdir/skel/usr-distfiles
mv etc $workingdir/skel
mv usr/local $workingdir/skel/usr-local
mv usr/obj $workingdir/skel/usr-obj
mv tmp $workingdir/skel
mv var $workingdir/skel
mv root $workingdir/skel
mergemaster -t $workingdir/skel/var/tmp/temproot -D $workingdir/skel -i
cd $workingdir/skel/
rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev
cd $workingdir/master/
mkdir s
ln -s /s/etc etc
ln -s /s/home home
ln -s /s/root root
ln -s /s/usr-local usr/local
ln -s /s/usr-X11R6 usr/X11R6
ln -s /s/usr-obj usr/obj
ln -s /s/usr-distfiles usr/distfiles
ln -s /s/tmp tmp
ln -s /s/var var

Create an Jail

Die Example below creates an IPv6-only Jail.

# define variables V 1.01
basejail="/usr/jails/basejail"
workdir="/usr/jails/pfs"
domain="calmar.me"
jail="development"
ipv4_adress="127.0.0.10"
ipv6_adress="2001:a60:f112:1:f:ff:fff:30"
D="/usr/jails/$jail"

#####################################
######## Create the Jail ############
#####################################

####  create directories ####
mkdir -p $D
mkdir -p $workdir/$jail 
#### Insert Mount Points ####
echo "$basejail/master      $D                 null    ro      0       0" >> /etc/fstab
echo "$workdir/$jail        $D/s               null    rw      0       0" >> /etc/fstab
mount -a
##### Copy Skelleton to Target Dir ####
cpdup $basejail/skel $workdir/$jail
#### Generate /etc/rc.conf entries ####
echo "# Jail ${jail}" >> /etc/rc.conf
echo "jail_${jail}_rootdir=\"$D\""     >> /etc/rc.conf
echo "jail_${jail}_hostname=\"${jail}.$domain\"" >> /etc/rc.conf
echo "jail_${jail}_ip=\"$ipv4_adress,$ipv6_adress\""  >> /etc/rc.conf
echo "jail_${jail}_devfs_enable=\"YES\""  >> /etc/rc.conf
####  Giving the Jail an Nameserve Entry #### 
echo "nameserver      2001:4860:4860::8844" > $workdir/$jail/etc/resolv.conf
#### Copying my ssh-Keys to the Jails and change listen IP of SSH ####
mkdir -p  $workdir/$jail/root/.ssh
cp -a /root/.ssh/authorized_keys  $workdir/$jail/root/.ssh/
echo "ListenAddress $ipv6_adress" >> $workdir/$jail/etc/ssh/sshd_config
####  Generating Jails /etc/rc.conf ####
touch $workdir/$jail/etc/rc.conf
echo "network_interfaces=\"\"" >> $workdir/$jail/etc/rc.conf
echo "hostname=\"${jail}.$domain\"" >> $workdir/$jail/etc/rc.conf
echo "sshd_enable=\"YES\"" >> $workdir/$jail/etc/rc.conf

Transferring a Jails to it's own PFS

In this example we transfer an existing Jail called tiger to a new PFS (/usr/local/jails/pfs/tiger)

Stop the Jail and unmount it

# /etc/rc.d/jail stop tiger
# umount /usr/jails/tiger/s
# umount /usr/jails/tiger/
# cd /usr/jails/pfs/
# mv tiger tiger2

Create a new PFS

# hammer pfs-master /pfs/tiger

Add the folowing Line to /etc/fstab before the Jails Entries

/pfs/tiger      /usr/jails/pfs/tiger    null    rw              0       0

Example:

.....
pfs/var.crash  /var/crash              null    rw              0       0
/pfs/var.tmp    /var/tmp                null    rw              0       0
proc                    /proc           procfs  rw              0       0
...
/pfs/tiger      /usr/jails/pfs/tiger    null    rw              0       0
....
/usr/jails/basejail/master      /usr/jails/tiger                 null    ro      0       0
/usr/jails/pfs/tiger        /usr/jails/tiger/s               null    rw      0       0

and mount it

# mount /usr/jails/pfs/tiger

Transfer the Data

# cpdup tiger2 tiger
# mount -a
# chflags -R noschg tiger2/
# rm -fr  tiger2/
# /etc/rc.d/jail start tiger

Updating

When updating you create an additional new master, which will be mounted read-only in every jail.

Update Host

# cd /usr/src/
# make -j4 buildworld
# make -j4 buildkernel
# make installkernel
# make installworld
# make upgrade

Create new Master (Master2)

basejail="/usr/jails/basejail"
mkdir  $basejail/master2
cd /usr/src
make installworld DESTDIR=$basejail/master2
cd $basejail/master2/
cpdup /usr/src usr/src
mkdir s
chflags -R 0 var
rm -R etc var root usr/local tmp usr/obj
ln -s /s/etc etc
ln -s /s/home home
ln -s /s/root root
ln -s /s/usr-local usr/local
ln -s /s/usr-X11R6 usr/X11R6
ln -s /s/usr-obj usr/obj
ln -s /s/usr-distfiles usr/distfiles
ln -s /s/tmp tmp
ln -s /s/var var

Stop Jails

# /etc/rc.d/jail stop tiger

Unmount Jails

# umount /usr/jails/tiger/s
# umount /usr/jails/tiger

Change Master

We change the master und Upgrade its Dports

basejail="/usr/jails/basejail"
cd $basejail/
mv master master_old
mv master2  master 
mv -v master_old/usr/dports master/usr
cd /usr/dports/
git pull
rsync -av --delete /usr/dports/ $basejail/master/usr/dports/

Replication of Jails

You can stream an read-only copy of your Master-PFS to any Dragonfly BSD reachable by ssh. The slave is Read-only, but in case of emergency it can be upgraded to become the new master.

Example:

Type

Name

PFS

Master

apollo.example.com

/pfs/mail

Slave

zeus.example.com

/pfs/mail

Get Shared-UUID of Master-PFS

To enable replication wie have to get the Shared-UUID of the Master.

apollo# hammer pfs-status  /pfs/mail/
/pfs/mail/        PFS #9 {
    ......
    shared-uuid=f9e7cc0d-eb59-10e3-a5b5-01e6e7cefc12
    ........
}    

Create Slave-PFS

Now we create an slave-PFS on zeus.example.com.

zeus# hammer pfs-slave /pfs/mail shared-uuid=f9e7cc0d-eb59-10e3-a5b5-01e6e7cefc12

Create SSH-Keys

In order to connect to apollo, we need to generate SSH-Keys for passwordless Authentificaton.

zeus# cd /root.ssh/private/ 
zeus# ssh-keygen -b 4096 -f apollo_rsa  -t rsa  # no passphrase
zeus# mv apollo_rsa.pub /root.ssh/public/

Generate a file called /root/.ssh/config on zeus.

Host apollo.example.com
    HostName apollo.example.com
    Port 22
    User root
    IdentityFile ~/.ssh/private/apollo_rsa

Now copy the file /root.ssh/public/apollo_rsa.pub to apollo.example.com, and add it to the authorized_keys.

apollo# cat apollo_rsa.pub >> /root/.ssh/authorized_keys

Now try to connect from zeus to apollo.

zeus# ssh apollo.example.com 

Connect the Slave-PFS to it's master

Add the folowing Line in /etc/crontab on zeus to enable constant mirrowing after every reboot.

@reboot root    /sbin/hammer mirror-stream root@apollo.example.com:/pfs/mail  /pfs/mail 

Dragonfly BSD/Advanced Jails (last edited 2015-07-18 06:46:36 by jackhammer)