Jason Friedland

Installing Mediatomb on Solaris 11 Express

2010-12-20T15:22:00.007+10:00

My home NAS is now running Solaris 11 for the 'express' purpose (sorry) of being able to use ZFS, and all the light, joy and happiness it brings to the world of storage. I'm using three 1TB SATA disks in a RAIDZ configuration, and using ZFS's built-in filesystem compression, CIFS and NFS sharing capabilities. Since I use a PS3 as my media center front-end I need a DLNA/UPnP media server and having used Mediatomb previously on Linux, that's what I'll walk through installing from source here today.

Before starting, make sure you have GNU C compiler (gcc) and make installed. This is easily achieved using the new package management tools in Solaris 11. Try man pkg to get started.

For Mediatomb to correctly identify certain files' MIME types (like image/jpeg, for example) you need to build the GPL'd version of the file utility for a library called libmagic.

tar zxvf file-5.04.tar.gz
cd file-5.04
./configure --prefix=/usr/local
make
make install

Check for libmagic library and headers:

ls /usr/local/include
ls /usr/local/lib

Next, download the source for Mediatomb and extract it.

tar zxvf mediatomb-0.12.1.tar.gz
cd mediatomb-0.12.1

The source code contains some apparently outdated prelink logic which breaks the build on newer versions of the OS, so we need to comment it out. You can either edit src/main.cc by hand or just use the following patch - save it to main.cc.patch0.

*** src/main.cc.orig    2010-12-20 13:13:11.080796210 +1000
--- src/main.cc 2010-12-20 13:13:30.478542756 +1000
***************
*** 141,146 ****
--- 141,147 ----

      Ref<Array<StringBase> > addFile(new Array<StringBase>());

+ /*
  #ifdef SOLARIS
      String ld_preload;
      char *preload = getenv("LD_PRELOAD");
***************
*** 155,160 ****
--- 156,162 ----
          exit(EXIT_FAILURE);
      }
  #endif
+ */

  #ifdef HAVE_GETOPT_LONG
      while (1)

And apply the patch:

patch src/main.cc < main.cc.patch0

Now configure, build and install:

./configure --enable-libmagic --with-magic-h=/usr/local/include --with-magic-libs=/usr/local/lib
make
make install

At this stage, you could add your config files and fire up the daemon, however I'm going to create service manifests so we can manage startup and shutdown of Mediatomb via the Solaris Service Management Framework (SMF). Save this file as /lib/svc/method/svc-mediatomb.

#!/bin/sh
. /etc/mediatomb.conf
LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/mediatomb -d \
    -u $MT_USER \
    -g $MT_GROUP \
    -P $MT_PIDFILE \
    -l $MT_LOGFILE \
    -m $MT_HOME \
    -f $MT_CFGDIR \
    -p $MT_PORT \
    -e $MT_INTERFACE \
    $MT_OPTIONS

Create /etc/mediatomb.conf, setting appropriate values for variables above. Something like the following would suffice:

MT_INTERFACE="rge0"
MT_OPTIONS=""
MT_PORT="50500"
MT_USER="media"
MT_GROUP="media"
MT_PIDFILE="/var/run/mediatomb.pid"
MT_LOGFILE="/var/log/mediatomb"
MT_HOME="/etc"
MT_CFGDIR="mediatomb"

Now for the service manifest - create this file as /var/svc/manifest/application/mediatomb.xml.

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type="manifest" name="mediatomb">
    <service name="application/mediatomb" type="service" version="1">
        <create_default_instance enabled="false"/>
        <single_instance/>
        <dependency name="network" grouping="require_all" restart_on="error" type="service">
            <service_fmri value="svc:/milestone/network:default"/>
        </dependency>
        <dependency name="filesystem" grouping="require_all" restart_on="error" type="service">
            <service_fmri value="svc:/system/filesystem/local"/>
        </dependency>
        <exec_method type="method" name="start" exec="/lib/svc/method/svc-mediatomb" timeout_seconds="60">
        </exec_method>
        <exec_method type="method" name="stop" exec=":kill" timeout_seconds="5">
        </exec_method>
        <property_group name="startd" type="framework">
            <propval name="ignore_error" type="astring" value="core,signal"/>
        </property_group>
        <stability value="Evolving"/>
        <template>
            <common_name>
                <loctext xml:lang="C">
                    UPnP Media Server
                </loctext>
            </common_name>
            <documentation>
                <manpage title="mediatomb" section="1" manpath="/opt/local/share/man"/>
                <doc_link name="mediatomb.cc" uri="http://mediatomb.cc"/>
            </documentation>
        </template>
    </service>
</service_bundle>

Then validate, import and run the service:

svccfg validate /var/svc/manifest/application/mediatomb.xml
svccfg import /var/svc/manifest/application/mediatomb.xml
svcs -a mediatomb
svcadm enable mediatomb

If all went well you should now be able to connect to the Mediatomb web interface on port 50500.

Scan for new devices in Solaris

2010-11-06T11:53:00.002+10:00

This is how to tell Solaris to look for new devices when booting - append a '-r' to the kernel boot arguments:

Select the Solaris entry GRUB menu that you want to boot, to edit, enter e
Select the "kernel /platform" line, to edit that again enter e
Add to the end of the 'kernel' line a space followed by -r
Press enter key to accept the change
Press b to boot

Via: http://blogs.sun.com/harcey/entry/solaris_x86_vmware_adding_a

Setting up a highly available NFS cluster

2010-08-04T20:11:00.010+10:00

The following is a proof-of-concept HA NFS cluster running on RHEL 5, using DRBD for block-level replication and heartbeat for failover.

In this example, I'll be setting up 2 nodes: nfs01 and nfs02. Ensure that ALL commands and setup is run against both nodes EXCEPT where stated otherwise.

Following is the partition table for a test install - the metadata partition /dev/sda7 and the data partition /dev/sda8 are unformatted and unmounted.

[root@nfs01 ~]# fdisk -l /dev/sda

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         274     2096482+  82  Linux swap / Solaris
/dev/sda3             275         535     2096482+  83  Linux
/dev/sda4             536        2610    16667437+   5  Extended
/dev/sda5             536         796     2096451   83  Linux
/dev/sda6             797        1057     2096451   83  Linux
/dev/sda7            1058        1074      136521   83  Linux
/dev/sda8            1075        2610    12337888+  83  Linux

Instructions for building the DRBD RPMs, including the kernel module can be found on the DRBD website.

The kernel in use is the latest update for RHEL 5.5 at the time of writing, version kernel-2.6.18-194.3.1.el5.x86_64.rpm - DRBD must be compiled against your specific version of kernel-headers.

Ensure at least the following RPMs are installed.

rpm-build-4.4.2.3-18.el5    
elfutils-0.137-3.el5        
elfutils-libs-0.137-3.el5   
flex-2.5.4a-41.fc6          
glibc-devel-2.5-49.el5_5.2  
glibc-headers-2.5-49.el5_5.2
kernel-2.6.18-194.3.1.el5   
kernel-devel-2.6.18-194.3.1.el5  
kernel-headers-2.6.18-194.3.1.el5
gcc-c++-4.1.2-48.el5
gcc-4.1.2-48.el5
make-3.81-3.el5

Use DRBD's built-in functionality to build both the userland and kernel module RPMs.

[root@nfs01 build]# tar zxf drbd-8.3.8.tar.gz 
[root@nfs01 build]# cd drbd-8.3.8
[root@nfs01 drbd-8.3.8]# ./configure --with-km=/usr/src/kernels/2.6.18-194.3.1.el5-x86_64/
[root@nfs01 drbd-8.3.8]# make km-rpm rpm
...
You now have:
/usr/src/redhat/RPMS/x86_64/drbd-bash-completion-8.3.8-1.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-pacemaker-8.3.8-1.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-km-2.6.18_194.3.1.el5-8.3.8-12.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-udev-8.3.8-1.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-utils-8.3.8-1.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-8.3.8-1.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-xen-8.3.8-1.x86_64.rpm
/usr/src/redhat/RPMS/x86_64/drbd-heartbeat-8.3.8-1.x86_64.rpm

Install DRBD and modprobe the new module - a reboot is given in the examples here to check the module loads on a system restart.

[root@nfs01 drbd-8.3.8]# rpm -ivh /usr/src/redhat/RPMS/x86_64/drbd-*
[root@nfs01 drbd-8.3.8]# modprobe drbd # or...
[root@nfs01 drbd-8.3.8]# reboot
...
[root@nfs01 drbd-8.3.8]# lsmod | grep drbd
drbd                  277784  0

Install the sample /etc/drbd.conf file.

# A comprehensively commented example of this file exists at:
# /usr/share/doc/drbd-utils-8.3.8/drbd.conf.example

resource r0 {

    protocol C;
    
    # Parse error here:
    # incon-degr-cmd "halt -f";
    
    startup {
        degr-wfc-timeout 120;    # 2 minutes.
    }

    disk {
        on-io-error   detach;
    }

    net {
    }

    syncer {
        rate 10M;
        # group 1;
        al-extents 257;
    }

    on nfs01 {                   
        device    /dev/drbd0;          
        disk      /dev/sda8;           # NFS data partition
        meta-disk /dev/sda7[0];        # 1024MB partition for DRBD metadata
        address   192.168.1.101:7788;
    }

    on nfs02 {                   
       device    /dev/drbd0;           
       disk      /dev/sda8;            # NFS data partition
       meta-disk /dev/sda7[0];         # 1024MB partition for DRBD metadata
       address   192.168.1.102:7788;  
    }

}

Install the sample /etc/exports file, and ensure the NFS service is stopped and is not set to launch on system startup.

/export/ 192.168.1.0/255.255.255.0(rw,no_root_squash,no_all_squash,sync)

[root@nfs01 drbd-8.3.8]# /etc/init.d/nfs stop
[root@nfs01 drbd-8.3.8]# chkconfig nfs off

Configure the NFS statd daemon to use the hostname for the floating IP.

[root@nfs01 ~]# echo >> /etc/sysconfig/nfs 'STATDARG="-n nfs.localdomain"'

Ensure there's no existing filesystem on your metadata partition, /dev/sda7 in this case.

[root@nfs01 ~]# dd if=/dev/zero of=/dev/sda7 bs=1M count=256
[root@nfs01 ~]# drbdadm create-md r0
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.

[root@nfs01 ~]# mkfs.ext3 /dev/sda8
[root@nfs01 ~]# drbdadm up all

[root@nfs01 ~]# cat /proc/drbd 
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by root@nfs01, 2010-06-18 16:52:07
 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:12217404

Do this only on node 1:

[root@nfs01 ~]# drbdadm adjust all 
[root@nfs01 ~]# drbdadm -- --force primary r0

[root@nfs01 ~]# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by root@nfs01, 2010-06-18 16:52:07
 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r----
    ns:537600 nr:0 dw:0 dr:537600 al:0 bm:32 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:11679804
 [>....................] sync'ed:  4.5% (11404/11928)M delay_probe: 102
 finish: 0:15:41 speed: 12,272 (10,336) K/sec

# Completed now:
[root@nfs01 ~]# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by root@nfs01, 2010-06-18 16:52:07
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----
    ns:12217401 nr:0 dw:0 dr:12217401 al:0 bm:746 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

Do this only on node 1:

[root@nfs01 ~]# mount /dev/drbd0 /export
[root@nfs01 ~]# mv /var/lib/nfs /export/
[root@nfs01 ~]# cd /var/lib/
[root@nfs01 ~]# mv nfs nfs.old
[root@nfs01 ~]# ln -s /export/nfs

Do this only on node 2:

[root@nfs02 ~]# cd /var/lib/
[root@nfs02 lib]# mv nfs nfs.old
[root@nfs02 lib]# ln -s /export/nfs

Ensure the EPEL repository is enabled and install heartbeat.

[root@nfs01 ~]# yum install heartbeat.x86_64

Edit the /etc/ha.d/ha.cf file.

logfacility local0
keepalive 2
deadtime 10
bcast eth0
node nfs01.localdomain nfs02.localdomain
auto_failback on

Edit /etc/ha.d/haresources - the IP address is the floating one.

nfs01.localdomain IPaddr::192.168.1.103/24/eth0 \
    drbddisk::r0 Filesystem::/dev/drbd0::/export::ext3 nfs

Add a secret to /etc/ha.d/authkeys file.

auth 3
3 md5 pa55word

chmod 600 /etc/ha.d/authkeys

Make sure the drbd and heartbeat servces are set to start on boot, and start them up:

/etc/init.d/drbd start
/etc/init.d/heartbeat start

chkconfig drbd on
chkconfig heartbeat on

That's a very brief and tersely documented example - you can now test failover by pulling a network cable :-)

Automount filesystems by label

2010-04-12T12:30:00.002+10:00

Here's a handy (and from what's in the autofs the man pages, undocumented) trick for auto-mounting ext3 filesystems by label. Say I had a removable USB drive formatted as ext3 with the label WD-250 I could add this line to my auto.misc file:

WD-250 -fstype=ext3 :LABEL=WD-250

The filesystem UUID also seems to work using this method.

Init script for Redmine

2010-03-30T14:24:00.005+10:00

With my new role at work as an Infrastructure Engineer in our Projects team, I've recently started using Redmine, a flexible project management web application written using Ruby on Rails framework, to keep myself and my projects organised.

Redmine is distributed as source and doesn't include any management scripts other than the Ruby ones used to manage Rails. In order to start and stop it automatically on boot on my Fedora 12 workstation, I'm using the following init script.

#!/bin/bash
#
# chkconfig: - 16 84
# description: Start up Redmine
#
# processname: redmine
# config: /etc/sysconfig/redmine

# source function library
. /etc/rc.d/init.d/functions

# Get network config
. /etc/sysconfig/network

[ "${NETWORKING}" = "no" ] && exit 0

# Defaults
DAEMON_HOME=/usr/local/redmine
DAEMON_ARGS="-e production -d"

# Daemon
NAME=redmine
RUBY=$(which ruby)

APP_PIDFILE=$DAEMON_HOME/tmp/pids/server.pid
DAEMON_PIDFILE=/var/run/$NAME.pid
DAEMON_LOCKFILE=/var/lock/subsys/$NAME

start() {
    echo -n $"Starting ${NAME}: "
        
    cd $DAEMON_HOME
    $RUBY script/server webrick $DAEMON_ARGS

    sleep 2

    status -p $APP_PIDFILE &> /dev/null && echo_success || echo_failure
    RETVAL=$?

    if [ $RETVAL -eq 0 ]; then
        touch $DAEMON_LOCKFILE
        cat $APP_PIDFILE > $DAEMON_PIDFILE
    fi

    echo
}

stop() {
    echo -n $"Shutting down ${NAME}: "
    
    killproc -p $DAEMON_PIDFILE
    RETVAL=$?

    [ $RETVAL -eq 0 ] && /bin/rm -f $DAEMON_LOCKFILE $DAEMON_PIDFILE

    echo
}

case "$1" in
    start)
        start
    ;;
    stop)
        stop
    ;;
    restart)
        stop
        start
    ;;
    status)
        status -p $DAEMON_PIDFILE $NAME
    ;;

    *)
        echo "Usage: $SCRIPTNAME {start|stop|restart|status}" >&2
        exit 3
    ;;
esac

Once you have it in place in /etc/init.d, run the following commands to install it, enable Redmine startup on boot and fire it up.

chmod +x /etc/init.d/redmine
chkconfig --add redmine
chkconfig redmine on
/etc/init.d/redmine start

Automatically update the Terminal window title in Mac OS X

2010-03-21T09:01:00.005+10:00

It seems Mac OS X doesn't provide the nice Bash functionality found in RHEL/CentOS/Fedora Linux that updates the terminal title after each command. For example, using Linux, if I do the following:

cd ~/Developer

the window title is updated to:

jason@rocksteady:~/Developer

While the Terminal app in OS X does allow you to customize the title to some extent by adding the TTY name, name of running process etc. I prefer the 'user@host:pwd' format.

After digging around in /etc/bashrc on CentOS 5, I found you can set the PROMPT_COMMAND Bash variable and have your title updated after every command.

Add the following to your .bash_profile in Mac OS X to get the same behaviour:

PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME%%.*}:${PWD/#$HOME/~}"; echo -ne "\007"'

Dstat plugin for squid proxy

2010-02-27T10:07:00.007+10:00

I've been using Dag Wieers' excellent system utility Dstat for a little while now and have recently starting exploring the possibility of extending it with my own plugins. Dstat is written in Python and so are its plugins, so that made the possibility all the more likely for me.

In case you don't know what Dstat is:

Dstat is a versatile replacement for vmstat, iostat, netstat, nfsstat and ifstat. Dstat overcomes some of their limitations and adds some extra features, more counters and flexibility. Dstat is handy for monitoring systems during performance tuning tests, benchmarks or troubleshooting.

Dstat allows you to view all of your system resources instantly, you can eg. compare disk usage in combination with interrupts from your IDE controller, or compare the network bandwidth numbers directly with the disk throughput (in the same interval).

Check the Dstat homepage for more details.

I was mostly interested in obtaining some useful metrics on a squid proxy and plugging them into Dstat. Using squidclient, you can get all sorts of interesting info using a command line interface and that's what I'm using to get these stats.

You'll need to ensure you have manager access enabled in your squid.conf like so:

http_access allow manager localhost  
http_access deny manager

The metrics I've chosen to extract are just a proof-of-concept really - I'll eventually tweak them to be somewhat more useful, and this O'Reilly article Eleven Metrics to Monitor for a Happy and Healthy Squid has always proven to be a good reference.

You can check out the dstat_squid.py plugin from my dstat-plugins GitHub repository, or copy it below.

# Dstat plugin for measuring various squid statistics.
# Author: Jason Friedland <thesuperjason@gmail.com>
#
# This plugin has been tested with:
# - Dstat 0.6.7
# - CentOS release 5.4 (Final)
# - Python 2.4.3
# - Squid 2.6 and 2.7
 
global squidclient_options
squidclient_options = os.getenv('DSTAT_SQUID_OPTS') # -p 8080
 
class dstat_squid(dstat):
    def __init__(self):
        self.name = 'squid status'
        self.format = ('s', 8, 100)
        self.vars = ('Number of file desc currently in use',
            'CPU Usage, 5 minute avg',
            'Total accounted',
            'Number of clients accessing cache',
            'Mean Object Size')
        self.nick = ('fdesc',
            'cpu5',
            'mem',
            'clients',
            'objsize')
        self.init(self.vars, 1)
        
    def check(self):
        if not os.access('/usr/sbin/squidclient', os.X_OK):
            raise Exception, 'Needs squidclient binary'
        return True
 
    def extract(self):
        try:
            for line in os.popen('/usr/sbin/squidclient %s mgr:info' 
                    % (squidclient_options), 'r').readlines():
                l = line.split(':')
                for item in self.vars:
                    if l[0].strip() == item:
                        self.val[item] = l[1].strip()
                        break
        except Exception, e:
            if op.debug > 1: print '%s: exception' (self.filename, e)
            for name in self.vars: self.val[name] = -1

Init script for transmission used in official Fedora 12 package

2010-02-15T14:28:00.006+10:00

Thanks to an RFE filed in Red Hat Bugzilla, my transmission-daemon init script which was originally written for RHEL 5 has made its way into the official Fedora 12 transmission-daemon-1.83-2.fc12.i686 package.

The header of the script shows:

# This is a modified version of the original init script at 
# http://jasonfriedland.blogspot.com/2009/07/init-script-for-transmission-daemon-on.html

Python script to time HTTP requests via a proxy server

2010-02-10T17:49:00.005+10:00

Like the title says - I just want to time HTTP requests via different proxy servers to compare response times.

I hadn't really used Python decorators before, but once you get your head around them they're an extremely useful design pattern.

#!/usr/bin/env python

# Script to time HTTP requets via a proxy server. 

# Sample output:
# proxy01 [ http://www.news.com.au ] took 222.687 ms
# proxy01 [ http://www.google.com ] took 134.649 ms
# proxy02 [ http://www.news.com.au ] took 1387.021 ms
# proxy02 [ http://www.google.com ] took 4197.468 ms

import urllib, time

proxies = ('proxy01', 'proxy02')
urls = ('http://www.news.com.au', 'http://www.google.com')

# Decorator for timing functions
def timer(f):
    def wrapper(*arg):
        t_start = time.time()
        r = f(*arg)
        t_finish = time.time()
        print '%s [ %s ] took %0.3f ms' % (proxy, url, (t_finish-t_start)*1000.0)
        return r
    return wrapper

@timer
def testProxy(proxy, url):
    proxy = 'http://username:password@%s.localdomain:8080' % (proxy)
    proxy_list = {'http': proxy}
    try:
        f = urllib.urlopen(url, proxies=proxy_list)
        html = f.read()
    except URLError, e:
        print e.reason

# Main
if __name__ == "__main__":
    for proxy in proxies:
        for url in urls:
            testProxy(proxy, url)

Idle CPU usage and kernel options in RHEL 5.4 VMs

2010-02-07T17:51:00.007+10:00

I was recently having (another) read through the VMware documentation on best practices for timekeeping in Linux guest virtual machines, and while the article claims that using the "divider" option in RHEL 5.4 kernels running as a VM is no longer required for accurate timekeeping, it does (quite significantly) reduce CPU usage for idle machines, as this graph clearly illustrates.At point 1. the ESX host has no running VMs, point 2. has an idle vanilla 2.6.18-164.11.1 kernel running on CentOS 5.4 and point 3. is the same machine with "divider=10" passed to the kernel at boot.

Converting FLAC to MP3

2010-01-12T19:15:00.004+10:00

It's been a loooong time since my last post (3 months!) which is testament to how busy I've been at work in the crazy lead-up to the New Year. It's not going to let up anytime soon, but I do endeavour to post more.

Here's a nifty Bash script to convert a directory of FLAC files to MP3 and ID3 tag them while we're at it.

Feel free to contribute!

Simple bash HandBrakeCLI wrapper

2009-09-07T21:24:00.003+10:00

Here's a simple bash shell wrapper for using HandBrake to (backup, and) encode DVD video to MP4 (this is somewhat equivalent to the 'Normal' preset).

#!/bin/bash
# Usage: handbrake.sh "Some Title"

TITLE=$1
MOVIE_DIR=/var/media/Movies

if [ -n "$TITLE" ]; then
    mkdir -p $MOVIE_DIR/"$TITLE"
    /usr/bin/HandBrakeCLI -i /dev/cdrom -o $MOVIE_DIR/"$TITLE"/"$TITLE".mp4 \
        -e x264 -b 1500 -a 1 -E faac -B 160 -L -R Auto -6 dpl2 \
        -f mp4 -m -2 -T -x ref=2:bframes=2:me=umh
    exit 0
else
    exit 1
fi

Configuring MediaTomb for the PS3

2009-09-07T18:24:00.015+10:00

Having just gone out and splurged on a shiny new PS3 Slim, I've had to make some changes on my file server to cater for what I previously thought was a severe limitation of the PS3 - lack of SMB support for connecting to network shares.

I've now replaced my previous media center - an XBox running the excellent XBMC - with the PS3, however the only remote shares supported by the full-HD, H.264-capable PS3 are UPnP/DLNA media servers.

As mentioned, I initially considered this a limitation but you really do get so much more with a UPnP server, such as transcoding (converting media formats on-the-fly and then streaming to the client) and using media metadata (such as ID3 tags in MP3 files) for searching and browsing.

Enter MediaTomb, "an open source (GPL) UPnP MediaServer with a nice web user interface, it allows you to stream your digital media through your home network and listen to/watch it on a variety of UPnP compatible devices".

Actually the web interface isn't that nice but it is functional, and you only have to visit it once.

My current file server is running Fedora 11 and you'll find MediaTomb in the Everything repo, but you won't have any trouble finding or installing the software on RHEL/CentOS (via EPEL) or Ubuntu.

There are plenty of resources to get you up and running quickly so in this post I wanted to share the specifics of my setup, especially:

modifying the import script to preserve a directory heirarchy
using transcoding to stream video within RAR archives, Flash video and other formats the PS3 doesn't understand

One of the biggest annoyances is, because your video files will most likely lack any useful metadata, they'll all just get lumped into one big folder, even if you have them neatly organised into seasons, categories or video-type on your fileserver.

Photos aren't quite so bad as they at least contain some metadata and MediaTomb will organise them by date by default.

This however is easily overcome by modifying the import.js script (in /usr/share/mediatomb/js on RHEL/CentOS/Fedora systems).

I use the following JavaScript (adapted from the workaround posted here) which replaces the addVideo() and addPhoto() functions:

function addMedia(obj, media_type) // media_type = Photos | Video
{
    // Change this to suit your folder naming scheme
    var parent_folder = media_type;

    var loc = obj.location.split('/');
    var mediaFound = false;
    var chain = new Array(media_type);

    for (i=0; i<loc.length-1; i++) {
        if (mediaFound) chain.push(loc[i]);
        if (loc[i] == parent_folder) mediaFound = true;
    }

    str = loc[loc.length-1];
    obj.title = str.substring(0,str.lastIndexOf('.'));
    addCdsObject(obj, createContainerChain(chain));
}

You'll need to ensure that you change the function calls in the main section of the script like so:

if (mime == 'video')
{
    addMedia(obj, 'Video');
}

if (mime == 'image')
{
    addMedia(obj, 'Photos');
}

And that your top level folders which contain your Photos and Video are named exactly "Photos" and "Video" respectively or adjust the parent_folder variable accordingly.

You should now see your files neatly organised just as they appear on the filesystem of your media server.

I'll follow up on this post in a day or so with the configuration and software required to get transcoding of RARs and FLVs working.

More fun with Linux RAID and LVM

2009-08-03T12:49:00.006+10:00

In an earlier post, I performed a test RAID5 + LVM set up using files instead of disks. Now I'm going to fail a 'disk', replace it in the array, and then expand the LVM logical volume to include a second RAID5 array.

In this example, I'll replace out /dev/loop2 as the faulty member. So, mark it as faulty, and remove it from the array. I'll also delete the loopback device.

mdadm /dev/md0 --fail /dev/loop2
mdadm /dev/md0 --remove /dev/loop2
rm /var/tmp/hdc.img
losetup -d /dev/loop2

Next, create a new disk, and add it to the array:

dd if=/dev/zero of=/var/tmp/hdc.img bs=1024 count=1048576
losetup /dev/loop2 /var/tmp/hdc.img 
fdisk /dev/loop2
partprobe
mdadm /dev/md0 --add /dev/loop2

And the rebuild will begin. Check its progress with:

mdadm --detail /dev/md0

Now follow the steps in my previous post to build a second RAID5 array and call it md1; we'll be using using it to extend our original LVM volume group.

[root@canard ~]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "vg_media" using metadata type lvm2

pvcreate /dev/md1
vgextend vg_media /dev/md1

Find the total number of extents you have available, resize the LV to that new number and grow the EXT3 filesystem for the free space on your device:

[root@canard ~]# vgdisplay vg_media | grep "Total PE"
  Total PE              1010

So we'll extend to 1010 physical extents.

lvextend -l 1010 /dev/mapper/vg_media-lv_media 
resize2fs /dev/mapper/vg_media-lv_media

If all went according to plan, you should now have a ~4GB LV with no remounting required.

[root@canard ~]# lvdisplay /dev/vg_media/lv_media
  --- Logical volume ---
  LV Name                /dev/vg_media/lv_media
  VG Name                vg_media
  LV UUID                iS01Vt-Hyo7-GIiA-Sxlu-rc5J-bX5g-Trd9Ln
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                3.95 GB
  Current LE             1010
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

Simple MySQL slave monitor

2009-08-01T11:10:00.006+10:00

Here's a simple Bash script to monitor a MySQL slave, checking that a) MySQL is running, b) the slave process is running, and c) that it's not too far behind the master.

We have a database slave that gets taken down every night for backups and this script checks that it came back up OK.

It's designed to be executed via cron and it runs on our production Solaris servers from around the time that the backups should finish. With a couple of minor mods (path to mysql, for instance) it runs just fine on Linux too.

#!/bin/bash

# A simple MySQL process/replication status monitor - checks for
# a Seconds_Behind_Master threshold and that both MySQL and the 
# slave process are running.

# Errors
ERR_SLAVE_STOPPED=1
ERR_SLAVE_OVER_THRESHOLD=2
ERR_MYSQL_STOPPED=3

ERR=0

# Backup vars - if a backup's running we don't bother alerting
BACKUP_DIR=/u07/mysql/exports

# Threshold settings
SBM_THRESHOLD=150 # Seconds behind master threshold 
PROCS_THRESHOLD=2 # Minimum number of MySQL processes

# Values
NUM_SBM=`/opt/coolstack/mysql/bin/mysql -e "SHOW SLAVE STATUS \G" \
    | grep Seconds_Behind_Master | cut -d ":" -f 2 | sed -e 's/\ //g'`
NUM_PROCS=`pgrep mysql | wc -l | sed -e 's/\ //g'`

# Tests
if [ "$NUM_PROCS" -lt $PROCS_THRESHOLD ]; then
    # Check backup is running
    if [ ! -f $BACKUP_DIR/backingup ]; then
        MSG="There are less than $PROCS_THRESHOLD MySQL processes running"
        ERR=$ERR_MYSQL_STOPPED
    fi
fi

if [ "$NUM_SBM" = "NULL" ]; then
    MSG="The slave replication process has faulted"
    ERR=$ERR_SLAVE_STOPPED
fi

if [ "$NUM_SBM" -gt $SBM_THRESHOLD ]; then
    MSG="The slave replication process is $NUM_SBM seconds behind the master"
    ERR=$ERR_SLAVE_OVER_THRESHOLD
fi

# Send email if errors
if [ $ERR -gt 0 ]; then
    echo $MSG | mailx -s "[CRON] MySQL Errors on `hostname`" you@example.com
    exit $ERR
fi

exit 0

Obtain your switch port ID on a Cisco device using CDP

2009-07-30T15:39:00.012+10:00

Here's a handy way to use tcpdump to capture CDP packets from your Cisco device, which will allow you to obtain the hostname, model and IOS version as well as the switch port you're patched into.

Quite useful for when you need to confirm the port but don't fancy a trip to the datacenter.

tcpdump -c1 -s0 -XX 'ether[20:2]=0x2000'

The tcpdump flags are:

-c count, capture 1 packet
-s snaglen, 0 don't use an offset
-XX hex and ASCII packet data, including header
ether[20:2]=0x2000 bytes 20 to 21 of the ethernet header, value of 2000 for CDP

Sample output follows showing that we're patched into FastEthernet2/0/11 on a 24 port Cisco Catalyst 3750 running IOS 12.2:

[root@canard ~]# tcpdump -c1 -s0 -XX 'ether[20:2]=0x2000'
13:01:38.634910 CDPv2, ttl: 180s, Device-ID 'sw01.klime', length 409
... snip ...
 0x0040:  0005 00c5 4369 7363 6f20 494f 5320 536f  ....Cisco.IOS.So
 0x0050:  6674 7761 7265 2c20 4333 3735 3020 536f  ftware,.C3750.So
 0x0060:  6674 7761 7265 2028 4333 3735 302d 4144  ftware.(C3750-AD
 0x0070:  5649 5053 4552 5649 4345 534b 392d 4d29  VIPSERVICESK9-M)
 0x0080:  2c20 5665 7273 696f 6e20 3132 2e32 2834  ,.Version.12.2(4
 0x0090:  3429 5345 322c 2052 454c 4541 5345 2053  4)SE2,.RELEASE.S
 0x00a0:  4f46 5457 4152 4520 2866 6332 290a 436f  OFTWARE.(fc2).Co
 0x00b0:  7079 7269 6768 7420 2863 2920 3139 3836  pyright.(c).1986
 0x00c0:  2d32 3030 3820 6279 2043 6973 636f 2053  -2008.by.Cisco.S
 0x00d0:  7973 7465 6d73 2c20 496e 632e 0a43 6f6d  ystems,.Inc..Com
 0x00e0:  7069 6c65 6420 5468 7520 3031 2d4d 6179  piled.Thu.01-May
 0x00f0:  2d30 3820 3135 3a34 3220 6279 2061 6e74  -08.15:42.by.ant
 0x0100:  6f6e 696e 6f00 0600 1663 6973 636f 2057  onino....cisco.W
 0x0110:  532d 4333 3735 302d 3234 5000 0200 1100  S-C3750-24P.....
 0x0120:  0000 0101 01cc 0004 c0a8 05fc 0003 0016  ................
 0x0130:  4661 7374 4574 6865 726e 6574 322f 302f  FastEthernet2/0/
 0x0140:  3131 0004 0008 0000 0029 0008 0024 0000  11.......)...$..
 0x0150:  0c01 1200 0000 00ff ffff ff01 0221 ff00  .............!..
... snip ...

My Linux tcpdump version was 'ported' from the equivalent Solaris snoop command.

Matching and replacing substrings using Bash

2009-07-30T12:35:00.003+10:00

The Bash shell has some pretty nifty tricks when it somes to substring matching and replacement. The documentation over at TLD is very comprehensive but here's the gist of it:

${string#substring}
Deletes shortest match of $substring from front of $string.

${string##substring}
Deletes longest match of $substring from front of $string.

${string%substring}
Deletes shortest match of $substring from back of $string.

${string%%substring}
Deletes longest match of $substring from back of $string.

So to provide an example when working with filenames, given the following files:

file123.txt
file456.txt
file789.txt

If I want to rename them all to be Document-123.txt, Document-456.txt etc. I could do the following:

for f in file*; do 
    mv $f Document-${f#file}
done

And to rename the extensions to .doc, it's as easy as:

for f in Document-*; do
    mv $f ${f%.txt}.doc
done

A more practical example might be to convert all FLAC files in a directory to MP3s and rename the extensions along the way. To achieve this you'd need flac and lame installed and do something like:

for f in *.flac; do 
    flac -cd "$f" | lame -h - "${f%.flac}.mp3"
done

Easy!

Creating a test RAID5 + LVM storage pool

2009-07-29T17:02:00.022+10:00

I'm currently planning an upgrade to my RHEL5 Linux storage server at home which will comprise three 1TB disks in a RAID5 configuration so there's at least some redundancy in case of a disk failure, and Logical Volume Manager (LVM) in case I decide to expand the pool down the track.

I thought it'd be a good idea to do a dry run using files instead of disks just to have a clear idea of what needs to be done.

There's a excellent walkthrough over at http://www.gagme.com/greg/linux/raid-lvm.php and that's what I based my configuration on, so check it out for a little more background info.

Create three zero-filled 1GB files:

for d in a b c; do
    dd if=/dev/zero of=/var/tmp/hd${d}.img bs=1024 count=1048576
done

Next, set up the loopback devices:

losetup /dev/loop0 /var/tmp/hda.img 
losetup /dev/loop1 /var/tmp/hdb.img 
losetup /dev/loop2 /var/tmp/hdc.img

Now we're going to use fdisk to create a 'Linux raid autodetect' partition on each 'disk':

fdisk /dev/loop0
fdisk /dev/loop1
fdisk /dev/loop2

The output should look like this, my commands in bold. Don't forget to do a partprobe once you're done.

[root@canard ~]# fdisk /dev/loop0
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-127, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-127, default 127): 
Using default value 127

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

partprobe

Now we're going to set up our RAID5 array on the md0 device with the 3 disks we just prepared:

mdadm --create /dev/md0 --level=5 --raid-devices=3 \
    /dev/loop0 /dev/loop1 /dev/loop2

And that's pretty much it for the RAID part. You can view that status of your md device using mdadm or viewing the proc entry, examples follow.

[root@canard ~]# cat /proc/mdstat 
Personalities : [raid6] [raid5] [raid4] 
md0 : active raid5 loop2[3] loop1[1] loop0[0]
      2047872 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [=========>...........]  recovery = 46.5% (476544/1023936) finish=4.3min speed=2112K/sec

[root@canard ~]# mdadm --detail /dev/md0 
/dev/md0:
        Version : 00.90.03
  Creation Time : Wed Jul 29 11:46:47 2009
     Raid Level : raid5
     Array Size : 2047872 (2000.21 MiB 2097.02 MB)
  Used Dev Size : 1023936 (1000.11 MiB 1048.51 MB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed Jul 29 11:46:47 2009
          State : clean, degraded, recovering
 Active Devices : 2
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

 Rebuild Status : 53% complete

           UUID : 1744c4be:0566a9ee:8a056ff6:bd165eb4
         Events : 0.1

    Number   Major   Minor   RaidDevice State
       0       7        0        0      active sync   /dev/loop0
       1       7        1        1      active sync   /dev/loop1
       3       7        2        2      spare rebuilding   /dev/loop2

As you can see, md is in the process of building the array. Now it's time to configure the LVM side of things.

First create a PV (physical volume), then a VG (volume group) consisting of your PV, then a LV (logical volume) on top of your VG.

It's probably a good idea to check out the LVM docs prior to commencing this step to gain a good understanding of what it is and does.

You could just as easily leave things here and format the md0 device as ext3, mount it, and you'd be done. But we're not doing that just yet. So, onwards:

pvcreate /dev/md0
vgcreate vg_media /dev/md0

Now display some info on your VG - the output should look a little like this - pay special attention to the number of PE (physical extents) you have available:

[root@canard ~]# vgdisplay vg_media
  --- Volume group ---
  VG Name               vg_media
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1.95 GB
  PE Size               4.00 MB
  Total PE              499 
  Alloc PE / Size       499 / 1.95 GB
  Free  PE / Size       0 / 0   
  VG UUID               2rsx3T-C1Lx-uMRf-kz1g-UyBP-tbTz-iC7K1f

OK so now we can go ahead and create the LV using the value for logical extents we obtained from the previous command:

lvcreate -l 499 vg_media -n lv_media

And again, display some information:

[root@canard ~]# lvdisplay /dev/vg_media/lv_media
  --- Logical volume ---
  LV Name                /dev/vg_media/lv_media
  VG Name                vg_media
  LV UUID                iS01Vt-Hyo7-GIiA-Sxlu-rc5J-bX5g-Trd9Ln
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1.95 GB
  Current LE             499
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

We're nearly done! All that's left to do is format and mount the LV:

mkfs.ext3 /dev/vg_media/lv_media
mkdir /mnt/lvtest
mount /dev/vg_media/lv_media /mnt/lvtest

You'll notice that df reports the parity space as being used already, so out of the 3GB we should have around 2GB available for use:

[root@canard ~]# df -h /mnt/lvtest/
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_media-lv_media
                      2.0G   35M  1.8G   2% /mnt/lvtest

Next I'll be failing a disk and expanding the LV to incorporate a second RAID5 array. As ever, stay tuned.

Installing and configuring SNMP on OpenSolaris

2009-07-26T10:20:00.011+10:00

SNMP is installed and run slightly differently on OpenSolaris than on Solaris 10, so here's the basics on what you need to know.

Firstly, you'll need to install the SUNWsmmgr package.

pkg install SUNWsmmgr

All of the config files, including snmpd.conf have moved to /etc/sma/snmp.

The service you need to run is now called sma.

svcadm enable svc:/application/management/sma:default

Stay tuned as I'm currently setting up iSCSI targets on my OpenSolaris server at home and will be running the initiator from a RHEL5 box.

Init script for transmission-daemon

2009-07-20T13:37:00.014+10:00

The RPM for the Transmission bittorrent client version 1.34 in EPEL, or 1.72 in Fedora 11 don't include an init script for the daemon. Since that's all I use, I wrote my own.

EDIT: Thanks to a helpful comment below, I've modified the script around the TRANSMISSION_HOME variable to fix config options being dropped on a restart.

Also, you can edit the settings.json file in the config dir to further customise your install. See the wiki for help editing config files.

#!/bin/bash
#
# chkconfig: - 16 84
# description: Start up transmission-daemon
#
# processname: transmission-daemon
# config: /etc/sysconfig/transmission

# source function library
. /etc/rc.d/init.d/functions

# Get network config
. /etc/sysconfig/network

[ "${NETWORKING}" = "no" ] && exit 0

# Defaults
TRANSMISSION_HOME=/var/lib/transmission

DAEMON_USER="transmission"
DAEMON_ARGS="-T --blocklist -g $TRANSMISSION_HOME/.config/transmission"

# Daemon
NAME=transmission-daemon

DAEMON=$(which $NAME)
DAEMON_PIDFILE=/var/run/$NAME.pid
DAEMON_LOCKFILE=/var/lock/subsys/$NAME
DAEMON_SCRIPTNAME=/etc/init.d/$NAME
DAEMON_LOGFILE=/var/log/$NAME.log

[ -x "$DAEMON" ] || exit 0

start() {
    echo -n $"Starting ${NAME}: "
    
    if [ -n "$TRANSMISSION_HOME" ]; then
        export TRANSMISSION_HOME
    fi

    su - $DAEMON_USER -c "$DAEMON $DAEMON_ARGS"

    sleep 2

    status $NAME &> /dev/null && echo_success || echo_failure
    RETVAL=$?

    if [ $RETVAL -eq 0 ]; then
        touch $DAEMON_LOCKFILE
        pidof -o %PPID -x $NAME > $DAEMON_PIDFILE
    fi

    echo
}

stop() {
    echo -n $"Shutting down ${NAME}: "
    
    killproc $NAME
    RETVAL=$?

    [ $RETVAL -eq 0 ] && /bin/rm -f $DAEMON_LOCKFILE $DAEMON_PIDFILE

    echo
}

case "$1" in
    start)
        start
    ;;
    stop)
        stop
    ;;
    restart)
        stop
        start
    ;;
    status)
        status $NAME
    ;;

    *)
        echo "Usage: $SCRIPTNAME {start|stop|restart|status}" >&2
        exit 3
    ;;
esac

Backing up F5 Big-IP device configuration

2009-07-15T19:14:00.014+10:00

At my place of work we have a number F5 Big-IP devices which are used for load-balancing, failover, SSL offloading, application health monitoring and persistence. They are, quite simply, awesome.

Each device forms one of an HA Active/Standby pair and naturally, we keep nightly backups of each pair's configuration.

You can get the full blurb from the F5 website, but Big-IPs have a Linux-based service console and while they offer a web GUI for easy configuration, they also sport a fully-featured CLI and Bash shell via SSH.

This makes it very simple to automate backing them up using standard Linux sysadmin tools.

I'm sharing with you two Bash scripts - one resides on your management server and the other is installed on all your F5s which is then called by your nightly management script.

You'll need to set up SSH keys to make this work.

Here's the management cron script:

#!/bin/bash

# This script is run on your management server. It executes the backup script on each 
# devices and rsync's the saved files over.

F5_ALL="host1 host2"

cd /home/backupadm/f5

for HOST in $F5_ALL; do
    echo "Backing up $HOST ... "
    ssh root@$HOST 'sh /root/bin/backup.sh'
    rsync -avz -e '/usr/bin/ssh' root@$HOST:/var/local/ucs/daily.* backups/
    echo
done

# Purge expired backups
find ./backups/ -name "daily.*" -mtime +30 -exec rm -f {} \;

And the F5 script:

#!/bin/bash

# Install this script as /root/bin/backup.sh on all F5 devices.

# See: https://support.f5.com/kb/en-us/solutions/public/3000/400/sol3499.html
# for details on backing up F5 configs.

cd /var/local/ucs

# Purge expired backups
find ./ -name "daily.*" -mtime +7 -exec rm -f {} \;

# This the bigpipe utility - man bigpipe
/bin/bigpipe config save daily.$(/bin/hostname).$(/bin/date +%F)

# Check state for config sync "active | standby" and pull if we're the standby
if [ $(/bin/bigpipe failover show | awk '{ print $2 }') = "standby" ]; then
    /bin/bigpipe config sync pull    
fi

You'll notice at the end of the last script that I perform an automatic pull of the Active device configuration to the Standby as well - this may or may not be necessary for your installation.

Simple disk threshold check

2009-07-15T18:18:00.011+10:00

Whew, it's been a while since my last post. I'll ease back into things with this very simple disk threshold check which you could run via cron. It works (has only been tested) on Linux.

It's... well, simple.

#!/bin/bash

# Threshold usage in %
threshold=90

# Admin email
email="you@example.org"

df -l | while read l; do
    usage=$(echo $l | awk '{ print $5 }' | sed -e 's/%//')
    if [ $usage -gt $threshold ]; then
        df -l | mail -s "[CRON] $(hostname) Disk Alert - ${threshold}% threshold exceeded" $email
        break
    fi
done

Parsing CLI options in Python

2009-06-17T11:55:00.011+10:00

Python makes performing common scripting tasks easy - the batteries are included, as they say.

Here's a simple script to connect to a host on a given TCP port using a specified timeout. It's designed to be run from the command line, and using the optparse module parsing options passed to the script is trivial.

The script returns a non-zero value on failure, and optionally prints an informative message; it was originally written to be called by other programs, such as shell scripts, to test connectivity prior to executing other routines.

The Python docs have more info on the optparse module.

Example usage is as follows:

$ ./conn.py --ip 192.168.0.1 --port 1234 --timeout 5 -v

And here's the script:

#!/usr/bin/env python

import socket, sys, optparse
 
p = optparse.OptionParser()
 
# CLI Options
p.add_option('-i', '--ip', action='store', type='string', dest='ip')
p.add_option('-p', '--port', action='store', type='int', dest='port')
p.add_option('-t', '--timeout', action='store', type='int', dest='timeout', default=1)
p.add_option('-v', '--verbose', action='store_true', dest='verbose', default=False)
 
(o, args) = p.parse_args()
 
# Require IP and port
if not o.ip or not o.port:
    sys.stderr.write("Try -h for help \n")
    sys.exit(255)
 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(o.timeout)
 
try:
    s.connect((o.ip, o.port))
except socket.error, e:
    if o.verbose:
        sys.stderr.write("Connect to %s on port %i - FAIL \n" % (o.ip, o.port))
    sys.exit(1)
else:
    if o.verbose:
        sys.stdout.write("Connect to %s on port %i - PASS \n" % (o.ip, o.port))
    sys.exit(0)
finally:
    s.close()

Use ipcalc to calculate a broadcast, network and netmask

2009-05-27T17:44:00.003+10:00

The ipcalc utility is included with the initscripts package on all RHEL/CentOS/Fedora systems. Here's a simple script to calculate the broadcast, netmask and network given an IP address in CIDR notation:

#!/bin/bash
 
echo -n "Enter IP address in CIDR notation: (eg. 192.168.0.0/24) "
read IP_ADDRESS
 
# Calculate network values
BROADCAST=$(ipcalc -b $IP_ADDRESS)
NETWORK=$(ipcalc -n $IP_ADDRESS)
NETMASK=$(ipcalc -m $IP_ADDRESS)
 
echo "Set up your ifcfg file with the following values:"
echo $BROADCAST
echo $NETWORK
echo $NETMASK

Using at for one-off scheduled tasks

2009-05-21T18:31:00.013+10:00

When you think of scheduled tasks on a Linux/Unix host, you think... cron, right?

For one-off jobs using at is much quicker, easier and simpler, plus there's no danger of you leaving an unwanted entry in your crontab once the job is complete.

For example, say you need to take down a server for maintenance at 2am. And if you're anything like me, you don't want to have to get out of bed.

[jason@bonito ~]$ at 2am tomorrow
/sbin/shutdown -h 2 "System going down for maintenance in 2 minutes."
job 2 at Fri May 22 02:00:00 2009

Easy. To view the details of the job, use atq.

[jason@bonito ~]$ atq
2 Fri May 22 02:00:00 2009

To view the job itself, that is how it will run as a shell script, use the -c flag followed by the job spec (which is 2 in our example). To remove a job, use atrm or the -r flag.

Time specifications are well documented - from the man page:

For example, to run a job at 4pm three days from now, use at 4pm + 3 days, to run a job at 10:00am on July 31, use at 10am Jul 31 and to run a job at 1am tomorrow, use at 1am tomorrow. The at utility also supports the POSIX time format (see -t option).

Send a binary file as an attachment to an email

2009-05-19T18:08:00.003+10:00

Here's a quick little one-liner I posted over at http://www.commandlinefu.com/ on how to send a binary file, such as an archive or image as an attachment to an email from the command line.

You'll need the sharutils package installed on a RHEL/CentOS/Fedora system and of course a working MTA.

So, here it is:

uuencode archive.tar.gz archive.tar.gz \
    | mail -s "Emailing: archive.tar.gz" user@example.com

SSH hostname autocomplete

2009-04-24T08:47:00.006+10:00

At my place of work I have over 200 servers to manage, and while most of our management/admin/update processes are automated in some way, I'm incredibly lazy (not in the pejorative sense of course) I use the following line in my ~/.bashrc to do hostname autocompletion for the ssh command.

This tip probably won't work for Ubuntu users whose ~/.ssh/known_hosts file uses hashed hostnames by default. In Red Hat and Fedora it works just fine.

complete -W "$(echo $(cat ~/.ssh/known_hosts | \
    cut -f 1 -d ' ' | sed -e s/,.*//g | \
    sort -u | grep -v "\["))" ssh

Bitbucket and pid, a simple PID file handler

2009-04-15T17:41:00.015+10:00

Whilst on my current bent of all things Python, Mercurial and RPM, I stumbled across Bitbucket "a place for you and your team to host and follow your Mercurial projects".

You can access your repositories via SSH or HTTPS and projects feature a Wiki, Bug tracker and Source code browser. Bitbucket is written using the Django web framework.

You get one free private repository and unlimited public ones that are limited to 150MB in size, with paid plans also being available.

I'll let you get acquainted with its more intricate features, but to do so I uploaded pid, a simple Python module I've written to handle PID (Process ID) files in my scripts.

You can browse the source code, view the Wiki (very spartan), or download a source RPM or Python egg.

It's simple to use where you need to ensure that a script never runs as 2 concurrent processes:

from pid import Pid, FileExistsError, FileDoesNotExistError
import sys

p = Pid('/tmp/test.pid')
 
try:
    p.start()
except FileExistsError, e:
    print e
    sys.exit(1)
  
# [ ... do stuff ...]

try:
    p.finish()
except FileDoesNotExistError, e:
    print e

Please feel free to contribute, ridicule or provide any other feedback as you see fit.

Verifying packages using Python's RPM API

2009-04-06T09:35:00.008+10:00

As part of our custom CentOS 5 SOE server build, I include a number of 3rd-party and user-contributed packages.

During the CD ISO creation process, I want to ensure that all the RPM packages I'm about to include have their header and payload MD5 checksum, and GPG signature verified, to ensure no un-signed or corrupt packages make their way into the build.

Here's an easy way to do this via a simple Python script - you'll need to import all your GPG keys into your RPM database (rpm --import) prior to running the check.

import sys, os, rpm

if len(sys.argv) < 2:
    sys.stderr.write("Usage: %s /path/to/package.rpm \n" % (sys.argv[0]))
    sys.exit(1)

package = sys.argv[1]
ts = rpm.TransactionSet()

try:
    fd = os.open(package, os.O_RDONLY)
except OSError, e:
    sys.exit(1)

try:
    header = ts.hdrFromFdno(fd)
    # At this point, all checks have passed
except rpm.error, e:
    sys.exit(1)

os.close(fd)
sys.exit(0)

For more info, see the Python RPM API docs.

Packaging Python cliutils for Fedora

2009-03-31T13:07:00.013+10:00

As promised, I've submitted Python cliutils for review to be included as a Fedora package.

I've only been using it for a couple of days so far, but I'm already finding the cliutils library really useful for everyday systems-admin scripting, where I'm frequently mix-n-matching Bash and Python.

With cliutils, I can do just about everything in Python and call out to shell scripts and commands with ease.

Here's an example:

lines = Process('grep blogger.com access.log') | Process('wc -l')

This could also be written as:

lines = sh.grep('blogger.com access.log') | sh.wc('-l')

Another handy feature is the @cliargs decorator, which parses command line arguments as method arguments as shown in this small script:

from cliutils import sh, cliargs

@cliargs
def doSearch(domain, logfile='domains.log'):
    """Usage: search.py domain [ --logfile file ]"""
    lines = sh.grep('%s %s' % (domain, logfile)) | sh.wc('-l')
    print "Lines with domain '%s': %s" % (domain, lines)

doSearch()

And you could run this script like so:

./search.py blogger.com --logfile access.log

There's also API documentation available and more examples and general usage docs on the project homepage.

You can download the SPEC file and SRPM for Fedora 10.

Installing SyntaxHighlighter for Blogger

2009-03-31T10:44:00.003+10:00

I've just 'installed' Alex Gorbatchev's excellent Syntax Highlighter for this blog. It's a snap to set up, is very well-documented and has configurable themes and highlighting options.

The output looks like this:


import socket, sys

if len(sys.argv) < 2:
    sys.stderr.write("Usage: %s args \n" % (sys.argv[0]))
    sys.exit(1)

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(1)

Very nice.

For more information on Themes, Syntaxes and Usage, see the Wiki.

Packaging the Grizzled Library for Fedora

2009-03-29T18:03:00.001+10:00

I've recently joined the Fedora project to do my bit for the free and open-source software community, by packaging 3rd party software for inclusion in the main Fedora project repository.

Software from the frequently-updated Fedora project often makes it downstream into RHEL; CentOS is a re-packaged distribution of RHEL so everybody wins.

Being a long-time user of and advocate for Fedora, RHEL and CentOS I've decided it's time to give something back, and since I've been using RPM much more heavily recently, I figured packaging would be a natural fit.

I'd like to focus on the broader area of 'useful Python packages for Linux systems adminstration' as this complements my day job nicely, so the first package I'll be submitting for inclusion is Brian Clapper's Python Grizzled Utility Library which contains a number of handy modules. The os.daemonize module is especially useful.

Brian's website also includes comprehensive API documentation making it a snap to start using the library.

You can download the SPEC file and the SRPM for Fedora 10 until of course the package is available in Fedora's package repository :-)

Installing Mercurial on RHEL/CentOS 3

2009-03-25T11:48:00.011+10:00

I've recently started using (and am subsequently hooked on) Mercurial for version control, both at work, for projects on which I collaborate, and at home as a simple way of isolating development changes to documents and files.

It was a simple case of doing a

yum install mercurial

on my Fedora 10 workstation, however I ran into some trouble when it came time to install Mercurial on one of the number of RHEL/CentOS 3 management servers we still operate and support, mainly due to Mercurial's requirement of a Python version >= 2.4.

RHEL 3 ships with Python 2.2 and as anyone who's attempted to upgrade the installed system version of Python on a RHEL-based system will attest, it's typically an operation fraught with danger.

A custom install of Python 2.5 to an alternate location is a safer option and extremely easy to achieve.

Here's how I did it.

Fetch and extract sources:

wget http://www.python.org/ftp/python/2.5.4/Python-2.5.4.tgz
wget http://www.selenic.com/mercurial/release/mercurial-1.2.1.tar.gz
tar zxvf mercurial-1.2.1.tar.gz
tar zxvf Python-2.5.4.tgz

Configure and build Python using /opt - you could use /usr/local or similar but I preferred to keep it out of my $PATH:

cd Python-2.5.4
./configure --prefix=/opt
make
su -c "make install"

Test Python installed properly:

/opt/bin/python
Python 2.5.4 (r254:67916, Mar 25 2009, 12:16:36)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Build Mercurial, using your Python 2.5:

cd ../mercurial-1.2.1
su -c "make install PYTHON=/opt/bin/python PREFIX=/opt"

Test Mercurial installed properly:

/opt/bin/hg --version
Mercurial Distributed SCM (version 1.2.1)
Copyright (C) 2005-2009 Matt Mackall and others

You may wish to symlink hg from somewhere in your $PATH:

su -c "ln -s /opt/bin/hg /usr/local/bin/hg"

Obviously you'll need a C compiler and associated development tools installed. You may find that the Python configure command complains of missing libraries, such as zlib-devel which can be installed via yum if required.

Good luck and refer to the install instructions on Mercurial's site if you get stuck.