Difference between revisions of "Cross compiling deb packages on Hudson"

This guide will help setting up cross compilation and packaging i386 and amd64 deb packages for the following distros:

1. Debian testing (Squeeze)
2. Debian unstable (Sid)
3. Ubuntu Karmic
4. Ubuntu Lucid

Link that might be useful: PbuilderHowto[1]

Setting up pbuilder

1. Firstly, install pbuilder

   sudo apt-get install pbuilder

2. Then replace /etc/pbuilderrc with the following config:

# Codenames for Debian suites according to their alias. Update these when
# needed.
UNSTABLE_CODENAME="sid"
TESTING_CODENAME="squeeze"
STABLE_CODENAME="lenny"
STABLE_BACKPORTS_SUITE="$STABLE_CODENAME-backports"

# List of Debian suites.
DEBIAN_SUITES=($UNSTABLE_CODENAME $TESTING_CODENAME $STABLE_CODENAME
    "unstable" "testing" "stable")

# List of Ubuntu suites. Update these when needed.
UBUNTU_SUITES=("lucid" "karmic" "jaunty" "intrepid" "hardy")

# Mirrors to use. Update these to your preferred mirror.
DEBIAN_MIRROR="ftp.de.debian.org"
UBUNTU_MIRROR="ubuntu.mirror.tudos.de"

# Optionally change Debian release states in $DIST to their names.
case "$DIST" in
    unstable)
        DIST="$UNSTABLE_CODENAME"
        ;;
    testing)
        DIST="$TESTING_CODENAME"
        ;;
    stable)
        DIST="$STABLE_CODENAME"
        ;;
esac

# Optionally set the architecture to the host architecture if none set. Note
# that you can set your own default (i.e. ${ARCH:="i386"}).
#: ${ARCH:="$(dpkg --print-architecture)"}

NAME="$DIST"
if [ -n "${ARCH}" ]; then
    NAME="$NAME-$ARCH"
    ARCHITECTURE="$ARCH"
fi
BASETGZ="/var/cache/pbuilder/$NAME-base.tgz"
# Optionally, set BASEPATH (and not BASETGZ) if using cowbuilder
# BASEPATH="/var/cache/pbuilder/$NAME/base.cow/"
DISTRIBUTION="$DIST"
BUILDRESULT="`pwd`/debian/$NAME/"
APTCACHE="/var/cache/pbuilder/$NAME/aptcache/"
BUILDPLACE="/var/cache/pbuilder/build/"

# Create here as pbuilder creates it with root owner.
mkdir -p "$BUILDRESULT"
chown -R tomcat6 "$BUILDRESULT"

if $(echo ${DEBIAN_SUITES[@]} | grep -q $DIST); then
    # Debian configuration
    MIRRORSITE="http://$DEBIAN_MIRROR/debian/"
    COMPONENTS="main contrib non-free"
    # This is for enabling backports for the Debian stable suite.
    #if $(echo "$STABLE_CODENAME stable" | grep -q $DIST); then
    #    EXTRAPACKAGES="$EXTRAPACKAGES debian-backports-keyring"
    #    OTHERMIRROR="$OTHERMIRROR | deb http://www.backports.org/debian $STABLE_BACKPORTS_SUITE $COMPONENTS"
    #fi
elif $(echo ${UBUNTU_SUITES[@]} | grep -q $DIST); then
    # Ubuntu configuration
    MIRRORSITE="http://$UBUNTU_MIRROR/ubuntu/"
    COMPONENTS="main restricted universe multiverse"
else
    echo "Unknown distribution: $DIST"
    exit 1
fi



EXTRAPACKAGES="aptitude"
#export DEBIAN_BUILDARCH=athlon
#export http_proxy=http://your-proxy:8080/
USEPROC=yes
USEDEVPTS=yes
USEDEVFS=no

# specifying the architecture passes --arch= to debootstrap; the default is
# to use the architecture of the host
#ARCHITECTURE=`dpkg --print-architecture`
# specifying the components of the distribution, for instance to enable all
# components on Debian use "main contrib non-free" and on Ubuntu "main
# restricted universe multiverse"
#COMPONENTS="main"
#specify the cache for APT 
#APTCACHE="/var/cache/pbuilder/aptcache/"
APTCACHEHARDLINK="yes"
REMOVEPACKAGES=""
#HOOKDIR="/usr/lib/pbuilder/hooks"
HOOKDIR=""

# make debconf not interact with user
export DEBIAN_FRONTEND="noninteractive"

DEBEMAIL=""

#for pbuilder debuild
BUILDSOURCEROOTCMD="fakeroot"
PBUILDERROOTCMD="sudo -E"
# use cowbuilder for pdebuild
#PDEBUILD_PBUILDER="cowbuilder"


# command to satisfy build-dependencies; the default is an internal shell
# implementation which is relatively slow; there are two alternate
# implementations, the "experimental" implementation,
# "pbuilder-satisfydepends-experimental", which might be useful to pull
# packages from experimental or from repositories with a low APT Pin Priority,
# and the "aptitude" implementation, which will resolve build-dependencies and
# build-conflicts with aptitude which helps dealing with complex cases but does
# not support unsigned APT repositories
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends"

#Command-line option passed on to dpkg-buildpackage.
#DEBBUILDOPTS="-IXXX -iXXX"
# Binary-only.
DEBBUILDOPTS="-b"

#APT configuration files directory
APTCONFDIR=""

# the username and ID used by pbuilder, inside chroot. Needs fakeroot, really
BUILDUSERID=1234
BUILDUSERNAME=pbuilder

# BINDMOUNTS is a space separated list of things to mount
# inside the chroot.
BINDMOUNTS=""

# Set the debootstrap variant to 'buildd' type.
# DEBOOTSTRAPOPTS[0]='--variant=buildd'
# or unset it to make it not a buildd type.
unset DEBOOTSTRAPOPTS

# Set the PATH I am going to use inside pbuilder: default is "/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"
export PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"

# SHELL variable is used inside pbuilder by commands like 'su'; and they need sane values
export SHELL=/bin/bash

# The name of debootstrap command, you might want "cdebootstrap".
DEBOOTSTRAP="debootstrap"

# default file extension for pkgname-logfile
PKGNAME_LOGFILE_EXTENTION="_$(dpkg --print-architecture).build"

# default PKGNAME_LOGFILE
PKGNAME_LOGFILE=""

# default AUTOCLEANAPTCACHE
AUTOCLEANAPTCACHE=""

Note: Built packages will be copied to "debian/" directory relative to current path. It is created right in this config to set its owner to tomcat6, so Hudson would be able to remove it (otherwise pbuilder creates it with root as the owner). You'll also probably want to change mirrors being used.

Creating chroot environments

Run the following:

sudo DIST=testing ARCH=i386 /usr/sbin/pbuilder --create
sudo DIST=testing ARCH=amd64 /usr/sbin/pbuilder --create

sudo DIST=unstable ARCH=i386 /usr/sbin/pbuilder --create
sudo DIST=unstable ARCH=amd64 /usr/sbin/pbuilder --create

sudo DIST=karmic ARCH=i386 /usr/sbin/pbuilder --create
sudo DIST=karmic ARCH=amd64 /usr/sbin/pbuilder --create

sudo DIST=lucid ARCH=i386 /usr/sbin/pbuilder --create
sudo DIST=lucid ARCH=amd64 /usr/sbin/pbuilder --create

Building packages

1. Add purge extension to Hudson. To do that add to .hg/hgrc in your working copy:

[extensions]
hgext.purge=

1. Allow Hudson run pbuilder using sudo. Add to your /etc/sudoers

   Defaults:tomcat6        env_keep+="DIST ARCH"

   And this line closer to /etc/sudoers end so it would not be overriden

   tomcat6 ALL=(root) NOPASSWD: /usr/sbin/pbuilder --build debian/glob2_*.dsc

2. Now you should add building steps in Hudson. I assume you're using web interface.
   1. purge will remove all the ignored and untracked files (including previously built packages) from working copy. So it should be the first one. "Execute shell command":

      hg purge --all

   2. And this one should be right after purge to get clean tarball for further building. "Invoke scons script":

      --config=force -j3 dist

   3. Now the building itself. "Execute shell command":

sudo DIST=testing ARCH=i386 /usr/sbin/pbuilder --build debian/glob2_*.dsc
sudo DIST=testing ARCH=amd64 /usr/sbin/pbuilder --build debian/glob2_*.dsc

sudo DIST=unstable ARCH=i386 /usr/sbin/pbuilder --build debian/glob2_*.dsc
sudo DIST=unstable ARCH=amd64 /usr/sbin/pbuilder --build debian/glob2_*.dsc

sudo DIST=karmic ARCH=i386 /usr/sbin/pbuilder --build debian/glob2_*.dsc
sudo DIST=karmic ARCH=amd64 /usr/sbin/pbuilder --build debian/glob2_*.dsc

sudo DIST=lucid ARCH=i386 /usr/sbin/pbuilder --build debian/glob2_*.dsc
sudo DIST=lucid ARCH=amd64 /usr/sbin/pbuilder --build debian/glob2_*.dsc