At Lanedo, we quite often deal with embedded hardware and need to compile packages for it. Most of the time, this is not done on the embedded hardware itself but on our more powerful desktop machines. And most of the time, the desktop machine has an Intel processor while the target device has for example an ARM processor, so we have to cross-compile the packages. Consider DirectFB for example, which we’ve recently been working with and because it’s a well behaved piece of software that is used to being cross-compiled.
You need two things for cross-compiling: a compiler that can produce binaries for the target platform, and the target platform’s libraries your binaries link against, which is at least lowlevel stuff like libc, libm etc. You can set up all of that yourself, but this is beyond the scope of this post. Typically when you have embedded hardware or a development board, the vendor provides a bootloader, kernel image and root filesystem for installation on the device and also a toolchain for installation on your build machine, which contains the cross-compiler and a sysroot which is a tree of files that match the root filesystem that is installed on your embedded hardware. When cross-compiling, you can link against the libraries in that sysroot as if they were the ones on your target system.
Let’s assume you have a toolchain and sysroot from your device’s vendor and they install into the following directories:
It’s perhaps not enough to build your packages for the target system, often software depends on auxillary or their own build tools (needed to run on your build system). For building DirectFB from git, we need a tool called fluxcomp and DirectFB itself builds a tool called directfb-csource which may be needed when building DirectFB applications. We assume we do not want to pollute our build system with this so we set up a proper development prefix:
mkdir /local/dfb mkdir /local/dfb/src
In order to set up the environment for that prefix, we create a file that contains all the variables needed:
# host-env.sh PREFIX=/local/dfb # set up the stuff we need anyway export PATH=$PREFIX/bin:$PATH export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH export ACLOCAL_FLAGS="-I $PREFIX/share/aclocal" export PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH # set up stuff some legacy (non pkg-config) packages might need export CPPFLAGS=-I$PREFIX/include export LDFLAGS=-L$PREFIX/lib # set the terminal title so we always see where we are echo -en "\033]0;$PREFIX - local x86 env\a"
Now we are ready to build our local build tools:
source host-env.sh cd flux ./configure --prefix=/local/dfb make make install cd ../DirectFB ./configure --prefix=/local/dfb make make install
and we are set for the actual task…
In order to do that, we need the compiler and sysroot from above mentioned toolchain. They both live in their own prefixes, which we set up to be used very similarly to the above local environment:
# cross-env.sh # This is where the cross compiler lives TOOLCHAIN=/opt/vendor/toolchain # Sets its binaries to be CC, LD etc. # (examples from an ARM cross build) export PATH=$TOOLCHAIN/bin:$PATH export CC=arm-linux-gnueabi-gcc export CXX=arm-linux-gnueabi-g++ export AR=arm-linux-gnueabi-ar export RANLIB=arm-linux-gnueabi-ranlib export LD=arm-linux-gnueabi-ld # This is where the libraries of the target platform live SYSROOT=/opt/vendor/sysroot # We do NOT set PATH and LD_LIBRARY_PATH here because we # can't run the binaries from SYSROOT on our build machine, # but we want to link against libraries that are installed # there. We also set pkg-config's default directory to the # one in the SYSROOT, and clear the environment from any # other local-machine architecture pkg-config path # directories. export PKG_CONFIG_LIBDIR=$SYSROOT/usr/lib/pkgconfig export PKG_CONFIG_PATH="" export ACLOCAL_FLAGS="-I $SYSROOT/usr/share/aclocal" export CPPFLAGS=-I$SYSROOT/usr/include export LDFLAGS=-L$SYSROOT/usr/lib # This is our local prefix where we will install all # cross-built stuff first, in order to verify that all # builds fine. We set it up as proper devel prefix because # later built packages might need the libraries we have # installed there earlier. We also don't want to pollute # SYSROOT with our own stuff. CROSS_PREFIX=/local/dfb/sysroot # We do NOT set PATH and LD_LIBRARY_PATH here because we can't # run the binaries from CROSS_PREFIX on our build machine export PKG_CONFIG_PATH=$CROSS_PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH export ACLOCAL_FLAGS="-I $CROSS_PREFIX/share/aclocal $ACLOCAL_FLAGS" export CPPFLAGS="-I$CROSS_PREFIX/include $CPPFLAGS" export LDFLAGS="-L$CROSS_PREFIX/lib $LDFLAGS" # set the terminal title and prompt so we always see where we are echo -en "\033]0;$CROSS_PREFIX - cross ARM env\a" export PS1="[cross ARM] $PS1"
Now we are ready to start the compiling. It is important to use source on both host-env.sh and then cross-env.sh because we need the tools we built into the local build-machine prefix in order to build the cross compiled packages:
source host-env.sh source cross-env.sh cd DirectFB ./configure --prefix=/local/dfb/sysroot --host=arm-linux-gnueabi make make install cd ../DirectFB-examples ./configure --prefix=/local/dfb/sysroot --host=arm-linux-gnueabi make make install
In this example we built DirectFB for the “arm-linux-gnueabi” platform. Sometimes, and depending on how your vendor’s toolchain and sysroot are built, it might be neccessary to additionally pass –with-sysroot=/opt/vendor/sysroot in order to make sure that the compiler considers this directory to be the root of the system, where things like libc live in the /lib or /usr/lib subdirectories. Not all toolchains need –with-sysroot so we left it out of the configure options in above example, to keep it simple. We also built DirectFB-examples which depends on the previously cross-compiled DirectFB. Depending on what is available on your embedded platform, you might have to pass additional options to DirectFB’s configure, such as –with-gfxdrivers=none or –without-mesa according to the target you’re building for.
If all that went fine and the files in /local/dfb/sysroot look like they should…
To The Device!
…we are ready to build and install the packages on our embedded device for testing.
We use exactly the same environment we used for cross-compiing, but we use slightly different configure options. Next you can either mount the device’s SD card, NFS-mount the device’s root filesystem or just have an empty directory to install to which you can later tar and untar with on the device. We will assume the device’s root file system is mounted at
So let’s compile for this mount point. Note that we cannot use –prefix=/media/user/device, because –prefix tells configure where the files will be installed on the running system. We compile the packages for –prefix=/usr and just make install them to the mount point:
source host-env.sh source cross-env.sh cd DirectFB # clean the source directory, so all traces of the previous prefix are gone, # or use a separate source directory for buliding for device installation ./configure --prefix=/usr --sysconfdir=/etc --host=arm-linux-gnueabi make make DESTDIR=/media/user/device install cd ../DirectFB-examples # clean the source directory, so all traces of the previous prefix are gone, # or use a separate source directory for buliding for device installation ./configure --prefix=/usr --sysconfdir=/etc --host=arm-linux-gnueabi make make DESTDIR=/media/user/device install
Now we can verify the installed files in /media/user/device, unmount the volume, perform a test run on the device. With a little luck, we didn’t mess up and everything works 🙂
If you found these steps useful and have any feedback about how we can improve them, feel free to add a comment below or contact us at Lanedo
Good luck cross compiling!