Scratchbox

From Chumby Wiki
Revision as of 07:19, 7 December 2011 by Bstern (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Description

This is a step-by-step documentation on how to install a Scratchbox development environment for Chumby.

This will allow everybody to create their own dev environment, compile Perl, build and test binaries for Chumby.

These instructions have been compiled on Ubuntu 9, and they would work just as fine on a Ubuntu 8.04 or Debian 4.0.

In our environment this runs on a XP box in a VMWare virtual machine. Installing the free VMWare server and Ubuntu should be a pretty easy task.

For the impatient, you can get here the packed environment described below.

Setting up the work environment

Installing Scratchbox

Installing a scratchbox environment on Ubuntu, or any other Debian distribution. Be sure to uncomment the sources for "universe"

  • add the following to /etc/apt/sources.list
deb http://scratchbox.org/debian stable main
  • Update the local repository cache:
sudo apt-get update
  • Install the scratchbox packages:
sudo apt-get install scratchbox-core scratchbox-libs
  • add yourself to the scratchbox group:
sudo /scratchbox/sbin/sbox_adduser username

If you are getting an error showing that the host kernel has vdso support turned on, which is incompatible with Scratchbox, you need to run:

 echo 0 > /proc/sys/vm/vdso_enabled
  • If running on an Ubuntu system, you might encounter a problem when running compiled ARM binaries under qemu, such as:
 mmap: Permission denied

We found the following hint very helpful:

For problems when building packages (mmap: permission denied when building), it’s necessary to reduce the mmap minimum address:

 echo 4096 > /proc/sys/vm/mmap_min_addr

To make these changes permanent across reboots, one can add these settings to /etc/sysctl.conf:

vm.vdso_enabled = 0
vm.mmap_min_addr = 4096

Then run

sudo sysctl -p


  • logout, login to grok the new group "sbox". If using X, you need to completely restart X for the environment to be set right.
  • You can now enter scratchbox:
 /scratchbox/login
 
 [sbox-: ~] > logout

Set up the HOST target

  • Install the following packages:
 scratchbox-toolchain-host-gcc
 scratchbox-devkit-doctools
 scratchbox-devkit-perl
 scratchbox-devkit-debian
 scratchbox-devkit-cputransp

They can be installed in one call as:

 apt-get install scratchbox-toolchain-host-gcc scratchbox-devkit-doctools \
 scratchbox-devkit-perl scratchbox-devkit-debian scratchbox-devkit-cputransp
  • Allow the sbox group to create/add new compilers. Refer to the Scratchbox Wiki for more information on steps 1-3 on their page.
 chgrp -R sbox /scratchbox/compilers
 chmod -R g+w /scratchbox/compilers
 mkdir /scratchbox/device_tools
 chgrp -R sbox /scratchbox/device_tools
 chmod -R g+w /scratchbox/device_tools


  • Login to scratchbox as a normal user, and run the menu:
 /scratchbox/login
 
 [sbox-: ~] > sb-menu
  • Setup a new target and call it HOST
  • Select the host-gcc compiler
  • Select the debian-etch, perl, and doctools devkits.
  • Select no cpu transparency method.
  • Select "Yes" to installing files, and check /etc and Devkits.
  • Select the target by choosing Yes, observe the shell restarting and placing us in the HOST context:
 Shell restarting...
 [sbox-HOST: ~] >



Install a toolchain

The following are the steps needed to install the Code Sourcery toolchain.

These steps closely mirror those specified on the Scratchbox wiki detailing setting up foreign toolchains.

Install the C compiler

All steps below have to be executed from within the HOST target.

  • Navigate to the compilers directory
 [sbox-HOST: ~] >
 cd /scratchbox/compilers
  • Download the Chumby compiler as specified in [GCC_4.3.3] and extract the toolchain:
 [sbox-HOST: /scratchbox/compilers] >
 wget https://sourcery.mentor.com/sgpp/lite/arm/portal/package4571/public/arm-none-linux-gnueabi/arm-2009q1-203-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
 tar xvjf arm-2009q1-203-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
 cd arm-2009q1
  • We now have to create a number of symbolic links, to get the toolchain to work with scratchbox:
 [sbox-HOST: /scratchbox/compilers/arm-2009q1] >
 ln -s arm-none-linux-gnueabi/libc/usr/include
 ln -s /scratchbox/compilers/arm-2009q1/arm-none-linux-gnueabi/include/c++ include/c++

Configure the toolchain

  • Fetch the sb-toolchains-extras repository, which will create the toolchain package:
 [sbox-HOST: /scratchbox/compilers/arm-2009q1] >
 pushd ~
 rm -rf sb-toolchain-extras
 darcs get --set-scripts-executable http://scratchbox.org/repos/1.0/sb-toolchain-extras
 popd
  • Create the configuration file (this step could take a long time, 5-10 min):
 [sbox-HOST: /scratchbox/compilers/arm-2009q1] >
 ~/sb-toolchain-extras/confhelper/create_toolchain_conf.py > \
 ~/sb-toolchain-extras/meta/alien-tc/arm-2009q1.conf
  • You need to edit ~/sb-toolchain-extras/meta/alien-tc/arm-2009q1.conf. Set the following keys:
 COMPILER_PACKAGE = Chumby CodeSourcery
 BINUTILS_FULLVER = 2.19.51.20090205
 # BINUTILS_FULLVER = 2.18.50.20080215 # This is the correct version number for 2008q3
  • Move to the sb-toolchain-extras directory, where we'll be building the toolchain, and create the checksums for the various packages:
 [sbox-HOST: /scratchbox/compilers/arm-2009q1] >
 cd ~/sb-toolchain-extras
 make CONFIG=meta/alien-tc/arm-2009q1.conf -C meta/alien-tc all-sums

Before we start configuring a target, we need to do the following as root (ie in another terminal window):

 echo 0 > /proc/sys/vm/vdso_enabled
 echo 4096 > /proc/sys/vm/mmap_min_addr

Build the packages

 [sbox-HOST: ~/sb-toolchain-extras] >
 make CONFIG=meta/alien-tc/arm-2009q1.conf -C meta/alien-tc

If you do encounter errors, you'll need to correct the problem and try compiling again. Because strace stubbornly insists on re-applying its patch, noticing it doesn't match up, then erroring out you'll need to remove the work directory's contents, then re-extract the file. E.g.

 pushd device_tools/strace/work/
 rm -rf strace-4.5.15
 tar xvzf ../download/strace-4.5.15.tar.gz
 popd

Post-creation cleanup

You'll need to do one last thing before you use the toolchain. The toolchain assumes it's living in the root, which it isn't. There's a flag to disable this, that needs to be set in the user's environment. Set this up by appending the following line to /scratchbox/compilers/arm-2009q1/target_setup.sh:

os.system('echo export SBOX_EXTRA_COMPILER_ARGS=-Wno-poison-system-directories >> ' + target + '.environment')

Create the new target

You should be able to switch to the new toolchain now:

  • Run sb-menu.
  • Setup a new target. For this example, we named the target Chumby.
  • Select the new compiler: arm-2009q1
  • Select only the cputransp devkit.
  • Choose an ARM emulator. The latest available version is qemu-arm-cvs-m.
  • Do not select a rootstrap (select < No >).
  • Install files. De-select all options except for /etc and C-library
  • Select the new target, observe that it reloads the shell onto the new target:
Shell restarting...
[sbox-Chumby: ~] >

Using the Scratchbox environment

Limitations of the Scratchbox environment

The biggest limitation is that pthread support is nonexistent. This means, for example, that any call to pthread_join() will cause the program to silently quit. Practically, this means that threaded perl tests will not run, meaning many perl modules must be installed using force. However, once installed, these programs will run just fine on a real Chumby.

Running Scratchbox

Everytime we start it, we need to do the following as root:

 echo 0 > /proc/sys/vm/vdso_enabled
 echo 4096 > /proc/sys/vm/mmap_min_addr

Compiling and configuring Perl

  • Make a work folder, download and expand the Perl source:
[sbox-Chumby: ~] >
mkdir perl
cd perl
wget http://www.cpan.org/src/perl-5.10.1.tar.gz
tar zxf perl-5.10.1.tar.gz
cd perl-5.10.1
  • Create the following symlink
ln -s /scratchbox/tools/bin/pwd /bin/pwd
  • Create the target folder for Perl, and start the Configure script:
[sbox-Chumby: ~/perl/perl-5.10.1] >
mkdir -p /psp/usr/bin
./Configure
Installation prefix to use? (~name ok) [/opt] /psp/usr

What installation prefix should I use for installing files? (~name ok)
[/psp/usr]

If this doesn't make any sense to you, just accept the default 'n'.
Build Perl for SOCKS? [n] 

If this doesn't make any sense to you, just accept the default 'n'.
Use the PerlIO abstraction layer? [y]

If this doesn't make any sense to you, just accept the default 'y'.
Build a threading Perl? [n] y

Use which C compiler? [cc] gcc

Directories to use for library searches? [/lib /usr/lib]    *leave here the default*

What is the file extension used for shared libraries? [so] 

Try to use long doubles if available? [n] 

What libraries to use? [-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc] 

We need to override here the default since the current toolchain has issues with this flag:

What optimizer/debugger flag should be used? [-O2] none
Any additional cc flags?
[-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe]

Any additional ld flags (NOT including libraries)? [none] 

Try to use 64-bit integers, if available? [n] 

Try to use maximal 64-bit support, if available? [n] 

What is your architecture name [arm-linux] 

Pathname where the public executables will reside? (~name ok) [/psp/usr/bin] 

If this doesn't make any sense to you, just accept the default 'n'.
Use relocatable @INC? [n] 

Pathname where the private library files will reside? (~name ok)
[/psp/usr/lib/perl5/5.10.1] 

Where do you want to put the public architecture-dependent libraries? (~name ok)
[/psp/usr/lib/perl5/5.10.1/arm-linux-thread-multi] 

Other username to test security of setuid scripts with? [none] 

Does your kernel have *secure* setuid scripts? [n]

Do you want to do setuid/setgid emulation? [n] 

Installation prefix to use for add-on modules and utilities? (~name ok) [/psp/usr]

Pathname for the site-specific library files? (~name ok)
[/psp/usr/lib/perl5/site_perl/5.10.1]

List of earlier versions to include in @INC? [none] 

Do you wish to wrap malloc calls to protect against potential overflows? [y] 

Do you wish to attempt to use the malloc that comes with perl5? [n] 

Pathname for the site-specific architecture-dependent library files? (~name ok)
[/psp/usr/lib/perl5/site_perl/5.10.1/arm-linux-thread-multi] 

Do you want to configure vendor-specific add-on directories? [n] 

Colon-separated list of additional directories for perl to search? [none] 

Install any extra modules (y or n)? [n]

Directory for the main Perl5 html pages? (~name ok) [none]

Directory for the Perl5 module html pages? (~name ok) [none]

Since you don't have a /usr/bin/perl I'm assuming creating one is okay.
Do you want to install perl as /usr/bin/perl? [y] n

Shall I use /scratchbox/compilers/bin/nm to extract C symbols from the libraries? [n] 

Do you wish to use dynamic loading? [y] 

Source file to use for dynamic loading [ext/DynaLoader/dl_dlopen.xs] 

Any special flags to pass to gcc -c to compile shared library modules? [-fPIC] 

What command should be used to create dynamic libraries? [gcc]

Any special flags to pass to gcc to create a dynamically loaded library? [-shared  ]

Any special flags to pass to gcc to use dynamic linking? [-Wl,-E] 

Build a shared libperl.so (y/n) [n]

Where do the main Perl5 manual pages (source) go? (~name ok) [none] 

Where do the perl5 library man pages (source) go? (~name ok) [none] 

Your host name appears to be "ubuntu-ws". Right? [y] 

What is your domain name? [.nonet] 

What is your e-mail address? [user@ubuntu-ws.nonet] 

Perl administrator e-mail address [user@ubuntu-ws.nonet] 
 
Do you want to install only the version-specific parts of perl? [n] 

What shall I put after the #! to start up perl ("none" to not use #!)?
[/psp/usr/bin/perl] 

Where do you keep publicly executable scripts? (~name ok)
[/psp/usr/bin] 

Pathname where the add-on public executables should be installed? (~name ok)
[/psp/usr/bin] 

Pathname where the site-specific html pages should be installed? (~name ok) [none] 

Pathname where the site-specific library html pages should be installed? (~name ok) [none] 
 
Pathname where the site-specific manual pages should be installed? (~name ok) [none] 

Pathname where the site-specific library manual pages should be installed? (~name ok) [none] 

Pathname where add-on public executable scripts should be installed? (~name ok)
[/psp/usr/bin] 

Use the "fast stdio" if available? [n] 

Try to understand large files, if available? [y] n

What is the extension of dynamically loaded modules [so]

Shall I ignore gethostname() from now on? [n]

Do you still want to use vfork()? [n] 

Doubles must be aligned on a how-many-byte boundary? [4] *with the new toolchain we will get 8 as default*

Use which function to generate random numbers? [drand48] 

What type pointer is the second argument to getgroups() and setgroups()? [gid_t]

Build Perl with MAD? [n] 
 

Accept the following path, even if it does not exist

What pager is used on your system? [/scratchbox/tools/bin/less] /usr/bin/less

Which compiler compiler (yacc or bison -y) shall I use? [yacc] 
What extensions do you wish to load dynamically? ...

What extensions do you wish to load statically? [none] 

We use nptl but we don't have the program that detects that it is there. In order to do this, we need to remove the definition of THREADS_HAVE_PIDS from the ccflags and cppflags definitions.

Edit the config.sh file, and remove all instances of /scratchbox/tools. This will cause, for example, /scratchbox/tools/bin to become /bin. If you're using vi or vim, you can do this with the regexp :%s/\/scratchbox\/tools//g

Press return or use a shell escape to edit config.sh: !vim config.sh

Run make depend now? [y]


After it completes, we can run

[sbox-Chumby: ~/perl/perl-5.10.1] >
make && make install

Perl will be installed in the given path:

/psp/usr/bin

Installing Expat

The Expat library is required by the XML parsing Perl modules.

  • Download, extract, and install expat:
[sbox-Chumby: ~] >
wget http://downloads.sourceforge.net/expat/expat-2.0.1.tar.gz?modtime=1181083143&big_mirror=0
tar xvzf expat-2.0.1.tar.gz
cd expat-2.0.1
./configure --prefix=/psp/usr
make && make install
cd ~

Installing GD

We need to install GD and some of its dependencies in the /usr too, since some steps in the CPAN install fail if the default library is not found.

We did not add support for the Fontconfig library.


GD needs libpng, libjpeg, and freetype (for GD::Graph).


  • Download, extract, and install zlib (needs to be installed in both targets):
[sbox-Chumby: ~] >
wget http://www.zlib.net/zlib-1.2.3.tar.gz
tar xvzf zlib-1.2.3.tar.gz
cd zlib-1.2.3
./configure
make && make install
./configure --prefix=/psp/usr
make && make install
./configure --shared --prefix=/psp/usr
make && make install
cd ..


  • Download, extract, and install libpng:
[sbox-Chumby: ~] >
wget http://prdownloads.sourceforge.net/libpng/libpng-1.2.39.tar.gz?download
tar xvzf libpng-1.2.39.tar.gz
cd libpng-1.2.39
./configure --prefix=/psp/usr
make && make install
cd ..


  • Download, extract, and install libjpeg (the URL will change, check it out first in a browser):
[sbox-Chumby: ~] >
wget http://site.n.ml.org/download/19980327233823/libjpeg/libjpeg-6b.tar.gz
tar xvzf libjpeg-6b.tar.gz
cd jpeg-6b
mkdir -p /usr/man/man1
./configure --enable-shared --prefix=/usr
make && make install
./configure --enable-shared --prefix=/psp/usr
make && make install
cd ..


  • Download, extract, and install freetype:
[sbox-Chumby: ~] >
wget 'http://downloads.sourceforge.net/freetype/freetype-2.3.11.tar.gz?modtime=1214734497&big_mirror=0'
tar xvzf freetype-2.3.11.tar.gz
cd freetype-2.3.11
./configure --prefix=/usr
make && make install
./configure --prefix=/psp/usr
make && make install
cd ..


  • Download, extract, and install GD. We need to install it both in the default /usr location and in the intended target:
[sbox-Chumby: ~] >
wget http://www.libgd.org/releases/gd-2.0.35.tar.gz
tar xvzf gd-2.0.35.tar.gz
cd gd-2.0.35
./configure --prefix=/usr
make && make install
./configure --prefix=/psp/usr
make && make install
cd ..
  • Download, extract, and install libgmp:
[sbox-Chumby: ~] >
wget ftp://ftp.gnu.org/gnu/gmp/gmp-4.3.1.tar.gz
tar xvzf gmp-4.3.1.tar.gz
cd gmp-4.3.1
./configure --prefix=/usr
make && make check && make install
./configure --prefix=/psp/usr
make && make install
cd ..
  • Download, extract, and install libpari:
[sbox-Chumby: ~] >
wget ftp://megrez.math.u-bordeaux.fr/pub/pari/unix/pari-2.3.4.tar.gz
tar xvzf pari-2.3.4.tar.gz
cd pari-2.3.4
./Configure --prefix=/usr
make && make install
./Configure --prefix=/psp/usr
make && make install
cd ..

Installing CPAN modules in the ARM Perl

If we ever need to reinitialize the CPAN configuration we can use:

o conf init connect_to_internet_ok urllist

Making sure that we did initialize the CPAN module list in the user home directory (see first step), we can now start installing modules in the ARM Perl. For all modules that use threading, you need to force the install.

[sbox-Chumby: ~/perl/perl-5.10.1] >
/psp/usr/bin/perl -MCPAN -e shell
 
cpan>
install Bundle::CPAN
reload CPAN // you can also exit and restart the CPAN here to avoid redefined warnings

force install Time::HiRes LWP::UserAgent Net::Daemon MIME::Lite Bundle::DBI Test::Simple

install Expect Test::Pod::Coverage Log::Dispatch::FileRotate Test::Exception File::Tail

install DBD::CSV Compress::Bzip2 String::CRC32 Mail::Sender Tie::IxHash 

install Math::BigInt Math::BigInt::FastCalc Math::BigRat

If planning to use Sys::Info, it has to be forced because of some broken tests.

cpan>
force install Linux::Distribution Sys::Info

We can also install some crypt modules.

Math::BigInt is used by most crypt modules. After installing it, it is recommended to upgrade a couple dependencies.

Crypt::DSA needs the Convert::ASN1, that has broken test code, so we need to force install it. Also the order of installing dependencies seems to be locking some out, so we will install them explicitly. When installing the Crypt::DSA, the t/03-keygen is taking a very long time (over an hour).

cpan>
install Math::BigInt Math::BigRat Math::BigInt::FastCalc

force install Convert::ASN1

If installing Math::Pari fails, drop to command line and install it manually. It would most likely complain that the GP/PARI source could not be downloaded, so it has to be specified manually

install Math::Pari
exit
[sbox-Chumby: ~/perl/perl-5.10.1] >
cd /home/user/.cpan/build/Math-Pari-2.010801
/psp/usr/bin/perl Makefile.PL  paridir=/home/user/Chumby/pari-2.3.4
make && make install
cd ~
cpan>
install Class::ErrorHandler Crypt::DES_EDE3 Convert::PEM Term::Gnuplot 
// This works in 2008q3, but fails in the 2009q1, so you might need to install it by hand.
// See the procedure used for XML::Parser below
install Math::GMP

force install Crypt::Primes Crypt::DSA

install Crypt::IDEA Crypt::CBC Crypt::Rijndael Crypt::Twofish Crypt::Blowfish Crypt::Blowfish_PP

To install XML modules, we have to first get the XML::Parser. This will not build at first, but it will extract in the .cpan/build. After that, we exit the CPAN shell, and configure it manually:

cpan>
install XML::Parser
exit
[sbox-Chumby: ~/perl/perl-5.10.1] >
cd /home/user/.cpan/build/XML-Parser-2.36
/psp/usr/bin/perl Makefile.PL EXPATLIBPATH=/psp/usr/lib EXPATINCPATH=/psp/usr/include
make && make install
cd ~

After installing XML::Parser manually, you can resume installing other XML modules. There are also some other modules that could prove useful (some are prerequisites that need to be force-installed).

Respect the order the modules are installed here.

[sbox-Chumby: ~] >
/psp/usr/bin/perl -MCPAN -e shell

cpan>
install IO::Scalar Text::Iconv Unicode::String Unicode::Map8 HTML::TreeBuilder
install XML::SAX XML::SAX::Writer XML::Simple Pod::POM

force install XML::XPathEngine

install XML::Handler::YAWriter XML::DOM XML::Writer
install Image::Info Image::Size Image::ExifTool
install Template Test::Differences Text::RecordParser Graph::Directed Bit::Vector
install XML::XPath XML::Twig XML::XSLT
  • work in progress*
// This will also have to be installed by hand
install XML::LibXML XML::LibXSLT
exit
[sbox-Chumby: ~/perl/perl-5.10.1] >
cd ../..
wget http://xmlsoft.org/sources/libxml2-2.7.6.tar.gz
tar zxf libxml2-2.7.6.tar.gz
cd libxml2-2.7.6
./configure --prefix=/psp/usr
make && make install
cd /home/user/.cpan/build/XML-LibXML-1.70
/psp/usr/bin/perl Makefile.PL LIBS='-L/psp/usr/lib' INC='-I/psp/usr/include'
  • end of work in progress*

We continue now with the install from CPAN:

force install SQL::Translator

install Spreadsheet::ParseExcel SQL::Translator::Parser::Excel Spreadsheet::WriteExcelXML 
install CGI Date::Calc ExtUtils::ParseXS ExtUtils::ModuleMaker
install File::Find::Rule File::Find::Rule::MP3Info MP3::Tag MP3::Find 
install GD GD::Graph

force install GD::Thumbnail

Install ImageMagick

  • Download, extract, and configure. This will also install the Perl module in the target build.
[sbox-Chumby: ~] >
wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.gz
tar xvzf ImageMagick.tar.gz
cd ImageMagick-6.5.7-2
./configure --prefix=/psp/usr --with-perl=/psp/usr/bin/perl

At the end of configure, you should be seeing:

Options used to compile and link:
 PREFIX          = /psp/usr
 EXEC-PREFIX     = /psp/usr
 VERSION         = 6.5.7
 CC              = gcc -std=gnu99
 CFLAGS          = -g -O2 -Wall -W -pthread
 MAGICK_CFLAGS   = -g -O2 -Wall -W -pthread
 CPPFLAGS        = -I/psp/usr/include/ImageMagick
 PCFLAGS         =
 DEFS            = -DHAVE_CONFIG_H
 LDFLAGS         =
 MAGICK_LDFLAGS  = -L/psp/usr/lib
 LIBS            = -lMagickCore -lfreetype -ljpeg -lz -lm -lpthread
 CXX             = g++
 CXXFLAGS        = -g -O2 -Wall -W -pthread


Now run make and install:

[sbox-Chumby: ~/ImageMagick-6.5.7-2] >
make && make install
cd ..

Installing id3v2

  • Change to your home directory
  • Download and extract id3v2:
 [sbox-Chumby: ~] >
 wget http://softlayer.dl.sourceforge.net/sourceforge/id3lib/id3lib-3.8.3.tar.gz
 tar xvzf id3lib-3.8.3.tar.gz
 cd id3lib-3.8.3
  • Patch the following files:

configure

Look for iomanip.h, replace it with iomanip


In all the modules listed below, look for the following line:

 int main( unsigned int argc, char * const argv[])

replace it with:

 int main( int argc, char * const argv[])

Also, you need to add before the specified methods a prototype:

examples/demo_convert.cpp

void PrintUsage(const char *sName);
void PrintVersion(const char *sName);

examples/demo_copy.cpp

void PrintUsage(const char *sName);
void PrintVersion(const char *sName);
void DisplayTags(ostream &os, luint nTags);

examples/demo_info.cpp

void PrintUsage(const char *sName);
void PrintVersion(const char *sName);
void PrintInformation(const ID3_Tag &myTag);

examples/demo_tag.cpp

void PrintUsage(const char *sName);
void PrintVersion(const char *sName);
void DisplayTags(ostream &os, luint nTags);


Some other edits:

include/id3/id3lib_strings.h

Add include:

 #include <string.h>


include/id3/writers.h

Uncomment existing include:

 #include <string.h>


  • Configure and build:
 ./configure --prefix=/psp/usr
 make && make install
 cd ..

Install command-line id3v2 tool

  • Change to your home directory
  • Download and extract id3v2:
 [sbox-Chumby: ~] >
 wget http://softlayer.dl.sourceforge.net/sourceforge/id3v2/id3v2-0.1.11.tar.gz
 tar xvzf id3v2-0.1.11.tar.gz
 cd id3v2-0.1.11
 make PREFIX=/psp/usr
 cp ./id3v2 /psp/usr/bin

To test the built binary:

 LD_LIBRARY_PATH=/psp/usr/lib ./id3v2

Installing vim

  • Change to your home directory
  • Download and extract ctags:
[sbox-Chumby: ~] >
wget http://prdownloads.sourceforge.net/ctags/ctags-5.8.tar.gz
tar xvzf ctags-5.8.tar.gz
  • Change to ctags directory and configure ctags, then compile and install it:
[sbox-Chumby: ~] >
cd ctags-5.8
./configure --prefix=/psp/usr
make && make install
cd ..
  • Download and extract ncurses:
 [sbox-Chumby: ~] >
wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.7.tar.gz
tar xvzf ncurses-5.7.tar.gz
  • Change to ncurses directory and configure ncurses, then compile and install it in both local and staging:
 [sbox-Chumby: ~] >
cd ncurses-5.7
./configure --prefix=/usr/local
make && make install
./configure --prefix=/psp/usr
make && make install
  • Change back to home directory, get and extract vim:
[sbox-Chumby: ~/ncurses-5.7] >
cd ..
wget ftp://ftp.vim.org/pub/vim/unix/vim-7.2.tar.bz2
tar xvjf vim-7.2.tar.bz2
  • Change to vim directory and configure and install vim:
[sbox-Chumby: ~] >
cd vim72
CFLAGS="-I/psp/usr/include" LDFLAGS="-L/psp/usr/lib" ./configure --prefix=/psp/usr --with-tlib=ncurses
make && make install


If you want reasonable defaults for vim, try putting these files into a file called vimrc on the root of the flash drive. This will be a template file that will be used by the Chumby evironment setup script to copy it in the home directory:


if $TERM == "vt100"
  set term=vt220
endif
set background=dark "A background for darkness, as all of my windows are blue"
if has("gui_running")
  set background=dark
endif
if has("syntax")
  exec "syntax on"
  augroup syntax  " doesnt work right now - need to find out from archives
  au  BufNewFile,BufReadPost * hi link Todo Error
  augroup END
endif
set cindent
set autoindent
set expandtab
set ignorecase
set ruler
set tabstop=4       "Make an indentation level every four columns"
set expandtab       "Convert all tabs typed into spaces"
set shiftwidth=4    "Indent/Outdent by four columns"
set shiftround      "Always indent/outdent to the nearest tabstop"
set laststatus=2    "Have two lines of status, including current filename"
set tw=75           "Automatically wrap at 75 characters"
set cinkeys=0{,0},!^F,o,O,e "default is: 0{,0},0),:,0#,!^F,o,O,e"   "This fixes perl comments"
set incsearch

Installing GNU Debugger

  • Change to your home directory
  • get and install gdb:
[sbox-Chumby: ~] >
wget ftp://mirrors.kernel.org/gnu/gdb/gdb-7.0.tar.gz
tar xvzf gdb-7.0.tar.gz
cd gdb-7.0
./configure --prefix=/psp/usr
make && make install
cd ..
  • If we are getting errors pointing to XML components, we should re-install expat, then retry:
[sbox-Chumby: ~] >
cd ../expat-2.0.1
./configure --prefix=/psp/usr
make && make install
cd ../gdb-7.0
./configure --prefix=/psp/usr
make && make install
cd ..

Installing Emacs

In the interest of fairness, Emacs is also relatively easy to compile on the Chumby.

  • Download and extract Emacs:
[sbox-Chumby: ~] >
wget http://mirrors.usc.edu/pub/gnu/emacs/emacs-22.2.tar.gz
tar xvzf emacs-22.2.tar.gz
  • Change to emacs directory and configure Emacs:
[sbox-Chumby: ~] >
cd emacs-22.2
./configure --prefix=/psp/usr
  • Emacs looks for the C runtime object files in /usr/lib/, but in Scratchbox they reside in /lib/. Create symlinks to get this working:
[sbox-Chumby: ~/emacs-22.2] >
ln -s /lib/crt* /usr/lib/
  • Compile and install Emacs:
[sbox-Chumby: ~/emacs-22.2] >
make && make install

Installing Subversion

To begin with, make sure you have expat installed, as mentioned in the steps for installing the Expat perl module.

  • Download, unpack, and configure zlib:
[sbox-Chumby: ~] >
wget http://www.zlib.net/zlib-1.2.3.tar.gz
tar xvzf zlib-1.2.3.tar.gz
cd zlib-1.2.3
./configure --prefix=/usr
make && make install
cd ..
  • Get and install openssl:
[sbox-Chumby: ~] >
wget http://www.openssl.org/source/openssl-0.9.8k.tar.gz
tar xvzf openssl-0.9.8k.tar.gz
cd openssl-0.9.8k
PERL=/psp/usr/bin/perl ./config --prefix=/usr
make && make install
cd ..
  • Get and install SQLite
[sbox-Chumby: ] >
wget http://www.sqlite.org/sqlite-amalgamation-3.6.19.tar.gz
tar xvzf sqlite-amalgamation-3.6.19.tar.gz
cd sqlite-3.6.19
./configure --prefix=/usr
make && make install
./configure --prefix=/psp/usr
make && make install
cd ..
  • Subversion uses mawk, which doesn't exist. Create a shortcut to awk:
ln -s /scratchbox/tools/bin/awk /bin/mawk
  • Download and unpack the subversion source:
[sbox-Chumby: ~] >
wget http://subversion.tigris.org/downloads/subversion-1.6.6.tar.gz
tar xvzf subversion-1.6.6.tar.gz
cd subversion-1.6.6
  • In the host, check out the latest versions of apr and apr-util:
user@ubuntu:~$
cd /scratchbox/users/user/home/user/subversion-1.6.6/
svn co http://svn.apache.org/repos/asf/apr/apr/branches/1.4.x apr
svn co http://svn.apache.org/repos/asf/apr/apr-util/branches/1.4.x apr-util

  • Back in the target, configure the apr and apr-util:
[sbox-Chumby: ~/subversion-1.6.6] >
cd apr
./buildconf
cd ..
cd apr-util
./buildconf
cd ..
  • Get neon, which is used to access http and https servers:
[sbox-Chumby: ~/subversion-1.6.6] >
wget http://www.webdav.org/neon/neon-0.29.0.tar.gz
tar xvzf neon-0.29.0.tar.gz
mv neon-0.29.0 neon
  • Configure, compile, and install subversion:
[sbox-Chumby: ~/subversion-1.6.6] >
mkdir sqlite-amalgamation
cp ../sqlite-3.6.19/sqlite3.c sqlite-amalgamation/
./configure --prefix=/psp/usr --with-ssl
make && make install

A warning about SVN being built without the Berkeley DB is expected.

Installing GNU Screen

The process of getting gnu screen installed is almost identical to the process of getting vim working:

  • Change to your home directory
  • Get and install ncurses:
[sbox-Chumby: ~] >
wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.7.tar.gz
tar xvzf ncurses-5.7.tar.gz
cd ncurses-5.7
./configure --prefix=/psp/usr
make && make install
cd ..
  • Get, extract, and configure screen:
[sbox-Chumby: ] >
wget http://ftp.gnu.org/gnu/screen/screen-4.0.3.tar.gz
tar xvzf screen-4.0.3.tar.gz
cd screen-4.0.3
CFLAGS="-I/psp/usr/include" LDFLAGS="-L/psp/usr/lib" ./configure --prefix=/psp/usr --with-tlib=ncurses
  • Compile and install screen:
[sbox-Chumby: ~/screen-4.0.3] >
make && make install
mkdir -p /psp/usr/etc
cp ./etc/etcscreenrc /psp/usr/etc/screenrc
# NOTE: termcap entry (./terminfo/screencap) should be installed manually.
cd ..

Installing tcpdump

  • Configure and compile pcap:
[sbox-Chumby: ~] >
wget http://www.tcpdump.org/release/libpcap-1.0.0.tar.gz
tar xvzf libpcap-1.0.0.tar.gz
cd libpcap-1.0.0
./configure --with-pcap=linux --prefix=/psp/usr
make && make install
cd ..
  • Configure, compile, and install tcpdump:
[sbox-Chumby: ~] >
wget http://www.tcpdump.org/release/tcpdump-4.0.0.tar.gz
tar xvzf tcpdump-4.0.0.tar.gz
cd tcpdump-4.0.0
LDFLAGS="-ldl" ./configure --prefix=/psp/usr
make && make install
cd ..

Installing GNU Netcat

The device already has Netcat, this is provided as an alternative.

http://www.catonmat.net/blog/unix-utilities-netcat/

[sbox-Chumby: ~] >
wget http://internap.dl.sourceforge.net/sourceforge/netcat/netcat-0.7.1.tar.gz
tar xvzf netcat-0.7.1.tar.gz
cd netcat-0.7.1
./configure --prefix=/psp/usr
make && make install

Installing NMap

http://nmap.org/

[sbox-Chumby: ~] >
wget http://nmap.org/dist/nmap-5.00.tgz
tar xvzf nmap-5.00.tgz
cd nmap-5.00
./configure --prefix=/psp/usr
make && make install

Installing Pipe Viewer

http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

wget http://pipeviewer.googlecode.com/files/pv-1.1.4.tar.bz2
bunzip2 pv-1.1.4.tar.bz2
tar xf pv-1.1.4.tar
cd pv-1.1.4
./configure --prefix=/psp/usr
make && make install
cd ..

Installing Lighttpd

http://wiki.dreamhost.com/Lighttpd_From_Source

Install first PCRE:

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.00.tar.gz
tar zxf pcre-8.00.tar.gz
cd pcre-8.00
./configure --prefix=/psp/usr
make && make install
cd ..

Now download and install Lighttpd:

wget http://www.lighttpd.net/download/lighttpd-1.4.23.tar.gz
tar zxf lighttpd-1.4.23.tar.gz
cd lighttpd-1.4.23
export PATH=/psp/usr/bin:$PATH
./configure --without-bzip2 --prefix=/psp/usr
make && make install

Now to copy over the files

mkdir /psp/usr/etc
mkdir /psp/usr/etc/init.d
mkdir /psp/usr/etc/lighttpd
mkdir /psp/usr/etc/sysconfig
sed -e 's/FOO/lighttpd/g' \
    -e 's/\/etc/\/psp\/usr\/etc/g' \
    -e 's/\/usr\/sbin/\/psp\/usr\/sbin/g' \
    doc/rc.lighttpd > /psp/usr/etc/init.d/lighttpd
chmod a+rx /psp/usr/etc/init.d/lighttpd

sed -e 's/\/etc/\/psp\/usr\/etc/g' \
  doc/sysconfig.lighttpd > /psp/usr/etc/sysconfig/lighttpd

sed -e 's/\/srv\/www/\/mnt\/usb\/www/g' \
    -e 's/\/var\/log\/lighttpd/\/mnt\/usb\/www\/log/g' \
    ./doc/lighttpd.conf > /psp/usr/etc/lighttpd/lighttpd.conf

When you configure the server to run off the USB you have to create the folders:

mkdir -p /mnt/usb/www/htdocs
mkdir -p /mnt/usb/www/errors
mkdir -p /mnt/usb/www/vhosts
mkdir -p /mnt/usb/www/log

Lighttpd has a nice utility to make sure our config file has no errors in it.

/psp/usr/sbin/lighttpd -t -f /psp/usr/etc/lighttpd/lighttpd.conf

Expect the following answer:

Syntax OK

You need to edit the lighttpd.conf config file and define the port the lighttp will be running (around line 140):

server.port = 8080

Testing the first run:

/psp/usr/sbin/lighttpd -D -f /psp/usr/etc/lighttpd/lighttpd.conf

This will start lighttpd for testing. The -D switch tells lighttpd not to go to the background like a daemon normally would. You should be able to hit http://DEVICE_IP:8080/ now and get a 404 - Not Found error page (because there's nothing in your doc root)

To use it as a daemon, you can start it as follows:

/psp/usr/sbin/lighttpd -f /psp/usr/etc/lighttpd/lighttpd.conf

You can also use the start script that was set up for you:

/psp/usr/etc/init.d/lighttpd start


Sychronizing the local environment with the staging

  • Make a copy of the package config, to make the package config database work.

This should be performed after installing any package, to make the local copy aware of the installed packages.

[sbox-Chumby: ~] >
mkdir -p /usr/local/lib/pkgconfig
cp /psp/usr/lib/pkgconfig/* /usr/local/lib/pkgconfig

Transferring applications and data to your Chumby

Packing Data in Scratchbox

After compiling all tools, we can tar/gz the /psp/usr and copy it to target.

Since the USB drives are FAT32, we tar it so as to resolve symlinks to real files. This will indeed result in a larger archive.

 [sbox-Chumby: ~] >
 cd /psp
 tar -zcf usr_arm_ext3.tgz usr
 tar -zchf usr_arm_fat32.tgz usr

Unpacking Data on Chumby

Since most of the USB drives are formatted as FAT32, the symlinks will not be created. For those drives, make sure the data was packed with the h param. There will be files instead of the symlinks in the tar file:

tar -zxf usr_arm_fat32.tgz

Check for the existence of /mnt/usb/usr/bin/perl. If it does not exist, make a copy of the current binary:

cp /mnt/usb/usr/bin/perl5.10.1 /mnt/usb/usr/bin/perl

Symlink the unpacked folder in /psp

ln -s /mnt/usb/usr /psp/usr

On Ironforge, this symlink will be destroyed every time when the device is reimaged, so you will need to create it again.

Environment setup script

Here is a simple script that can be used to give you a rather nice development environment on the Chumby.

We named the file vimrc with no leading dot because cp would skip it.

Save the following in the file: /mnt/usb/setdev

#!/bin/sh
if [ ! -e /psp/usr ]; then
  if [ -e /mnt/storage/usr ]; then
    ln -s /mnt/storage/usr /psp/usr
  else
    if [ -e /mnt/usb/usr ]; then
      ln -s /mnt/usb/usr /psp/usr
    else
      echo "Could not find /mnt/storage/usr or /mnt/usb/usr"
      exit 1
    fi
  fi
fi
if [ ! -f /psp/usr/bin/perl ]; then
  if [ -f /psp/usr/bin/perl5.10.1 ]; then
    # We run this from the USB as FAT32 and Perl was not correctly unpacked
    echo "Making a working copy of Perl as /psp/usr/bin/perl"
    cp /psp/usr/bin/perl5.10.1 /psp/usr/bin/perl
    sync
  else
    echo "Could not find perl in /psp/usr/bin"
    exit 1
  fi
fi
if [ ! -e /tmp/home ]; then
  mkdir /tmp/home
fi
export HOME=/tmp/home
export PATH=/psp/usr/sbin:/psp/usr/bin:$PATH
export LD_LIBRARY_PATH=/psp/usr/lib:$LD_LIBRARY_PATH
export PS1="\[\e[36;1m\]\u\[\e[33;1m\]:\[\e[0m\]\w\[\e[32;1m\] > \[\e[0m\]"
export TERM='ansi'
export PERL_PATH=/psp/usr/bin/perl
if [ -e /psp/usr/vimrc ]; then
  cp /psp/usr/vimrc /tmp/home/.vimrc
fi

You can now use it as:

source setdev

Or you can append this to the script, to open a new shell:

exec /bin/sh

Get it already built

For the impatient, you can get here the packed environment described below.