Distcc cross compiling for Raspbian

If you plan on compiling a lot of software for Raspbian and want to speed up
the process, the following guide will turn an x86 Linux computer into an
ARM cross compiler. It's also much easier than most cross compile setups.

1. Configure slaves

1.1 Acquiring cross toolchain

	You could probably use any working prebuilt cross toolchain for distcc
	compiling, but I built my own toolchain with crosstool-ng.
	
	Arch Linux:
	
		On Arch, crosstool-ng is in AUR (http://aur.archlinux.org/packages.php?ID=23246)
		so just use your favourite AUR manager to get it:

			yaourt -S crosstool-ng

	Debian based:
		
		crosstool-ng is not available from repositories so you'll have
		to build it manually. Refer to manuals like
		http://biffengineering.com/wiki/index.php?title=HowToSetupCrossCompileEnvironment
		Sections "1.2 Download" and "1.3 Setup/install crostool-ng"

	Now we'll build the cross toolchain. You can use the .config I put on
	pastebin, or just make your own config with 'ct-ng menuconfig'.
	Getting my .config from http://pastebin.com/Rh6tvh32 and building the toolchain:
	
		mkdir ~/cross-rasp
		cd ~/cross-rasp
		wget http://pastebin.com/raw.php?i=Rh6tvh32 -O .config
		ct-ng build

	After ct-ng is finished, you should have the directory ~/x-tools/arm-unknown-linux-gnueabi.
	The built toolchain is lacking tuple-less variants (cc, gcc, ..).
	I put up a script on pastebin to fix that problem, http://pastebin.com/NR7qiJA4 .

		cd ~/x-tools/arm-unknown-linux-gnueabi/bin
		wget http://pastebin.com/raw.php?i=NR7qiJA4 -O link
		chmod +x link
		./link

1.2 Configure distcc

	We'll need to install distcc and configure it on all slaves doing the
	compilation.

	Arch Linux:
		
		Install distcc:

			Pacman -S distcc
		
		In "/etc/conf.d/distccd", you need to change DISTCC_ARGS to allow the
		network or invidual hosts to connect, for example allowing my
		whole LAN:
	
			DISTCC_ARGS="--user nobody --allow 192.168.0.0/24"

		Edit "/etc/rc.d/distccd", Add this line after sourcing of the
		functions file. Remember to change your_user accordingly.

			PATH=/home/your_user/x-tools/arm-unknown-linux-gnueabi/bin:$PATH
		
		Start the daemon:
		
			/etc/init.d/distccd start

		You probably want to autostart this in your rc.conf
		

	Debian Based:
		
		Install distcc:

			apt-get install distcc

		In /etc/default/distcc, you need to change ALLOWED_NETS to
		allow the network or invidual hosts to connect. 
		You also need to change STARTDISTCC to actually start it.

		Edit "/etc/init.d/distcc", change the PATH variable to include
		the cross toolchain path first,  for example on my debian
		machine it becomes (rembember to change "your_user"):
		
			PATH=/home/your_user/x-tools/arm-unknown-linux-gnueabi/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

		Start the daemon:
			
			/etc/init.d/distcc start

	All:
		Check how many processing units you have available with the 'nproc' command.
		Add the values from all the slaves together. This is going to
		be used later with 'make'


2. Configure Raspbian master:

	Install distcc:
		
		apt-get install distcc
	
	Edit "/etc/distcc/hosts"

	Comment out the line with +zeroconf. Add space-delimited list 
	of slaves on your network. I only have one PC doing the compiling 
	so I have a line with:

	192.168.0.101

	Which is the local IP-address for the Arch slave I have.


4. Compile

	When compiling we need to edit PATH so it uses distcc instead of the
	local compiler.

		export PATH="/usr/lib/distcc/:${PATH}"
	
	
	I put up a small "Hello World"-program on pastebin, http://pastebin.com/cMVyx3AR
	We can use this for testing:

		wget http://pastebin.com/raw.php?i=cMVyx3AR -O hello.c
	
	Now we'll compile it with a lot of verbosity so we'll see if it's
	working:
	
		DISTCC_VERBOSE=1 gcc -c hello.c -o hello.o

	Look for lines like "compile hello.c on 192.168.0.101 completed ok"
	
	That's it!

	You just need to export that path everytime you're compiling.

	When using make, use the total number of processing units available for the 
	-j paramater. This is the number you got with 'nproc' and added together earlier.
	Example for my one slave:
	
		make -j4

Notes

	There's also distcc-pump, which also distributes preprocessing. I
	haven't had any luck getting this to work on raspbian, so I'll leave
	this for those who want look into it further. Process should be
	something like this:

	Edit "/etc/distcc/hosts". Add ",cpp,lzo" after the hosts you want to
	do preprocessing, eg.:
	
		192.168.0.101,cpp,lzo

	Start distcc-pump:
		
		eval `distcc-pump --startup`

	Export the path:

		export PATH="/usr/lib/distcc/:${PATH}"

	Now you should do the compilation you want. After it is finished run:

		distcc-pump --shutdown

References

	http://www.gentoo.org/doc/en/distcc.xml
	http://archlinuxarm.org/developers/distcc-cross-compiling
	http://inkhornnet.wordpress.com/2011/06/18/debian-squeeze-distcc-pump-and-kernel-builds/

Contact

	Bane^ @ freenode