[distcc] Crosscompiling with distcc on Gentoo

[m***@gmx.de]

Hi,

My environment:
Two computers:
a) Desktop PC running Gentoo Linux on a AMD Phenom(tm) II X6 1090T Processor
b) A beaglebone single board computer running Gentoo Linux on a ARMv7 Processor rev 2 (v7l)

a) has the IP-address 192.168.178.21
b) has the IP-address 192.168.178.25

The task:
I want to emerge packages (that is compiling them from source by
using Gentoo package manager) via "pump emerge".

The problem:
It does not work.... ;/

I followed different howtos from different parts of this century ;)
googled from the net how to accomplish this...but no luck.
Often there was no clear distinction between hot and server.
I also reinstalled distcc several times.

My current setup is as follows:
I build a cross compiler toolchain on a), the server, named
armv7a-softfp-linux-gnueabi-*

On the beaglebone

ls -l /usr/lib/distcc/bin

shows

lrwxrwxrwx 1 root root 15 2012-10-15 19:21 armv7a-softfp-linux-gnueabi-c++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 2012-10-15 19:21 armv7a-softfp-linux-gnueabi-g++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 2012-10-15 19:21 armv7a-softfp-linux-gnueabi-gcc -> /usr/bin/distcc
-rwxr-xr-x 1 root root 83 2012-10-14 15:11 armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:38 c++ -> armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:39 cc -> armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:39 g++ -> armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:39 gcc -> armv7a-softfp-linux-gnueabi-wrapper

and

armv7a-softfp-linux-gnueabi-wrapper

has the contents:

#!/bin/bash
exec /usr/lib/distcc/bin/armv7a-softfp-linux-gnueabi-g${0:$[-2]} "$@"


On that machine, the server I started distccd with

/usr/bin/distccd --daemon --pid-file /var/run/distccd/distccd.pid --port 3632 --log-file=/tmp/NFS/distcc.solfire.log --log-level critical --allow 192.168.178.25 -N 15

(the logfile is placed in a directory, which is nfs-mounted on machine
"b" so I have access to the log from both nmachines)


On the beaglebone, machine "b", the client, the command

distcc-config -get-host

will print

192.168.178.21,cpp,lzo

and

distcc-config --get-log

will print

DISTCC_LOG=/tmp/distcc.log

Then I wrote a little C-program called test.c and did a

pump make test

, which lead to the following contents of distcc.log:

distcc[15967] (dcc_trace_version) distcc 3.1 armv7a-softfp-linux-gnueabi; built Oct 15 2012 19:19:29
distcc[15967] (dcc_recursion_safeguard) safeguard level=0
distcc[15967] (main) compiler name is "distcc"
distcc[15967] (dcc_get_hostlist) not reading /root/.distcc/hosts: No such file or directory
distcc[15967] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[15967] (dcc_parse_hosts) found tcp token "192.168.178.21,cpp,lzo"
distcc[15967] (dcc_parse_options) got CPP option
distcc[15967] (dcc_parse_options) got LZO option
distcc[15967] (dcc_exit) exit: code 0; self: 0.000000 user 0.010000 sys; children: 0.000000 user 0.000000 sys
distcc[15967] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files
distcc[15990] (dcc_trace_version) distcc 3.1 armv7a-softfp-linux-gnueabi; built Oct 15 2012 19:19:29
distcc[15990] (dcc_recursion_safeguard) safeguard level=0
distcc[15990] (main) compiler name is "armv7a-softfp-linux-gnueabi-gcc"
distcc[15990] (dcc_set_path) setting PATH=/usr/lib/distcc/bin:/usr/local/bin:/bin/:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/armv7a-softfp-linux-gnueabi/gcc-bin/4.5.4:/root/bin:/root/bin
distcc[15990] (dcc_scan_args) scanning arguments: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15990] (dcc_scan_args) found input file "test.c"
distcc[15990] (dcc_scan_args) found object/output file "test"
distcc[15990] (dcc_scan_args) compiler apparently called not for compile
distcc[15990] (dcc_lock_host) got cpu lock on localhost slot 0 as fd4
distcc[15990] exec on localhost: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15990] (dcc_note_state) note state 5, file "test.c", host "localhost"
distcc[15990] (dcc_spawn_child) forking to execute: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15990] (dcc_spawn_child) child started as pid15991
distcc[15991] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[15991] (dcc_trace_version) distcc 3.1 armv7a-softfp-linux-gnueabi; built Oct 15 2012 19:19:29
distcc[15991] (dcc_recursion_safeguard) safeguard: 1
distcc[15991] (dcc_recursion_safeguard) safeguard level=1
distcc[15991] (main) compiler name is "armv7a-softfp-linux-gnueabi-gcc"
distcc[15991] (dcc_set_path) setting PATH=/usr/local/bin:/bin/:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/armv7a-softfp-linux-gnueabi/gcc-bin/4.5.4:/root/bin:/root/bin
distcc[15991] exec on localhost: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15991] (dcc_note_state) note state 5, file "(NULL)", host "localhost"
distcc[15991] (dcc_spawn_child) forking to execute: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15991] (dcc_spawn_child) child started as pid15992
distcc[15992] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=2
distcc[15991] (dcc_collect_child) cc child 15992 terminated with status 0
distcc[15991] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[15991] compile (null) on localhost completed ok
distcc[15991] elapsed compilation time 0.443756s
distcc[15991] (dcc_exit) exit: code 0; self: 0.000000 user 0.000000 sys; children: 0.280000 user 0.120000 sys
distcc[15991] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files
distcc[15990] (dcc_collect_child) cc child 15991 terminated with status 0
distcc[15990] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[15990] compile test.c on localhost completed ok
distcc[15990] (dcc_unlock) release lock fd4
distcc[15990] elapsed compilation time 0.626678s
distcc[15990] (dcc_exit) exit: code 0; self: 0.000000 user 0.010000 sys; children: 0.280000 user 0.120000 sys
distcc[15990] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files


Using nmap on the beaglebone with

nmap 192.168.178.21 -p 10-32000 gives

prints:

Starting Nmap 5.51 ( http://nmap.org ) at 2012-10-15 20:06 CEST
Nmap scan report for noname (192.168.178.21)
Host is up (0.00019s latency).
Not shown: 31987 closed ports
PORT STATE SERVICE
22/tcp open ssh
111/tcp open rpcbind
2049/tcp open nfs
3632/tcp open distccd
MAC Address: 00:15:F2:18:B0:20 (Asustek Computer)

therefore distcc is listening on the PC but does not react in any way
-- eberything was compiled on the beaglebone (localhost).

How can I convince the distccd on the PC to get its job done?

Thank you very much in advance for any help!

Best regards,
mcc

[Fergus Henderson]

Post by m***@gmx.de
Hi,
a) Desktop PC running Gentoo Linux on a AMD Phenom(tm) II X6 1090T Processor
b) A beaglebone single board computer running Gentoo Linux on a ARMv7
Processor rev 2 (v7l)
a) has the IP-address 192.168.178.21
b) has the IP-address 192.168.178.25
I want to emerge packages (that is compiling them from source by
using Gentoo package manager) via "pump emerge".
It does not work.... ;/
I followed different howtos from different parts of this century ;)
googled from the net how to accomplish this...but no luck.
Often there was no clear distinction between hot and server.
I also reinstalled distcc several times.
I build a cross compiler toolchain on a), the server, named
armv7a-softfp-linux-gnueabi-*
On the beaglebone
ls -l /usr/lib/distcc/bin
shows
lrwxrwxrwx 1 root root 15 2012-10-15 19:21
armv7a-softfp-linux-gnueabi-c++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 2012-10-15 19:21
armv7a-softfp-linux-gnueabi-g++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 2012-10-15 19:21
armv7a-softfp-linux-gnueabi-gcc -> /usr/bin/distcc
-rwxr-xr-x 1 root root 83 2012-10-14 15:11
armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:38 c++ ->
armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:39 cc ->
armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:39 g++ ->
armv7a-softfp-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 35 2012-10-15 19:39 gcc ->
armv7a-softfp-linux-gnueabi-wrapper
and
armv7a-softfp-linux-gnueabi-wrapper
#!/bin/bash
On that machine, the server I started distccd with
/usr/bin/distccd --daemon --pid-file /var/run/distccd/distccd.pid
--port 3632 --log-file=/tmp/NFS/distcc.solfire.log --log-level critical
--allow 192.168.178.25 -N 15
(the logfile is placed in a directory, which is nfs-mounted on machine
"b" so I have access to the log from both nmachines)
On the beaglebone, machine "b", the client, the command
distcc-config -get-host
will print
192.168.178.21,cpp,lzo
and
distcc-config --get-log
will print
DISTCC_LOG=/tmp/distcc.log
Then I wrote a little C-program called test.c and did a
pump make test
distcc[15967] (dcc_trace_version) distcc 3.1 armv7a-softfp-linux-gnueabi;
built Oct 15 2012 19:19:29
distcc[15967] (dcc_recursion_safeguard) safeguard level=0
distcc[15967] (main) compiler name is "distcc"
distcc[15967] (dcc_get_hostlist) not reading /root/.distcc/hosts: No such file or directory
distcc[15967] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[15967] (dcc_parse_hosts) found tcp token "192.168.178.21,cpp,lzo"
distcc[15967] (dcc_parse_options) got CPP option
distcc[15967] (dcc_parse_options) got LZO option
distcc[15967] (dcc_exit) exit: code 0; self: 0.000000 user 0.010000 sys;
children: 0.000000 user 0.000000 sys

The lines above indicate that you don't have "localhost" in your ,cpp,zlo"

Post by m***@gmx.de
distcc[15967] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files
distcc[15990] (dcc_trace_version) distcc 3.1 armv7a-softfp-linux-gnueabi;
built Oct 15 2012 19:19:29
distcc[15990] (dcc_recursion_safeguard) safeguard level=0
distcc[15990] (main) compiler name is "armv7a-softfp-linux-gnueabi-gcc"
distcc[15990] (dcc_set_path) setting
PATH=/usr/lib/distcc/bin:/usr/local/bin:/bin/:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/armv7a-softfp-linux-gnueabi/gcc-bin/4.5.4:/root/bin:/root/bin
armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16
-mfloat-abi=softfp test.c -o test
distcc[15990] (dcc_scan_args) found input file "test.c"
distcc[15990] (dcc_scan_args) found object/output file "test"
distcc[15990] (dcc_scan_args) compiler apparently called not for compile

That is the critical line which explains why this particular compiler
invocation was not distributed to a remote host. Your makefile ran the
command "$(CC) $(CFLAGS) test.c -o test", which does two things:
compilation *and* linking. Distcc does not attempt to distribute linking,
it will only distribute compilation. If you do compilation and linking in
a single command, distcc will not attempt to distribute that.

Instead of using a single command that combines compilation and linking

$(CC) $(CFLAGS) test.c -o test

you should use separate commands

$(CC) $(CFLAGS) -c test.c
$(CC) $(CFLAGS) test.o -o test

and then distcc should be able to distribute the compilation command (the
first of those two).

distcc[15990] (dcc_lock_host) got cpu lock on localhost slot 0 as fd4

Post by m***@gmx.de
distcc[15990] exec on localhost: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe
-march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15990] (dcc_note_state) note state 5, file "test.c", host "localhost"
armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16
-mfloat-abi=softfp test.c -o test
distcc[15990] (dcc_spawn_child) child started as pid15991
_DISTCC_SAFEGUARD=1
distcc[15991] (dcc_trace_version) distcc 3.1 armv7a-softfp-linux-gnueabi;
built Oct 15 2012 19:19:29
distcc[15991] (dcc_recursion_safeguard) safeguard: 1
distcc[15991] (dcc_recursion_safeguard) safeguard level=1
distcc[15991] (main) compiler name is "armv7a-softfp-linux-gnueabi-gcc"
distcc[15991] (dcc_set_path) setting
PATH=/usr/local/bin:/bin/:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/armv7a-softfp-linux-gnueabi/gcc-bin/4.5.4:/root/bin:/root/bin
distcc[15991] exec on localhost: armv7a-softfp-linux-gnueabi-gcc -O2 -pipe
-march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp test.c -o test
distcc[15991] (dcc_note_state) note state 5, file "(NULL)", host "localhost"
armv7a-softfp-linux-gnueabi-gcc -O2 -pipe -march=armv7-a -mfpu=vfpv3-d16
-mfloat-abi=softfp test.c -o test
distcc[15991] (dcc_spawn_child) child started as pid15992
_DISTCC_SAFEGUARD=2
distcc[15991] (dcc_collect_child) cc child 15992 terminated with status 0
distcc[15991] (dcc_collect_child) cc times: user 0.000000s, system
0.000000s, 0 minflt, 0 majflt
distcc[15991] compile (null) on localhost completed ok
distcc[15991] elapsed compilation time 0.443756s
distcc[15991] (dcc_exit) exit: code 0; self: 0.000000 user 0.000000 sys;
children: 0.280000 user 0.120000 sys
distcc[15991] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files
distcc[15990] (dcc_collect_child) cc child 15991 terminated with status 0
distcc[15990] (dcc_collect_child) cc times: user 0.000000s, system
0.000000s, 0 minflt, 0 majflt
distcc[15990] compile test.c on localhost completed ok
distcc[15990] (dcc_unlock) release lock fd4
distcc[15990] elapsed compilation time 0.626678s
distcc[15990] (dcc_exit) exit: code 0; self: 0.000000 user 0.010000 sys;
children: 0.280000 user 0.120000 sys
distcc[15990] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files
Using nmap on the beaglebone with
nmap 192.168.178.21 -p 10-32000 gives
Starting Nmap 5.51 ( http://nmap.org ) at 2012-10-15 20:06 CEST
Nmap scan report for noname (192.168.178.21)
Host is up (0.00019s latency).
Not shown: 31987 closed ports
PORT STATE SERVICE
22/tcp open ssh
111/tcp open rpcbind
2049/tcp open nfs
3632/tcp open distccd
MAC Address: 00:15:F2:18:B0:20 (Asustek Computer)
therefore distcc is listening on the PC but does not react in any way
-- eberything was compiled on the beaglebone (localhost).
How can I convince the distccd on the PC to get its job done?
Thank you very much in advance for any help!
Best regards,
mcc
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc