John
2012-12-20 00:11:59 UTC
I have two Arch Linux machines on a GigaLAN both running the distccd package from the distro which is properly configured. I can compile using just distcc without pump, but I cannot compile using pump and would love to figure out what's going wrong.
% systemctl status distccd.service
distccd.service - A distributed C/C++ compiler
Loaded: loaded (/usr/lib/systemd/system/distccd.service; disabled)
Active: active (running) since Wed, 2012-12-19 18:56:56 EST; 14min ago
Docs: man:distccd(1)
Main PID: 6081 (distccd)
CGroup: name=systemd:/system/distccd.service
├─6081 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6082 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6087 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6088 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6098 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6099 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6102 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6103 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6106 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6107 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
└─6110 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
1) Workstation (Intel i7-3770K) with 8 logical CPUs.
2) Laptop (Intel C2D) with 2 logical CPUs.
On my workstation:
% cat /etc/distcc/hosts
localhost/9,cpp,lzo
192.168.0.102/3,cpp,lzo
On my laptop:
% cat /etc/distcc/hosts
localhost/3,cpp,lzo
192.168.0.101/9,cpp,lzo
Just the workstation:
% make -j8 bzImage 496.11s user 25.67s system 712% cpu 1:13.26 total
Trying to use pump/distcc
On the laptop...
% eval pump --startup
On the workstation...
% eval pump --startup
% pump make -j12 bzImage CC=distcc
__________Using distcc-pump from /usr/bin
__________Using 2 distcc servers in pump mode
make[1]: Nothing to be done for `all'.
HOSTCC scripts/basic/fixdep
CHK include/generated/uapi/linux/version.h
...
CC arch/x86/xen/setup.o
CC arch/x86/realmode/rm/video-vga.o
CC arch/x86/mm/fault.o
CC arch/x86/realmode/rm/video-vesa.o
distcc[25252] ERROR: compile arch/x86/realmode/rm/video-vesa.c on 192.168.0.102/3,cpp,lzo failed
distcc[25252] (dcc_build_somewhere) Warning: remote compilation of 'arch/x86/realmode/rm/video-vesa.c' failed, retrying locally
distcc[25214] ERROR: compile arch/x86/realmode/rm/video-vga.c on 192.168.0.102/3,cpp,lzo failed
distcc[25214] (dcc_build_somewhere) Warning: remote compilation of 'arch/x86/realmode/rm/video-vga.c' failed, retrying locally
distcc[25252] Warning: failed to distribute arch/x86/realmode/rm/video-vesa.c to 192.168.0.102/3,cpp,lzo, running locally instead
distcc[25214] Warning: failed to distribute arch/x86/realmode/rm/video-vga.c to 192.168.0.102/3,cpp,lzo, running locally instead
CC mm/mempool.o
distcc[25252] (dcc_please_send_email_after_investigation) Warning: remote compilation of 'arch/x86/realmode/rm/video-vesa.c' failed, retried locally and got a different result.
distcc[25252] (dcc_note_discrepancy) Warning: now using plain distcc, possibly due to inconsistent file system changes during build
CC kernel/panic.o
distcc[25214] (dcc_please_send_email_after_investigation) Warning: remote compilation of 'arch/x86/realmode/rm/video-vga.c' failed, retried locally and got a different result.
CC arch/x86/realmode/rm/video-bios.o
CC arch/x86/xen/multicalls.o
PASYMS arch/x86/realmode/rm/pasyms.h
LDS arch/x86/realmode/rm/realmode.lds
LD arch/x86/realmode/rm/realmode.elf
RELOCS arch/x86/realmode/rm/realmode.relocs
OBJCOPY arch/x86/realmode/rm/realmode.bin
AS arch/x86/realmode/rmpiggy.o
...
OBJCOPY arch/x86/boot/setup.bin
BUILD arch/x86/boot/bzImage
Setup is 17036 bytes (padded to 17408 bytes).
System is 3429 kB
CRC fa62427
Kernel: arch/x86/boot/bzImage is ready (#10)
__________Warning: 3 pump-mode compilation(s) failed on server, but succeeded locally.
__________Distcc-pump was demoted to plain mode. See the Distcc Discrepancy Symptoms section in the include_server(1) man page.
__________Shutting down distcc-pump include server
pump make -j12 bzImage CC=distcc 373.40s user 18.70s system 455% cpu 1:26.10 total
% systemctl status distccd.service
distccd.service - A distributed C/C++ compiler
Loaded: loaded (/usr/lib/systemd/system/distccd.service; disabled)
Active: active (running) since Wed, 2012-12-19 18:56:56 EST; 14min ago
Docs: man:distccd(1)
Main PID: 6081 (distccd)
CGroup: name=systemd:/system/distccd.service
├─6081 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6082 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6087 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6088 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6098 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6099 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6102 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6103 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6106 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
├─6107 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
└─6110 /usr/bin/distccd --no-detach --daemon --allow 192.168.0.0/24 --log-level warning --log-file /tmp/distccd-x86_64.log
1) Workstation (Intel i7-3770K) with 8 logical CPUs.
2) Laptop (Intel C2D) with 2 logical CPUs.
On my workstation:
% cat /etc/distcc/hosts
localhost/9,cpp,lzo
192.168.0.102/3,cpp,lzo
On my laptop:
% cat /etc/distcc/hosts
localhost/3,cpp,lzo
192.168.0.101/9,cpp,lzo
Just the workstation:
% make -j8 bzImage 496.11s user 25.67s system 712% cpu 1:13.26 total
Trying to use pump/distcc
On the laptop...
% eval pump --startup
On the workstation...
% eval pump --startup
% pump make -j12 bzImage CC=distcc
__________Using distcc-pump from /usr/bin
__________Using 2 distcc servers in pump mode
make[1]: Nothing to be done for `all'.
HOSTCC scripts/basic/fixdep
CHK include/generated/uapi/linux/version.h
...
CC arch/x86/xen/setup.o
CC arch/x86/realmode/rm/video-vga.o
CC arch/x86/mm/fault.o
CC arch/x86/realmode/rm/video-vesa.o
distcc[25252] ERROR: compile arch/x86/realmode/rm/video-vesa.c on 192.168.0.102/3,cpp,lzo failed
distcc[25252] (dcc_build_somewhere) Warning: remote compilation of 'arch/x86/realmode/rm/video-vesa.c' failed, retrying locally
distcc[25214] ERROR: compile arch/x86/realmode/rm/video-vga.c on 192.168.0.102/3,cpp,lzo failed
distcc[25214] (dcc_build_somewhere) Warning: remote compilation of 'arch/x86/realmode/rm/video-vga.c' failed, retrying locally
distcc[25252] Warning: failed to distribute arch/x86/realmode/rm/video-vesa.c to 192.168.0.102/3,cpp,lzo, running locally instead
distcc[25214] Warning: failed to distribute arch/x86/realmode/rm/video-vga.c to 192.168.0.102/3,cpp,lzo, running locally instead
CC mm/mempool.o
distcc[25252] (dcc_please_send_email_after_investigation) Warning: remote compilation of 'arch/x86/realmode/rm/video-vesa.c' failed, retried locally and got a different result.
distcc[25252] (dcc_note_discrepancy) Warning: now using plain distcc, possibly due to inconsistent file system changes during build
CC kernel/panic.o
distcc[25214] (dcc_please_send_email_after_investigation) Warning: remote compilation of 'arch/x86/realmode/rm/video-vga.c' failed, retried locally and got a different result.
CC arch/x86/realmode/rm/video-bios.o
CC arch/x86/xen/multicalls.o
PASYMS arch/x86/realmode/rm/pasyms.h
LDS arch/x86/realmode/rm/realmode.lds
LD arch/x86/realmode/rm/realmode.elf
RELOCS arch/x86/realmode/rm/realmode.relocs
OBJCOPY arch/x86/realmode/rm/realmode.bin
AS arch/x86/realmode/rmpiggy.o
...
OBJCOPY arch/x86/boot/setup.bin
BUILD arch/x86/boot/bzImage
Setup is 17036 bytes (padded to 17408 bytes).
System is 3429 kB
CRC fa62427
Kernel: arch/x86/boot/bzImage is ready (#10)
__________Warning: 3 pump-mode compilation(s) failed on server, but succeeded locally.
__________Distcc-pump was demoted to plain mode. See the Distcc Discrepancy Symptoms section in the include_server(1) man page.
__________Shutting down distcc-pump include server
pump make -j12 bzImage CC=distcc 373.40s user 18.70s system 455% cpu 1:26.10 total