Discussion:
[distcc] Distcc pump mode
yudhistir reddy
2014-03-11 14:13:06 UTC
Permalink
Hi,

I am trying to build using distcc pump mode .

I set DISTCC_HOSTS .

i am starting the pump mode with

pump mk jkernel-jobs WITH_DISTCC=yes

I am getting below waring

distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to run
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally


When we start pump mode , is distcc not setting this variable ?



Regards
Yudhistir
Fergus Henderson
2014-03-12 21:34:13 UTC
Permalink
The pump script will set that environment variable before invoking "mk".
But perhaps "mk" is resetting the environment? Or something else between
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to run
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
yudhistir reddy
2014-03-13 06:40:24 UTC
Permalink
Thank you. Is there any way i can display the list of environment variables
it sets once pump script starts . mk is just the wrapper to call make . We
are not doing anything in that . But still i will dig into it more (mk) .
distcc normal mode is working fine and i see 10% build time reduction.

Is there any work around for this . I can export these variable for some
fixed port like 3632 for tcp . Pls let me know any other variables i need
to set to make this pump mode work . Actually i am running out of time
because i need to show my colleagues and friends , how distcc is working
and improving build time . For the time being , i need some work around for
pump mode to work , then later i will debug more on this .

Any help on this much appreciated .

Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking "mk".
But perhaps "mk" is resetting the environment? Or something else between
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to run
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
Fergus Henderson
2014-03-13 09:16:47 UTC
Permalink
On Thu, Mar 13, 2014 at 6:40 AM, yudhistir reddy
Post by yudhistir reddy
Thank you. Is there any way i can display the list of environment
variables it sets once pump script starts .
Yes, try "pump env".
Post by yudhistir reddy
mk is just the wrapper to call make . We are not doing anything in that .
But still i will dig into it more (mk) . distcc normal mode is working fine
and i see 10% build time reduction.
Is there any work around for this . I can export these variable for some
fixed port like 3632 for tcp . Pls let me know any other variables i need
to set to make this pump mode work . Actually i am running out of time
because i need to show my colleagues and friends , how distcc is working
and improving build time . For the time being , i need some work around for
pump mode to work , then later i will debug more on this .
Any help on this much appreciated .
Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking "mk".
But perhaps "mk" is resetting the environment? Or something else between
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to
run under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
--
Fergus Henderson <***@google.com>

"Defend the user, exclude no one, and create magic." -- Eric Schmidt.
Vanush "Misha" Paturyan
2014-03-13 09:18:03 UTC
Permalink
Post by yudhistir reddy
Thank you. Is there any way i can display the list of environment variables
it sets once pump script starts . mk is just the wrapper to call make . We
are not doing anything in that . But still i will dig into it more (mk) .
distcc normal mode is working fine and i see 10% build time reduction.
I would suspect that whichever way mk is started by the shell it
resets the environment. Can you show this "mk" script (or at least its
first few lines)?

Withouth looking at pump itself: is it possible to do someting like
"pump env" (this should show the environment), and then add an env
invocation into "mk" script, just before calling make, and compare
outputs?

Misha.
Post by yudhistir reddy
Is there any work around for this . I can export these variable for some
fixed port like 3632 for tcp . Pls let me know any other variables i need
to set to make this pump mode work . Actually i am running out of time
because i need to show my colleagues and friends , how distcc is working
and improving build time . For the time being , i need some work around for
pump mode to work , then later i will debug more on this .
Any help on this much appreciated .
Regards
Yudhistir
Any help on this much appreciated .
Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking "mk".
But perhaps "mk" is resetting the environment? Or something else between
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to run
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
yudhistir reddy
2014-03-13 10:21:46 UTC
Permalink
sample output..

src >pump env
__________Using distcc-pump from
/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin
__________Using 4 distcc servers in pump mode
LOGNAME=
_=/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin/pump
BLOCKSIZE=K
INCLUDE_SERVER_PORT=/tmp/distcc-pump.iA44t8/socket
OSTYPE=linux
SSH_CONNECTION=<ip address> 58236 <ip address> 22
DISTCC_HOSTS=s-server01/32,cpp,lzo s-server02/32,cpp,lzo
s-server03/32,cpp,lzo s-server04/32,cpp,lzo
HOSTTYPE=x86_64-linux
EDITOR=vi
__________Shutting down distcc-pump include server

It is diplaying the INCLUDE_SERVER_PORT and its creating the file there in
/tmp directory .

otherway:
eval 'pump --startup'
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'
Post by yudhistir reddy
Post by yudhistir reddy
Thank you. Is there any way i can display the list of environment
variables
Post by yudhistir reddy
it sets once pump script starts . mk is just the wrapper to call make .
We
Post by yudhistir reddy
are not doing anything in that . But still i will dig into it more (mk) .
distcc normal mode is working fine and i see 10% build time reduction.
I would suspect that whichever way mk is started by the shell it
resets the environment. Can you show this "mk" script (or at least its
first few lines)?
Withouth looking at pump itself: is it possible to do someting like
"pump env" (this should show the environment), and then add an env
invocation into "mk" script, just before calling make, and compare
outputs?
Misha.
Post by yudhistir reddy
Is there any work around for this . I can export these variable for some
fixed port like 3632 for tcp . Pls let me know any other variables i
need
Post by yudhistir reddy
to set to make this pump mode work . Actually i am running out of time
because i need to show my colleagues and friends , how distcc is working
and improving build time . For the time being , i need some work around
for
Post by yudhistir reddy
pump mode to work , then later i will debug more on this .
Any help on this much appreciated .
Regards
Yudhistir
Any help on this much appreciated .
Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking
"mk".
Post by yudhistir reddy
Post by Fergus Henderson
But perhaps "mk" is resetting the environment? Or something else
between
Post by yudhistir reddy
Post by Fergus Henderson
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to
run
Post by yudhistir reddy
Post by Fergus Henderson
Post by yudhistir reddy
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes
from
Post by yudhistir reddy
Post by Fergus Henderson
Post by yudhistir reddy
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
yudhistir reddy
2014-03-13 10:54:32 UTC
Permalink
Hi,

I followed this

1.started pump mode like
eval 'pump --startup'
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'

2. didnot start mk command yet .
I want to see the variable value

echo $INCLUDE_SERVER_PORT .

It is showing nothing . So all pump mode variables are not getting exported
properly . I think something is wrong with exporting . I am using bash
shell .

Any suggestions , why its not exporting variables properly .

P.s : mk didnot start yet .

Regards
Yudhistir






On Thu, Mar 13, 2014 at 3:51 PM, yudhistir reddy
Post by yudhistir reddy
sample output..
src >pump env
__________Using distcc-pump from
/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin
__________Using 4 distcc servers in pump mode
LOGNAME=
_=/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin/pump
BLOCKSIZE=K
INCLUDE_SERVER_PORT=/tmp/distcc-pump.iA44t8/socket
OSTYPE=linux
SSH_CONNECTION=<ip address> 58236 <ip address> 22
DISTCC_HOSTS=s-server01/32,cpp,lzo s-server02/32,cpp,lzo
s-server03/32,cpp,lzo s-server04/32,cpp,lzo
HOSTTYPE=x86_64-linux
EDITOR=vi
__________Shutting down distcc-pump include server
It is diplaying the INCLUDE_SERVER_PORT and its creating the file there in
/tmp directory .
eval 'pump --startup'
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'
Post by yudhistir reddy
Post by yudhistir reddy
Thank you. Is there any way i can display the list of environment
variables
Post by yudhistir reddy
it sets once pump script starts . mk is just the wrapper to call make .
We
Post by yudhistir reddy
are not doing anything in that . But still i will dig into it more (mk)
.
Post by yudhistir reddy
distcc normal mode is working fine and i see 10% build time reduction.
I would suspect that whichever way mk is started by the shell it
resets the environment. Can you show this "mk" script (or at least its
first few lines)?
Withouth looking at pump itself: is it possible to do someting like
"pump env" (this should show the environment), and then add an env
invocation into "mk" script, just before calling make, and compare
outputs?
Misha.
Post by yudhistir reddy
Is there any work around for this . I can export these variable for some
fixed port like 3632 for tcp . Pls let me know any other variables i
need
Post by yudhistir reddy
to set to make this pump mode work . Actually i am running out of time
because i need to show my colleagues and friends , how distcc is working
and improving build time . For the time being , i need some work around
for
Post by yudhistir reddy
pump mode to work , then later i will debug more on this .
Any help on this much appreciated .
Regards
Yudhistir
Any help on this much appreciated .
Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking
"mk".
Post by yudhistir reddy
Post by Fergus Henderson
But perhaps "mk" is resetting the environment? Or something else
between
Post by yudhistir reddy
Post by Fergus Henderson
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget
to run
Post by yudhistir reddy
Post by Fergus Henderson
Post by yudhistir reddy
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes
from
Post by yudhistir reddy
Post by Fergus Henderson
Post by yudhistir reddy
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
Fergus Henderson
2014-03-13 11:04:10 UTC
Permalink
Post by yudhistir reddy
Hi,
I followed this
1.started pump mode like
eval 'pump --startup'
That should use back quotes:
eval `pump --startup`
NOT
eval 'pump --startup'
Post by yudhistir reddy
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'
2. didnot start mk command yet .
I want to see the variable value
echo $INCLUDE_SERVER_PORT .
It is showing nothing . So all pump mode variables are not getting
exported properly . I think something is wrong with exporting . I am using
bash shell .
Post by yudhistir reddy
Any suggestions , why its not exporting variables properly .
P.s : mk didnot start yet .
Regards
Yudhistir
On Thu, Mar 13, 2014 at 3:51 PM, yudhistir reddy <
Post by yudhistir reddy
sample output..
src >pump env
__________Using distcc-pump from
/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin
Post by yudhistir reddy
Post by yudhistir reddy
__________Using 4 distcc servers in pump mode
LOGNAME=
_=/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin/pump
BLOCKSIZE=K
INCLUDE_SERVER_PORT=/tmp/distcc-pump.iA44t8/socket
OSTYPE=linux
SSH_CONNECTION=<ip address> 58236 <ip address> 22
DISTCC_HOSTS=s-server01/32,cpp,lzo s-server02/32,cpp,lzo
s-server03/32,cpp,lzo s-server04/32,cpp,lzo
Post by yudhistir reddy
Post by yudhistir reddy
HOSTTYPE=x86_64-linux
EDITOR=vi
__________Shutting down distcc-pump include server
It is diplaying the INCLUDE_SERVER_PORT and its creating the file there
in /tmp directory .
Post by yudhistir reddy
Post by yudhistir reddy
eval 'pump --startup'
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Thank you. Is there any way i can display the list of environment variables
it sets once pump script starts . mk is just the wrapper to call make . We
are not doing anything in that . But still i will dig into it more (mk) .
distcc normal mode is working fine and i see 10% build time reduction.
I would suspect that whichever way mk is started by the shell it
resets the environment. Can you show this "mk" script (or at least its
first few lines)?
Withouth looking at pump itself: is it possible to do someting like
"pump env" (this should show the environment), and then add an env
invocation into "mk" script, just before calling make, and compare
outputs?
Misha.
Post by yudhistir reddy
Is there any work around for this . I can export these variable for some
fixed port like 3632 for tcp . Pls let me know any other variables i need
to set to make this pump mode work . Actually i am running out of time
because i need to show my colleagues and friends , how distcc is working
and improving build time . For the time being , i need some work around for
pump mode to work , then later i will debug more on this .
Any help on this much appreciated .
Regards
Yudhistir
Any help on this much appreciated .
Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking "mk".
But perhaps "mk" is resetting the environment? Or something else between
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you forget to run
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
yudhistir reddy
2014-03-13 11:53:14 UTC
Permalink
Sorry for that , that was silly mistake by me .

Here is the use case , i did :

1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket


2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to Normal
mode ) .
I see the warning messages in distcclog as like below

distcc[12968] Warning: INCLUDE_SERVER_PORT not set - did you forget to run
under 'pump'?
distcc[12968] (dcc_build_somewhere) Warning: failed to get includes from
include server, preprocessing locally

3. build finished
4. checked this variable INCLUDE_SERVER_PORT which is set to
/tmp/distcc-pump.LIlIcs/socket .


So through out build , INCLUDE_SERVER_PORT variable is set to
/tmp/distcc-pump.LIlIcs/socket .


But i donot know how , pump mode is not working and throwing those errors .


Regards
Yudhistir
Post by Fergus Henderson
Post by yudhistir reddy
Hi,
I followed this
1.started pump mode like
eval 'pump --startup'
eval `pump --startup`
NOT
eval 'pump --startup'
Post by yudhistir reddy
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'
2. didnot start mk command yet .
I want to see the variable value
echo $INCLUDE_SERVER_PORT .
It is showing nothing . So all pump mode variables are not getting
exported properly . I think something is wrong with exporting . I am using
bash shell .
Post by yudhistir reddy
Any suggestions , why its not exporting variables properly .
P.s : mk didnot start yet .
Regards
Yudhistir
On Thu, Mar 13, 2014 at 3:51 PM, yudhistir reddy <
Post by yudhistir reddy
sample output..
src >pump env
__________Using distcc-pump from
/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin
Post by yudhistir reddy
Post by yudhistir reddy
__________Using 4 distcc servers in pump mode
LOGNAME=
_=/shared_location/Linux/Ubuntu-12.04/x86_64/distcc/3.1/bin/pump
BLOCKSIZE=K
INCLUDE_SERVER_PORT=/tmp/distcc-pump.iA44t8/socket
OSTYPE=linux
SSH_CONNECTION=<ip address> 58236 <ip address> 22
DISTCC_HOSTS=s-server01/32,cpp,lzo s-server02/32,cpp,lzo
s-server03/32,cpp,lzo s-server04/32,cpp,lzo
Post by yudhistir reddy
Post by yudhistir reddy
HOSTTYPE=x86_64-linux
EDITOR=vi
__________Shutting down distcc-pump include server
It is diplaying the INCLUDE_SERVER_PORT and its creating the file there
in /tmp directory .
Post by yudhistir reddy
Post by yudhistir reddy
eval 'pump --startup'
export INCLUDE_SERVER_PID='13821'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.ygmgYX'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.ygmgYX/socket'
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Thank you. Is there any way i can display the list of environment
variables
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
it sets once pump script starts . mk is just the wrapper to call
make . We
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
are not doing anything in that . But still i will dig into it more
(mk) .
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
distcc normal mode is working fine and i see 10% build time
reduction.
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
I would suspect that whichever way mk is started by the shell it
resets the environment. Can you show this "mk" script (or at least its
first few lines)?
Withouth looking at pump itself: is it possible to do someting like
"pump env" (this should show the environment), and then add an env
invocation into "mk" script, just before calling make, and compare
outputs?
Misha.
Post by yudhistir reddy
Is there any work around for this . I can export these variable for
some
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
fixed port like 3632 for tcp . Pls let me know any other variables
i need
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
to set to make this pump mode work . Actually i am running out of
time
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
because i need to show my colleagues and friends , how distcc is
working
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
and improving build time . For the time being , i need some work
around for
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
pump mode to work , then later i will debug more on this .
Any help on this much appreciated .
Regards
Yudhistir
Any help on this much appreciated .
Regards
Yudhistir
Post by Fergus Henderson
The pump script will set that environment variable before invoking
"mk".
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Post by Fergus Henderson
But perhaps "mk" is resetting the environment? Or something else
between
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Post by Fergus Henderson
"mk" and "distcc" might be unsetting it?
Post by yudhistir reddy
Hi,
I am trying to build using distcc pump mode .
I set DISTCC_HOSTS .
i am starting the pump mode with
pump mk jkernel-jobs WITH_DISTCC=yes
I am getting below waring
distcc[3342] Warning: INCLUDE_SERVER_PORT not set - did you
forget to run
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Post by Fergus Henderson
Post by yudhistir reddy
under 'pump'?
distcc[3342] (dcc_build_somewhere) Warning: failed to get
includes from
Post by yudhistir reddy
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Post by Fergus Henderson
Post by yudhistir reddy
include server, preprocessing locally
When we start pump mode , is distcc not setting this variable ?
Regards
Yudhistir
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
Vanush "Misha" Paturyan
2014-03-13 12:01:19 UTC
Permalink
Post by yudhistir reddy
Sorry for that , that was silly mistake by me .
1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket
2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to Normal
mode ) .
I see the warning messages in distcclog as like below
can you show us your "mk" wrapper?

Misha
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
yudhistir reddy
2014-03-13 13:29:06 UTC
Permalink
# May need to override this (eg. for logging)
exec_cmd() {
exec "$@"
}

# This is my official name
MYNAME=mk

Mydir=`dirname $0`

# for customization
unset mk_cmd_hooks mk_setup_hooks mk_finish_hooks

# override this for more complex requirements
mk_target_machine() {
MACHINE=$1
export MACHINE
}

. $Mydir/sb-env.sh

# for backwards compatability
mk_cmd_hook() { :; }
add_hooks mk_cmd_hooks mk_cmd_hook

run_hooks mk_setup_hooks
run_hooks mk_cmd_hooks
run_hooks mk_cmd_hooks_lifo LIFO
run_hooks mk_finish_hooks

# we generally cannot afford to consume the command line
# since virtually any option might be for $cmd
# but we'll make an exception for these
while :
do
case "$1" in
--) shift; break;; # stop trying to interpret anything
--doc) __doc;; # does not return
--help) __help;; # does not return
--machine) mk_target_machine $2; shift 2;;
*) break;;
esac
done

case $Myname in
mk|mk-*|*make)
nice=${nice:-nice}
cmd=${cmd:-${REAL_MAKE:-/usr/bin/make}};;
*) cmd=${cmd:-$Myname.real};;
esac
# at last! do some work
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Sorry for that , that was silly mistake by me .
1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket
2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to Normal
mode ) .
I see the warning messages in distcclog as like below
can you show us your "mk" wrapper?
Misha
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
yudhistir reddy
2014-03-13 14:11:03 UTC
Permalink
can you pls explain more on this

*cpp*: Enables distcc-pump mode for this host. Note: the build command must
be wrapped in the pump script in order to start the include server.

normal pump mode command i use :

pump mk kernel -j30 WITH_DISTCC=yes
or
eval `pump --startup`
mk kernel -j30 WITH_DISTCC=yes
pump --shutdown

Regards
Yudhistir



On Thu, Mar 13, 2014 at 6:59 PM, yudhistir reddy
Post by yudhistir reddy
# May need to override this (eg. for logging)
exec_cmd() {
}
# This is my official name
MYNAME=mk
Mydir=`dirname $0`
# for customization
unset mk_cmd_hooks mk_setup_hooks mk_finish_hooks
# override this for more complex requirements
mk_target_machine() {
MACHINE=$1
export MACHINE
}
. $Mydir/sb-env.sh
# for backwards compatability
mk_cmd_hook() { :; }
add_hooks mk_cmd_hooks mk_cmd_hook
run_hooks mk_setup_hooks
run_hooks mk_cmd_hooks
run_hooks mk_cmd_hooks_lifo LIFO
run_hooks mk_finish_hooks
# we generally cannot afford to consume the command line
# since virtually any option might be for $cmd
# but we'll make an exception for these
do
case "$1" in
--) shift; break;; # stop trying to interpret anything
--doc) __doc;; # does not return
--help) __help;; # does not return
--machine) mk_target_machine $2; shift 2;;
*) break;;
esac
done
case $Myname in
mk|mk-*|*make)
nice=${nice:-nice}
cmd=${cmd:-${REAL_MAKE:-/usr/bin/make}};;
*) cmd=${cmd:-$Myname.real};;
esac
# at last! do some work
Post by yudhistir reddy
Post by yudhistir reddy
Sorry for that , that was silly mistake by me .
1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket
2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to
Normal
Post by yudhistir reddy
mode ) .
I see the warning messages in distcclog as like below
can you show us your "mk" wrapper?
Misha
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
Fergus Henderson
2014-03-13 18:37:52 UTC
Permalink
Post by yudhistir reddy
# May need to override this (eg. for logging)
exec_cmd() {
}
# This is my official name
MYNAME=mk
Mydir=`dirname $0`
# for customization
unset mk_cmd_hooks mk_setup_hooks mk_finish_hooks
# override this for more complex requirements
mk_target_machine() {
MACHINE=$1
export MACHINE
}
. $Mydir/sb-env.sh
What's in sb-env.sh?
That looks suspicious...
Post by yudhistir reddy
# for backwards compatability
mk_cmd_hook() { :; }
add_hooks mk_cmd_hooks mk_cmd_hook
run_hooks mk_setup_hooks
run_hooks mk_cmd_hooks
run_hooks mk_cmd_hooks_lifo LIFO
run_hooks mk_finish_hooks
# we generally cannot afford to consume the command line
# since virtually any option might be for $cmd
# but we'll make an exception for these
do
case "$1" in
--) shift; break;; # stop trying to interpret anything
--doc) __doc;; # does not return
--help) __help;; # does not return
--machine) mk_target_machine $2; shift 2;;
*) break;;
esac
done
case $Myname in
mk|mk-*|*make)
nice=${nice:-nice}
cmd=${cmd:-${REAL_MAKE:-/usr/bin/make}};;
*) cmd=${cmd:-$Myname.real};;
esac
# at last! do some work
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Sorry for that , that was silly mistake by me .
1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket
2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to Normal
mode ) .
I see the warning messages in distcclog as like below
can you show us your "mk" wrapper?
Misha
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
yudhistir reddy
2014-03-14 11:25:38 UTC
Permalink
Thank you so much for helping me . This issue is resolved . Now i am
testing the pump mode .

Regards
Yudhsitir
Post by Fergus Henderson
Post by yudhistir reddy
# May need to override this (eg. for logging)
exec_cmd() {
}
# This is my official name
MYNAME=mk
Mydir=`dirname $0`
# for customization
unset mk_cmd_hooks mk_setup_hooks mk_finish_hooks
# override this for more complex requirements
mk_target_machine() {
MACHINE=$1
export MACHINE
}
. $Mydir/sb-env.sh
What's in sb-env.sh?
That looks suspicious...
Post by yudhistir reddy
# for backwards compatability
mk_cmd_hook() { :; }
add_hooks mk_cmd_hooks mk_cmd_hook
run_hooks mk_setup_hooks
run_hooks mk_cmd_hooks
run_hooks mk_cmd_hooks_lifo LIFO
run_hooks mk_finish_hooks
# we generally cannot afford to consume the command line
# since virtually any option might be for $cmd
# but we'll make an exception for these
do
case "$1" in
--) shift; break;; # stop trying to interpret anything
--doc) __doc;; # does not return
--help) __help;; # does not return
--machine) mk_target_machine $2; shift 2;;
*) break;;
esac
done
case $Myname in
mk|mk-*|*make)
nice=${nice:-nice}
cmd=${cmd:-${REAL_MAKE:-/usr/bin/make}};;
*) cmd=${cmd:-$Myname.real};;
esac
# at last! do some work
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Sorry for that , that was silly mistake by me .
1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket
2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to
Normal
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
mode ) .
I see the warning messages in distcclog as like below
can you show us your "mk" wrapper?
Misha
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
yudhistir reddy
2014-03-14 11:48:03 UTC
Permalink
I have a query ,

Client machine(where build is running) : 16 CPU dual socket ( logically 32
Cores )

Server1 (distccd running ) : 16 CPU dual socket ( logically 32 Cores )

Server2 (distccd running ): 16 CPU dual socket ( logically 32 Cores )

How much -j value i can give so that it would give best performance .
for running single build on client ?
for running two concurrent builds on client ?


might be silly question , but that would be help me reducing my testing
time to give demo .


Regards
Yudhistir






On Fri, Mar 14, 2014 at 4:55 PM, yudhistir reddy
Post by yudhistir reddy
Thank you so much for helping me . This issue is resolved . Now i am
testing the pump mode .
Regards
Yudhsitir
Post by Fergus Henderson
Post by yudhistir reddy
# May need to override this (eg. for logging)
exec_cmd() {
}
# This is my official name
MYNAME=mk
Mydir=`dirname $0`
# for customization
unset mk_cmd_hooks mk_setup_hooks mk_finish_hooks
# override this for more complex requirements
mk_target_machine() {
MACHINE=$1
export MACHINE
}
. $Mydir/sb-env.sh
What's in sb-env.sh?
That looks suspicious...
Post by yudhistir reddy
# for backwards compatability
mk_cmd_hook() { :; }
add_hooks mk_cmd_hooks mk_cmd_hook
run_hooks mk_setup_hooks
run_hooks mk_cmd_hooks
run_hooks mk_cmd_hooks_lifo LIFO
run_hooks mk_finish_hooks
# we generally cannot afford to consume the command line
# since virtually any option might be for $cmd
# but we'll make an exception for these
do
case "$1" in
--) shift; break;; # stop trying to interpret anything
--doc) __doc;; # does not return
--help) __help;; # does not return
--machine) mk_target_machine $2; shift 2;;
*) break;;
esac
done
case $Myname in
mk|mk-*|*make)
nice=${nice:-nice}
cmd=${cmd:-${REAL_MAKE:-/usr/bin/make}};;
*) cmd=${cmd:-$Myname.real};;
esac
# at last! do some work
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
Sorry for that , that was silly mistake by me .
1. eval `pump --startup`
checked pump mode variables , i can see INCLUDE_SERVER_PORT set to
/tmp/distcc-pump.LIlIcs/socket
2. started build
mk kernel -j30 WITH_DISTCC=yes
Build finished with pump mode ( But actually it is turing back to
Normal
Post by yudhistir reddy
Post by Vanush "Misha" Paturyan
Post by yudhistir reddy
mode ) .
I see the warning messages in distcclog as like below
can you show us your "mk" wrapper?
Misha
--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 1.37
Computer Science Department
NUI Maynooth
__
distcc mailing list http://distcc.samba.org/
https://lists.samba.org/mailman/listinfo/distcc
Loading...