[distcc] Do not allow localhost compile

[Avuton Olrich]

Hello,

I've grazed all FAQ and documents and couldn't find my answer so I
depend on your great knowledge:

I have a really slow machine and two faster ones, I am compiling on
the slow machine and I would rather it be idle than compiled on. I do
not have it in the get hosts but I do see a few cc1plus in the top
output and 0% idle. Here's the simple question: Is localhost added by
default?

I'm running 2.18.3-r6 (gentoo) and thanks for your time.

avuton

[Martin Pool]

Post by Avuton Olrich
Hello,
I've grazed all FAQ and documents and couldn't find my answer so I
I have a really slow machine and two faster ones, I am compiling on
the slow machine and I would rather it be idle than compiled on. I do
not have it in the get hosts but I do see a few cc1plus in the top
output and 0% idle. Here's the simple question: Is localhost added by
default?

No, it's not used by default. However, there can be some jobs that
can't be distributed and distcc will run those locally, because the only
other alternative would be to not compile them at all.

In this situation you might try running with about -j8 so that there is
plenty of room to run things remotely. But for C++ that might causes
your slow machine to run out of memory, so you might not be able to go
that high.

[Avuton Olrich]

Post by Martin Pool

Post by Avuton Olrich
output and 0% idle. Here's the simple question: Is localhost added by
default?

No, it's not used by default. However, there can be some jobs that
can't be distributed and distcc will run those locally, because the only
other alternative would be to not compile them at all.
In this situation you might try running with about -j8 so that there is
plenty of room to run things remotely. But for C++ that might causes
your slow machine to run out of memory, so you might not be able to go
that high.

Well, with a bit of experimentation, the -j8 works marvelously. It
still does compiling on the localhost but it doesn't seem to take up
as much of the time.

Many thanks for all the replys, alot of them which were very helpful.

avuton

[Daniel Kegel]

Post by Avuton Olrich
I have a really slow machine and two faster ones, I am compiling on
the slow machine and I would rather it be idle than compiled on. I do
not have it in the get hosts but I do see a few cc1plus in the top
output and 0% idle. Here's the simple question: Is localhost added by
default?
I'm running 2.18.3-r6 (gentoo) and thanks for your time.

Yes, kind of.
I think you want to apply Michael Donohue's "Preprocessing is not free!" patch,
and set --localslots=0. (I haven't tried this value myself, and
it's not quite the intended use, but it's plausible.)
You can get the patch from here:
http://www.mail-archive.com/***@lists.samba.org/msg02699.html

Let us know if it works for you.

The patch works well for me,
and I'm about ready to say that both of Michael's patches should
be applied by the distcc maintainer.
If it works for you, too, that's one more vote in favor.

[Martin Pool]

Post by Daniel Kegel

Post by Avuton Olrich
I have a really slow machine and two faster ones, I am compiling on
the slow machine and I would rather it be idle than compiled on. I do
not have it in the get hosts but I do see a few cc1plus in the top
output and 0% idle. Here's the simple question: Is localhost added by
default?
I'm running 2.18.3-r6 (gentoo) and thanks for your time.

Yes, kind of.
I think you want to apply Michael Donohue's "Preprocessing is not free!" patch,
and set --localslots=0. (I haven't tried this value myself, and
it's not quite the intended use, but it's plausible.)

Although that patch should probably be merged, and might help, I don't
think it has any effect on when cc1plus runs on localhost, which was the
original question. That will basically be either when delegation fails,
or when there is a compiler option that can't be distributed.

[Martin Pool]

Post by Martin Pool

Post by Daniel Kegel

Post by Avuton Olrich
I have a really slow machine and two faster ones, I am compiling on
the slow machine and I would rather it be idle than compiled on. I do
not have it in the get hosts but I do see a few cc1plus in the top
output and 0% idle. Here's the simple question: Is localhost added by
default?
I'm running 2.18.3-r6 (gentoo) and thanks for your time.

Yes, kind of.
I think you want to apply Michael Donohue's "Preprocessing is not free!" patch,
and set --localslots=0. (I haven't tried this value myself, and
it's not quite the intended use, but it's plausible.)

Although that patch should probably be merged, and might help, I don't
think it has any effect on when cc1plus runs on localhost, which was the
original question. That will basically be either when delegation fails,
or when there is a compiler option that can't be distributed.

Oops, I meant to add: you could try setting DISTCC_VERBOSE=1
DISTCC_LOG=/tmp/distcc.log to get some information about why compiles
are being run locally.

[Jean Delvare]

Hi all,

[Martin Pool]

Post by Martin Pool
Although that patch should probably be merged, and might help, I don't
think it has any effect on when cc1plus runs on localhost, which was the
original question. That will basically be either when delegation
fails, or when there is a compiler option that can't be distributed.

The former case could probably be improved by trying to redistribute to
a different host rather than falling back to localhost (which we have no
real reason to do, that I can see). I think this was discussed already
on this list, but I don't remember anyone proposing a patch to do this.
I have been giving it a try, patch is attached (against the latest Arch
version of distcc). Seems to work just fine for me, I would appreciate
if others could try it, especially people which are actually interested
in this feature. Comments would be very welcome too, of course.

Thanks,

[Donohue, Michael]

It would be nice if this were configurable, so that users can decide how
many retries they want to have until distcc falls back on the localhost.


One advantage to this approach is that the existing behavior can be made
the default value, remote-attempts=1, but users who do not want any
local compilation can set a very high remote attempts value, to prevent
falling back on the localhost, unless there is some massive network
failure.

Michael



-----Original Message-----
From: distcc-bounces+mdonohue=***@lists.samba.org
[mailto:distcc-bounces+mdonohue=***@lists.samba.org] On Behalf Of
Jean Delvare
Sent: Wednesday, June 01, 2005 1:11 PM
To: ***@lists.samba.org
Cc: Avuton Olrich
Subject: Re: [distcc] Do not allow localhost compile

Hi all,

[Martin Pool]

Post by Martin Pool
Although that patch should probably be merged, and might help, I don't
think it has any effect on when cc1plus runs on localhost, which was

the

Post by Martin Pool
original question. That will basically be either when delegation
fails, or when there is a compiler option that can't be distributed.

The former case could probably be improved by trying to redistribute to
a different host rather than falling back to localhost (which we have no
real reason to do, that I can see). I think this was discussed already
on this list, but I don't remember anyone proposing a patch to do this.
I have been giving it a try, patch is attached (against the latest Arch
version of distcc). Seems to work just fine for me, I would appreciate
if others could try it, especially people which are actually interested
in this feature. Comments would be very welcome too, of course.

Thanks,

[Jean Delvare]

[Michael Donohue]

Post by Donohue, Michael
It would be nice if this were configurable, so that users can decide
how many retries they want to have until distcc falls back on the
localhost.

Actually I wanted to mention that possibility in my previous post, but
for some reason forgot to do so. Question is, is this really needed or
not?

Post by Donohue, Michael
One advantage to this approach is that the existing behavior can be
made the default value, remote-attempts=1, but users who do not want
any local compilation can set a very high remote attempts value, to
prevent falling back on the localhost, unless there is some massive
network failure.

After some more thinking, I can't see how it would make a difference. If
I read the code properly, we end up defaulting to localhost anyway if
the list of available remote hosts end up being empty, and that will
happen if all hosts fail. If there is a network failure and all hosts
are unreachable, they will end up being all (temporarily) blacklisted.
Whether they are as the result of the same compilation failing many
times (that will happen with my patch if we don't set a retry limit), or
as the result of several compilations failing once (that will happen
otherwise) doesn't seem to change anything, unless I am mising
something.

NB: Michael, could you please use an e-mail client which doesn't break
the threads?

Thanks,