Multithreading/Parallel Processing with PHP

[Schroeder, AJ]

Group,

I have googled this topic and I know this has been asked before, but there
seems to be no good answer except "PHP doesn't support multi-threading" I am
new to PHP so I might be using the wrong terminology.

I have inherited support of php scripts that backup network gear via SNMP.
They currently work fine, and outside of the lack of good overall program
structure and flow, the scripts perform quite well.

The script is back-ended by postgres, and when it executes, it grabs a list
of device names from the database, gets the count of devices, then performs
a while loop to back up every device - unelegant but effective. However, the
amount of network gear that is being backed up is increasing, and I would
like to "multithrea d" the script to backup more than one device at a time so
that it doesn't take 45 + minutes to back up all of the switch gear.

Is this possible with PHP? Personally I don't think so since PHP seems to be
a "top-down" language, but maybe the gurus in this group could lend some
insight. I have looked at the "tick" functions, but those don't seem to be a
good fit.

Any help or ideas are appreciated,

AJ Schroeder

[Chris Hope]

Re: Multithreading/Parallel Processing with PHP

Schroeder, AJ wrote:
[color=blue]
> I have googled this topic and I know this has been asked before, but
> there seems to be no good answer except "PHP doesn't support
> multi-threading" I am new to PHP so I might be using the wrong
> terminology.[/color]

That is correct. PHP has no support for multi threading.
[color=blue]
> I have inherited support of php scripts that backup network gear via
> SNMP. They currently work fine, and outside of the lack of good
> overall program structure and flow, the scripts perform quite well.
>
> The script is back-ended by postgres, and when it executes, it grabs a
> list of device names from the database, gets the count of devices,
> then performs a while loop to back up every device - unelegant but
> effective. However, the amount of network gear that is being backed up
> is increasing, and I would like to "multithrea d" the script to backup
> more than one device at a time so that it doesn't take 45 + minutes to
> back up all of the switch gear.
>
> Is this possible with PHP? Personally I don't think so since PHP seems
> to be a "top-down" language, but maybe the gurus in this group could
> lend some insight. I have looked at the "tick" functions, but those
> don't seem to be a good fit.
>
> Any help or ideas are appreciated,[/color]

You can fork a process using the PHP CLI (assuming you're on a *nix box
and it soundsl ike you probably are) like you can with most other
scripting languages.

Here's an article I wrote a while back which covers forking and has some
basic examples:

http://www.electrictoolbox.com/article/php/process-forking/

--
Chris Hope | www.electrictoolbox.com | www.linuxcdmall.com

[Colin McKinnon]

Re: Multithreading/Parallel Processing with PHP

Chris Hope wrote:
[color=blue]
> Schroeder, AJ wrote:
>[color=green]
>> I have googled this topic and I know this has been asked before, but
>> there seems to be no good answer except "PHP doesn't support
>> multi-threading" I am new to PHP so I might be using the wrong
>> terminology.[/color]
>
> That is correct. PHP has no support for multi threading.
>[/color]
<snip>[color=blue]
>
> You can fork a process using the PHP CLI (assuming you're on a *nix box
> and it soundsl ike you probably are) like you can with most other
> scripting languages.
>[/color]

Or you (if you're not interested in the process after it starts) can ask the
OS to start up a new process, even if it's only web accessible:

for ($x=0; $x<count($devic es); $x++) {
$url="http://localhost/snmp_grab.php?d ev=" . $device[$x];
$cmd="curl '$url'"; // or {"php -q /docroot/snmp_grab " . $device[$x]}
// or lynx, or w3m, or ...
`echo "$cmd" | at now`;
}

HTH

C.

[notquitejack]

Re: Multithreading/Parallel Processing with PHP

In the past, I've found it useful to have one PHP script spawn other
processes that can run simultaneously. I'm not sure that it will
improve speed, but at least things run in parallel.

foreach ($array as $value){
$command="/usr/local/bin/php -q script.php $value 1>/dev/null 2>&1
&";
`$command`;
}

[Chung Leong]

Re: Multithreading/Parallel Processing with PHP

The easiest way is to do an internal HTTP look-up, basically letting
the web server handle the multitasking for you. Open a bunch of HTTP
connections to http://localhost/????.php, save the handles into an
array, then keep fgets() each until feof() return true for all of them.
Interprocess communication is simple, as the child script only need to
echo messages to the output stream.

[Schroeder, AJ]

Re: Multithreading/Parallel Processing with PHP


"Chris Hope" <blackhole@elec trictoolbox.com > wrote in message
news:dhi3ss$v5h $1@lust.ihug.co .nz...[color=blue]
> Schroeder, AJ wrote:
>[color=green]
>> I have googled this topic and I know this has been asked before, but
>> there seems to be no good answer except "PHP doesn't support
>> multi-threading" I am new to PHP so I might be using the wrong
>> terminology.[/color]
>
> That is correct. PHP has no support for multi threading.
>[/color]

Since it is a scripting language (albeit a higher level language than say
<shudder> VBScript) is there a chance that PHP would support
multi-threading? I don't even think that it could support it, but that is my
limited knowledge.
[color=blue][color=green]
>> I have inherited support of php scripts that backup network gear via
>> SNMP. They currently work fine, and outside of the lack of good
>> overall program structure and flow, the scripts perform quite well.
>>
>> The script is back-ended by postgres, and when it executes, it grabs a
>> list of device names from the database, gets the count of devices,
>> then performs a while loop to back up every device - unelegant but
>> effective. However, the amount of network gear that is being backed up
>> is increasing, and I would like to "multithrea d" the script to backup
>> more than one device at a time so that it doesn't take 45 + minutes to
>> back up all of the switch gear.
>>
>> Is this possible with PHP? Personally I don't think so since PHP seems
>> to be a "top-down" language, but maybe the gurus in this group could
>> lend some insight. I have looked at the "tick" functions, but those
>> don't seem to be a good fit.
>>
>> Any help or ideas are appreciated,[/color]
>
> You can fork a process using the PHP CLI (assuming you're on a *nix box
> and it soundsl ike you probably are) like you can with most other
> scripting languages.
>
> Here's an article I wrote a while back which covers forking and has some
> basic examples:
> http://www.electrictoolbox.com/artic...ocess-forking/
>[/color]

Yes, I am using a Redhat machine for the PHP scripting. Although I have not
read the entire article, it looks as if that might be what I am looking for.
I just have to find out if the admin before me installed PHP with forking
support. Something tells me he didn't. :(

Thank you for the advice,

AJ Schroeder

[Schroeder, AJ]

Re: Multithreading/Parallel Processing with PHP


"Colin McKinnon" <colin.deleteth is@andthis.mms3 .com> wrote in message
news:dhiva9$lsh $1$8300dec7@new s.demon.co.uk.. .[color=blue]
> Chris Hope wrote:
>[color=green]
>> Schroeder, AJ wrote:
>>[color=darkred]
>>> I have googled this topic and I know this has been asked before, but
>>> there seems to be no good answer except "PHP doesn't support
>>> multi-threading" I am new to PHP so I might be using the wrong
>>> terminology.[/color]
>>
>> That is correct. PHP has no support for multi threading.
>>[/color]
> <snip>[color=green]
>>
>> You can fork a process using the PHP CLI (assuming you're on a *nix box
>> and it soundsl ike you probably are) like you can with most other
>> scripting languages.
>>[/color]
>
> Or you (if you're not interested in the process after it starts) can ask
> the
> OS to start up a new process, even if it's only web accessible:
>
> for ($x=0; $x<count($devic es); $x++) {
> $url="http://localhost/snmp_grab.php?d ev=" . $device[$x];
> $cmd="curl '$url'"; // or {"php -q /docroot/snmp_grab " .
> $device[$x]}
> // or lynx, or w3m, or ...
> `echo "$cmd" | at now`;
> }
>
> HTH
>
> C.[/color]

That won't work for my backup scripts, but that'll work for another web app
that I am writing...

Great suggestion! Thank you!

AJ Schroeder