Sending Lots of Emails in the BACKGROUND?

[Good Man]

Hi

I'm building a 'job posting' site of sorts.

When a job is available in a particular state, I want the system to send an
email to everyone who is 'watching' that state.

I know how to do this, but I need to figure out a way to 'send the emails
in the background' - ie: if an administrator adds a job to Wyoming, I don't
want them to have to wait for the php script to finish selecting and e-
mailing everyone watching the state before the administrator can move on
and do other things.

I originally thought that passing this task to the PHP command line would
do the trick, but it turns out that a script like...

exec("my php script that emails people");
echo "complete!" ;

....does not echo "complete!" until the thousands of people have all had
their info passed to the mail server (a long time).

I suppose that I could get rid of this by just lumping everyone together in
the Bcc: field of a single e-mail message, but at this point I'd like to
keep that as a last resort and go with the personalization ("Hi James... a
job has been posted in Wyoming").

Does anyone have any thoughts as to how I can send the emails without
preventing the administrator from doing other tasks until the script has
finished executing?

[Alvaro G Vicario]

Re: Sending Lots of Emails in the BACKGROUND?

*** Good Man wrote/escribió (Thu, 23 Jun 2005 00:03:14 -0500):[color=blue]
> Does anyone have any thoughts as to how I can send the emails without
> preventing the administrator from doing other tasks until the script has
> finished executing?[/color]

Do you have access to a crontab facility? Writing a shell script* that
sends needed mails and configuring it to execute at regular intervals can
be a good option.

(*) Shell scripts can be written in PHP too.

--
-- Álvaro G. Vicario - Burgos, Spain
-- http://bits.demogracia.com - Mi sitio sobre programación web
-- Don't e-mail me your questions, post them to the group
--

[Daniel Tryba]

Re: Sending Lots of Emails in the BACKGROUND?

Good Man <heyho@letsgo.c om> wrote:
[snip][color=blue]
> exec("my php script that emails people");
> echo "complete!" ;
>
> ...does not echo "complete!" until the thousands of people have all had
> their info passed to the mail server (a long time).[/color]
[snip][color=blue]
> Does anyone have any thoughts as to how I can send the emails without
> preventing the administrator from doing other tasks until the script has
> finished executing?[/color]
[snip]

You need to fork and background that process (see
http://nl2.php.net/manual/en/function.pcntl-fork.php or simply append a
'&' to the program/script you exec (making sure _any_ output is
redirected to eg /dev/null)).

Better yet would be to create a que mechanism, add email and body to a
database and have a script periodically check if there are entries to be
sent. That way there is only 1 process trying to spam^Wsend email you
have to look after.

[Good Man]

Re: Sending Lots of Emails in the BACKGROUND?

Alvaro G Vicario <alvaro_QUITAR_ REMOVE@telecomp uteronline.com> wrote in
news:zd6ozxpk35 46.cv8l860w5hpi $.dlg@40tude.ne t:
[color=blue]
> *** Good Man wrote/escribió (Thu, 23 Jun 2005 00:03:14 -0500):[color=green]
>> Does anyone have any thoughts as to how I can send the emails without
>> preventing the administrator from doing other tasks until the script
>> has finished executing?[/color]
>
> Do you have access to a crontab facility? Writing a shell script* that
> sends needed mails and configuring it to execute at regular intervals
> can be a good option.
>
> (*) Shell scripts can be written in PHP too.[/color]

i don't want to do a crontab, because ideally i'd like 'instant'
notification when a job is posted... but i suppose an hourly crontab might
not be TERRIBLE...

[Good Man]

Re: Sending Lots of Emails in the BACKGROUND?

Daniel Tryba <partmapsswen@i nvalid.tryba.nl > wrote in
news:42ba86cb$0 $36228$c5fe704e @news6.xs4all.n l:
[color=blue]
> Good Man <heyho@letsgo.c om> wrote:[color=green]
>> Does anyone have any thoughts as to how I can send the emails without
>> preventing the administrator from doing other tasks until the script
>> has finished executing?[/color]
> [snip]
>
>
> Better yet would be to create a que mechanism, add email and body to a
> database and have a script periodically check if there are entries to
> be sent. That way there is only 1 process trying to spam^Wsend email
> you have to look after.[/color]

you clever devil, thanks for the suggestion!

[Alvaro G Vicario]

Re: Sending Lots of Emails in the BACKGROUND?

*** Good Man wrote/escribió (Thu, 23 Jun 2005 08:37:28 -0500):[color=blue]
> i don't want to do a crontab, because ideally i'd like 'instant'
> notification when a job is posted... but i suppose an hourly crontab might
> not be TERRIBLE...[/color]

Or even a Once Every Fifteen Minutes if necessary, given the script is
designed to cause little load if no work left and to stand ocasional
overlappings.


--
-- Álvaro G. Vicario - Burgos, Spain
-- http://bits.demogracia.com - Mi sitio sobre programación web
-- Don't e-mail me your questions, post them to the group
--

[Google Mike]

Re: Sending Lots of Emails in the BACKGROUND?

GM,

I assume you're using Linux here, right? It works much better that way
for this kind of thing. The way I do it is to have my PHP fork off
another PHP page to do the work. In my case, I have a work order
ticketing system. I don't want to hold up the user while the mail
server is busy, so I fork off a process in RAM and then return the user
immediately back with a message that the email will be sent to the user
shortly. On my web server in a given day, you may see a couple dozen of
these forked files in RAM, waiting for the slow mail server in our
corporate intranet to respond over the T1 connection. The queue
mechanism is better because of error control, but my mechanism is the
poor man's version. You can build the queue with a database and a cron
job that calls the PHP page to read the database, send the email, and
go back to sleep again until the next cron wakeup time. Your task,
then, would be to only file these entries in the database.

[R. Rajesh Jeba Anbiah]

[FAQ] Bulk mailer in PHP (Was Re: Sending Lots of Emails in the BACKGROUND?)

Q: How can I send bulk mails with PHP?
A: PHP may not be the right choice for sending bulk mails. PHP's
built-in mail() function is written to use sendmail binary; and higher
performance is not usually expected. It is also suggested that qmail is
faster than sendmail; to make PHP to use qmail instead of sendmail, one
has to work on the make and installation process. There are many (C
based) applications available to speed up sendmail.

Another suggestion is to sort the email addresses based on domain names
and deliver them using BCC. Most of the time, this will make the mail
to be delivered in junk folder as many spam guards trap BCCed mails.

Many commercial bulk mailers such as LISTSERV guarantee high speed
delivery as they seem to use higher-end hardwares and resources.

Refer:
1. http://www.php.net/mail
2. ftp://cs.utk.edu/pub/moore/bulk_mail..._mailer.README

+++++
@todo Grammar cleanup.

[Jerry Stuckle]

Re: Sending Lots of Emails in the BACKGROUND?

Good Man wrote:[color=blue]
> Hi
>
> I'm building a 'job posting' site of sorts.
>
> When a job is available in a particular state, I want the system to send an
> email to everyone who is 'watching' that state.
>
> I know how to do this, but I need to figure out a way to 'send the emails
> in the background' - ie: if an administrator adds a job to Wyoming, I don't
> want them to have to wait for the php script to finish selecting and e-
> mailing everyone watching the state before the administrator can move on
> and do other things.
>
> I originally thought that passing this task to the PHP command line would
> do the trick, but it turns out that a script like...
>
> exec("my php script that emails people");
> echo "complete!" ;
>
> ...does not echo "complete!" until the thousands of people have all had
> their info passed to the mail server (a long time).
>
> I suppose that I could get rid of this by just lumping everyone together in
> the Bcc: field of a single e-mail message, but at this point I'd like to
> keep that as a last resort and go with the personalization ("Hi James... a
> job has been posted in Wyoming").
>
> Does anyone have any thoughts as to how I can send the emails without
> preventing the administrator from doing other tasks until the script has
> finished executing?[/color]

I've implemented a queue mechanism in the past; it works fine, and you
can control the rate your messages are sent independent of the UI. I
did it all in PHP and MySQL with a cron job to kick off the queue runner
every 15 minutes (sufficient for my purposes).

Later I rewrote the queue runner in C, but didn't have any noticeable
effects. The PHP code for this was so small it really didn't matter much.

The only thing you want to be careful of is that you don't have two
queue runners going at the same time (i.e. one starts before the
previous one finishes). Depending on timing and exactly how you wrote
the code, you could get duplicate emails sent. You can protect it in
the code - but I found it much easier to just check to see if another
copy was running in a startup script and not start the job if there is.

--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attgl obal.net
=============== ===