How Can I Throttle Down CFMAIL's Speed?
(and why on Earth would I do this??)
Its strange to discuss a procedure that makes a ColdFusion function less efficient. Especially in light of how well the ColdFusion MX 6.1 CFMAIL tag works. Nonetheless, some realities in this world are inescapable, and doing this will fix one of them.
A Bit of Background, First...
Years ago I used commercial shared hosting. Those systems have notorious resource issues, for reasons that are fairly well known. Also at that time (this is the ColdFusion 4.0/4.5 era) CFMAIL had a reputation for being buggy when the server was under heavy load. So what was necessary was a way to "throttle down" the speed at which ColdFusion transmitted mail, so a taxed, shared server could handle it.
Flash forward to The Present Day...
ColdFusion can now reliably handle large volumes of mail. But I still throttle it down: Beginning some time in 2003 many large dialup vendors began instituting short-term, temporary blocks on senders who pumped what the provider considered to be too much mail to users of their networks (the idea is that spammers don't do mail retries for extended periods, making this a part of the spam control process). And it didn't take much to trigger the filters. Only a few dozen emails to any provider was enough to get blocked for a short time.
Services like AOL, MSN and Yahoo in particular were identified as doing this. The typical behavior was for mail to be refused at the receiving server as a nonexistent address, when in fact the user existed. Resends later in the day worked. The solution discussed on the Ipswitch Imail list was to crank up the retry attempt count and interval so the retry period exceeded the blockage period.
During this time I became aware that my older, throttled-down systems were having no problems. It turned out that the throttled speed had a byproduct: it kept the mailing rates underneath the trigger threshold of this new antispam 'radar'.
On to the code (finally!)
The idea is to send mail slowly and to have it sent at this slower speed unattended, so its not necessary to break a list up into pieces and whatnot. What we do is take advantage of an HTML meta refresh to re-execute our template, and ColdFusion to manage the operation until its completion.
|
Once you get this under your belt, check out our improved version that introduces failure recovery! |
<cfsilent>
<!---
Run Rate is the number of seconds between refresh. To
optimize this make this setting about 2 seconds longer
than the ColdFusion Server's mail spool fetch rate.
Otherwise mail will pile up in the spooler and partially
defeat the purpose of trickling out the mail
--->
<cfset variables.RunRate=17>
<!---
Query Run is the number of query rows (email addresses)
that will be processed on each execution of this template
--->
<cfset variables.QueryRun=20>
<!---
pull the email addresses. Cache this query since its
going to be re-used every few seconds.
--->
<cfquery
datasource="#request.myDSN#"
name="MailList"
cachedwithin="#CreateTimeSpan(0,1,0,0)#">
SELECT
mylist.myEmail
FROM mylist
WHERE
mylist.myEmail IS NOT NULL
ORDER BY
mylist.myPrimaryKey ASC
</cfquery>
<!---
Set the starting point of the mail run to the first
record of query output
--->
<cfparam
name="url.CurrStart"
default=1
type="numeric">
<!---
Do some stuff to prevent page caching, which was found to
be a problem on some client browsers
--->
<cfheader
name="Expires"
value="Sun, 06 Nov 1994 08:49:37 GMT">
<cfheader
name="Pragma"
value="no-cache">
<cfheader
name="cache-control"
value="no-cache, no-store, must-revalidate">
</cfsilent>
<!---
Are we there yet?
--->
<cfif url.CurrStart GT MailList.RecordCount>
<!---
yes. Inform the user
--->
<html><head><title>FINISHED</title></head>
<body ONLOAD=history.go(1)>
<h1>All Done</h1>
<b>Close this browser window!</b>
<p><b>Do NOT press your BACK button</b> to leave.</p>
</body></html>
<cfelse>
<!---
No. Calculate the *next* starting point, and some other
values used in the display below
--->
<cfset variables.NextRun=url.CurrStart+variables.Queryrun>
<cfset variables.NextShow=url.CurrStart+(variables.QueryRun-1)>
<cfif variables.NextShow GT MailList.RecordCount>
<cfset variables.NextShow=MailList.RecordCount>
</cfif>
<!---
display the "in-progress" page
--->
<html>
<cfoutput>
<!---
This next line re-runs the template automatically at the run rate,
with the incremented starting value. Note the UUID inserted into
the url. This is another part of ensuring the browser doesn't
cache this page.
--->
<meta
http-equiv="REFRESH"
content="#variables.RunRate#;
URL=#cgi.script_name#?CurrStart=#variables.NextRun#
&UniqueURL=#UrlEncodedFormat(CreateUUID())#">
</cfoutput>
<head><title>Low Volume Mass Mailer</title></head>
<body ONLOAD=history.go(1)>
<!---
Send the actual mail. This is a very simple version, with
hardcoded values in many places where, in a real application,
you'd be plugging in default values or output from other
queries (such as the mail server name, the sender address and
the email message text.
--->
<cfset variables.CharSet="text/html; charset="&chr(34)&"utf-8"&Chr(34)>
<cfmail
to="#MailList.myEmail#"
from="blah@blah.com"
subject="SnailMail"
server="mail.blah.com"
query="MailList"
maxrows="#variables.QueryRun#"
startrow="#url.CurrStart#"
type="HTML">
<h1>YOUR EMAIL TEXT GOES HERE</h1>
<cfmailparam
name="Reply-To"
value="blah@blah.com">
<cfmailparam
name="Message-ID"
value="<#CreateUUID()#@mail.blah.com>">
<cfmailparam
name="Content-Type"
value="#variables.CharSet#">
<cfmailparam
name="Mime-Version"
value="1.0">
</cfmail>
<!---
give the user a visual cue as to where the operation is at the moment
--->
<cfoutput>
<p>Sending messages #url.CurrStart# thru #variables.NextShow#<br>
<b>Total to send: #MailList.RecordCount#</b></p>
</cfoutput>
<!---
This is a good place to do some extra calculations so you can show
the user how many messages have been processed, how many are left
and an extimated time to completion
--->
<p>Leave this system alone and wait for this job to complete.</p>
<p>When it finishes, you will be notified.</p>
</body></html>
</cfif>
The technique has obvious size limits. It clearly won't run on truly large lists, but for the small timer it should be fine. We have clients with 5000+ mailing lists that have no problems with it or their large volume of AOL/MSN recipients. Finally, the run rates here are only suggestions. You may find that higher numbers will work fine, thereby increasing this system's capacity to handle mail in a more timely fashion.
Hope this helps,
--------------
Matt Robertson
--------------
|
Programmers' Tools
