The Robertson Team


  Programmers' Tools >  The Free Stuff >  Free ColdFusion Tutorials >  How Can I Throttle Down CFMAIL's Speed?

How Can I Throttle Down CFMAIL's Speed?

(and why on Earth would I do this??)

Its strange to discuss a procedure that makes a ColdFusion function less efficient.  Especially in light of how well the ColdFusion MX 6.1 CFMAIL tag works.  Nonetheless, some realities in this world are inescapable, and doing this will fix one of them.

A Bit of Background, First...
Years ago I used commercial shared hosting.  Those systems have notorious resource issues, for reasons that are fairly well known.  Also at that time (this is the ColdFusion 4.0/4.5 era) CFMAIL had a reputation for being buggy when the server was under heavy load.  So what was necessary was a way to "throttle down" the speed at which ColdFusion transmitted mail, so a taxed, shared server could handle it.

Flash forward to The Present Day...
ColdFusion can now reliably handle large volumes of mail.  But I still throttle it down:  Beginning some time in 2003 many large dialup vendors began instituting short-term, temporary blocks on senders who pumped what the provider considered to be too much mail to users of their networks (the idea is that spammers don't do mail retries for extended periods, making this a part of the spam control process).  And it didn't take much to trigger the filters.  Only a few dozen emails to any provider was enough to get blocked for a short time.

Services like AOL, MSN and Yahoo in particular were identified as doing this.  The typical behavior was for mail to be refused at the receiving server as a nonexistent address, when in fact the user existed.  Resends later in the day worked.  The solution discussed on the Ipswitch Imail list was to crank up the retry attempt count and interval so the retry period exceeded the blockage period.

During this time I became aware that my older, throttled-down systems were having no problems.  It turned out that the throttled speed had a byproduct: it kept the mailing rates underneath the trigger threshold of this new antispam 'radar'.

On to the code (finally!)
The idea is to send mail slowly and to have it sent at this slower speed unattended, so its not necessary to break a list up into pieces and whatnot.  What we do is take advantage of an HTML meta refresh to re-execute our template, and ColdFusion to manage the operation until its completion.

Once you get this under your belt, check out our improved version that introduces failure recovery!

Run Rate is the number of seconds between refresh.  To
optimize this make this setting about 2 seconds longer
than the ColdFusion Server's mail spool fetch rate. 
Otherwise mail will pile up in the spooler and partially
defeat the purpose of trickling out the mail

<cfset variables.RunRate=17>
Query Run is the number of query rows (email addresses)
that will be processed on each execution of this template

<cfset variables.QueryRun=20>
pull the email addresses.  Cache this query since its
going to be re-used every few seconds.


   FROM mylist
      mylist.myEmail IS NOT NULL
      mylist.myPrimaryKey ASC
Set the starting point of the mail run to the first
record of query output


Do some stuff to prevent page caching, which was found to
be a problem on some client browsers

   value="Sun, 06 Nov 1994 08:49:37 GMT">
   value="no-cache, no-store, must-revalidate">

Are we there yet?

<cfif url.CurrStart GT MailList.RecordCount>
   yes.  Inform the user
   <body ONLOAD=history.go(1)>
   <h1>All Done</h1>
   <b>Close this browser window!
Do NOT press your BACK button</b> to leave.

   No.  Calculate the *next* starting point, and some other 
   values used in the display below

   <cfset variables.NextRun=url.CurrStart+variables.Queryrun>
   <cfset variables.NextShow=url.CurrStart+(variables.QueryRun-1)>
   <cfif variables.NextShow GT MailList.RecordCount>
      <cfset variables.NextShow=MailList.RecordCount>

   display the "in-progress" page

   This next line re-runs the template automatically at the run rate,
   with the incremented starting value.  Note the UUID inserted into
   the url.  This is another part of ensuring the browser doesn't
   cache this page.


   <head><title>Low Volume Mass Mailer</title></head>
   <body ONLOAD=history.go(1)>
   Send the actual mail.  This is a very simple version, with
   hardcoded values in many places where, in a real application,
   you'd be plugging in default values or output from other
   queries (such as the mail server name, the sender address and
   the email message text.

   <cfset variables.CharSet="text/html; charset="&chr(34)&"utf-8"&Chr(34)>


   give the user a visual cue as to where the operation is at the moment

   <p>Sending messages #url.CurrStart# thru #variables.NextShow#<br>
   <b>Total to send: #MailList.RecordCount#</b></p>
   This is a good place to do some extra calculations so you can show
   the user how many messages have been processed, how many are left
   and an extimated time to completion

   <p>Leave this system alone and wait for this job to complete.</p>
   <p>When it finishes, you will be notified.


The technique has obvious size limits.  It clearly won't run on truly large lists, but for the small timer it should be fine.  We have clients with 5000+ mailing lists that have no problems with it or their large volume of AOL/MSN recipients.  Finally, the run rates here are only suggestions.  You may find that higher numbers will work fine, thereby increasing this system's capacity to handle mail in a more timely fashion.

Hope this helps,
Matt Robertson

The Robertson Team,
1.559.360.1717 is a leader in ColdFusion web hosting and managed dedicated servers.