Identifying Performance Bottlenecks

By: Chris Lewis

There are many elements that are involved in email distribution that can affect performance.  When you have to send out a few thousand emails performance does not really matter as much, but if you are in the ten's or hundred's of thousands then performance is crucial.  The flow of an email distribution system is:

SQL or file-based Database with email addresses and other info (SQL Server, Access, Oracle, etc.)
Email merging and sending software (Campaign Enterprise)
Mail Transport system (SMTP) like PowerMTA, Ironport, etc.

Bottlenecks can occur in these systems, as a result of the merges within an email itself, or depending on how these systems are placed in a network, the connections between them.  The way to diagnose this is to start from the back and work your way up front.

Checking the MTA
The #1 bottleneck we have found historically is the MTA.  Campaign can send emails faster but it is wait for the MTA to accept the emails. In order to find out if this is the bottleneck you will need to send the emails to a different MTA that will accept emails at an extreme speed.  Then, if the sending speed increases due to this switch, then you know it is either the MTA or the connection to the MTA system.  The simplest way to do this in Campaign Enterprise is to:

1. Create an SMTP Connection that sends the emails to a folder (EML files) instead of the MTA.  Do do this, after logging in as an administrator, go the the Administrator link, choose SMTP Connections, then create a new SMTP connection and select EML Files, and set a scratch folder on a disk the EML files can be sent to.
2. Change your Campaign's sending SMTP setting to this new EML files setting
3. Run the campaign. You can stop it after a few thousand emails or 1 minute, it should be sufficient to get a performance
4. Go to the reports for that campaign and see what your performance is in the Overall Report

If your speed increase after this test, then you can assume the issue is your MTA or the path to it and that should be addressed.  Many MTAs are busy doing other things, they could have throttling agents in place for mass-mailings, etc.  So if this happens to be the case, then there is plenty to check out.  Sometimes when the MTA is on the same box as the database and the Campaign Enterprise, it is true that communications between these system can be really fast, but it may end up being slower overall because the computer has to manage all these processes even with multiple cores.

Writebacks During Send
If you are writing-back to the database after each email goes out, then you should probably disable this code for the test above if there is no improvement.  It is possible though this write-back activity could be the bottleneck, so if you can leave it in for this test and reset the data after you run the campaign that would be best.

Merge Field Performance
If you have a lot of merge fields in the message, especially if it is a large message, then that can cause a bottleneck.  Remove all the merges from the email and send a test Campaign to the EML files.

Email Size
If your performance is going down sometimes a your emails may be unknowingly getting bigger and require more transmission time.  Reduce your email to a simple 1 line email and test again using the EML files setting.  If the performance speeds up this may be the problem.

Embedded Images
If you have embedded images in your email, that can slow down transmission of the emails.  Also Campaign Enterprise always does a scan for embedded images in an email.  If you are not using embedded images you can go to the Miscellaneous tab for the Campaign and check the box to ignore the embedded images scan.

Your Database
If extreme rare cases your database may be the issue.  If none of the steps above help performance then it may be your SQL server or single database file.  Best way to test this is just to keep dumbing-down the SQL statement you are using until it is very simple.  Usually single database files are very fast, like MS Access, but SQL servers can be busy or on a different computer on a busy network.  We found many times an SQL server really slows down when batch processes are being done on them, like backups or mass data updates.

If you find that none of these are helping, we can definitely help further by logging into you system for an analysis.