By: Chris Lewis
From time to time we get questions concerning email address verification. Though many SMTP servers allow the verification of email addresses there are probably just as many that don't allow it. So, email verification at it's best is about 50% accuracy using the method the internet RFC guru's designed. Thanks to SPAMers this built in verification system is not very usable, especially with the big email providers.
Campaign Enterprise and Email Marketing Director do not have this email verification capability mainly because doing mass email verification can cause your SMTP server to become blacklisted. Many SPAMers use this technique with fabricated email lists in hope of getting valid email addresses (ie: firstname.lastname@example.org, email@example.com,...). So while it is an important thing to keep your list clean it should be done as if you were a normal one-at-a-time emailer that gets a bounce when the email is bad, then you don't send to it again. When you send out the first email campaign on a new list you will receive bounces if the email addresses are not valid. It is important to receive these emails and mark them as bounces to not send to them again. There are two reasons for this: First, you don't want your campaign to waste time sending to these addresses again. If your list is small, this may not matter, but if your list is big, or you are paying for each email sent out through a service, it is especially important. Second, if you keep sending emails that bounce your SMTP server will acquire a bad reputation will eventually get blacklisted.
There are verifications services that specialize in the field of email list cleaning that have good internet reputations, are known for being good citizens of the internet, and are allowed mass verifications of lists. One of them we recommend is http://freshaddress.com. You send them a list and they will make sure each address is usable, and they also do other services like email forwarding if the email address has changed.
If you use sending services like SMTP.C
Taking the time or spending the money to get your list clean will have many benefits over the cost. If you try to save money here and do it marginally you may end up spending a great deal more in time and money trying to get your SMTP server off blacklists.
By: Chris Lewis
Sometimes it's best to go back to basics to understand some of the issues that can hamper your email distribution efforts. One of those basics is just how email is sent through the internet to the destination. You may set up the perfect email and it may never reach your destination because it was blocked from getting there. Most of the "hoops" that need to be jumped through is because of SPAM and open SMTP relays (servers that will allow anyone to send from it). Below is a quick explanation of how the email system works and how your email interacts with sending and receiving SMTP servers. Here is the path an email takes:
1. Your email client sends an email to your SMTP server, say it was sent to firstname.lastname@example.org
2. Your SMTP server grabs the domain name in the email, in this case THEIRDOMAIN.COM
3. Your SMTP server looks up the MX records for THEREDOMAIN.COM from your DNS Server (this is the MXLookup you find in MXToolbox.com website)
4. In the order of priority, your SMTP server attempts to connect to the first destination SMTP server based on the first MX record.
5. If the first destination SMTP is unavailable or busy, your SMTP server will move on to the next destination SMTP server, and so on.
6. Once your SMTP server has connected to a destination SMTP server, before the email is transferred, the full return-path (or FROM) email address to is sent to the destination SMTP server
7. If the destination SMTP is setup correctly, it will take the return-path email address and extract the domain name (in our case ARIALSOFTWARE.COM if we were sending the email)
8. Now the destination SMTP starts authenticating the email by one or more methods. The most common are below:
10. Exceptions - Sometimes these methods will create a SPAM score, and if the score is exceeded then the email is rejected. So it is possible for one of these to fail but the email still makes it through
So, the answers to why an email does not make it is within this flow. The MXToolbox.com website is a big combination of different tests which may confuse the issues. Blacklisting ONLY has to do with the "sending" SMTP server and has nothing to do with a particular domain because you could use the same SMTP server to send out emails for multiple domains…so the IP address is king. If you try to send out emails through your SMTP server that is blacklisted, most of the time the emails will be rejected by the destination SMTP server.
So, to effectively use the MXToolbox.com tool for blacklisting detection, you need to do two things. First, you need to get the SMTP server they are using in Campaign Enterprise or Email Marketing Director. You need to translate that SMTP server setting to an IP address if it is not already one. Then you go to the Blacklisting tab and put that IP address in and then see if your SMTP server is blacklisted.
One last spin on this…if the SMTP server setting they are using is an "internal" IP address, like 10.10.10.1 or 192.168.1.100, then you will have to find out what the "external" IP address of the SMTP server is to see if it is truly blacklisted in the world.
By: Chris Lewis
I lightly addressed this issue last month but many asked for a better description of the SPF authentication method, so here it is. Setting up a new SMTP server is pretty easy, especially if you are using an IIS SMTP server. Many have set up SMTP servers on Webservers just so that when someone fills out a webform that this information is emailed to you. What can happen though is that even though the email is sent to your SMTP server, it never makes it to your Inbox. The emails seem simple enough and could not even come close to SPAM, so what is the problem? The problem is usually it is that YOUR email provider (the receiving system of email), like Gmail, that is dumping those emails because it could not authenticate the domain part of the email. If you don't have an SPF record setup for your domain in DNS, it is possible that 90% of your emails will never make it to recipients and you will never know it because it is not considered a bounce!
Here is what is happening:
- You send an email with your email client, mass mailer, or website form to your sparkling new SMTP server
- Your SMTP server happily sends the email off to the destination SMTP server
- The destination SMTP server sees your email is coming from email@example.com and before accepting the emails asks the DNS system for the SPF record for the yourdomain.com domain
- If the SPF record does not exist on your DNS, the destination SMTP server is going to consider your SMTP server as not authorized to send on behalf of yourdomain.com and will reject the email
- If the SPF record is found for yourdomain.com, the IP address in the SPF record is compared to that of your SMTP server that is attempting to deliver the email. If they match, the email is accepted. If not, the email is rejected.
So, if you are setting up an SMTP server to send one email a day or 1 million, you need to have an SPF record in your DNS server for the domain you are sending from.
SPF is just one of the ways email messages are authenticated as being legitimate senders for the domain. The advantage the SPF method has over other authentication methods is that the SPF record is easy to enter into the DNS, the email message is not modified as in the DKIM method, and it is basically a perfect way to stop SPAM. The disadvantages of this method is that every email addresses domain you have needs to have an SPF record, so if you are sending emails on behalf of others, there is a lot to manage.
SPF, DKIM, and other email authentication methods are many times used in combination with one other. Most of the time these methods are used in SPAM scoring, and though an email's successful delivery is not absolutely based on the presence of these SPF records, it adds a TON of good SPAM scoring points to the equation. Below is a simple example of a SPF record entry in a DNS system. This can be added in the TXT section of the DNS record.
For HOST: normally use the @ sign which means use the domain name for the DNS entry.
For TXT: v=spf1 a ip4:XXX.XXX.XXX.XXX ip4:XXX.XXX.XXX.XXX include:yourdomain.com ~all
Change the XXX IP addresses to the IP addresses your SMTP server using to send.
Obviously this SPF record could have a lot of different variations on formatting, but this will give you an example of a simple format and shows you just how easy it is to set this record up. I hope this helps your deliver-ability!
By: Chris Lewis
Well, not really for dummies, but knowing more about this anti-spam system will help you get your emails into the Inbox. Most of the time, the question is, "should I use DKIM for my email sending..." YES! "Will it ensure my emails get into Inboxes?" NO! But, it helps a ton. DomainKeys Identified Mail (DKIM) is a method for associating domain name to an email message thereby allowing a person, role, or organization to claim some responsibility for the message.
Here is how it works:
- You send an email using your SMTP server.
- Your SMTP server that has authority to send in behalf of your domain adds a DKIM signiture header to your email and sends it
- The destination SMTP receives the email, sees a DKIM signature in the email, and then looks up the public signature on your domain using via DNS
- If the signature in your email authenticates with the public signature in your email, then the email is "good"
Now, saying that the email is "good" means that the destination SMTP knows it is from you, but the destination mail server will still SPAM score your email by the content. DKIM adds a ton of "good" SPAM scoring advantage to your email and thus will greatly improve it's inbox-ability.
So, if you are not using DKIM, use it! Click here is a great link in Wikipedia on the subject
Understand this will not have anything to do with your email sending client, like Outlook or Campaign Enterprise. The system works because it is on a SMTP server level of activity and is based on authority that can only be established by domain owners.
There are several different accepted methods in use now that keep spammers from being successful. One of those methods is the SPF designation. In order for your emails not to be rejected by the recipients mail system we recommend that you create a Sender Policy Framework (SPF) record for your domain. An SPF record is a type of Domain Name Service (DNS) record that identifies which mail servers are permitted to send email on behalf of your domain.
The purpose of an SPF record is to prevent spammers from sending messages with forged From addresses at your domain. Recipients can refer to the SPF record to determine whether a message purporting to be from your domain comes from an authorized mail server.
For example, suppose that your domain example.com uses Gmail. You create an SPF record that identifies the Google Apps mail servers as the authorized mail servers for your domain. When a recipient's mail server receives a message from firstname.lastname@example.org, it can check the SPF record for example.com to determine whether it is a valid message. If the message comes from a server other than the Gmail's mail servers listed in your domain's SPF record, the recipient's mail server can reject it as spam.
If your domain does not have an SPF record, some recipient domains may reject messages from your users because they cannot validate that the messages come from an authorized mail server.
To Create an SPF record for your domain, you will have to be able to edit your domains DNS information and use a format that specifies your mail servers.
By: Chris Lewis
The choice of the SMTP server you use is as if not more important that the email marketing tool you us. You can create the best looking, most effective email ever but if your SMTP server setup is not set up well then it is like you never sent it.
SMTP servers are the email work-horses of the internet. In the world of amazing technology, it is sometimes hard to be impressed, but I am constantly amazed by the thought and foreknowledge of the whole SMTP server framework. As time has moved on, the SMTP framework has changed, and the biggest change is that of SPAM control. There are to main problem areas we see daily with SMTP servers: How to connect to your SMTP server, and how "good" your SMTP actually functions.
SMTP Connection - The problems that are encountered when connecting to SMTP servers are varied and usually include these:
SMTP Providers - There are specialized companies out there that will act as your SMTP server. They will ensure your emails going out are appropriately make to conform to SPAM rules, they monitor their servers to make sure they are not blocked, and they will help you resolve issues that do come up is there are any problems with email recipients. Now it is true this will cost you some money, but if you add up all the time you or someone you will hire to work on these issues I think the value is there. If you are sending 100,000 emails a month, then the prices are really reasonable. If you send over a million, then that is maybe when you can start thinking of having your own SMTP server, but it really depends on the connectivity, your budget, and your expertise. One of our partners is SMTP.com (http://arialsoftware.smtp.com) and they have been a great partner with us for years. There are others, but I think they are the best, so check them out if this seems like a solution for you.
Bounced email is a subject that comes up very often in our tech support department. In this blog we will try to define all the aspects of email bounced and how to deal with them. But first we need a reason...
Dealing with bounced emails is very important. Many times bounced emails are just ignored because it does not "cost" anything to just keep sending to the same email addresses. The problem is that many mail providers like AOL, Hotmail, Gmail are all sensitive to emails sent to them that do not exist. For instance, if you are sending 1000 emails to AOL addresses and hundreds of them does not exist, AOL will consider you a spammer because you are not a "good citizen" and will at some point not allow your SMTP server to send ANY more email to them. When you do this to all the major email providers all the time you can see why many of the emails you send out will just not make it to the REAL end users. So overall, you need to deal with bounces to keep your reputation clean.
As explained in a previous blog, bounces can happen immediately or delayed. Immediate bounces happen when your SMTP server rejects the email the instant it is submitted. These records can be marked as "bad" in Campaign Enterprise and then you would filter these addresses on the next send or just delete them completely. The delayed bounced emails are the tricky ones. Here is how a bounce is generated:
1. You send an email using your SMTP server
2. You SMTP server verifies the format of the email address and accepts the email message into it's queue
3. Sometime later, your SMTP server looks up the MX record for the domain of the email address (like AOL.COM)
4. If the domain does not exist, then your SMTP server sends a bounced email to your bounced email account
5. If the domain exists, your SMTP server now contacts the destination SMTP server
6. The email message is submitted to the destination SMTP server
7. The destination SMTP can immediately reject the email message, and if it does, then your SMTP server sends a bounced email to your bounced email account.
8. If the destination SMTP does accept your email message, it can then reject it later if it finds a problem. In that case, the destination email address will send a bounce email to your bounced email account.
So, from all of this, you can see that the actual bounced email you receive can come from many different sources. The "Wild West" reference in the title refers to this process and also to the actual contents of the bounced email: There are no rules about the content. This makes if very difficult for us. To properly record a bounce we need to know two major things: The original email address we sent to, and what the error was. Sounds easy? Nope. With all the email systems throughout the world, there are that many different bounced email formats, and they change, all the time. So harvesting information from bounced emails is truly an "art" and is a good reason to keep updating your Campaign Enterprise product because we strive to keep up with all these formats.
In tomorrow's blog, we will discuss the differences between "soft" and "hard" bounces because it is a big subject in how to deal with these.
By: Chris Lewis
Email bounces that occur when you send out email from any email software sending program can occur in two different way:
An Error During Sending - During sending, if your SMTP server deems a certain email address as permanently undeliverable, then your SMTP server will return a 500-series error code which tells Campaign Enterprise or Email Marketing Director to hard bounce that email immediately. These kind of immediate hard bounces usually only occur for two reasons: The email address was malformed or the mail system you are sending through has "authority" over the domain of the email being sent and can just right on the spot if the email address is good or not. We see this happen a lot with Exchange servers where you both you the Exchange Server for receiving and sending emails. If your company domain is hosted on the Exchange server then the SMTP server of the Exchange server will only allow valid email addresses to be sent with those domains.
Returned Email - This is the most common way bounces are recorded because most of the time your SMTP server does not have the immediate authority to say whether or not an email address is valid. In this scenario your SMTP server simply relays the email message you sent to the SMTP server that has authority for the domain (from the MX record). When your SMTP server is talking to the target SMTP server, the target SMTP server might say "that email box does not exist" which is a 500 series error. This triggers your SMTP server to send the email back to you (or your bounce account). This can take seconds to hours to happen.
So, overall, bounces are somewhat difficult to deal with since they may happen from several different source, and since there is no standard for the formatting of bounces, it become the task of our software to decipher a bounced email which may have all or just part of the original message. In the next article, we will discuss the different ways to deal with the returned emails.
If you have said to yourself "I want to save money over an email ASP, I am tired of paying per email costs." Good for you.
One of the many advantages of using email marketing software is that you never pay a per-campaign fee or per-email fee. Managing your own email campaigns is far more cost-effective in the long run, and many of our customers find that they pay for the software after just two or three mailings.
That doesn't mean an email ASP isn't a viable choice in many circumstances. There are many high quality email ASPs in the industry who offer an honest service and excellent support. (Our choice is Email Labs, a Silicon Valley-based company that offers quality email marketing services.)
Most companies switch to email marketing software (bringing it in-house) only after they realize how easy it is to conduct email marketing on their own. Our clients send out hundreds of millions of emails each year using our Campaign Enterprise software, and it's usually the marketing person managing the campaigns, not a technical person.
(In other words, you don't have to be a database genius, a mail server geek, or an Internet guru to make this work really well. All you need is the same kind of understanding necessary to operate a direct mail campaign: selecting recipients, reading click-thru reports, writing effective messages, and so on.)
Just how much money can Campaign Enterprise save you over using an email service? That all depends on the volume you're sending. If you have 50,000 subscribers that you email once a week, you're sending 2.5 million emails a year. Given that email ASPs charge somewhere around a penny per email (on average), you'll spend $25,000 for that year. Campaign Enterprise, in contrast, is a fraction of that dollar amount, and you only buy it once. Once you make the initial purchase, you can send an unlimited number of future emails at no additional cost.
The Hidden Costs Of Using Email ASPs
There's more to this than just the up-front cost. Using an email ASP can cost you significant sums in database import and export overhead. Consider this: when you're using an email ASP, you have to send them your email list. That means a database person in your organization has to spend time selecting and packaging the data for exporting. Once the data is exported, it has to be delivered to the email ASP for importing into their database. And that's only the beginning...
Once the email campaign has been delivered, the click-thru counts, unsubscribe flags and bounce counts have to be exported from the ASP's database, delivered back to you, and then imported back into your database so that the source records are up to date. All this takes time and money, of course: with every database import, export or delivery that takes place, you're footing the bill.
Campaign Enterprise, on the other hand, requires no database importing, exporting or delivery. Our software talks directly to your master database. Once you tell it who to email (by selecting your database, tables and filters), it reads the records directly. Better yet, it writes back to your master database in real time, keeping your database fresh with information indicating clicks, unsubscribes, bounces and delivery confirmations. Once again, no importing or exporting required. Your database engineers will love it!
Not only does this save you tremendous time and effort, it also means your database is up-to-the-minute fresh with the latest status of every email recipient. Within seconds after someone clicks a link in your email message, your database will be updated with that event. You'll know exactly who clicked which link, on a record-by-record basis.
The Hidden Cost of Communication
There's another hidden cost of using email ASPs that disappears when you're using Campaign Enterprise software: the communications overhead of telling your ASP what you want done.
Have you ever noticed that the more people you get involved in a marketing project, the more the cost and complexity escalates? Have you noticed how much longer it takes to get things done? ... and how mistakes are far more common? That's what happens when you're trying to tell another company how to run your email campaigns. Even when they mean well, it's simply harder to make it work smoothly.
Now, instead, imagine conducting email marketing campaigns in house, with you and your small team in control. You don't have to tell someone else how to do what you want: you just do it yourself in Campaign Enterprise. Voila! Suddenly, your campaigns are happening more quickly, with fewer errors, and with much less overhead. You're in control, and you don't have to spend time telling someone else how you want things done.
Saving Money Is Just The Beginning
In reality, saving money is just a small example of the many reasons for choosing email marketing software over a service provider. It's a big reason, yes, you will save a lot of money by using Campaign Enterprise. But the bigger reason is how much more revenue you'll earn by using the software. Once you explore and recognize how Campaign Enterprise can enhance your relationships with customers, generate increased trust, and rapidly escalate your follow-up sales figures, you'll really start to see the tremendous benefits of managing your own email marketing with Campaign Enterprise software.