Tag: email marketing

Deliverability 101, Delivery Essentials

Deliverability 101: Spam traps

Brad

It's a trap!Recently we presented a basic guide to blacklists using a rather colorful example ripped from today’s headlines. In that post we described what happens when you’re on a blacklist, but as a sender you probably want to know how to avoid getting blacklisted in the first place. And if you’re already blacklisted, you’ll certainly want to find out how you got there. Understanding spam traps can help with both.

What is a spam trap?

While the name might conjure thoughts of being lured into a sticky situation via canned meat, a spam trap is actually an email address. Sometimes referred to as honeypots, spam traps are addresses that exist for the purpose of identifying senders who are not following best practices.

A spam trap isn’t used by a real person to send or request email. Each trap is monitored by the trap operator (typically a blacklist or mailbox provider) and any mail sent to the address can cause the sender to be put on a blacklist. While the potential exists for blacklisting based on a single message sent to a trap, providers most often look for patterns of repeated hits. This could mean multiple mails to the same trap address, mail to multiple distinct trap addresses, or both.

There are two main types of spam traps in the wild: pristine and recycled. Pristine traps were created for the sole purpose of being a spam trap. These addresses have never been used by a real person and have never requested any emails. If you send mail to purchased lists or scrape addresses from the web, there’s a good chance you’ll run into this type of trap.

Recycled traps, by contrast, are the type most commonly seen by legitimate email marketers. These email addresses did, at some point in the past, belong to a real person. That person likely sent emails, signed up for mailing lists, and provided the address to others as their point of contact. Then, for whatever reason, that person abandoned the address – maybe due to an organizational change or migration to a new mail provider.

Once the recipient abandoned this address it sat dormant for some period of time (generally at least 6 months), during which time the address would have rejected all mail. After that period, the address was reactivated and became an active spam trap.

How do spam traps get on my list?

Since spam traps are designed to identify senders not using best practices, it stands to reason that failure to follow best practices typically leads to their presence in your database. Mailing to recipients who have not given opt-in permission, sending to old or outdated lists, and lack of proper bounce handling are some of the most common reasons spam traps end up within your list. In addition, typographical errors at the time of address collection can introduce traps into your list – particularly with less accurate address collection methods such as point-of-sale address transcription or collecting addresses via telephone.

What happens if I have spam traps in my list?

When you send mail to a spam trap address, the trap monitor will note the sender of the message and typically take some action against that sender. Most trap monitors also maintain their own blacklists, and in many cases these blacklists are publicly used by many ISPs and mailbox providers to filter mail. In short, sending to spam traps will probably get you on a blacklist, and that blacklisting will probably get your mail rejected by at least one major email provider.

I’m not being blocked. Why does it matter if I have spam traps in my database?

If you receive word (from your ESP, delivery monitoring service, or a trap owner) that you are sending to spam trap addresses, it’s tempting to gloss over the warning if you’re not seeing any large-scale delivery issues. A word of advice? Don’t ignore spam traps.

The presence of spam traps in your contact database is an indicator of an underlying issue with either your email acquisition practices or your list maintenance protocols. When you have spam traps in your list, you are sending mail to contacts that don’t want it or never requested it. This means that alongside the traps, you are also mailing real people who will (at best) ignore your message or (at worst) report you to their mailbox provider or a third-party spam filtering service. Even if the spam traps haven’t gotten your mail blocked (yet), you can bet the spam complaints and low engagement are keeping you out of your recipients’ inboxes.

How do I get the spam traps out of my list?

Removing spam traps from your list is, by design, a difficult process. A spam trap doesn’t (usually) bounce or reject mail. It doesn’t provide any signs or signals that it’s a trap. The trap operator doesn’t want you to be able to spot the traps in your list, because then you could simply remove the traps and not address the underlying issue.

If you have traps in your database, the best place to start is typically contacts who haven’t engaged with an email (opened or clicked) recently. Typically, we recommend targeting contacts who haven’t opened in 6-12 months. Send a confirmation request to those non-openers, asking them to confirm they are real and they still want your messages. Once that message is sent, you’ll want to suppress from your list anyone who doesn’t respond. It’s also a good idea to repeat this process at least once a year.

In conjunction with addressing spam traps already in your list, you want to make sure you cut off traps at the source. Check your list acquisition practices to be sure all of your incoming recipients have opted in for your mailings. Add CAPTCHA to any public-facing web forms to prevent automated sign-ups. Think about adding a confirmation step to your opt-in process. This could be a traditional confirmed opt-in (COI) where recipients have to click a link to be confirmed, or it could be a “soft confirmation” that considers an open to be a confirmation action.

If you can take actions that make it harder for spam traps to end up in your list, you’ll proactively decrease your risk of dealing with the difficult process of culling your list to get rid of them later.

Have a war story or questions about dealing with spam traps? Leave a comment or shoot me an email to chat!

– BG

Industry Updates, Laws and Regulations

CRTC levies first CASL fine against an individual

Brad

Last week the CRTC, the Canadian regulatory body tasked with CASL enforcement, issued notice of yet another notice of action for violations of the Anti-Spam Law. The Commission imposed a penalty of $15,000 against William Rapanos, alleging that messages sent by Mr. Rapanos in mid-2014 were in violation of multiple provisions of CASL.

This decision is noteworthy because it represents the first time a CASL penalty has been levied against an individual. All previous actions to this point have been issued against companies or corporate entities: names like Compu-Finder, Porter Airlines, PlentyofFish, and Kellogg Canada are among those hit with prior penalties.

The CRTC decision indicates that messages from Mr. Rapanos were sent without the recipient’s consent, did not clearly indicate the sender of the message, made it difficult or impossible to contact the sender, and (in at least one case) included no unsubscribe method.

Another interesting tidbit is that the Spam Reporting Centre received a total of 58 complaints about Rapanos’ messages. These complaints were mostly unique, with 50 different recipients lodging complaints to the SRC.

In discussions about CASL, I’ve heard quite a few people theorize that the CRTC is only looking for large-scale violations and penalties against smaller senders or individuals are unlikely. William Rapanos may have thought the same thing. Or he may have thought the effort and cost involved in CASL compliance weren’t worth it. Then 50 people complained, and now he’s on the hook for $15,000.

I think this decision – and the resulting penalty – proves to Mr. Rapanos and to all of us that compliance is definitely worth it.

Have questions about CASL compliance? Disagree that compliance is paramount for every sender? Leave a comment or email me to keep the discussion going!

Delivery Essentials

Know your Role: Why you should avoid role addresses

Brad

dicerole I’ve had a lot of discussions with clients who have (or should have) concerns about the role addresses in their lists. Most discussions revolve around the presence of these addresses in the subscriber database and why Real Magnet prevents them from being loaded by default.

To provide some context, let’s clarify what we call “role addresses.” They’re typically defined as any address that is assigned not to a specific individual, but instead to a role within an organization. These addresses include, but are not limited to:

  • sales@
  • info@
  • admin@
  • hostmaster@
  • abuse@

…and the list goes on and on. There is no definitive list of role address prefixes, because domain admins can set up role addresses for literally anything. However, there are really just a couple dozen role addresses that are most commonly seen, including those examples above.

Even when these addresses are provided via a clear opt-in method, they are still more likely to cause delivery issues. As a result most providers block them from being imported into email lists. In fact, some ESPs have reported that role addresses are 2-3 times more likely to unsubscribe or bounce than non-role addresses, and lists containing high numbers of role addresses typically see sharp declines in engagement.

So what’s the big deal? Why exactly are role addresses problematic? Most (but not all) of the problems with role addresses hinge on the fact they often route mail to a group of people instead of a single contact. Let’s look at some of the biggest concerns:

  1. Permission is often impossible to confirm. With multiple recipients at the same email address, opt-ins are murky at best. Let’s say you get an opt-in request from sales@example.com, which includes everyone on the Sales team at that domain. That request was made by only one of the people who actually receives those mailings. Even with a confirmed opt-in, it’s possible that the other recipients of that address do not want to receive your mailings. They are more likely to report the message as spam or unsubscribe, which brings us to…
  2. Unsubscribes get applied too broadly or reversed too easily. With a role address, one of the recipients of the message may choose to unsubscribe even while others want to continue receiving your mailings. If you use an unsubscribe method that is linked to a custom recipient ID (as most ESPs provide), that person will be unsubscribing the group instead of just his own address.In addition, recipients often send an email request to unsubscribe (or report spam). This email will likely originate not from the role address, but from their personal address that actually receives the mail. If the recipient fails to include message headers or the headers have been distorted by their email client, it is sometimes tough to know what address actually received the mailing. The personal email address may be added to a suppression list, but the recipient will continue to receive your mailings.
  3. Roles (and the people who fill them) change regularly. Let’s say you have a role address that directs to a team of employees, and let’s also say that each of those employees has made it clear they want to receive your emails. Great! Now what happens when a new person joins that team? Or a member of another department is added to that role address group to be kept abreast of team-related emails? To avoid all issues, you’d have to get a new consent each time someone new joins that address. That’s certainly possible, but not really practical and nearly impossible to track.
  4. Bounces are more difficult to process. If one of the recipients of your email leaves the organization and their address goes dark, there’s a chance the bounce response never reaches you (depending how the org handles bounces and routing). But even if you do receive the bounceback, you may not know how to process it. The address returning the bounce will not be the one that’s in your mailing list, which will cause most automated bounce handling to choke. If you continue to send to invalid addresses domains may block all your mail or, even worse, one of those addresses may roll over into a spam trap and get you blacklisted.

Now that you know the problems with role addresses, how do you avoid these issues? If you’re building a new list, you should require that recipients enter a non-role address. There are a couple of ways to do this depending on your technical resources. The most effective method is to implement real-time address checks that will reject role address submissions, but this also requires the most technical overhead. Many senders choose instead to simply inform their subscribers that role addresses cannot be used. Then, those addresses will be discarded upon list import and will not receive email.

For senders who already have a list containing role addresses, the next steps may be a bit less clear. If your list contains only a few of these addresses, it may be best to reach out to the individuals to ask them to provide a different address. This can be accomplished through a one-to-one email, phone call, or even a face-to-face conversation.

When you have a larger number of role addresses, we typically recommend sending a bulk campaign asking these recipients to update their information. Most senders do this via email (outside their ESP in many cases) or a postcard encouraging users to update their preferences to continue receiving mail.

Typically, recipients who want your emails will be willing to update their information. However, if you have some recipients who can only use a role address, many ESPs will evaluate on a case-by-case basis whether or not those addresses can be allowed in your list.

– BG

Delivery Essentials

Is my ESP lying to me?

Brad

Collodi_PinocchioIf you use an email service provider, you most likely have tracking reports that tell you the disposition of each message you send. These reports usually indicate the message falls into the broad categories of “delivered” or “bounced“. While many ESPs use more detailed categories, you just want to see if your email made it to the recipient or not…right?

Of course. So what happens when you send out that nice shiny new email and you get a response rate far lower than what you were expecting? Naturally, you check in with some of your best recipients to make sure they got the message. Your tracking shows delivered, but when you reach out they say they didn’t see the message at all. Not in the inbox. Not in the spam folder. Not even in quarantine…now what?

Why is your ESP telling you the message was delivered when it clearly wasn’t?

To answer this common question, let’s dive a bit deeper into what that “delivered” status really means.

When you hit the Send button at your ESP, your mail server will attempt to hand off your message to the mail servers for each recipient. The initial contact between the sending mail server (your ESP) and the receiving mail server (your recipient’s email provider) is often referred to as the “handshake.”

At the time of this handshake, the ESP server will attempt to hand off the message to the receiving server. When this happens, there are a few potential outcomes:

  1. The receiving mail server rejects the message due to the address not existing, the sender being blocked, or other errors considered permanent. These are hard bounces, and usually the receiving server returns a code in the format 5xy, where x and y are additional digits that indicate the specific type of hard bounce. This error typically causes a bounced status in your ESP reporting.
  2. The receiving mail server returns a temporary bounce or deferral. These bounces indicate the mail cannot be delivered at this time, but the sending server should try again later. These are soft bounces, and are typically accompanied by a 4xy error code. These can generate a bounced status in your ESP reporting if the subsequent delivery attempts are not successful. If the later sends do make it through, these will show as delivered. 
  3. The receiving mail server accepts the message for delivery. This is considered a successful delivery, and is accompanied by the code 250 OK. These are reported by your ESP as delivered. 

Once this handoff takes place, the sending server (your ESP) has no further visibility into the delivery of the message. There could be additional spam filters in place after the message is accepted, or individual user settings could cause the message not to be delivered, with no further notification to the sender.

While it’s not extremely common, even major ISPs have been known to have messages “dropped on the floor” if the sender’s reputation is not up to their standards. This is the (highly technical) term for a message that is accepted by the receiving server, but then essentially disappears. It’s not returned to the sender, but it’s also not delivered to the recipient’s inbox or spam folder. It’s simply deleted.

So how do you find out what really happened?

That can be the tricky part. Since the information is not shared with the sender or ESP, the only way to find out for sure what happened to one of these messages is to check the mail logs for the receiving server. In most cases, this will require working with IT staff on the recipient’s side who can search for the message(s) in question and provide a definitive answer on what happened to the message and why.

If the recipient’s IT team isn’t an option, you can also check the content of your message, as well as the reputation of your domain and the domains of any links within the message body. In many cases the initial handoff looks primarily at the reputation of the mail server (IP address), while the subsequent filters can include message content, link URLs, domain reputation, and other factors.

Check out the Resources page for links to some of the most popular reputation tools, and feel free to comment with any additional questions.

– BG

Delivery Essentials, Events

Shield your sender reputation

Brad

capshield05UPDATE: The webinar is over, but don’t worry! You can download the recorded version here. 

Do you know how a blacklist works? How about a blocklist? Did you know there’s a difference?

Or that complying with anti-spam laws doesn’t guarantee a good reputation?

If not, don’t worry – most senders have lots of questions when it comes to sender reputation.

Chances are, you’re probably doing something right now that could get your IP address or domain blacklisted, which could have a major impact on your email deliverability. And even if you’re not, there’s likely more you could be doing to safeguard your sender reputation.

Tomorrow afternoon, I’m hosting a webinar designed to break down how blacklists work, what happens when you’re listed, and some steps you can take to help ensure your reputation is in top form. It’s at 2pm EST, and you can sign up at the link below.

http://www.realmagnet.com/land/protect-sender-reputation-tips-trends-avoid-deliverability-pitfalls-2/

If you can’t make the webinar but still have questions about reputation or blacklists, feel free to post in the comments here or email me!

– BG

Delivery Essentials

Spam filters change. Deal with it.

Brad

“But I didn’t change anything!”

courtesy http://shop.cnc-design.fi/
courtesy http://shop.cnc-design.fi/

We’ve all heard it (and maybe even said it). When your mail suddenly starts landing in the spam folder with one or more mailbox providers, the first response is often to point the finger at the mailbox provider or even your own Email Service Provider, since you just know nothing changed with your mailing program. However, even when you don’t think anything has changed, there are often many unseen factors that can make or break your email deliverability.

Sure, whether or not your mail reaches the inbox – or even gets rejected by the recipient – is often affected by things that are easy to see. Changes like sending to a new list, adding IP addresses or domains, or trying out all-new message content are easy to pinpoint when delivery issues arise. But they’re not the only factors that cause failures.

The performance of your mailings can change almost daily, and can be one of the biggest factors in how your mail gets delivered. User engagement, both positive and negative, plays a huge role in inbox rates. If users are opening your mail, moving it into the inbox or Primary tab, assigning a label, responding, etc., that positive engagement is more likely to improve your inbox delivery. If users ignore your message, move it to a bulk folder, or lodge a spam complaint, it could spell bulking or even rejection due to negative engagement.

You probably already know all of this, right? Every email blogger in the world has drilled engagement into your head. But don’t let your eyes gloss over just yet…

What you may not be considering is that these behavior patterns can change without any major change on your part. Maybe the offers in the past couple of emails haven’t been as appealing to your customers, so they’re not opening. Or your business is more seasonal, and engagement rates are lower due to the time of year. And what about other factors that may increase spam complaints, like general email fatigue around holiday seasons? All of these factors can affect where your message lands, though they may not be the most obvious at first glance.

But let’s assume for a minute that you’re right – absolutely nothing on your end has changed. Your open rates are identical, content is constant, and no seasonal malaise has taken hold. Even if true, that’s only one side of the coin. Mailbox providers, ISPs, and spam filter operators regularly change their filtering criteria, which could send your mail from inbox to spam folder at the flip of a switch. Some providers, like Gmail, are “smart” about these changes and base the adjustments on observed user behavior and complex algorithms. In fact, many of the factors that impact delivery at Gmail can change almost daily, based on mailing patterns.

At the same time, many providers make more arbitrary changes – based on observed data as well, but not quite as fluid as those at Gmail et al. These changes might include lowering a spam complaint threshold, or even turning on new spam trap addresses. Often the major blacklist providers like Spamhaus will monitor spam trap addresses for traffic before actually making them active spam traps. Even though you are mailing an address that wasn’t a spam trap yesterday, it might become one today.

As a sender, you simply need to take precautions to ensure your email program is resistant to these changes. If a small change in engagement or filtering criteria is enough to derail your inbox rates, then it’s likely you weren’t following best practices before the change. If you are getting clear permission, monitoring and targeting your most engaged contacts, and cleaning your list of outdated and dormant subscribers, you’re on the right track. But to get the most effective insurance against these changes, you should be watching engagement and list hygiene regularly. Check engagement quarterly or even monthly. Measure which types of content or sending frequency generate the most (and least) engagement.

Even so, there will be times that even those best practices are not enough to navigate the muddy waters of email delivery. Fortunately for you, there’s a whole segment of the industry who specialize in email delivery, privacy and compliance.

To misquote a famous tome, “Those who can, do. Those who can’t, hire an expert.”

– BG

Deliverability 101

Deliverability 101: Shared or Dedicated IP?

Brad

As children, most of us probably learned that it’s nice to share with others. It’s one of those timeless lessons that often carries throughout adulthood – but does it also apply to mail server IP addresses? It can, but there are a number of factors to consider to accurately make the determination whether a shared or dedicated IP environment is better for you.

The Basics

The question of shared vs. dedicated IP is most often asked when choosing an email service provider (ESP). While some ESPs specialize in one or the other, many offer both shared and dedicated IP options for senders depending on their sending patterns.

In a shared IP environment, one or more IP addresses are arranged into pools that are shared among multiple senders. At any given moment, mail from multiple senders is likely to be sending over one or all of the IPs in the shared pool. This also means any mail you send will be spread across some or all of the IPs in that pool.

A dedicated IP environment will provide you with an IP address that sends mail for only your organization. No other traffic from any other sender will use that mail server and it will be easily identifiable as ‘your’ IP.

Which Option is Right for You?

Hopefully we all have a clear understanding of what constitutes a shared vs. dedicated IP setup, but the burning question remains: which should you choose? There are a few factors that help make this determination:

  1. Sending volume.
    When you send from a dedicated IP, you need to be sure that you have enough sending volume to establish and maintain a reputation as a “known” sender at the ISPs you are attempting to reach. Volume recommendations will vary depending on list composition and other factors, but we typically suggest around 500,000 in average monthly volume as a good rule of thumb. This will help to ensure that getting a few extra complaints or bounces one day won’t completely derail your deliverability.
  2. Sending frequency.
    When considering a dedicated IP, the consistency of your send volume is every bit as important as the volume itself. In order to maintain that reputation with the ISPs, you need to send consistently and regularly. For example, if you send 1 million emails per month, but your sends are split 800,000 on the 3rd of the month and 200,000 on the 17th, you may not see the full benefits of a dedicated IP. We suggest at least 100,000 emails every week as a good starting point.
  3. Reputation.
    Even if you aren’t paying attention to reputation as a factor in your decision, you can bet your (potential) ESP is. If you are a sender who is following best-of-the-best practices, gets confirmed opt-in for every subscriber, and sends consistent high volume, you are not likely to see benefits from a shared IP pool. Since shared IP pools send mail for multiple marketers, the positive or negative reputation of each can impact delivery for every sender on the pool.

    While many ISPs are placing more weight on domain-based reputation, the IP address sending the mail is still a major factor at most every mailbox provider. As such, your mail delivery rate could be negatively impacted by the performance of others on the shared pool. Most ESPs have monitoring in place to ensure senders do not take actions that will seriously harm the shared IP reputation, but there will always be a higher level of shared risk than with a dedicated IP. However, by the same token, if you are not getting clear permission, sending irregular volume, or otherwise not following best practices, you are likely to see issues no matter which IP option you choose.

If you have the volume and/or consistency to support it, there are few cases where a dedicated IP setup would not be recommended. But if you are a smaller-volume sender, or only need to send sporadically, you can certainly find success on a shared IP pool with the right ESP.

– BG

Deliverability 101

Get Ready for Deliverability 101!

Brad

Question BoxOn May 27th, I presented a webinar with WhatCounts titled Deliverability 101: Back to the Basics. This webinar covered many of the core ideas and concepts of deliverability including bounces, mail filters, spam traps, and inbox delivery. The turnout for the session was excellent, and the content drew so many questions that time wasn’t available to answer them all.

Starting next week, I’ll be diving deeper into some of those same topics, both here and at the WhatCounts blog. Some of the posts will be based on questions received during the webinar, and questions or comments received on the posts as they progress. This will be an ongoing series designed to provide an easy point of reference for anyone in the email industry to get a basic education in the science (or is it art?) of deliverability.

If you’d like to download the WhatCounts webinar, you can find it here.

Also, if you’d like a bit of insight into my daily delivery activities and thoughts on the industry, check out this Q&A with Ashley Hinds of DataValidation.com.

– Brad Gurley