Measuring a Predictive Model’s Email Marketing Results - Part II

On Monday, I began a discussion about how Istobe evaluates the ROI from email marketing campaigns based on our predictive models. At the end of my post, I promised a discussion about other factors that we take into account when evaluating the lift. And…voila. Today we unveil those factors: the email influence zone and opt-outs, and we discuss how Istobe accounts for them in our lift calculations.

Email influence zone
Sometimes referred to as decay rate in the catalog industry, the email influence zone (EIZ) - not unlike the catalog influence zone (CIZ) - is essentially the time period after an email is sent. And we assume that each succeeding day after the email is received has less effect than the day before. Thus, the moniker decay rate. Catalogers have believed for years that their catalogs have a carry-over influence: the catalog accounts for many web purchases. In fact, this is very reason that catalogers are loathe to cut the number of catalogs that they ship. Even to those customers who have never purchased from the catalog itself. We believe this is also true of email marketing.

Basically, the idea behind the EIZ is that an email offer has an effect on online purchases that have no other obvious origin and which relate to the product that we predicted. For example, if our models predict that shoes are the likely next product for a particular customer and that customer purchases shoes online five days after receiving an email that advertises shoes, then we can assume that the email - and our product recommendation - influenced the customer’s purchase. Our model gets credit for a small percentage of this purchase even though the purchase didn’t come directly from an email click-through. The EIZ period that we calculate differs per client depending on the frequency with which our clients send emails.

Opt-outs on the Istobe watch
If we’re going to give ourselves some of the credit for purchases that occur in non-email channels, we also have to take a hit for bad events that occur during our watch. The bad event that Istobe tracks carefully is email opt-out. We track whether the opt-out rate goes up during our watch. If it does, we have to assume that next-best offer has somehow turned customers off. If the opt-out rate does go up, we deduct a portion of our lift because we believe that we were responsible for that incline in opt-out rate. We’re responsible for that small piece of customer attrition.

Taken together with the variables I spoke about last time, these are just four factors that we constantly adjust in determining how successful we are on behalf of clients. And we’re always looking for new ways to perceive actual lift. If you have new ideas for evaluating predictive-model efficacy, please email me. I’d love to talk about them.

Tags: email, Email Timing, Lift Calculation, Personalized Marketing, Predictive Analytics, ROI measurment

Add Personalized Product Recommendations to Your eCommerce Site

Istobe is a powerful recommendation engine that makes it easy to add recommendations to your eCommerce site. How powerful? Shoppers who click on Istobe recommendations spend 20-50% more than the average visitor.

This entry was posted on Thursday, July 24th, 2008 at 8:14 pm and is filed under Email Marketing, Email Timing, Lift Calculation, Personalized Marketing, Predictive Analytics. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.