The Istobe eCommerce Analytics Blog

Over the past two weeks, I’ve been watching the fallout from two important conferences taking place on the West Coast (where I would much rather be right now given the latest round of snow in Boston.)  The first annual Predicitive Analytics World ’09 was staged in San Francisco on Feb 18th and 19th and eTail West 2009 took place last week and drew in a host of e-Marketers from around the country. One common theme that I seem to be hearing from the blogs and articles I’ve read from both: make sure you’re using your customer data effectively and, most importantly, keep it simple.
Read Simplicity is Key for Marketing Analytics »

Kevin Hillstrom has an excellent post today entitled Customer Retention: The Myth where he says improving customer retention might sound like a good idea but is actually quite difficult to do. I agree with his assertion that retention rate, average order value, and orders per buyer are difficult to improve in the short term, but I, perhaps unlike Kevin, I think that there can be real immediate gains to be had by trying.

Read Customer Retention: a Tough Needle to Move? »

As marketers, we only have a few levers to really impact the bottom line. We can acquire customers more cheaply, acquire more valuable customers, increase customer purchasing amount and frequency, and improve retention rate. We only have so many resources to devote to moving these levers.  Which one should get attention first?

Read Which Marketing Lever is Biggest? »

I have to admit that it’s been a while since I’ve written on this Istobe blog, but for good reason. My wife and I welcomed a little girl into the world (Vivian June) on December 28th and have been battling sleep deprivation and dirty diapers ever since (and yes, she’s definitely worth it).  The bad news is that I just haven’t had much time to read all of the news feeds and blog posts that I would have liked to over the course of the last few weeks. So, as I looked blurry eyed through my blog posts and news items this morning, I noticed a few articles that point out some important marketing trends for retailers in 2009:

Read Marketing Trends For Retailers in 2009 »

We all agreed long ago that the advantage of pay per click ads over more traditional advertising was that we could use conversion rates to measure ROI accurately. As a result, we amped up spending on ads that were paying off and ruthlessly eliminated those that failed to reach breakeven.  

What we didn’t consider, however, is that many of us have been calculating breakeven wrong.

Read Measuring PPC ROI: Ur Doin It Wrong »

As the economy turns ever uglier, it might be a good time to take a long hard look at your business to accurately assess the health of your customer base. Businesses with strong healthy customers are in prime position to weather the economic storm and take market share from flailing compeititors. But if you work for a company with a growing share of weak, low value, high cost customers, it might be time to start getting that resume togther.

Here are the three signposts that indicate your customer base is weakening:

Read How Healthy is Your Customer Base? Here are 3 Metrics to Find Out »

Continuing our research into how to navigate the social and micro-blogging networks, specifically twitter, I want to dicuss the various sites and tools which can be used to draw a map of the twitterverse.  The tools fall into categories of search and survey.

Search. Like every other internet based network there are several solutions for searching the twitter world.  Each search tool is oriented towards a specific goal.  For searching against keywords there are 2 good options: Twitter Scan and Monittor.

Read Twitter Analytics »

One question that we’re repeatedly asked by our customers is how they compare to the “average” online marketer or retailer in terms of conversion rates, number of campaigns, customer acquisition, etc.  Many times it’s hard to pinpoint exact numbers with much accuracy because of differences in business model and marketing channels, but a new survey that just came out on eMarketer gives a fairly nice overview of how current online retailers (limited to the US) are doing in regards to email marketing.

Read Better than average? »

I want to take a minute and build off of Matt’s post from yesterday. While lift, confidence, and support may sound like terms that are more applicable to therapy sessions, they are actually the metrics that we use to rely the trust we have in our models. Many people we talk with are familiar with these terms on a certain level, but when pressed, their understanding boils down to the following: higher is usually good and lower is usually bad. I wanted to use this post to define these metrics a little more thoroughly and talk about how they’re calculated.  Hopefully, readers will come away with a better understanding of what they mean and exactly how they’re used.

Support

In order to talk about more complex terms like lift and gain, we need to first start with the basics: support.  Support, sometimes referred to as the cover, is the number of data points (customers, transactions, etc.) that meet a set of rules and/or assumptions.  If I do a market basket analysis and find that customers who buy milk also buy cereal, the support would be the number of customers in the sample set where this holds true. Obviously, you can only estimate the value of the support number when given the size of the total sample population which is why we have our next metric: confidence.

Confidence

Since a rule with a support of 900 looks good when the sample size is 1,000 and not so good when the sample size is 1,000,000, we need a way to easily figure out whether or not our support is significant. Confidence is a ratio that takes the support number and divides it by the number of instances where the rule may hold true (or to be more exact - where the antecedent of our rule holds true).  For instance, in our milk/cereal example above, confidence would be the total number of customers who bought milk and cereal divided by the total number of customers that bought milk.  While it’s true that the higher the confidence the more reliable the rule, it is important to note that knowing the total sample size and the support value as well as the confidence is necessary to get an accurate picture of the rules significance in regards to the total population.

Benchmark

I define benchmark here because it makes it easier to explain both gain and lift.  Benchmark is the total number of items (customers, transactions, etc.) that meet an outcome divided by the total number of items in the database. Let’s go back to the milk/cereal example. Since cereal is the outcome that we are trying to predict, the benchmark would be the total number of transactions where cereal was purchased over the total number of transactions in the database. In layman’s terms, if we were randomly picking 100 items out of the database, it is the percentage of those items where the outcome would hold true. Benchmark is valuable because it puts a lower bound on the value of a model. If a model can do better than the benchmark value, then it provides real value to the customer.

Lift

The most common term that is used in statistics and especially analytics is lift.  Lift is a way to measure how much better a model is over benchmark. It is defined as the confidence divided by the benchmark and any value that is greater that one suggest that there is some usefulness to the rule. Many applications show lift in a chart. In these instances, the total population is divided into deciles - ten even groups - into which members are placed based on their predicted probability of response. The highest responders are put into decile 1, etc.  Lift is then calculated for each of these deciles and plotted on a line chart.

Hopefully, this provided a little more insight into how we calculate the value of a model. Next time, I’ll run through a complete example to show how these are calculated in practice.

One of the toughest problems facing organizations with many customers and large numbers of transactions is determining when a customer has left the company for good. Unlike contractual businesses where the customer is locked in for a given period of time, most companies have to make educated guesses about whether or not a customer is worth spending their marketing dollars pursuing.  While there are complex data analysis models that can be more than 80% accurate in predicting when a customer will buy next, many times a quick and dirty calculation can give your marketing executive a better indication of who to contact in the future.

We sometimes use the following metric to give an initial suggestion of whether or not a customer is still active in a given period. To clarify, period here can mean any observable time length - days, weeks, months, even years can work depending on your type of business and how long your typical customer stays engaged:

Probability (Active) = (T/N)ⁿ

where:

T = last period when the customer made a purchase

N = total periods in your observation time

n = total number of purchases during observation time

To give you an example of how it’s used, let me take the following example out of the book Managing Customers for Profit: Strategies to Increase Profits and Build Loyalty by V. Kumar (highly recommended) to illustrate the power of its simplicity. Let’s say I have two customers with the following purchasing pattern over a twelve month time horizon (X’s indicate when a customer made a purchase):

The probability for each customer to purchase in time period 12 is calculated as follows:

Customer 1 = (7/12)⁴ = 11.58%

Customer 2 = (8/12)² = 44.44%

The initial results may seem counter-intuitive - why does Customer 1 have a lower chance of purchasing again when he/she has 4 purchases in the observation time while Customer 2 only has 2 purchases? Well, as you can see from the table, Customer 2 appears to buy every 4 months and is poised for another purchase in month 12 while Customer 1 was purchasing every 2 months, but hasn’t made a buy in over 5 months so is unlikely to buy again in month 12.  Nice, huh?

While the above metric doesn’t take into account all important information, it does provide a fast and easy way for marketers to get a better indication of whether a customer should be put into the active marketing group or the re-activation group.