eCommerce Analytics

I’m a big fan of Seth Godin, but his latest blog post, Ads are the new online tip jar, seems a bit shortsighted to me.  Seth argues that to support their favorite blogs readers should click on an ad to provide some small financial support to the blogger.  Taken to its logical conclusion, however, this strategy will only harm bloggers and other content providers in the long term.

Read Ads the New Online Tip Jar? Maybe in Small Quantities »

What’s not to like about fantasy football? It raises the importance of the unimportant football games (i.e., those games in which your team is not participating). It allows for some of the finest trash talking on the planet - with few repercussions for losing the Sunday match-up as long as you win the weekly war of words. And, most importantly, fantasy football gives Istobe yet one more opportunity to crunch data in an effort to predict outcomes. So in the next two weeks, we plan on crunching the last seven years of NFL player data in an attempt to predict this year’s fantasy workhorses.

Read Get Ready for the Ignite Fantasy Football Draft Predictions »

I noticed that the RRW Consulting blog alluded to an article on Friday that I have been promoting to my peers: a research report by the Aberdeen Group (abstract here) that discusses the importance of email personalization. The one-to-one marketing emphasis in the article is precisely the kind of email targeting that we espouse here at Istobe. Today, I want to expand on one aspect of the Aberdeen report that we spend extra time on at Istobe: the importance of the buying cycle in determining what kind of email message to send your customers.

In the Aberdeen article, Ian Michiels mentions that web analytics provide great clues to assessing where customers are in the buying cycle. For example, if a customer invests a vast amount of time clicking about a product group, that customer is likely doing research and is in the market to buy a product in that area. A discount offer, Michiels says, would likely get this customer - who is now highly qualified and advanced in the buying cycle - to act on their desire and make a purchase.

I totally agree with this sentiment. But as Chris mentioned in detailing his experience with GPS systems at Amazon, there is another way to do this. Customers can clue you into what they want via their clickstream. But even if you don’t have clickstream data, transaction histories, once supercrunched, can give you a leg up on finding customers who will likely buy next. In other words, this supercrunching can help you locate the customers that will likely buy before they locate you.

How does this work? Well, other customers have come before them and laid out patterns that aren’t perceptible to you and I but are very perceptible to Istobe’s predictive models. Istobe’s models throw out those customers that are not likely to buy again and then work with those who are. From there, Istobe’s models assign the products that are likely to be purchased by these likely buyers.

I won’t argue that this method is more statistically powerful than clickstream data, which is a solid indicator of future behavior. But I will argue that clickstream data takes vast amounts of resources to capture and use, a difficult proposition for online retailers who are just dipping their toes into analytics. And using transactional data to predict who will buy next is a more proactive approach. So what do you get from that proactivity? Probably a two- to three-month head start on your competition. You can focus on targeting your “most likely” customers with act-now offers while your competition waits for these customers to visit their web site.

Something that we constantly talk about around Istobe is why more small companies aren’t interested in implementing customer analytics. Especially given that these companies are pressed now more than ever to keep up with big companies that run customer analytics algorithms as part of their hourly routine. And in an era when a tome like Supercrunchers proclaims a new dawn for predictive models in business. Even with a value proposition that demonstrates a solid return per thousand customers, we often have problems convincing customers that our mojo is good and that data mining is more than just voodoo. Well, today I prepared a really simple sample revenue calculation to quantify the value of Istobe’s predictive models. Take a look at how much more an Istobe-based email campaign makes on a yearly basis.

The rule of any business venture is it must either make money or cut costs for clients. If MIT Sloan didn’t drill this into us, we got a refresher into this the other day when Doug, Chris, and I met with Chris Merrill, founder of Thrive Networks, The Orchid List, and Owner of Ass Industries. He broke that down pretty plainly for us when talking about our need to educate our customers. Well, here is as plain a case as I can make for the revenue increase that Istobe brings to the table when we mine customer data to determine what product they want to buy next.

The Revenue Increase

Assuming a company that emails 500,000 customers weekly, 50 emails a year, and $1.90 per email open, the Istobe predictive models allow a company to make $412,300 more than just basic email blasts and nearly $80,000 more than a baseline targeting approach. There is a full calculation below but let’s talk about what some of the numbers are before you examine them.

The Setup

First, let’s take a very conservative number of 4% as the increase in predictive power of our models over a general baseline. In this case, we assume the baseline to be the company offering the same product that a customer already bought. As creatures of habit, consumers are typically likely to buy the same thing from a company that they just bought. Let’s take dog food at a grocery store for example. Whoever buys dog food probably has a dog and there is a good chance that they’ll keep buying dog food. It’s a safe bet. The same goes for internet retail. If I buy a shirt from an internet retailer, it’s likely I’ll go back there to at least look for another shirt - assuming I didn’t hate the first shirt and the customer service was adequate. Now, the pitfalls of offering the same product is something I won’t go into in detail here; suffice it to say that offering the same thing that a customer is likely to buy again and again from your company is not the best business move. Really, your company should be looking to expand the share of your customers’ wallets. And this means offering them new products that they currently buy elsewhere. Like I said, a topic for another day. So Istobe does about 4% better than offering the dumb alternative, which is offering the same product again. 4% doesn’t sound like a lot but let’s play this through.

Now, let’s assume that a client uses our predictions to send out targeted email. It’s been assumed that normal behavioral targeting enhances your open rate by 40%. This means that just offering your customers the product that they bought last time increases your open rates by up to 40%. Well, I’m not so sure that I trust those studies so let’s just play it conservative and say that targeting spurs a 20% open rate increase. What that means ultimately is that Istobe predictions improve the targeting - the 20% increase - by 4%.

As a base open rate, we’ll use 3.5%. That’s a pretty conventional rate when sampled from our customers. When you apply the Istobe predictive power (4% increase) and the lift from general 1:1 targeting (20%), Istobe’s open rate is 4.4% and the baseline targeting rate is 4.2%. The normal open rate, remember, is 3.5%. This means that, if we take 1000 customers, Istobe will get 44 opens, baseline targeting will get 42, and the traditional email blast will get 35.

Here, we’ll use $1.90 per open as the amount of money that our fictitious company earns per open. Ultimately, not all opens are sales and this figure essentially backs out the complications associated with the website, ordering mechanisms, sizing problems, etc. In other words, it’s a way to understand how much each open is worth on average.

When you take into account that each email open is worth $1.90, and that Istobe has 44 opens per 1000 customers, Istobe’s predictions make our fictitious company $82.99 per 1000 customers per week. Baseline targeting makes $79.80 and regular email blasts make $66.50. Calculated for 500,000 emails per week for the course of a year, we get the figures that I began with at the top.

The Calculations

Trigger marketing is a good idea. Customers are not always ready to buy when you want to sell so it’s important to nuture leads through periodic communication triggered by events such as website interaction, inquiries, birthdays, etc. The more relevant and well timed that communication, the more likely you can keep the customer engaged until he or she is ready to pull the trigger, so to speak.

Michael Thompson of ClickTactics/ClickSquared makes good case for trigger marketing in his recent DMNews article Keys to trigger-based e-mail marketing. What caught my attention most was:

Timing and relevancy are the foundation on which successful marketing programs are built.Missing on either front can often spell disaster for a campaign.

Much as a dancer needs both rhythm and knowledge of the right steps to dance, marketers must hit both “right message” and “right time” to move the customer to action. By aligning the content and timing of e-mail messages with customer needs, you’re increasing the relevance, response and, ultimately, revenue from your direct marketing programs.

There is no question that communicating the right message at the right time is critical in trigger marketing. Often, however, more resources are devoted to implementing the data collection systems required to track customer behavior than to determining the marketing rules that will make trigger marketing a success. This is not surprising since vendors make their money selling analytics systems, leaving analysis to hapless marketers awash in a hurricane of raw data.

Faced with the task of developing trigger rules and content, many marketers go with intuition. Did the customer just abandon a shopping cart? Send an email asking them to finish the purchase! Did he click a PPC link for one product but then look at another before leaving? Send an email pushing the first product! Or should it be the second? Or both? And do it immediately! Or maybe give them a day to think it over!

The fact is that using human intuition, while better than nothing, rarely produces optimal trigger marketing rules. The optimal rules are there, though, hidden in gigabytes of customer data collected in analytics systems waiting to be uncovered. Few marketers, however, have access to the statisticans and data miners needed to unlock the secrets.

We’re looking to change that and I suspect that others are too. Trigger marketing will play an increasing role in marketing and it will always require elements of human intuition. Our bet, however, is that it will require data crunchers even more.

We talk a lot about the nuts and bolts of predictive analytics on this blog - how we build customer analytics models, how we interpret them, and how we establish whether or not they’re working in production. However, one item we have yet to tackle - and a very important item at that - is how to integrate data mining and/or predictive models into your organization and your company’s routines. Indeed, the black box in the middle should be the necessary data integration (getting the data into the right format for the models) and running a bevy of models against the data to see which one projects as the most effective. But it’s the before and after this black box that really make or break any attempt to incorporate predictive models into a company’s processes: data collection and incorporating data mining results into your processes.

Do you need to collect more data than you already have?
The main problem that companies run into is the need to collect more data in order to build the necessary predictive models. For example, if a company wants a model that tells them which of their products a specific customer might buy next, does it already collect the data necessary to support a model that does that? Ultimately, the question is: Is it worth the time and cost needed to invest in collecting more data that may not yield any better results when supercrunched? New collection methods take months to set up and sour clients on predictive models before the fun has even begun.

We believe that it’s important to try to work first with the data that our clients have and add new sources of data over time. This is a handy tip for business users who really want data mining to work in their organization: start small by using the data you have and get more sophisticated as you go. I know it may be tempting to put in that clickstream collection database right now so that you can use online behavior to segment and market to your customers. But trust me, get the buy-in from your organization first by proving that predictive customer analytics work. Then ask for the stars.

Are you ready to completely change the way that you do direct marketing?
Some companies would have you subscribe to a whole new way of doing business in order to use their models. Customer-centricity is my favorite new business philosophy. Though I firmly agree with the need for firms to be customer-centric, is it realistic to expect companies that use offers, coupons, and holidays to draw new and existing customers to their websites - and have for years - to change their direct marketing approach to accommodate a new set of tools? Not really. Company culture and routines are slow-developing and even slower-changing.

So the question at the end is really: What are you going to do with all this newfound predictive power? How are you going to fit in customer-centric model results - We’re 99% sure that Brad Pitt will buy two Baby Bjorns with lumbar support - to your next email blast that has a summer theme and features hats and sunscreen? And what about the extra creative necessary to relay that customer-centric message? You’re going to have to make a new email blast featuring the Baby Bjorn with lumbar support, in addition to the summer-themed email.

Well, I’m pretty sure that the summer themed email blast was probably going to draw some business but I’m also positive that luring customers with what they want when they want it is a lucrative way to market. The best practice is to initially skim the cream off of the predictions, to take those that have the highest probabilities of succeeding, and run with them. Collect a group of customers that has the highest probability of buying the Baby Bjord - Mr. Pitt among them - and send an intermittent email to just that group. Now watch your open rates and clickthroughs soar.

I want to take a minute and build off of Matt’s post from yesterday. While lift, confidence, and support may sound like terms that are more applicable to therapy sessions, they are actually the metrics that we use to rely the trust we have in our models. Many people we talk with are familiar with these terms on a certain level, but when pressed, their understanding boils down to the following: higher is usually good and lower is usually bad. I wanted to use this post to define these metrics a little more thoroughly and talk about how they’re calculated.  Hopefully, readers will come away with a better understanding of what they mean and exactly how they’re used.

Support

In order to talk about more complex terms like lift and gain, we need to first start with the basics: support.  Support, sometimes referred to as the cover, is the number of data points (customers, transactions, etc.) that meet a set of rules and/or assumptions.  If I do a market basket analysis and find that customers who buy milk also buy cereal, the support would be the number of customers in the sample set where this holds true. Obviously, you can only estimate the value of the support number when given the size of the total sample population which is why we have our next metric: confidence.

Confidence

Since a rule with a support of 900 looks good when the sample size is 1,000 and not so good when the sample size is 1,000,000, we need a way to easily figure out whether or not our support is significant. Confidence is a ratio that takes the support number and divides it by the number of instances where the rule may hold true (or to be more exact - where the antecedent of our rule holds true).  For instance, in our milk/cereal example above, confidence would be the total number of customers who bought milk and cereal divided by the total number of customers that bought milk.  While it’s true that the higher the confidence the more reliable the rule, it is important to note that knowing the total sample size and the support value as well as the confidence is necessary to get an accurate picture of the rules significance in regards to the total population.

Benchmark

I define benchmark here because it makes it easier to explain both gain and lift.  Benchmark is the total number of items (customers, transactions, etc.) that meet an outcome divided by the total number of items in the database. Let’s go back to the milk/cereal example. Since cereal is the outcome that we are trying to predict, the benchmark would be the total number of transactions where cereal was purchased over the total number of transactions in the database. In layman’s terms, if we were randomly picking 100 items out of the database, it is the percentage of those items where the outcome would hold true. Benchmark is valuable because it puts a lower bound on the value of a model. If a model can do better than the benchmark value, then it provides real value to the customer.

Lift

The most common term that is used in statistics and especially analytics is lift.  Lift is a way to measure how much better a model is over benchmark. It is defined as the confidence divided by the benchmark and any value that is greater that one suggest that there is some usefulness to the rule. Many applications show lift in a chart. In these instances, the total population is divided into deciles - ten even groups - into which members are placed based on their predicted probability of response. The highest responders are put into decile 1, etc.  Lift is then calculated for each of these deciles and plotted on a line chart.

Hopefully, this provided a little more insight into how we calculate the value of a model. Next time, I’ll run through a complete example to show how these are calculated in practice.

Many companies use surveys to get a handle on how customers perceive them and to find areas in which they can improve. Oftentimes, though, these surveys produce a lot of data but not a lot of insight. After all, if you ask your customers to rate you in 30 different product or service areas, how do you know which are critical and which are just nice-to-haves?

The key to analyzing survey data to find which areas are important is to be sure the survey has at least one all-encompassing “outcome” question that identifies whether you are successful in meeting the customer’s needs. This is usually something like “How would you rate our product/service overall?” or “Would you recommend us to a friend?”. We then use the responses to the other questions to find which individual product or service areas most directly affect the outcome.

This is most easily done using common data mining techniques. Using logistical regression or regression trees, it becomes easy to find the two or three individual areas that drive the overall customer perception of the company. For example, we might find that of the 100 different attributes available, a restaurant’s overall rating is driven primarily by speed of service, staff friendliness, and location.

Armed with information about the few attributes of your business which define your customers perception, you can better focus your resources to drastically improve the customer experience. Otherwise, it’s too easy to find yourself sweating over the unimportant details.

Someone asked me the other day, in response to my assertion that one-to-one marketing on a massive scale was the wave of the future, how a company could possibly send out so many personally-tailored emails. Being in the local Irish Pub, The Burren, I almost laughed Guinness out of my nostrils. But I couldn’t avoid the underlying message. One-to-one marketing never really has been embraced because no one really thinks that they do customer segmentation very well, that there are too many obstacles to customer segmentation for it to be entirely useful. Ultimately, this means that few believe they have homogenous enough segments to deliver the personalized goods.

What this also means is that one-to-one marketing is complex due to the fallacies of profiling. I once worked at a company that had such in-depth profiles for each segment that the profiles read like a Faulknerian novels. At this company, I learned that our target female customer in the 35-40 range probably once wanted to visit France but was now stuck with two kids in middle America and made meatloaf once a month for a husband she rarely saw. She obviously consoled herself by buying our software.

What’s my point with all this? This kind of profiling is for low-transaction sales, nothing more. Direct marketing units with high transaction rates should never take the tack of email blasting a segment based on their demographics. Nevermind writing fanciful biographies for said segment. Instead, direct marketing should ignore demographic profiles and concentrate on profiles that accomplish an immediate business goal (see below). Given the immediate needs that direct marketing normally serves, it needs to have a shortsighted, tactical approach, not the strategic approach that profiling represents. Below I look at the goal of getting rid of an overstock of shorts via the email channel. In doing so, I explore two, important dimensions of personalization: what the segment is willing to buy cross-referenced by when that segment most likely opens email.

The Group that Will Likely Buy Shorts Next

The truth is, the customer is not out there to buy from your company. They’re out there to purchase the product they want next and you’re merely there as a direct marketer to insinuate yourself into the buying equation. So which segment of your customers is likely to buy shorts next because that’s the group you want to reach when your shorts have been sitting in inventory for way too long and the leaves are already falling from the trees. So is this a profiling problem? In other words, is it time to blast every demographic who might wear shorts. I suppose you could. But then you’re likely to turn some people off. If you ran your customers past transactions through a classification data mining task, what you’d come up with is a list of people who are likely to buy discounted shorts at that time of the year. In fact, you’d probably come up with a few segments that demonstrate such a propensity. And they would definitely cut across your demographic profiles. You’ll have some moms buying shorts for their sons and some dads buying shorts for next summer’s Hawaii trip.

When Is the Best Time to Reach My Shorts Group?

Almost everyone out there sends me email blasts on Tuesdays and Thursdays. Why? Well, the general belief is that it adheres to the customers’ work/open schedule. I have seen elsewhere that most emails are opened on Sundays. That’s a compelling argument. But I tend to believe that each of your potential shorts purchasers has a more personalized schedule as to when they open and read emails. And that leads to the answer to the question in the subtitle. There are many times to best reach your customers who will buy your clearance shorts. The best web article I’ve read on this is by Bill Nussey of silverPOP who argues for tuning your send times per customer based on their last-recorded response. Couldn’t agree more. In fact, I believe that timing is the hidden axis of personalization. I would actually alter Nussey’s belief just slightly. And that’s simply to say that I would average their responses - and give the most recent responses just a bit more weight - to triangulate on the time your shorts buyers are most likely to open your email. For ease of use, you can bucket this into days or half-days so you don’t have to schedule an email every hour. If you record response data to your email blasts (opens), then this really shouldn’t be a problem.

So what do you ultimately have? You have customers that are most likely to want discount shorts and you have the best time to contact each of them. Now that’s personalized marketing.

Cross-sell should work particularly well for a company that has one best-of-breed product and a bevy of products on the come-up with which it really wants to gain traction. The reason for this: more accurate prediction of which groups of customers will buy a second product and when. You may even be able to determine which product is the likely second choice of those customers, meaning that you know which of your immature products to pitch them. By leveraging data mining, your best-of-breed products can really slingshot your other products to stardom.

If you have a best-of-breed product, then it’s likely that the majority of your customers were introduced to your company via this product. They became your customers because of this product, are likely customers who seek best-of-breed products, and are unlikely to buy one of your offerings which is not best-of-breed. In fact, they might not even think of you when they think of a product category in which you have a lesser offering. When these best-of-breed customers go looking for a new product category (in which you do have an offering, albeit an inferior one), you won’t be top-of-list. This is where timing is key to you. Knowing when your customers will buy a second product is a proxy for when they have money to buy and a proxy for when they’ve turned their attention to another product category. Thanks to data mining techniques such as neural networks and decision trees, you know when they have money before your best-of-breed competitors, which means you have a headstart - an early opportunity - to suggest a product category that may just be entering your customers’consideration set.

Once you get this timing down, you can figure out the best way to tout the integration or synergies that make your product better. Heck, you might even offer a discount on your non-best-of-breed product. But it’s the timing that gets you there. And never forget that these are your customers and you understand them - their behavior - better than your best-of-breed competitors. Use it to your advantage by investing in some state-of-the art database analytics techniques.