In my previous post, I talked about “5 Things To Do When Selecting A Predictive Marketing Vendor (Part 3 of 5)“.
You’re entering uncharted ground now and you’ve got to get your troops in line. You’ve decided to take the plunge and enlist a predictive technology for your B2B marketing and sales army.
Maybe the leads are weak, or there are too many leads, or you don’t know which ones should get assigned to what nurture track. In short, you’ve got a problem on your hands. You’re starting to look into some predictive analytics technologies for marketing which may work for you. Now, kick-off your evaluation by understanding the basics of what kind of test you can run to make sure the predictive marketing solutions you are looking at will actually solve your problem. It is time to get back on track to winning The Marketing War.
Why run a POC?
The goal of the proof of concept (aka a “POC”) process is to run an effective, thorough and concise comparison of different vendors. This will help you validate the technology and find out how the different vendors stack up in relevant capabilities for your use case.
How does a POC work?
In the POC, your vendor will use your data as well as their own data to create predictions and insights about your customers and segments. You will usually prove out one or two use cases at this stage. You will then measure the predictions against your actual results to check for accuracy.
How long will a POC take?
A POC can be run in about two to four weeks, depending on the complexity of your use case and the quality of the data used to build predictive models.
What will I need to begin a POC?
You will need your customer and pipeline data. You will need a “Positive” population, a sample of your “Universe of Prospects”, and a “Test Population”. Each of these sets of data are defined below:
Typically, several hundred Positives or “wins” will suffice. You will use different Positive sets depending on your use case. For example, a lead-based model requires leads which have converted to opportunities or closed won. For an account-based model, you will use accounts that have purchased from you. For an up-sell model, you will use accounts that have purchased more from you than their original deal amount.
Universe of Prospects:
This will be a sample of the leads or accounts in which you are trying to locate the Positives (the good leads or accounts). For example, if you were trying to find a great lead from your ongoing inbound lead generation, you would use a sample of your inbound to compare against.
This will be a sample of leads resembling your Universe of Prospects with Positives mixed in. Here, you must already have found the needle in the haystack — you must already know which prospects were actually Positives. This will serve as your test of accuracy for the vendors. Exceptional models should be able to predict against a test file with a very high level of accuracy.
The length of your sales cycle is critical to deciding on how big and how fresh your samples should be. For instance, if it takes one quarter for a prospect to become a Positive, your Test Population should be at least one quarter old and should not be in your Positive set of your Universe of Prospects.
How do I decide what is a Positive?
You want to use a state that takes about one or two quarters to achieve to define your positives. If it takes under two quarters to get a win, use wins. Otherwise use the status furthest down the pipeline you can achieve within two quarters. This will give you a model for earlier indication.
In most businesses, you want to keep to this timeframe because changes in your business will be reflected in who buys your product. If you’re modeling for something that will occur too far in the future, after many changes to company and marketing strategy, the model will not be effective. For example, should you have an 18-month sales cycle and choose to model for wins the prediction will only come to fruition in 1.5 years.
How do I know which vendor I should enlist to join me in the Marketing War?
In other words, how do I decide which vendor has performed the best? This is where the Test Population comes into play. Each vendor should be able to offer you a sample of the prediction. Comparing the accuracy of the prediction at the threshold relevant for your use case will give you the answer.
For example, if you are looking to find the top 20% of leads in your inbound lead generation everyday, you will want to measure which vendor most accurately predicted what would close by comparing the percentage of Positives they identified in the top 20% of your Test Population.
Now, that you’ve got the basics down, it’s time to take the next step. Who will you enlist to go to war with you? The choice is yours.
Check out the other posts in this 5 part series:
- Part 1: “Why Select A Predictive Marketing Solution?“
- Part 2: “How To Set Predictive Marketing Goals“
- Part 3: “5 Things To Do When Selecting A Predictive Marketing Vendor“
- Part 4: “Win The Marketing War: Get The Proof For Predictive Marketing“
- Part 5: “Marketing Mindfulness: 3 Critical Steps For A Successful Proof of Concept“