In my previous post, I discussed how to “Win The Marketing War: Get The Proof For Predictive Marketing (Part 4 of 5)“.
You’ve been meditating on how to take your marketing to the next level. You’ve decided that predictive marketing is for you. You’ve done your reading and research. Now, you are ready to conduct a proof of concept (POC) in order to move closer to reaching marketing nirvana. Make sure to practice mindfulness in your POC project. Here are a few things to pay attention to:
Step 1: Set Goals and Timelines
Right from the start you should pay attention to your POC kickoff process. In the kickoff you
- Review your own goals for predictive marketing.
Ask for the vendor’s insight and recommendations. Validate their expertise by finding out what experience they have with your specific goals.
- Provide the vendor with your data.
Your vendor should ask for several sets of data. A set of positives, a set to represent your universe of prospects, and a test set as described in my previous post. Allow the vendor to ask questions about the data and about your marketing and sales processes. They should be a partner with you in this project.
- Outline the expected timeline.
Make sure you and your vendor are on the same page for how long the POC should take. Have a shared timeline for what steps will happen when.
Step 2: Review Model Results & Insights
After the vendors have done their magic, matched their data with your data, built predictive models and found insights, they should share these with you. The vendor should present:
- The data that went into the model.
Your vendor should reveal what part of your data was used in modeling and why. You should also receive information about validations and statistics run on your data to enhance it for the model building process.
- The strongest insights and indicators in the model.
Your vendor should reveal what data they have added to the equation and what is most relevant to your model and business case. This provides a window into the models validity and granularity. For example, if you are selling an IT solution, you should see information about data storage solutions…the more detailed the better. It is more valuable to know a company is using Microsoft SQL server version X than it is to know they have a database (who doesn’t?).
- The model’s performance.
This is a tricky one. Vendors tend to measure performance differently so it is hard to compare performance if you don’t have a statistics expert. To make things simple, you should insist on a specific metric of your choice (see options below in Step 3). You should provide feedback to the vendor on what you’ve seen, the good, the bad, and the intriguing or ugly.
Step 3: Measuring and Comparing Results
The key to this step is deciding what to measure. This should be your decision, not the vendor’s. Choose the metrics, which are significant to your business case. Below is a guide to consider how to measure results.
- Don’t use a bad metric.
An example of a bad metric is “lift of group A leads compared to the lift of the rest.” There is a problem here — if group A is the group of the best leads as defined by the vendor, the groups may very likely be different sizes. It is easy to find the best 100 leads on your list, and of course they will perform better than a group of 1000 or 10,000. This is a false metric. Stay away!
- Use a good metric instead.
A good metric would be “the percent of wins in the top 20% of my leads.” All vendors will provide numbers that can easily be compared since we are talking about groups of the same size. The top 20% is large enough set to let you measure accurately, and the number of wins (or positives) in those 20% can be compared directly between vendors.
- Have a scorecard of important features.
Measuring model performance is important, however, so are all the additional features and data you should receive from your vendor. Perhaps the quality of certain marketing automation and CRM integrations is important to you. Perhaps there are certain actionable data points you want to have from the insights and analytics in order to provide an “air-traffic control” for your leads to guide them down nurture tracks. Make sure to include this in your evaluation.
Be mindful and make sure to consider these 3 steps while you’re going through a proof of concept for a predictive marketing solution. Now go forth and seek nirvana!
Image Credit: 121-training.net
Check out the other posts in this 5 part series:
- Part 1: “Why Select A Predictive Marketing Solution?“
- Part 2: “How To Set Predictive Marketing Goals“
- Part 3: “5 Things To Do When Selecting A Predictive Marketing Vendor“
- Part 4: “Win The Marketing War: Get The Proof For Predictive Marketing“
- Part 5: “Marketing Mindfulness: 3 Critical Steps For A Successful Proof of Concept“