Rob Reul and Cy Wegman are consultants at Predictum Inc., specializing in optimizing products and manufacturing with industrial statistics and analytics.
See below for episode links, timestamps, contact info and transcript:
EPISODE LINKS:
Predictum’s Unilever DataLab Win: https://youtu.be/wnOCqKvrwMU
Unilever DataLab Ecosystem: https://www.unilever.com/news/news-search/2023/introducing-unilevers-datalab-ecosystem-pioneering-collaborative-rd-for-superior-innovation/
Rob and Cy’s Work Profiles: https://predictum.com/company/#leadership
Rob’s LinkedIn: https://www.linkedin.com/in/rob-reul-69809a2/
Cy’s LinkedIn: https://www.linkedin.com/in/cywegman/
CONTACT NATHANIEL:
Feedback – https://forms.office.com/r/mAHmTU1eeP
PODCAST LINKS:
Apple Podcasts: https://podcasts.apple.com/us/podcast/knowledge-company-now/id1793779377
Spotify: https://open.spotify.com/show/5K5jj9nW3YTicH2y4ufI3s?si=78ca8866bd7a4a98
YouTube Playlist: https://youtube.com/playlist?list=PLvi4c35PeqnmiuDt4tycVE9FUnXLMt5IG&si=6QWqDuR9tpYJhARv
SOCIAL LINKS:
LinkedIn (Nathaniel Leies): https://linkedin.com/in/nathanielleies
LinkedIn (Predictum Inc.): https://linkedin.com/company/predictum
LinkedIn (CoBaseKRM): https://linkedin.com/company/cobasekrm
Episode Transcript
(0:00) Episode Trailer
Rob Reul:
Well, to conduct a choice experiment basically you hopefully know why you think consumers are buying your product you ought to have done enough of that research to establish what they want what are the needs what benefits are they trying to uh realize as a result of giving you money for their product.
As is often the case it’s a large list. Right? There’s a lot of things people will like in your product because you’ve got sound R&D and you’ve got strong engineers and you built a feature-rich product. The problem is you don’t know which things in the product they want and value most so what you do is you create a way to ask those questions and get them to make tradeoffs.
For a layman I always describe it as a game of clue: Professor Plum with the candlestick in the kitchen. Once you do that enough in a smart experiment you can tease out the relative attractiveness of what consumers want most and then the real power comes when you put money on them.
(1:13) Introduction
Nathaniel Leies:
I’ve had a question on my mind for some while I think that anybody who’s been in a product ownership role will have entertained this question at some point or another.
What does it actually take to create a product that your customers really want? The features that you put into your product… the price that customers are willing to pay… even the perceived value of your product that the customer might have when they evaluate yours against that of one of your competitors.
In this episode I’ve invited two guests who’ve spent the last 30 years trying to answer these questions: Cy Wegman and Rob Reul.
Cy Wegman spent the last 30 years of his career before joining Predictum Inc at Proctor and Gamble. He ultimately ended up leading Proctor and Gamble’s statistical and analytics programs across manufacturing divisions. Cy is also part of the Prism Society which represents the top 2% of all Proctor and Gamble engineers that have risen through the company.
Rob Reul is the president and CEO of Isometric Solutions. Rob brings over 30 years of consulting work across many consumer goods industries helping them solve many similar questions and deliver millions of dollars in savings.
I’d like to encourage you to watch to the end as Predictum will be announcing that we’ve participated in Unilever’s data lab ecosystem and won its hackathon called the 5S challenge in partnership with Capgemini.
This is really a big accomplishment. Cy and Rob were at the heart of this win for the hackathon and we’d love to share the story with you.
As always you can follow us on YouTube Spotify and Apple podcasts you can contact us on predictum.com/nowledge company now.
(2:52) Marketing analytics and its role in business
Nathaniel Leies:
Rob and Cy, what does marketing analytics in the way that businesses practice it mean to both of you? Not just the technical definition, but what role does marketing analytics serve in the organization?
Cy Wegman:
When I think about the market analytics and how it impacted things that I worked on at Proctor and Gamble, you could and and have worked on projects that were you know executed flawlessly, did great technical breakthroughs, but the the product itself didn’t sell. So it’s like, “wrong product; right process; right innovations for that wrong product.”
So the things that really interest me in the in the marketing analytics is trying to define and find that right product, so if you get the right product and and find out what the consumers really want then you have a much better chance of being very successful in innovating and in getting success in your product itself and getting market share and expanding the market itself.
Nathaniel Leies:
Rob let me turn it over to you.
Rob Reul:
Cy put his finger right on it.
It’s real simple for me. Analytics in the marketing realm is all about making sure you make money and there’s three big swaths of places where you look to see if they exist:
Are you designing the product the customer or consumer most want right? And then, are you promoting that product in the commercialization activities such that people try it? Okay? Because if you if you design the product that most want and then they try it, they’re going to conclude it’s better if your models are working.
And then all products have a natural useful life and then competition or — hopefully — you come in with something better. So managing that transition of maturation and optimizing in such a way that you cannibalize your own products with your better products before competition does… managing all three of those things with all your products ongoing is marketing analytics to me.
(05:08) Why businesses should care about marketing analytics
Nathaniel Leies:
So you both have sort of alluded to the fact that we can ideally create a really knockout product. We can do great science we can manufacture it extremely well. But who draws the line if the customer doesn’t want the product? And the answer, I think, is the customer draws the line.
Are enough companies performing marketing analytics in the way that you’ve described it right now? And if they’re not why should organizations and individuals care about doing this type of marketing analytics?
Cy Wegman:
I think everybody understands that they want to use data and of course we we’re having a plethora of new tools you know that are coming out you know especially you hear all the things around AI machine learning and these types of things.
And I think it’s uh imperative that companies really do this with the latest innovations. And it’s worth a lot of money. Some companies that we have worked with and looked at… they have not had as much skill in this area as I think that they could have had, and we’ve been able to go in and help them do a much better job.
Rob Reul:
Yeah, so the simple ad I’d introduce here comes comes to us from a guy named John Waker who well over 100 years ago ran the largest, first ever department store.
Massive place. People could go and shop and buy things and he spent a fortune on advertising when you asked him about it. And he was talking about what he spent and he said “My problem is I know half of it doesn’t work. I just don’t know which half.” And most organizations have this problem.
Everywhere I’ve worked, they all have this problem because they haven’t had an ability to figure out what is working and what isn’t working.
And that’s some of the magic of what Cy and I like to bring to the table, which is experimental analytics which is not forensic. And we’ll talk about that a little bit later. But the money to be saved is is massive.
(07:12) Choice experimentation
Nathaniel Leies:
So in terms of what customers want and what they don’t want… In terms of advertising spend… Rob you often talk about the role of choice experimentation and that choice experimentation is a gold standard.
So, for our listeners, if they haven’t heard of this before, what is choice experimentation just in simple language and why is it powerful say to a conventional method that maybe marketers would have heard of like AB testing?
Rob Reul:
Yeah, well the conventional method to basically navigate with analytics the business, we look at as forensic reporting.
Very sophisticated tools. Slicing and dicing to tell you what happened last month… last four months… last year… this month. A lot of data engineering to make that capability available to the decision makers in the business.
What’s different is rather than make data about the future based on data from the past, we make those decisions based on what people looking at the future want in their future.
And that’s what the experimentation is and so choice experiments is the gold standard for for doing that kind of work. Experimental designs, sophisticated experiments, done at scale to help better predict the future.
Nathaniel Leies:
Take us through a simple step-by-step process of how does somebody actually conduct a choice experiment. What would that look like for the marketing department?
Rob Reul:
Well, to conduct a choice experiment basically you hopefully know why you think consumers are buying your product. You ought to have done enough of that research to establish what they want, what are the needs, what benefits are they trying to realize as a result of giving you money for their product… and as is often the case, it’s a large list. Right?
There’s a lot of things people will like in your product because you’ve got sound R&D, and you’ve got strong engineers, and you built a feature-rich product. Problem is, you don’t know which things in the product they want and value most.
So, what you do is, you create a way to ask those questions and get them to make trade-offs. And for a layman, I always describe it as a game of clue. Professor Plum with the candlestick in the kitchen. Or was it Colonel Mustard in the bath with a hammer? Or was it Cy with a chainsaw? You never know, right, until you ask!
But once you do that enough in a smart experiment, you can tease out the relative attractiveness of what consumers want most. And then, the real power comes when you put money on it, so we can monetize those utilities and we can tell the client how much more a feature in a product is worth to a consumer.
(10:03) How to run choice experiments with surveys
Nathaniel Leies:
One of the vehicles by which we are able to deduce… play the game of clue as you will… is through surveys and being able to make choices between different product configurations.
So maybe a classic example is that I’m looking for a phone.
This one has a bigger screen size where option two has a smaller screen size. But with option one, longer battery runtime. Option two, not so much battery runtime.
You can create hundreds of thousands of configurations, but the power of of these surveys where users choose different product configurations is every time they do another run of the survey or another set of options, you get a little closer and closer to some statistical signal about what the customer actually does prefer… what attributes they like the most.
Rob Reul:
Yeah, you’re you’re on it.
That’s right. We design an experiment to show a sequence of or a series of combinations and you certainly can’t show any one individual all the combinations.
But if you design the research space sufficiently — and we do — you can hit all the edges and you can look at all the key combinations and look at pair wise comparisons and all those ways to stitch together the tapestry.
So we’ll talk to thousands of customers in what we think of as a thought experiment. “Given this which would you choose,” right? Or even better… “Given these options, what do you think most people would choose,” which is a prediction versus a preference, right?
So we can do it in some very smart ways to tease out what a company should offer to the consumer. You can cut that into pieces if you have subject variable data which is men, women, older, younger, east coast, west coast, whatever. Right?
And you can use that data to find different populations that are most attractive. So maybe you have to split your product.
(11:57) Conducting pricing research
Nathaniel Leies:
You can actually map out different price ranges and the attitude that the customer might have at a different price ranges.
What is the the method which achieves these attitudes and pricing ranges and and tell me a little bit about your experience developing these sorts of… um… let’s say preference mappings.
Rob Reul:
Yeah okay, so there’s several different ways actually you can do a price uh research. Van Westendorp is one of the most common. That’s the one the you know… better whatever um… the the key is there’s a economic relationship between a price and the volume. Elasticity is what you want to unpack, right? And you understand at what point is that specific feature set it’s the CVP (cost volume profit) is that going to get us the most profit which is the endgame in almost all cases.
It’s about long-term sustainable profit so we… I just did a bunch of work for for a client and what we found was they happily are the least expensive product in their category. But it’s a big category and they make a profit. And what we found was they could actually raise their prices a fair bit but still be the least expensive in the category.
So they’re going to get chosen if it’s a price buyer, right? And they’re going to make more profit and all they’ve done is change their prices.
Cy Wegman:
Yeah, the other thing I would throw in there about what Rob was talking about, and he mentioned this before, is these what we call subject variables.
So basically we want to find these different categories. It might be a price buyer. It may be a luxury buyer. And if we set up the experiment appropriately, many times we can get those consumer segments out and we can identify that, hey, 20% of the population are price buyers, you know? 10% are these luxury buyers. These people are really brand loyal to this other brand. Don’t worry about them.
We we can figure a lot of that out in the analysis that we do.
(14:09) Appliance repair case study
Rob Reul:
Nathaniel, here here’s an example and I did a talk on this it’s that’s pretty available on the web.
I had a client who was one of the largest re largest national retailers in the country, and they were a big provider of uh appliances, okay? And one of their business was being able to repair their appliances.
They sold everything you’d get in the house: kitchens, laundry, all that stuff, right? And the business that would became quite lucrative for them was repairing the appliances they sold because the warranty from the manufacturer was typically one year and the typical inhome life of an appliance is 14 years. So there’s 13 years it’s not under warranty and they all fail about year seven, you know? Plus or minus, right?
So people want to eek it out before they replace it, but they’re expensive to repair and so the study was what can we do to better serve the customers with what they want.
And at the end of the day, through segmentation like Cy’s talking about, the research said “offer more premium repair… like in time… at a much greater price.” Oh, your refrigerator doesn’t work? We’ll come today for $800 and guarantee it’s fixed. Your dryer doesn’t work? Okay, well, we can come next week on Tuesday for $200. Right? And it used to be every repair we’ll put you next on the schedule and it was the same amount of money plus parts if you will.
But what they figured out was they could offer the more premium service to those who wanted to pay for it and those who wanted to wait were happy to wait.
They ended up with a 24% take rate on every call and it saved them… I don’t know if you can put this in here or not… but it saved him $7 million in profit a month.
Nathaniel Leies:
That’s incredible.
Rob Reul:
It is. And that kind of opportunity is available to companies who go big and want to explore the marketing analytics power.
(16:15) The cost of not using marketing analytics
Nathaniel Leies:
There’s so much power in the story that you just shared Rob, so I thank you for sharing it.
Let’s flip this and look at the reverse side of the coin.
What is the cost of not doing analytics well?
Cy Wegman:
I think it I think it can be enormous. You know, I think the real thing is what’s your competition doing? So if your if your competition is not doing good analytics and you know the whole industry is not not doing good analytics, then you’re going to maintain your your place in that position.
But I had a colleague that says “the one who does the math the best wins,” and if somebody really delves into the analytics and does it well (and that’s including all various aspects that especially into prediction and you know optimization) you know they’re they’re going to clean up the competition as far as like delivering.
(17:09) Why Toaster Strudle missed an early go-to-market
Rob Reul:
Oh, I watched a manufacturing line be unable to make the product that the consumers wanted, and they went through three senior operational executives to try to make that thing happen. And this was when I was at Pillsbury. And they… this is great actually… I’m not sure if Cy knows this, but we never fixed it till they plucked a guy from Proctor and Gamble to come and run it, right?
And it was amazing what he did. But basically he took the machine apart. The product was something called Toaster Strudel and it was a terrific product, but the problem is the way the machine worked and the heat evaporation of the dough.
The the resulting product… you could never get it into a toaster cuz it was all… you know… fluffed up in different ways. They couldn’t make a flat one, which is what they sold, so it was a delicious product but it wasn’t fit for use. Right? They verified it but they couldn’t validate it. They couldn’t get it in the toaster and it cost them a fortune to figure that thing out. So, had they wired those things down sooner, they would have saved millions and millions of dollars and they would have got the product to market probably two years earlier and made epic dent.
You can still buy the product today but this is a a dated story… but yeah you you got to have it on critical things… tough decisions… you got to have good data.
Nathaniel Leies:
That was that’s a terrific story, Rob. Yhey should have– uh Jerry Seinfeld– did he reach out to you for his movie? You know which movie I’m talking about?
Rob Reul:
No!
Nathaniel Leies:
He did a very silly comedic movie about toaster strudles and pop-tart. I think it was called “Toasted” and Jerry Seinfeld was the main character. I’m lamenting that he didn’t reach out to you. I think you would have been perfect in there.
Rob Reul: You couldn’t get it in the toaster… yeah. True story.
So, what I would say is analytics is a new competitive battleground for companies.
Smart analytics…
(19:10) “Data engineering is not data analytics”
Rob Reul:
…and this puts a lot of people off… but I don’t think of data engineering as analytics.
We see a lot of companies that have spent a fortune on data lakes and data reservoirs and data seas and like, they have all this data, right?
And it it’s a tremendous amount of work to retain all that information. Like everything they ever bought at what price point, shipped it from where, what, you you name it.
But turning it into a monetizing entity takes a lot of work and I don’t know if anybody ever designed such a thing for the purposes of modeling, but if they have they’re in better shape than if it was simply for reporting. And that’s a big distinction that I’d like to I’d like to make.
Ddo you have the data? Can you report it in a way to inform a decision maker? Is the data renderable in such a way so that that decision maker can make predictive models? Are they forensic? Is it just historic? Or are there ways to actually fashion in future experimentation with that data to help inform the decisions?
Cy Wegman:
We used to… when we were first training people… in their own own area… we’d give them a simulation of their process and say “go solve this problem.”
You couldn’t do it. And these are even people who designed the line. But when we go through and we build these models based on experiments and then we use optimization techniques… that that can be a lot. I mean tens of millions of dollars on products or maybe even hundreds of millions of dollars.
I mean maybe as a rule of thumb, I like to say if you go into a place that is is not using these techniques, you can typically you know get about a 10% reduction in your ongoing cost.
And again it depends on the product that you’re making, but the the the savings and the amount of improvement you can make are substantial. And if you do this in the marketing, it’s even bigger. You’re talking real money as opposed to “how can I just make my products and processes more efficient?”
(21:28) Winning the Unilever DataLab 5-S Challenge
Nathaniel Leies:
Recently you both participated in the Unilver DataLab 5-S challenge as members of Predictum. It was quite a tremendous undertaking and so for folks who have not heard about Unilver’s DataLab ecosystem, I just invite you to talk a little bit about the ecosystem and how you as Predictum participated in it.
Rob Reul:
So let me just take a quick whack at it for you.
Unilver… massive company. Tremendous investment in analytics. Massive data lake. Many, many, many, many powerful brands and we being one of the few preferred providers were invited to take a look at all the data that they have and come up with innovative ways to help them make their business stronger.
And they were looking for basically three things: (1) something that was a whole new way of doing it (2) something that didn’t cost them a lot to implement and then (3) lastly something that had legs so if we could do it for one brand we could take it to many of the other brands.
And a lot of what we did, we talked about today was basically was smart marketing analytics with experiments to understand what the consumers most want in a product… What will they pay premium for? What segments could we find that would pay the most for it and if we optimize the formulation of these products, could we make them more profitable?
And we pretty much hit on every single one of those things, and for that work we were granted the win.
So we’re we’re pretty excited about that.
Cy Wegman:
Yeah, I would add to that Rob did a great job of you know problem formulation. So this is a very fast thing. This was not like, “you’ve got six weeks.” It’s like, “you have two weeks,” and so we had to come up with an idea and actually execute uh the program and the experimentation and get it back analyzed and then recommendations with a report all in just a few weeks.
So Rob did an excellent job of of formulating the problem. And that, actually, when you get down to solving a lot of these problems, is the key… is how do you formulate the problem, talking before about any kind of small amount of effort that gave big results, but this is one you know it was like very fast very quick and big savings.
Rob Reul:
Instantly and millions of dollars. It was big it was really big. And one more thing… we’re just, you know, a couple of guys who’ve been in the game a while. We know where to hit. We know how to size it up and we know how to hit. Great win for Predictum. Absolutely. Absolutely.
(24:30) Unpacking Unilever’s challenge
Nathaniel Leies:
So let’s talk shop. Walk me through the problem that Unilever invited you on behalf of Predictum to solve and then what steps did you take?
Rob Reul:
Well, they they didn’t frame any problem at all… It’s like, “here’s what we have. What can you do?” They didn’t do any of that. And so we spent probably a third of the time studying what they have and what the problems could be and where we could help which is why Cy talked about the problem formulation because that’s everything.
You got to be working on the right thing and that’s often how sometimes you have a client say “Hey this is my problem help me fix this.” Well, that’s half of it. Let me work on that… other times a client says “Why should I spend any money on marketing analytics?” Let us take a look and we’ll show you where and how and why. Because we don’t know, but we know where to start looking.
What we did know is Unilver was very… they are of the belief that superior products is their future. They need to be able to deliver unmissable benefits in their products which are… you can’t miss them. They are profound. They are delivering on jobs consumers want done. That’s the doctrine, and so you can’t get a more prescription of what the target looks like. A product that delivers on jobs consumers want done and so then the exercise was, “well what are the components of delivering on those things?” How do consumers experience them and what are they willing to pay for them?
At the back end of that is and how did you validate in R&D that you’re hitting that target?
(26:55) Why Predictum stood apart
Nathaniel Leies:
So, we’ve talked a little bit about marketing analytics, the value that it provides, how it helps to make the right product, and the R&D part.
We’ve also talked a little bit about the Unilever DataLab Ecosystem and the 5-S challenge that, Rob and Cy, you both enjoyed being the winners of that challenge.
So let’s just take a step back and reflect.
As Predictim Inc., what makes Predictum the most suited to solve these problems at the intersection of innovation, modeling, and experimentation?
Cy Wegman:
Well, I would say it’s it’s, number one, our experience.
We’ve worked across many industries, and as I mentioned, during my time at Procter & Gamble, I worked across all divisions as the global owner of empirical modeling and optimization. So this is an area we know deeply—we have the tools, the capability, and the experience to solve these kinds of problems.
We can step in, very rapidly diagnose the problem, figure out where the highest leverage areas are and then make the recommendations and then also to help implement those… and not just saying “okay, just solve this problem and good luck.” After this, we can help beyond that too.
So all that experience that we’ve had over our years and the technical expertise, this is something we’ve both done for our whole careers.
It isn’t like, “let’s pick this up,” for the last two-to-three years because it’s a hot topic. It’s that we’ve done this over decades and I think that really puts us in a position that most people don’t have.
Rob Reul:
I would add to put a funnier point on it: scar tissue.
We’ve been around a long time and done these things for a long time.
We’ve tried a lot of things and we’ve seen what other people have tried and we know what doesn’t work. We know what’s going to hurt you later, so we skip those things. We go right to what we know works. And the other thing that I would say differentiates us is that our job is, ultimately, not to do more marketing analytics.
Let me see if I can get this right… this is Phil’s quote, so you’re going to have to cut this…
“We want we want to bring more data to the marketing scientists that they already have rather than become our clients data scientists.”
Right? We want to help our clients do this work themselves. Our goal is not to be the outsource shop that does this work. We’re very mindful of that and we worry a lot about the knowledge transfer. We like to do the work with our clients collaboratively so they have the technology and the tools on their desktop to be working these problems with us, so that they can do them themselves on an ongoing basis and maintain their own models. And always be available to come back around. “Oh, you’re stuck? The model isn’t working?” Let’s take a look, right? So it’s that philosophy that allows us to best support those that like working with us, and I think that’s why they do.
(29:44) Building analytical capability
Nathaniel Leies:
In both of your estimation what does it actually take to build true analytics capability? Is it people, tools, culture? Take a step back and imagine this from an organization that is trying to do analytics and has a little bit of capability… or maybe somebody who’s realizing they’re new and they need to do something from scratch for the first time. What does it take to build that capability?
Cy Wegman:
The culture has to change.
It’s like, this is just how we do business. It’s like, we have accounting because it’s necessary. Well, you need to have data analytics in whatever field you’re in because it’s necessary. It’s part of the organization. It’s organic to the organization… to the culture. And that means from the top to the bottom.
So, if the management says “Yeah, go do data analytics and we’re not giving you any money for training and we’re not giving you any money for a computer that works or software,” well, yeah you’re kind of like… it isn’t that you can’t do anything.
I mean, I was doing it with pencil when I started. So you can still do things, but not in today’s environment.
That’s not going to cut it.
So changing that culture, providing the support and the necessary tools, and encouragement to do that, and then you reward you know the people who are doing this type of work…
…you can have varying uh levels of experience and and skill within the organization. So it isn’t like everybody has to be a data analyst but everybody needs to know how to interpret it and how to use it.
And you’re going to have a select few that are going to be like, “this is what we do,” and then that permeates the organization. And that is what it the means for making decisions inside the company.
(31:37) Hopes for new generations of analysts
Nathaniel Leies:
The next generation of people who work with data the next generation of analysts scientists what do you hope that they will do that builds upon the foundation that that you have laid?
Cy Wegman:
Well, people laid foundations upon which we’re working. When we started our careers, and we’ve continued to go down this avenue… when I was a Proctor and Gamble we’d train thousands of engineers and scientists on how to do these techniques. And at Predictum, that’s a big thing that we do, is training. So I just, I love personally love getting in and interacting with the trainees and classes and then there’s also a lot of coaching we do. One-on-one knowledge transfer. We’re working with people like Rob talked about before and I think that what we really like is when we see those people who who have taken that training or taken that knowledge that we’ve passed on to them utilize it and really flourish and take it to the next level.
I mean, technology is going to continue to improve and knowledge is going to increase past, you know, the times that we finally uh hang up our shoes.
People can really flourish in this and I think that that’s always exciting when I see people that I’ve worked with or worked for me or have trained and they they really excel in the area and and that always makes me feel good that you know we’ve pushed the technology and those people themselves have been success successful.
Rob Reul:
To me it comes down to asking good questions and being hungry and seeking knowledge and making sure that it’s a curious culture. And the new breed… those that follow like Cy said, were curious… hungry… want to learn… ask good questions. That’s the key, especially in the wave of AI. Best use of AI that I’ve seen and practiced myself is being able to ask smart questions to basically study the space… whatever it is I’m looking at and try to understand what’s been done and what else could be done and bringing things from other areas together to increase your intelligence.
Nathaniel Leies:
Rob. Cy. Thank you so much for sitting with me.
Rob and Cy:
Of course. Our pleasure!
