Understanding Prototyping to Learn

Hosts

Understanding Prototyping to Learn – Transcript

 

Greg Engle

Today, I want to take a deeper dive into one of the skills that you wrote about in your book, I know we covered in an earlier episode. I want to take a deeper dive into prototyping to learn.

 

Bob Moesta

Okay, one of my favorite topics.

 

Greg Engle 

First, can you give a brief description of what you mean by prototype? What does prototyping mean to you?

 

Bob Moesta

Prototyping and prototyping to learn are connected in the context I’m going to build something or make something or think through something, so I can find out what I don’t know. So, prototyping to learn, Ryan singer talks about a spiking, how do I spike something up? So I can then figure out what I don’t know around it, then when I figure out what I don’t know, how do I prototype to actually fill that space? What do I know? What don’t I know? And then how do I build a set of prototypes to learn what I don’t know? Then answer those questions. 

 

That gets back to divergent prototypes, and convergent prototypes. Divergent prototypes helped me see the spectrum, almost the boundaries of where something might not work, or where it will fail. Convergent is about helping me optimize against or manage the tradeoffs, between a whole bunch of different factors and cost and performance, etc. It’s a series of actions where I build sets of prototypes to help me understand what to do. What’s interesting is the way that both I was taught, and the general notion of prototyping is that we use something called prototyping to verify. We build a prototype to verify our hypotheses or what we think is the best thing. 

 

Then when we get the results, we go ‘Oh, now what should we do?’ It’s easier to build sets than it is to build one and then build another and then build another because we don’t build a cumulative knowledge.

 

Greg Engle

You’ve lost me a little bit. So let me let me see if I can unpack a little bit and figure things out. You talked about divergent and convergent prototypes. You explain them a little bit, but I want you to explain a little bit more, in what situations would you use divergent prototypes.

 

Bob Moesta

Here’s a good example is in sales, I use divergent prototypes as a wide thing, 2, 3, 4, 5 things I might take to a customer that was very different so they can use that contrast to help understand what they want. This is where we can use it in many different ways and they can eliminate to then help them pick. 

 

Greg Engle

But how different do they have to be? 

 

Bob Moesta

That’s the key in divergent prototyping: they have to be very different, where most times people make small little changes. For example, three different proposals of ways we can work, for example, I worked with a company called Auto books, and what we were able to do is build out three really different ways in which they can engage with auto books. 1. if you want custom, you want your logo on the software that we’re going to plug into your bank, the fact is, and you want to be have your logo on it, and you want to make sure it does this, this and this, and it’s connected to your system in this way, it’s eight weeks, it’s a million dollars, and we need these kinds of people involved. 

 

But if you want to try it, we can white label it, and have no logo on it, but see if people will use it, and we can do that in two weeks. But the price difference is that you don’t pay anything upfront, but our fees are more on the transactional side for the small thing, and they’re less on the bigger thing. So that contrast of prototypes helps them understand is this more about speed? Is this more about money? Is this more about making sure the brand is there? Is this more about trying it out? What do they want to do? Part of this is, the diversion prototypes are more about helping people eliminate what they don’t want, then help us build the criteria of what they want.

 

Greg Engle 

So you’re talking about factors or things or levels, you can change in that, so many different levels should you be changing in that divergence? 

 

Bob Moesta

The very most fundamental way is the phrase that was used in contrast to create meaning? How do we have three or four different things, because it’s not one thing, it’s not two, but it’s a minimum of always three for me. When you get to really understanding it, there’s a whole body of knowledge called Design of Experiments from RA fisher in the 1920s. I learned it from Dr. Taguchi, who is one of my mentors. We learned how to actually frame systems or frame situations and then basically understand the variables we can change in it. You can have 2, 3, 4 levels of each variable but ultimately, can we then build a wide range of prototypes, so we can learn. And that’s really kind of the origin of where a lot of the prototyping the learning stuff came from. You have something called control factors, which are inside the system, the things that we can control and set, how do we know which the best place is to set that parameter. An example here is very early in my career, I worked on injection molding of a mirror case for the rearview mirror for a car. 

 

And one of the problems we had is that was shrinking too much. And what would happen is, it turned out that it would shrink, and the lens of the glass would snap into place. But when it got hot enough that it would expand, so part of it was we could upgrade the materials, but the other thing is we could mold it a little bit differently, so it wasn’t so sensitive to temperature. Part of it was then going in and saying, Well I know how to create the perfect dimensions, but I want to make sure that I can create the right dimensions and have it not expand or contract through temperature. Ultimately, we were able to solve the problem without increasing the price, in some cases, we increased the productivity of the machine because the things we thought were going to have an implement didn’t.

 

Greg Engle

So divergent prototypes. Are you trying to find the tradeoffs in a customer system? What are you trying to find in a in that last example? More of manufacturing?

 

Bob Moesta

I would say it’s on the supply side. I think of prototyping as a tool I can use on either supply or demand side.

 

Greg Engle 

Okay, so now you have to unpack those two words.

 

Bob Moesta

Demand side is what causes people to say today’s the day they’re going to try something new or pull your product or service into their lives. The supply side is how do we build that product to be cost effective and deliver what the customer wants, at the same time make sure that we can do it in a consistent way.

 

Greg Engle 

Prototypes doing both of those all the time, depending on how I set them up.

 

Bob Moesta

That’s right. Part of this is to realize most people are trying to do one thing, they’re trying to do it all at the same time. Or they’re trying to say like, I know what the best thing is, let’s go test it. They don’t understand the difference between trying to see a range of possibilities on the supply side. And the range of possibilities on the demand side. And they might not be the same types of prototypes I use on the demand side versus the supply side. So, there’s some work we do that’s around jobs to be done around prototyping. It’s the prototyping job where we know what the job is. And then what we do is we bring a wide range of prototypes or samples to the table. 

 

And as we talk people through their context and the outcome, they want then we show them the product, they can go ‘Oh, that one I would never use because of this’ and ‘that one I would use but you know, I think it’s this way’ so it allows them to start to talk about fit in a very discreet and concrete way.

 

Greg Engle 

That’s what we call benchmarking, understanding of how people choose between the things that could be in the competitive side.

 

Bob Moesta

That’s right. Our version of benchmarking isn’t about how good or how bad we are. Our version of benchmarking is about prototyping to learn so we can know what the right criteria are, when they’re in this context.

 

Greg Engle

Because many different things can fulfill a job. I’m not just making a peanut butter and jelly sandwich, and that’s my job as a peanut butter jelly sandwich. It’s a food.

 

Bob Moesta

This is where you can compare an apple to an orange. Most people want to compare it just the way it tastes. But when you think about an apple, I don’t have to peel it, I can eat it right away, I can eat it without a knife, I can cut it. Part of it gets back to what’s the context you’re in? When would you pull an apple? And when would you pull an orange, and you start to realize it’s different contexts.

 

Greg Engle

That’s divergent. What about convergent?

 

Bob Moesta

Convergent is on the other side of this, which is some people will call it Optimize, but I would think about it as being able to manage and identify the tradeoffs. In any system, there are always constraints or things make it stronger, it gets heavier, so I can’t make it lighter, making it stronger and lighter is different, it’s very hard to do. So, where are those things that are bounding us, that we have to realize we have to make a trade off of either it can do this, or I can do that, but I can’t do both. I can make the performance better, but it’s going to cost me more. I want to get the performance up and the cost down, that’s where I would do convergent prototyping to help me see the prototype the way in which to do both.

 

Greg Engle

It sounds like those two things are why we’re doing the prototyping. What answers we’re trying to get out. 

 

Bob Moesta

It’s all about framing questions. 

Greg Engle  

Now, what are the types of prototypes? I know there’s physical prototyping, that I have a product, and I’m going to show people. There are all the different types of prototypes.

 

Bob Moesta

There are dimensions of prototyping as I think about it. 1. There’s a comprehensive prototype, meaning it includes everything, or 2. There’s focus prototype, which is focused on one aspect of it. So I understand how it works.

 

Greg Engle 

So, when do you use a comprehensive?

 

Bob Moesta

Often, I need comprehensive when I go to the customer, because the customer will judge something differently if it’s not complete. Another part is I use comprehensive when I’m trying to figure out how things interact, or when I assemble things. So, a lot of times I can put the brakes on the car, I can put the engine in the transmission, but then I get them all to work independently. But now I need to know how they work together. So, I have to do a comprehensive prototype to help me understand how the things I don’t know about when they’re there, VS focused is, I’m going to work on the transmission and make sure that it’s actually as efficient and effective as possible, given the input it gets from the engine and the output I want to be able to get to the to the wheels. The other part of this is that there’s physical prototypes.

 

Greg Engle 

How is it different than the other you talked about? So you talked about comprehensive.

 

Bob Moesta

Comprehensive and focused. Focused are the ones where I’m going to focus on the transmission and optimize it to fit into the environment of the engine and the wheels and be able to understand how to optimize. I have responsibility for the transmission, I can’t do anything about the engine, the I have to accept the inputs from the engine. Part of this is to realize that I have very strict boundaries around it. It’s being able to say, Do I need to focus prototype? Or do we need a comprehensive prototype? Focused prototyping is faster, so this gets back to it from a time perspective and a knowledge perspective, and what I know and don’t know, most of the time, what I’m doing is a lot of focus prototypes to then do a comprehensive to pull it together.

 

Greg Engle 

But the focus prototype seems like that’s more, in your words earlier ‘supply side’? I can’t really see that it’s not complete enough for them to do anything. If I’m talking about food, I might be talking about a sweetener, or preservative systems, and I’m breaking those apart, and the customer doesn’t care.

 

Bob Moesta

They might be able to perceive the differences, part of it is being able to understand the preservative is there to make sure that bacteria doesn’t grow. We really want to make sure that the preservative doesn’t affect the taste or the texture of something, that’s why we might end up doing a concrete comprehensive, but we need to make sure that this preservative works in our cookie.

 

Greg Engle 

So that goes with the philosophy of the divergent convergent, it’s what are you trying to do? Are you looking at the complete thing that you have to give to somebody? Or is it more for your learning?

 

Bob Moesta

I think the other dimension on this is what we would call physical versus analytical. Analytical is where I might model it, and physical is where I would make it and test it in the real world. So, what you start to realize is that, you need a whole range of those focused and comprehensive as well as, analytical and physical. A lot of this comes from Alrick and their book around product development in terms of I learned this in the 80s around how to be very clear about my prototyping strategy, to get what I need to break the market and I need to prototype both on the supply side and the demand side, to then put it together.

 

Greg Engle

But what I’m hearing is, as a manager, as a person guiding project teams, as a person in charge of development, I need to make sure my teams understand what the purpose of the prototype is, because there’s many different decisions I have to make, I can’t just say go make a prototype,

 

Bob Moesta

The other part is you start to realize there’s the role of prototypes. There’s a learning prototype, a communication prototype, a milestone prototype. An example, a milestone prototype is the number one mistakes a lot of people make when they go build something, and they try to bring it back to management to show them where they’re at, they never bring the old prototype and the new prototype to the table to say, here’s the progress we made, they just bring the new one. So, all they know is how far away they are, not how much progress they made. Communication prototypes are, in a lot of cases help the interface between the engine and the transmission, I need to communicate about what the interface look like, and what does it really means. Part of this is being able to use it to have language and conversation around the critical aspects of the interface between things.

 

Part of it is being very purposeful, and how we take the time to understand when we’re going to build something, or we’re going to try something, what are we trying to do? What is the purpose of it? What is the best strategy? Should we be using divergent sets? Or convergent sets? Should we be comprehensive? Or should we be focused? And should we do physical or analytical? There’s decisions you have to make as you go into this, to me as a manager, what I’m always asking is what don’t we know? What do we need to learn from the prototypes? What’s going to be the best and fastest strategy to get the right set of prototypes together?

 

Greg Engle 

Those are broad questions. So, I want to break it down to a little bit more concrete questions people can ask, what are you trying to learn? You have to ask the team that.

 

Bob Moesta

It has to start with what don’t you know?

 

Greg Engle 

That’s actually a really hard question for people, you can get away with asking that question, and you can help people coach. But if you don’t know those things, you still you need to get on what we know first before we can absolutely see what we don’t know. What we don’t know is night vision goggles. You have to be really aware of what you know, to know what you don’t know. You have to be self-aware first. What question are you trying to answer? What will this prototype tell you? Why are you doing it? Who is it for? And what do you hope to learn from it? And if you can do that, you’re going through the questions of, does it need to be physical? Does it need to be analytic? Or conceptual? Does it need to be focused? Because if it’s for my learning, for our internal learning, I usually don’t need a comprehensive one, unless I’m close to launch. I mostly need to be more focused, what are we trying to do if it’s internal thing? If it’s external, then I’m probably going to want to be more divergent, more comprehensive, and more physical, because to have a customer or someone that’s going to go use it, have to use their imagination. It’s hard because they tell you things that they don’t really mean.

 

Bob Moesta

I think the other thing to realize is that different industries have different standards of how they can make changes. Trying to change a thing in a car, it’s really hard. Part of it is that’s why we do a lot of analytical focus prototypes, and we’re doing a lot of stuff in parallel to then kind of bring it together and integrate. Versus when I’m making pickles, I can make a batch, put a salt in, make another batch, a lot of people don’t think through a good prototyping strategy because it’s just easy to make another batch. This is where I would say people are lazy because they think that the fact is, I’m just one prototype away. And this is going to be it. And then they get it. It’s awesome, we’ll do one more. They end up creating 300, 400 prototypes, but they’re always one prototype away from getting there.

 

Greg Engle

And what we’d say is, if you’re talking about one prototype, that’s not prototyping. That’s manufacturing. Prototyping should be multiple things because you have to learn.

Even with pickles, that still takes a long time because it’s food. But think about software, software is very quick. It’s relatively easy with little regulation. So prototyping, in that a lot of people just fly by it. But think of all the mistakes people have made because that. 

 

Bob Moesta

So this is where my belief is that if you look at the amount of code that’s generated, Deming would always say, how do we understand the quality of the code, and you start to realize that 80% of the code that’s generated doesn’t end up in the product, because nobody stopped through it. Some software companies where 80% of the code ends up in it, and they give their engineers way more time to think about what they’re building and why. And they break things down into smaller scopes, so part of this is, right now software seems easy, but I think it’s more of a maturity problem of it’s just easier to build it and try it, than think about it, because they don’t know how to think much about it.

 

Greg Engle 

So we’re almost up on the listener fatigue timing, which is about 20 minutes. So I want to give you one thing, and this is going to kind of piss you off a little bit, because I’m going to give you a question that should take you 20 minutes to answer, I’m going to tell you, you have to answer it in a minute and a half. Is AB testing, prototyping?

 

Bob Moesta

So AB testing was known as the worst way in which to test anything, I think of it as kindergarteners use AB testing, because they don’t know how to frame the system, they go ‘Okay, I have this thing, and I can actually change this and this and this, and here’s that, here’s a B, and let’s see which one’s better’. Problem is you build a house of cards, because at some point in time, you figure out which one’s better and you do five, six iterations of A B testing, and you get to the one that you think is the best it, you keep it, but nothing stays the same. So, when we need to reduce the cost, or we need to change this. It literally falls apart because you don’t know why that thing works. So to me AB testing is better than nothing, but at the same time, it’s a very immature way in which to prototype to learn.

 

Greg Engle

So in a spirit of how we do things I don’t really have homework in this episode. So don’t call them questions. Take the time to stop and think about why you are building something because we’re always in such a hurry to build things. And we don’t understand why we’re doing it. What context is it going to live in? So I want people to take this step back and listen to this episode. Think about the questions we were asking, are they setting up prototypes to be successful? Or are they just doing them to produce work. And if you’re just doing to produce work, please start over.

 

Go through to really know what you’re trying to accomplish by using prototypes. Because the worst thing is to spend money on things that are worthless when you can be spending money and time on things that are very useful. And we want to make sure prototyping is very useful because it is.

 

Bob Moesta

It is very powerful. And to be honest, the better you get at it, the more you can work on things. Everything gets better when you know how to prototype a superpower.

 

Greg Engle

As always, thank you for listening. Hope you found this helpful.