When does brainstorming work best?

Conventional wisdom says brainstorming works best when people from different departments or groups come together to think of new ideas or solve problems. But research from Sarah Kaplan, senior fellow at Wharton’s Mack Institute for Innovation Management and a professor of strategic management at the Rotman School of Management at the University of Toronto, shows that diversity is not enough. What is also necessary is in-depth expertise of the topic at hand. Those two factors together result in truly innovative ideas and also yield the highest economic value.

Knowledge@Wharton recently spoke to Kaplan, who is a professor of strategic management at the Rotman School of Management at the University of Toronto and a former Wharton professor, to discuss her research. What follows is an edited version of that conversation.

Focus of the Research

I have been conducting research … on innovation, specifically as looked at through the patenting of technological innovations. I published a paper recently with a former doctoral student at the Rotman School, Keyvan Vakili — who’s now at the London Business School — called “The Double-Edged Sword of Recombination in Breakthrough Innovation.”

I want to tell you a little bit about the double-edged sword that we found. Basically, in research on innovation, there has been an idea that breakthrough innovations, those with the highest impact, are ones that are produced through … what we call “re-combination” or combination processes of distant and diverse knowledge.

What we found in this study was that this is only one creative process and there are other creative processes that contribute to breakthrough innovation. Specifically, we found that different types of creativity contribute to the novelty in the knowledge space — how novel the idea is relative to the economic value that is created Twitter .

In the study, we examined patents in the sphere of nanotechnology because it’s an emerging and exciting new field with lots of breakthrough ideas for over about 20 years and found few patents that were both novel in terms of knowledge and also high impact in terms of economic value.

Patents that were both novel and had economic value were the most valuable. And that was only about 1% of the total patents, so it’s very rare to have a breakthrough in knowledge and a breakthrough in economic value. But when you have that, you get the highest impact patent.

“Patents that were both novel and had economic value were the most valuable. And that was only about 1% of the total patents.”

Key Takeaways

For practitioners, the important thing is to remember that not all creativity happens through combination processes. We have this idea out in the world that … bringing distant and diverse knowledge together is the way to get creative insights, and that’s certainly true. However, what we discovered is that there’s an equally important process of the deep dive, of deep knowledge in one domain.

So, if you are only designing your R&D processes or your new product development processes around that kind of diversity of ideas and combination, you may be in trouble because you don’t have the deep knowledge you need.

This relates to the idea of brainstorming. Everyone thinks we’re going to solve a creative problem by coming together and brainstorming.

What this would say is brainstorming is not enough without the deep knowledge development that you would need in a particular domain to understand what the issues are so that you can break away from existing ways of thinking.

Surprises

This study was actually quite surprising and we found something that we did not expect to find at all. Prior studies of innovation — in particular, studies that look at patenting of scientific inventions — have only focused on measuring breakthrough innovations according to measures of economic value. In the patent world, a measure of economic value is how many times it then gets cited by other patents. What that means is any time you go to get a patent in the patent office, you are required to list prior patents that were essential to the development of your ideas. And the more times your patent gets cited as a prior patent that is essential to the development of ideas, the more valuable that patent is.

And so, in the field of innovation studies, we tend to measure breakthrough innovation as those patents that get many, many citations from subsequent patents saying that it’s really a foundational idea. That is the way historically that the field has always measured innovation.

We actually adapted a computer science method called “topic modeling” to look at the text of the patents themselves, to understand the language in the patent so that we could see when there were shifts in language that would allow us to understand when new novel ideas are being developed. So when the language changes based on this methodology, we are able to say, “That’s a breakthrough idea,” because it’s talking about the domain in a fundamentally different way.

“Brainstorming is not enough without the deep knowledge development that you would need in a particular domain to understand what the issues are so that you can break away from existing ways of thinking.”

What we did in this study is introduce this alternative measure of breakthroughs that focused on knowledge breakthroughs as opposed to just the economic value as measured by the number of times a patent would be cited. And what’s interesting about that is that prior studies had always assumed that if you got a high level of citations it was because the idea was very novel. And so, they just said, “Because the idea’s novel, it gets more citations and therefore, it’s a breakthrough innovation.”

What we found in our study is, in fact, that most of the patents that do get highly cited are not necessarily novel — truly novel from a language standpoint, from a knowledge standpoint — that is, they’re not breaking new ground in the knowledge space.

And so, we break down the set of assumptions that people have made before: that novelty automatically leads to high levels of citations. And by doing that, we’re able to then show that there are different creative processes that lead to novelty than those that lead to generating citations. That’s really the surprising part for me was finding out that the assumptions that the field had been making up until now about the direct connection between novelty and citations were not as neatly linked as people had assumed.

Practical Implications

When we think about any organization that is trying to promote innovation, it’s typically been recommended that you try to create processes that bring together distant and diverse knowledge. So we hear lots and lots of research about diverse teams, you know, bringing marketing and R&D and engineering and all the different groups together to generate innovation.

And while that is clearly very important, what we are finding from this research is that you also need different processes that allow you to do the deep dive into one knowledge space. So if we’re thinking about an R&D organization, that means really valuing the “R” part of R&D, the research part that says we’re going to really dive deeply into an area before we even know specifically what the product might be or the service might be because we have to understand an area deeply enough in order to be able to identify the key problems, challenges or anomalies in the field.

Once you have those insights, coming together in this process of combining different ideas makes a lot of sense. But if you just go straight for combination and diverse teams, you may be missing out on the highest impact ideas because you haven’t done what I consider to be the pre-work, which allows you to have that in-depth insight into innovation.

This corrects a misperception held by organizations, the public and media about how creativity is always about the brainstorming among diverse teams and the like. But what we’re finding is that you also need this deep dive information to get the most novel ideas.

This connects to an idea that — for people who have read [T.S.] Kuhn’s work on scientific paradigms where he talks about changes in paradigms — breaks in paradigms really require this deep dive. And so, what we’ve been able to do in this study is contrast the creative process around combination with the creative process around a deep dive in knowledge.

“What we found in our study is, in fact, that most of the patents that do get highly cited are not necessarily novel.”

Other Research

There are two things that set the research apart from other analysis. One is the methodology. I mentioned that we used a computer science technique called “topic modeling.” Topic modeling is a technique that was developed to improve search algorithms.

For example, we’re going to a search engine and we want to put a term in to get search results. [Topic modeling] was developed to help us improve those search results, so that we’re getting the results that we want. We then took that technique and said, “You know, what topic modeling is really about is, it takes what they call a bag of words, a body of text and in the case of our study, the body of text were the texts of the patents that we were looking at, and it infers from that body of text by the co-location of all the different words, what are the key underlying topics in the data?”

This is interesting because most of the time in the social sciences, when we want to categorize themes or topics, we come up — as analysts — with the topics and then we look at the texts or we look at the products or we look at whatever it is and we do the categorizing ourselves. What’s nice about this computer science technique is it allows the words to speak for themselves and tells us what the categories are without us having to impose our own frames of reference on the data. We then identify the topics that were in the body of texts of the patents that we studied and could identify which patent was the source patent of the new topic. So as topics emerged in the field … we can see which patent it is that is the actual source of that idea.

That’s methodologically an important contribution. It’s the first paper in the social sciences that has taken topic modeling and used it in a statistical regression analysis of the kind that we are doing. And it’s part of a vanguard of scholars who are now beginning to use these computer science techniques to analyze texts, understand social science phenomena in new ways.

The other contribution is theoretical, which is that we’ve gone back to the creativity research and found that there are two schools of thought in creativity research. One school of thought is what we call the “tension view,” and that’s the view that in order to get creative ideas, we have to break from our current way of thinking and therefore, we need diverse ideas, we need different ways of thinking, we need to combine or in the language of the field, recombine ideas from all sorts of different domains in order to break us out of our current mental models.

“It’s the first paper in the social sciences that has taken topic modeling and used it in a statistical regression analysis of the kind that we are doing.”

All of the theories around combination and diversity have been based on this tension view, which says that new knowledge is in tension with old knowledge. But there’s another view of creativity — if you go back to the psychology literature — called the “foundational” view. The foundational view says, “No, what you have to do is go deep into one domain in order to see the anomalies, and it’s only when you do that that you can really break out of the existing way of thinking.”

So, the other contribution of this paper is to go back to those insights from psychology and creativity and say, “Wait a minute, we’ve forgotten about the other half of the creative process.” We’ve spent all of our time focused on combination and diversity and less time focused on what the deep dive into existing fields can help us produce in terms of breakthrough ideas.

Follow-up Research

My co-author on this project and I have a follow-up project that is looking at what specific recommendations we might be able to make to organizations about how to design their innovation processes; it’s looking at the organization design for innovation [and] the team structures so that we can understand which types [work best]. You need a combination of experience and less experienced researchers, you need research in different domains. We’re going to look at the team structure and we’re also going to compare different technological domains because nanotechnology, in particular, … is more of a chemistry-based type of innovation process.

We’re going to compare that process to a more engineering[-type] of innovation process like looking at MRI machines that are used for imaging and comparing that to stem cells, which is more of a biological innovation process. We’re going to look at different domains where different creative processes might be at play and then understand what types of team structures are best at producing the breakthrough novel ideas, the ideas that can be highly cited and then the magic 1% that are both novel and the most highly cited out there … which is a way of measuring economic value.

We’ve got a whole project that will get deeper into those specific organizational recommendations. That is one project that I’m doing as a follow-on to this research.