Maximize the Wisdom of the Crowd for Complex Problems

Crowdsourcing is a powerful tool for innovation. All kinds of industries have embraced this approach to collaborative problem-solving to tap into a much wider pool of experiences and perspectives than a few experts could provide.

However, despite its many advantages, crowdsourcing can have disappointing results if not managed properly. A notable example is BP’s crowdsourcing initiative to find solutions that would alleviate the damage caused by the Deepwater Horizon oil rig disaster, one of the largest oil spills in history. Despite receiving 43,000 ideas, BP could not identify a single groundbreaking suggestion that would help mitigate the situation. Proposed solutions were either unworkable or mere modifications of existing solutions.

Crowdsourcing requires careful design, robust quality control mechanisms, and consideration of solutions’ potential pitfalls to deliver on its potential and ensure successful outcomes. Most important, our research has found that the optimal approach to guiding crowds depends on the type of problem a company is trying to solve. Specifically, our results show that crowds can be guided to solve complex problems more successfully and generate more innovative ideas when participants are encouraged to share their knowledge about the problem with others and to build on knowledge others have shared.

Two Instruction Approaches for Guiding the Crowd

We studied two approaches to instructing participants on how to solve a crowdsourcing problem. To understand the methods’ effectiveness, we conducted a field experiment that involved deploying 20 crowdsourcing events for 20 different organizations, from small startups to large companies across a range of industries. The companies that applied the first approach — called idea-sharing guidance — instructed participants to share their own ideas and build on one another’s. This is the approach that has traditionally been used to guide crowdsourcing efforts. The companies that applied the second, novel approach — referred to as integrative solutions generation — instructed participants to share any pertinent knowledge they had about the problem (such as trade-offs, examples, and assumptions) and to combine others’ knowledge with their own to generate comprehensive and integrative solutions rather than just solitary ideas. In each case, the instructions for participants comprised a few sentences that were placed prominently on the crowdsourcing platform, and there was no active moderation of the crowdsourcing process.

An innovation-focused executive at each company crafted the problem statement for their crowdsourcing event, which could be simple (for example, asking for ideas for refining an existing product or extending it to a new market) or complex (such as asking the crowd to imagine a new product for a new market). The executives were also tasked with evaluating the quality of the crowd-generated ideas: They rated how new they considered the ideas (novelty) and how easy those ideas would be to implement (usability), each on a 7-point Likert scale. Different online forums dedicated to discussing each company’s products were targeted for each event, while ensuring there was no overlap in participation. The crowd participants were motivated both extrinsically (to win small cash prizes) and intrinsically (the resulting solutions, when implemented, would benefit the companies whose brand or products they were interested in).

Across the 20 crowdsourcing events, 1,720 registered participants made a total of 2,428 posts. Of those posts, 1,198 were attempts (ideas or integrative solutions) to solve the problem. Out of those, the executives of the respective crowdsourcing events rated 297 nonduplicate ideas.

We found that the effectiveness of each approach was dependent upon the complexity of the problem presented. Idea-sharing guidance may yield better ideas from a crowd when members of it are presented with a simple problem. However, when problems are more complex, the integrative solutions generation approach to guidance yields better solutions. Thus, the guidance provided to a crowd needs to align with the complexity of the problem to be solved.

Complex Problems Call for an Integrative Solutions Generation Approach

Through our research, we learned that when a crowdsourced problem is complex — one that requires participants to identify new products and markets simultaneously — guiding the crowd with integrative solutions generation instruction yields ideas that are more innovative (more novel and more easily implementable). Integrative solutions generation encourages crowd participants to make their pertinent knowledge explicit. Thus, they may start questioning their assumptions and reflecting on their perspective of the problem. In doing so, the crowd may collectively end up redefining the problem itself. Further, by incorporating others’ problem perspectives and knowledge when crafting their solutions, crowd participants diverge from their idiosyncratic thoughts and integrate their ideas with others’ ideas to form more innovative solutions.

We found that organizations that used crowdsourcing to address complex problems were able to find better ideas when they guided participants to apply the integrative solutions generation approach. For example, a professional sports organization asked the crowd, “What new revenue streams can [the organization] develop to generate funds for its central organization?” This problem is complex because it is an open call for ideas that does not specify products or a market. It requires holistic solutions and cannot be addressed without considering the complex ecosystem in which the organization operates, including consumers, players, business partners, and other stakeholders.

This problem statement was broadcast on several topic-relevant online forums for sports enthusiasts. The crowd developed multiple ideas simultaneously, incorporating one another’s shared perspectives of the problem. Members of the crowd began to challenge the underlying implicit assumption that revenues had to be tied to exploiting the organization’s authority in ranking the sport’s players. In this way, others in the crowd were encouraged to think outside the box and share their knowledge (facts and examples) about other sports disciplines that might be relevant to the problem at hand.

Crowd members often provided sources (in the form of web links) supporting their perspectives and knowledge. The conversation then evolved into the topic of partnerships based on licensing and merchandising.

For example, one integrative idea likened the sports organization to a venture capitalist, shifting the focus to a completely different value proposition: “Initiate a player-equity model which invests in the development of up-and-coming youth players, eventually capitalizing on their success — perhaps with partial exclusive rights to endorsements. Model after the venture capital industry, planning for a large number of no-gos, a few 1x return, and rare 10-100x returns.” This example illustrates how the crowd redefined the initial problem statement by challenging implicit assumptions about building on the sport’s player rankings and then incorporating partnerships and licensing as ideas from other sports contexts. Participants also identified trade-offs and developed solutions that accounted for both the advantages and disadvantages. Overall, this process resulted in ideas that holistically solved the problem by considering all of its elements simultaneously and clarifying the boundaries of the problem. The solutions generated were rated highly novel and usable by the executive of the organization.

We also found that the approach of providing idea-sharing guidance is much less effective when crowdsourcing solutions to a complex problem. When the crowd is used to solve a complex problem, participants may need to collectively redefine it rather than just share their perspectives. Participants who share and refine only their own ideas, per the instructions given to them, do not have the opportunity to successfully integrate others’ perspectives and ideas into their own. Such a mismatch between guidance and the problem shifts the burden of cognitively processing the problem onto individual crowd participants instead of facilitating a collective solving process. When participants are unable to understand the problem from multiple perspectives or fail to integrate others’ perspectives, they are more likely to generate ideas that do not solve the problem.

We observed low performance that could be attributed to this form of mismatching. For example, a U.S. toy company broadcast this problem: “What new and disruptive products, services, and/or business models can [our company] pursue to grow its core brands [and] build new franchises, markets, and customer bases?” This problem statement is considered to be complex because it asks for new products and markets simultaneously. The ideas generated by the crowd, which was given idea-sharing guidance, were underdeveloped and rated as having low innovation quality by the company’s executive.

Simple Problems Call for an Idea-Sharing Approach

When the crowd is asked to solve simpler problems, such as developing new products for existing markets or finding new markets for existing products, being guided to simply share ideas leads to better ideas. The crowd can be tacitly influenced by others’ ideas without incurring the cognitive cost of having to explicitly integrate others’ knowledge or perspectives.

Several companies in our study achieved high performance using the instructions for idea-sharing guidance to solve simple problems. One example is a telecommunications company that broadcast this simple problem: “What new and innovative services/products can [our company] provide to its customers using new mobile technologies?” This problem is simple because it focuses on existing customers and leaves the door open only for innovation for services and products. As in the previous examples, the problem statement was broadcast on multiple topic-relevant internet forums. With idea-sharing guidance, the crowd developed innovative ideas without having to redefine the problem or engage too deeply with other crowd members.

Asking the crowd to share and integrate one another’s perspectives and consider redefining the problem levies a cognitive cost that is unnecessary when solving simpler problems. For such problems, crowd participants who followed instructions to integrate others’ perspectives often proposed solutions that were rated poorly on their ease of implementation. Thus, companies may be better served by asking members of the crowd to simply share their ideas, leaving the decision of whether to use and improve on others’ ideas up to the participants.

A mismatch between the type of problem and the guidance given to the crowd led to low performance in several companies we studied. For example, a U.S. software company broadcast this problem statement: “What technology could [our company] offer its retail and brand clients that would enable consumers to enjoy all the benefits of online shopping?” This problem statement is relatively simple, since it asks only for new technologies. However, the crowd was given integrative solutions generation guidance rather than idea-sharing guidance — a mismatch that yielded below-average ideas compared with the crowdsourcing events for which the instructions matched the problem complexity.

Subtle but powerful guidance in the form of instructions represents an elegant way to manage crowds. Instructing participants to share and integrate their perspectives of a problem and combine their ideas (integrative solutions generation) when the broadcast problem is complex leads to the generation of ideas that are more innovative. On the other hand, when the broadcast problem is less complex, guiding participants to simply practice idea-sharing can yield solutions that are more innovative.