Technologies Capturing Collective Wisdom – IT Trends Series I

The expression "the wisdom of crowds" has become a catch-phrase among the Web 2.0 set (and a book title too), and a host of technologies have emerged to help extricate this wisdom from the background noise of the Internet. However, the logic that the collective knowledge of a large group of people can be powerful if aggregated and distilled becomes even more powerful if the group already has an abnormal concentration of relevant wisdom. In other words, although some of the Web 2.0 technologies developed to support this movement can accommodate a poor signal to noise ratio, the technologies are even more effective with a good initial signal. The challenge is to find a group that naturally has a good concentration of wisdom.

Most companies already have such a group, one with deep if dispersed knowledge on the most relevant questions: their own employees. Many CEOs are now discovering that the Web 2.0 techniques work especially well when used within their own organization. The fact that vast expertise lies untapped within organizations has of course been known for a long time, but technology is starting to offer new ways to actually tap it. Here are a few:

Wikis
A wiki is sometimes described as "the simplest database that could work" but the typical use is in a collaborative editing environment. An underlying philosophy of wikis is "make it easy to fix mistakes instead of trying to prevent them from being made." In other words, participants are encouraged to contribute and edit by lowering the barrier to entry; quality then emerges from the group effort. Powerful tools for tracking revision histories generally make it safe for everyone to be given a hand in editing.

The power of wikis is that they enable lots of participants, each with perhaps a small piece of the overall picture, to add their pieces and then over time figure out how they all fit together. The success of Wikipedia, perhaps the best known of all wikis, attests to the effectiveness of this process. But wikis are increasingly being used within smaller groups, sometimes just among a handful of people to collaboratively edit a document, but also within companies and other organizations trying to do precisely what we described earlier: harness the wisdom of their own particular crowd. Companies have used wikis internally to generate commentary and discussion among regionally or globally dispersed groups on major events or actions (m&a, organizational changes, etc.) allowing senior management an efficient forum for rapid feedback. The applications for wikis are many and with appropriate guidelines and oversight can work at all levels within organizations for broad collaborative efforts.

Open Innovation
The trend toward open collaboration in engineering has been most prominent in the software industry as the "open source" movement, but the underlying philosophy that more eyes are better is spreading to other disciplines as well. "Open Innovation" is a model in which very specific engineering challenges are shared with the public on the off chance that somebody outside the organization might have a good answer to a hard problem. Proctor and Gamble has been an early champion of open innovation, and the practice has spread to other organizations. InnoCentive.com is a web site catering to both companies with problems to solve and individuals with ideas to share. "Seekers" and "Solvers" they call them.

But once again organizations don't necessarily have to turn to outsiders to find untapped expertise. Sometimes the open innovation can happen within the organization. This is especially true in large organizations with geographically diverse operations, but even many small companies can face difficulties in efficiently matching up challenges with the expertise they already possess. The open innovation solution is to make sure the problems are widely disseminated, and that potential "solvers" are given the means and incentive to seek out places they can help. At least one large pharmaceutical company has adopted this strategy, building a company-wide database where employees can both list problems they need help on or search for interesting challenges to work on.

Predictive Markets
A predictive market, like a stock or commodities market, allows traders to buy and sell shares based on their beliefs about future events. A predictive market differs from these other types of exchanges primarily in that its value lies more in the information it forecasts than it does in the monetary value of the shares being traded. A popular use for predictive markets in recent years has been to forecast political elections (for example, www.intrade.com), but this particular application suffers from an ailment common to many such public predictive markets: it is merely aggregating opinions which are largely based on another public source, namely polls. (It is telling that high stakes election markets thrived up until the early part of the 20th century, when scientific polling became widespread.)

Where predictive markets can have their highest value is scenarios in which the participants have information which is relevant, accurate and exclusive. Once again employees can, depending on the prediction desired, be just such a group. Forecasts of important data such as product release dates, production capacity requirements, and even the success of new products are all areas in which employees should, as a group, have as accurate a view as anyone. Corporate predictive markets first started appearing in the 1990s as home-grown experiments; today they are increasingly relied upon and several startups have sprung up to help companies implement them. (See, for example, www.inklingmarkets.com) The reason predictive markets are, or can be, more effective than simply asking employees what they think is that markets avoid a host of common human behaviors that can taint traditional survey data. In a predictive market one does not worry about disparaging a coworker or contradicting a supervisor, nor can managers (subconsciously or otherwise) selectively filter information.

All three of these trends—wikis, open innovation, and predictive markets— are ways to harness collective wisdom that can be equally and possibly more effective when used specifically within an organization. For any of them to succeed, however, there must be incentive to participate. The incentive could be financial, but peer recognition and the ability to have a “voice” can be sufficient motivation. Even in that case, though, the technology platform should be designed to maximize the many beneficial aspects of the collaboration.