Subprime failure and Prediction markets

What do the subprime meltdown and Iraq have in common?

Massive failure due to forecast errors.

I have written in prior posts about my respect for Nassim Taleb's book Black Swan, which speaks to the enduring inability of humans to recognize the fallibility of forecasts and linear thinking.

With respect to the subprime mess, Merrill Lynch, Morgan Stanley, and Citibank, alone, have announced $36+bn in write-downs to date. The most technically advanced companies in the country failed to live up to their raison d'etre: to price and manage risk.

Their failure is a powerful reminder that sophisticated business processes, risk management models, and management teams are no panacea if the assumptions that architect their systems are wrong.

For example, risk management is based on the premise that events in financial markets exhibit normal/Gaussian distributions. Value-at-risk models calculate the maximum loss not exceeded with a given probability/confidence interval over a given period of time. For example, the risk manager will report that with a 95% confidence the maximum capital at risk is $x. The losses in the financial sector are a powerful reminder of how rare events blow up the models and with them the business processes, risk controls, and balance sheets of their creators.

In supply chains, forecast errors compound across the supply chain in a phenomena known as the bullwhip effect resulting in excess inventory and a costly failure to match supply with demand.

Examples of forecast errors are legion - product ship dates, sales forecasts, demand forecasts, value-at-risk models, elections, stock price predictions, etc. A common remedy to forecast errors is to increase the liquidity of guesses - the greater the number of independent predictions the more accurate, in aggregate, the final prediction.

We can see this phenomena in today's social web - the web is rewriting the rule book on how we program content- rather than a top-down, command economy approach, where programmers decide what we should read, watch, and discuss - users are leverage social media sites to "reprogram" content. Communities, like liquid markets, vote with their time, comments, and clicks and the best of the web gets pushed to the top. Innovative companies are leveraging the wisdom of the community to ensure a better match between supply and demand.

So, where am I going with this post? Prediction markets are nascent business tools that allow employees to buy or sell certain business events - probability of product shipping on time, unit volumes to ship in the quarter, annual bookings, prioritizing new business ideas - and thereby allow their employers to improve the quality of information factored into predictions. The formal chain of command is infamous for distorting and hiding information - it is no wonder that CEOs are flying blind...their business systems are based on flawed models and their teams are incapable of accurately reporting the true state of affairs. Chinese whispers corrupts information moving up the chain and smart people are stuck making decisions with bad, misleading data.

We will never be able to predict the future, however, all of us should consider how we can open up our decision making processes to allow for non-biased, comprehensive input that allows the wisdom of our organizations to weigh in on key decisions - better inputs enable better capital and resource allocation decisions and can help avoid disaster.

While prediction markets are much less complex than advanced prediction algorithms, in the spirit of less is more, the front line sales people, developers, mortgage loan officers, etc will always have better information than the central corporate staff. What seems to be failing corporate America is an open framework for capturing that knowledge in a non-biased, confidential manner.

For centuries soldiers have complained that the central staff had no idea what was happening on the ground - the science, tools, and applications, however, exist today that allows for the harvesting of the collective wisdom of the group.

I predict, yes am I aware of the irony, that in 2008 corporate American will come to adopt one of the mainstays of web 2.0 - ie applications that leverage the tacit or explicit wisdom of a community. Prediction markets are needed to help unlock the tacit knowledge of organizations and to lessen the colossal forecast errors now reverberating through our economy.