Wisdom of Crowds – or stupidity of the mob?

A couple of days ago I came across a review of Andrew Keen‘s book The Cult of the Amateur. As in Keen‘s (must read!) debate with Chris Anderson (podcast) he is making the point that today’s internet — Web 2.0 — is killing our culture because the traditional gatekeepers are being removed, opening cultural, economic, and political life up to amateurs. With a negative impact on quality, as the crowd would only be able to produce mediocrity. To quote Keen in his own words from his debate with Anderson:

“I still think that the wisdom that I value — the scarcity, to put it in economic terms — is not in the crowd, but in people with talent and experience, whether they exist in political life, in economic life or cultural life.”

To defend the pre-Web 2.0 state of affairs that Keen prefers two conditions would have to be true:

  1. The people in the gatekeeper positions are people with talent and experience, enabling them to discover all the other talent in an effective way and foster it efficiently
  2. The result of this process is output of high quality

Just focusing on publishing as one example of the cultural life this is clearly not the case (as previously discussed):

  • the traditional process of screening new authors and estimating the sales is clearly neither effective nor efficient: a lot of talent goes undiscovered, as the process of discovery is not only based on talent and not all talent is getting through; a lot of other talent is overestimated (at least from a sales point of view), resulting in return rates of about 40% on average
  • as everybody familiar with the bestseller lists can attest they are not really a beacon for literary excellence, since the publishing houses are not operating in an economical vacuum and therefore have to publish what sells — to the crowds, who are by Keen‘s terms responsible for mediocrity in the Web 2.0 environment; I argue that because economic rational dictates what gets published and what doesn’t today the same is already true in the pre-Web 2.0 world

These obvious shortcomings in today’s publishing are at the heart of what we are trying to solve with quilp — to be launched shortly — and what I believe the technologies around Web 2.0 are best suited to enable people to do. I therefore couldn’t disagree more with Keen‘s argument, neither with his underlying observation that the current process is effective and ensures quality nor with the assumption that quality has to suffer if this process is opened up to amateurs.

I believe that there is a tremendous amount of unused creative potential out there that we can tap into and open up to everybody by providing the right tools as an enabler for sharing and evaluating ideas without outdated bottlenecks.

Strangely enough, at the same time that Keene favors a few gatekeepers over selections by the crowds he cites an article from the Wall Street Journal to prove his point: of the 900,000 registered users at Digg.com, only 30 were responsible for submitting one-third of the postings on the home page. Isn’t this the selection by the few experts that he is so fond of? Or is it a question about the authority of being an expert? But who is better suited to determine who the expert is in a certain field — a few employees at a publisher or newspaper or a crowd of people especially interested in a specific topic?

I think what Keen really wants to stress with this quote is the room for manipulation he is seeing in amateurs vs. professionals, their hidden agendas. If I recall the events that lead to the current war in Iraq correctly, however, it is the professional media that has been mislead the most and the independent amateurs that were giving a voice to a more pluralistic debate. That the professional media should be immune to hidden agendas especially if they are owned by arms industry magnates like in France (Lagardere, Dassault) is highly questionable, as Jürgen Altwegg points out in the Frankfurter Allgemeine Zeitung (July 4th, 2007); the internet, as Altwegg — a professional journalist by Keene‘s standards — argues, gives more room for tough questions, putting more pressure on journalists in turn to address these issues.

As Surowiecki points it out in his excellent book The Wisdom of Crowds there are certain elements required to make a crowd’s decision wise. Arguing against or in favor of crowdsourcing in general terms therefore doesn’t make a lot of sense. The design of the tools enabling this process is crucial for the value it manages to provide.

P.S. Funny enough, I only came across the review of Keen‘s book debating the value of crowdsourcing through one of my favorite crowdsourcing tools: using the affinities with other readers which I discovered through my favorites on del.icio.us I can now quickly filter millions of documents, as other gatekeepers which I determined as being relevant to my interests are doing the filtering for me.

Update (July 7th, 2007):
There has been quite some coverage of this topic recently – some interesting links:

Update (July 10th, 2007):
“A Luddite argument is one in which some broadly useful technology is opposed on the grounds that it will discomfit the people who benefit from the inefficiency the technology destroys. An argument is especially Luddite if the discomfort of the newly challenged professionals is presented as a general social crisis, rather than as trouble for a special interest.” Clay Shirky in Andrew Keen: Rescuing ‘Luddite’ from the Luddites (July 9th, 2007)

And here another post by Clay Shirky, quoting Scott Bradner: The Internet means you don’t have to convince anyone else that something is a good idea before trying it. The upshot is that the internet’s output is data, but its product is freedom.(July 10th, 2007)

Update (July 20th, 2007):
The Good, the Bad, And the ‘Web 2.0’ (Andrew Keen and David Weinberger in Wall Street Journal; July 18th, 2007)


2 Responses to Wisdom of Crowds – or stupidity of the mob?

  1. Peter Beermann says:

    100% agree! Furthermore I think that Keen’s argumentation is not very realistic. Let’s face it, the major principle these gatekeepers stick to while making their selection is not quality but profit orientation. They will prefer content that is supposed to bring the expected ROI. In most of the cases they won’t take any chances by going for very unpredictable content even though they might value it due to their expertise in this field. The crowd itself on the other side is free from profit orientated thinking. They will choose the content they value most. The new technology is what enables them to do so. The question is, if this crowd chosen content is from an expert’s view of higher quality than the pre-selected content? Not necessarily, but at least it is what people chose. Isn’t that a major criterion for the quality of content? Additionally a learning effect will take place while choosing this content. By confronting people with bad content they will learn to identify quality criteria. Thus they will become trained experts themselves. People will therefore have the ability to recognize the Mozarts and Van Goghs of our time themselves and not value them because they were told to!

  2. Good point, Peter. Content with sales *expectations* below 5,000 won’t even get out even if it is perceived to be of high quality, as smaller print runs aren’t economically viable. Ergo all the 4,999 people within many niches who would love to get their hands on it don’t even get served.

    The internet has the potential to change that. And it gives room for evaluating content beforehand to better grasp its sales potential. As the Rowling example shows, many (astronomically) underestimated Harry Potter’s sales potential as there wasn’t really a control segment around in publishing to base the estimates on. Nowadays a sorcerer kid comes in many shapes and forms as this segment seems to be big enough for me-toos.

    The current (single expert based) bottlenecks might succeed to some degree in evaluating something well-established (although the 40% return rates even here suggest they don’t) but have obvious limitations in paving the way for the unknown because of the costs associated with the decision to publish. As the costs of publishing online are virtually zero the threshold for testing many different creative ideas is removed. What’s better than that?

    With regard to the quality aspect of this equation I’d like to quote science fiction author Theodore Sturgeon, who when confronted with the question ‘Isn’t ninety percent of science fiction crap?’ by an interviewer replied: ‘Ninety percent of everything is crap – but the remaining ten percent are worth dying for.’ Let a diverse crowd of people decide where their ten percent lie and don’t deprive those who think differently of their freedom of choice. Judging by the expert decision of twelve editors at different publishing houses who rejected Rowling millions of people around the world are making the wrong decision by bying Harry Potter – enough rejections that would have lead many unknown authors to give up. If we can give authors the tools to reach their audience and let the readers decide for themselves without limitations to creativity, then the work we are putting into building this platform for authors and readers has been worthwhile.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: