Thursday, October 27, 2005

What the Wiki?

It seems to be pick-on-Wikipedia-day, so I figured I'd add my inconsequential voice into the fray. After all, the very fact that I have a blog means I'm a trendster. Before I trash Wikipedia, or at least the theoretical underpinning behind it, I'd like to note that I've found its articles on mathematics and statistics extremely useful.

The Register is running an article about Wikipedia, and ostensibly how it's a different kind of "open" than Linux. The article, however, really only touches briefly on this, preferring rather to focus on how Wikipedia differs from a traditional encyclopedia and why these differences are bad.

What disappoints me is that Mr. Orlowski (the author) misses the opportunity to highlight what I believe is the major fallacy behind Wikipedia. I refer, of course, to the idea of "Collective Intelligence," the theory that if you get enough regular schlubs together, you end up with a virtual Leonardo Da Vinci. To this I say: Hogwash!

I've seen very little rigorous discussion of Collective Intelligence (or the more soundbite-friendly "Hive Mind"). What I've seen, however, is very different than the purported effect behind Wikipedia's brilliance. The basis for the theory of Collective Intelligence seems to be that if you ask people who have a basic familiarity with, but no concrete data on, some thing, their estimates about that thing will tend to be distributed around the true value.

Note how different this is than, "if you have enough people reading and correcting an article, its quality and accuracy will gradually improve." Mr. Orlowski gives the lie to this with his quote of Carlo Graziani with which many will sympathise:

I'm relieved to see other people are also wary of information that they get from a source whose organizing principle appears to be that twenty jackasses make an expert. Although after reading your take on Wikipedia, it appears that the actual situation is worse - the output produced by twenty jackasses plus one expert is indistinguishable from what would be produced by twenty-one jackasses.

Unfortunately, I can't track down the original article, so I don't know to whom Mr. Graziani is responding.

The important point is that collective intelligence merely implies that as you ask more people to estimate some quantity, there's a good chance that the average will more closely approach the true value. That's all. You get no guarantees. You get nothing qualitative, nothing descriptive.

That's not to say that Wikipedia is an evil that must be wiped from the face of the Earth. As I wrote at the outset, I've found some of their articles very useful. The idea of a collaborative encyclopedia is fascinating. Supporting the idea with the misapplication of a set of empirical observations is just foolish.


Scott said...

To my utter surprise and amazment I've found Wikipedia a valuable (and highly addictive) resourse.

I've not questioned anything I've read/learned there so far.

Mike Marsh said...

Ah, a fellow Debianista, or Debianer, or whatever we're supposed to be called.

Yes, Wikipedia is very useful. I don't debate that, only the theory behind why it should be useful. It seems that the editorial controls have reduced or prevented some abuses, but it's also telling that these controls were needed.

I'd argue that the reason Wikipedia has some high-quality articles is that it is now well enough known to attract a broad base of expertise. Rather than benefitting from some sort of hive-mind, it's benefitting from the actual experts noting its existence and potential usefulness, and providing additional material for it themselves. This is a great model for a publicly available resource, and the naturally suspicious might say that the original rationale for Wikipedia was more of a marketing strategy to bootstrap the current model.