Over the past 4 months we’ve diligently tracked some of the nations most prominent pundits. Not everyone welcomed having the card marked but we continue to track them anyway. Our objective was to find out how accurate these pundits were and whether their accuracy (or lack of it) actually matter to readers.

The results to date show Nikki Finke in the lead as while she has only a 55% accuracy record she’s successfully predicted a number of rank outsiders giving her a healthy return. Celebrity blogger, Perez Hilton, isn’t far behind in third position overall but DListed trails badly with a 33% accuracy ratio.

In the world of politics David Brooks is top dog with an impressive 62.5% accuracy ratio but additionally a number of well picked outsiders. Pat Buchanan has achieved an impressive 80% success ratio however many of his punts were odds on favorites so he still only ranks in 6th position. Chris Mathews however is in the red with a 41.67% hit rate. Next time Chris offers up some sage advice keep in mind that you would do better trusting a coin toss!

In the tech world we have expertly written Venture Beat edging out former Wall Street analyst Henry Blodget’s Silicon Alley Insider. Venture Beat managed to hit an impressive 66.67% accuracy rate. The runt of the tech litter however was Mike Arrington’s TechCrunch with came bottom of the overall table with a woeful 20% accuracy rate, making them about as accurate as Barack Obama’s 10-pin bowling arm!

However the second test was to find out how much readers care about accuracy. Well using Compete to analyze traffic to entertainment blogs shows Perez Hilton remains the standout leader, followed by DListed and then Nikki Finke. All three blogs appear to have had fairly consistent traffic levels over the past 12 months. One can deduce that while readers of entertainment blogs seek celebrity gossip, inaccurate celebrity gossip seems just as good as the real thing.

Similar conclusions can probably be drawn in the tech world were TechCrunch leads the pack with more than double its nearest rivals user numbers. However VentureBeat and Alley Insider appear to still be growing strongly in the past 6 months while TechCrunch appears to have plateaued. Perhaps in the long run in tech credibility rather than confident speculation will win out in the end?

For politics it is harder to track each pundits ups and downs however a clear loser over the past 4 months was Chris Matthews who was pulled from MSNBC’s election coverage. However, there the reason was due to MSNBC’s concern about its perceived liberal bias as opposed to Matthew’s patchy record as a pundit.

This marks the end of Pundit Watch’s four month experiment. After this point we will no longer be recording new predictions however we will retain the leaderboard to see how the remaining predictions play out. Additionally, we have just launched partner accounts (with Reuters as our launch partner) so we will be inviting the nine pundits to join us there.

By Nigel Eccles (Chief News Junkie)

Bear Sterns is doing just fine shouted Jim Cramer just before they got bought by JP Morgan for $2. Google could hit $2,000 claimed Henry Blodget of Silicon Alley Insider just as their stock price peaked and started its long slide to under $500. Hillary Clinton is going to lose in New Hampshire claimed just about every commentator before her dramatic come back.

These are just some of the examples the recent examples of ‘experts’ failing miserably to predict future events. But we still listen to them. Are these isolated examples or does the seer/sucker theory hold true (”No matter how much evidence exists that seers do not exist, seers will find suckers”)?

The Pundits

Well now we will find out. Today we are launching PunditWatch into private beta. PunditWatch’s mission is to track the predictions of public commentators to see how they perform. Initially we are going to select commentators with three each from the fields of Politics, Entertainment and Technology. The pundits are:

• Politics: Chris Matthews (MSNBC), Pat Buchanan (MSNBC, NBC), David Brooks (PBS, NYT)
• Entertainment: Perez Hilton, Nikki Finke (Deadline Hollywood Daily), DListed
• Technology: TechCrunch, VentureBeat, Silicon Alley Insider

There were plenty more we wanted to add but after exhaustive investigation we found that they don’t ever in fact make any real testable predictions! We do however plan to add a second wave of pundits to this list so do comment with your suggestions.

How it all works

We have created a Hubdub account for each of the 12 pundits (for example the TechCrunch user id is techcrunch_pw).

1. Every Monday morning we will post up a round of all of their previous week’s predictions, a look at their performance and also a call for readers to be on the look out for them spotting new predictions.
2. During that week each time a pundit makes a prediction we will place H$10 on it from their Hubdub account.
3. If the question doesn’t exist we will create it with an estimate of the correct probabilities. We will then invite some of the top Hubdub forecasters to make predictions in order to set the prices. Then when the prices have settled down we will place a prediction for the pundit from their account.
4. We will update the blog with a post to say that the prediction has been placed.

Pundits can make anything from zero to three predictions per week. If they make more than three then we will ignore the later ones. After analysing many of these pundit’s predictions it is clear we are going to have to give some interpretation to their predictions to make them testable (hey, if they don’t like that then they can just start making real concrete predictions!). We will have to feel these out as we go along. One point is that if a pundit reports rumors then we will treat these as a prediction. If they don’t think the rumors are likely to be true then why report them?

Call for spotters

We’re keeping an eagle out for these pundits but it is quite possible we miss something. If you spot any of them making a prediction please let us know and we will follow up.