Optimizing For Pearls, Not Sand

by Jeff Atwood

on June 13, 2011

In March 2010, we rebalanced our reputation system to favor answers.

While we value good questions (and asking a great question is absolutely an art), we want to explicitly encourage people to provide the best possible answers. Without people interested in providing good answers, the questions are moot. We know that answers have more intrinsic value than questions, and the reputation balance should reflect that.

The question asker already enjoys a substantial benefit beyond reputation gain from upvotes on their question — namely, they get great answers to their question! Thus, the asker shouldn’t need as much reputation gain.

In November 2010, we began to actively block low quality questions, too:

We believe asking questions on our site is a privilege, not a right. If, after a few fair attempts, you haven’t been able to prove that your contributions to a particular Stack Exchange make it at least … not-worse … then we reserve the right to refuse your questions. If we don’t do our part to cull the bad questions, then we risk alienating the true experts who provide what really matters: the answers!

Last month we made voting more visible and added 10 additional "question-only" daily votes to encourage people to vote more on questions, so we can better discern their value.

Users intuit that answers are the real unit of work in any Q&A; system and tend to favor answers in their voting.

Continuing in that same vein, we have two more changes to formally announce today:

  1. We now limit users (and IP addresses) to a maximum of 6 questions per day and 50 questions per month.

  2. Downvotes on questions no longer cost the casting user 1 reputation, so they are effectively "free".

Perhaps you've noticed a theme here. Incoming questions are a universal constant, all around us in countless billions. But answers -- truly brilliant, amazing, correct answers -- are as rare as pearls. Thus, questions are merely the sand that produces the pearl. If we have learned anything in the last three years, it is that you optimize for pearls, not sand.

Consider the question Does torture work well as an interrogation technique? on Skeptics. Is this a brilliant question? Is it even an original question? No, it's just a mundane grain of sand question that could have been asked by anyone at any time. What makes it remarkable is the incredible answer on that question by Larian LeQuella with over 100 upvotes.

Sand, meet pearl.

That's why we're determined to keep question quality high, even at the cost of refusing a little sand. It's true that you can't have Q&A; without questions, but having the wrong sorts of questions is far more dangerous. The fastest way to kill any Q&A; site is to flood it with low-quality questions. I think Mark Trapp summed it up best in this meta answer:

To put it another way, when I go to a Stack Exchange home page, I see a list of questions. If most of those are terrible questions with little to no indication that I'd be wasting my time by reading them, the value proposition of visiting and participating is diminished: I have better things to do.

Compare that to answers on a specific question: I've made a conscious choice to look into what I think is an interesting question. I already made the decision that the question is worth my time. If I find the answers to be useless, I have a few different options, as an interested party, to register my displeasure, including writing my own answer. Being able to write your own answer is key: if your answer is good enough, it'll rise above the junk answers and everyone will be better off for it.

There is no such action for question lists. I can't say "these questions suck, show me this question I just thought up instead": that'd be silly. So, it's imperative the question list have a high signal-to-noise ratio, and removing the penalty for those users who do take the time to read a question and later find it to be useless so they can down-vote is conducive to that.

Fundamentally, answers can be filtered in ways that questions cannot. While there is a tension between having "enough" questions and a bunch of amazing, highly skilled answerers twiddling their thumbs waiting around for something to do, in the long run we'd much rather err on the side of having interesting and on-topic questions for these folks to sink their teeth into.

We feel that the world is awash in questions, but not answers. Answers are the real unit of work in any Q&A; system. Therefore, the only logical thing to do is to maximize the happiness and enjoyment of answerers. If this means aggressively downvoting or closing unworthy and uninteresting questions, so be it. Without a community of people willing to answer questions, it really doesn't matter if there are questions at all, does it?