Pruning season

by Joel Spolsky

on September 14, 2010

Soon, we're going to close down one of the 24 Stack Exchange sites that were created over the last two months. Sadly, the Gadgets Stack Exchange will go dark soon. A lot of the questions on that site were about Apple gear and Android gear; those questions can be migrated to the (beta) Apple site, or the (private beta) Android site, as appropriate. Gadgets will be closed and will no longer be accessible, except for our usual creative commons data dumps where an archive will be made available.

This isn't an arbitrary decision; we did a ton of thinking and questioning before we decided that the gadget site just didn't have enough momentum to get out of beta.

Q: Why do sites even need to get shut down? Are we running out of bits on the Internet?

First of all, because it's what we said we would do in the original Stack Exchange 2.0 announcement:

Why is the plan to close down sites that don’t get enough traffic?

This harks back to our corporate goal to “make the Internet a better place to get expert answers to your questions.” A ghost town, without traffic, does not get people answers, but it does draw a few people away from other sites that might do so. We do not believe that the Internet benefits from putting up placeholder sites with negligible traffic that do not attract high quality communities. And we want the Stack Exchange brand to be synonymous with great community Q&A; sites, even if we don’t necessarily cover every topic under the sun.

A site that's not really functioning is a trap for the unwary. The few users who do, accidentally, land there are lured into asking questions which will not be answered or will not be answered well. Even if are a few people around, if they don't have enough collective expertise to give good answers, the site is a net negative for human knowledge.

Eventually, a site that doesn't have critical mass becomes a spam attractor and a public nuisance, and we don't want to be behind that. We'd rather close the site and channel users to other sites which are working.

Q: What criteria are used to decide if a Stack Exchange has "critical mass"?

We're looking at lots and lots of metrics, but the most important ones are people and questions.

People: Do we have a lot of people visiting the site? Are a lot of people signing up? How many people are answering questions? How many page views does the site generate?

Questions: Are questions getting answered? Are they answered well? Are they answered quickly? Are a lot of answers accepted, indicating that the person who asked them was satisfied? Are a lot of answers upvoted, indicating that some third party thought they were quality answers?

Our philosophy is that if a site is getting a lot of traffic, that's all we need to know... it must be doing something right. If it's not getting a lot of traffic, it may still be valuable, as long as the few people who go there are getting great answers to their questions (which, thanks to our architecture, is really easy to measure). So, essentially, a site needs either traffic or good answers, but if it has neither, we don't think it will work.

Q: Is there any editorial judgment involved?

Before we pulled the trigger, we thought about why the gadgets site wasn't working nearly as well as its 23 siblings. Looking at the questions on the site, it's clear that there are too many kinds of gadgets, and our audience is too small to be able to answer detailed questions about all of them.

Think of it this way. There are probably tens of thousands of different kinds of cell phones, but only about 50 people who answer questions on the Gadgets site. What are the chances that one of those 50 actually knows how to automatically record voice calls on the Nokia Series 40? What are the chances that one of the 50 even has a Nokia Series 40?

A site needs to have a wide enough swath of active experts to cover the entire domain it purports to cover. Stack Overflow itself has a huge domain, but a huge number of highly active experts, so questions get pounced on, no matter how esoteric. Many of the smaller Stack Exchanges only have a few experts but the domain is narrow enough that they can really answer just about anything. But having a wide domain and a shallow pool of experts results in not enough peanut butter on the sandwich. That's what we think happened to Gadgets, and thats why we think that narrower sites like Apple and Android are likely to do better, even if it means that we don't have a place to discuss garage door openers.

To answer the question: in principle, the only thing we're looking at in deciding whether to close a site is metrics, but we're also using our brains to see if there's something behind those metrics before we pull the trigger.

Q: What about the other 23 sites? Are they likely to get out of beta?

The other sites are all currently producing very high quality answers very reliably. As of now (and of course this might change), there are no other sites that are even close to getting cut.