Marshall wrote:No, we shouldn't. That would be a rhetorical question, when questioner already knows the facts were not tested by a formal survey. But, we could run a survey here. Dan?

But the poster might say something to the effect of "it's been my experience that...", which is perfectly fine if presented as such. Also, the questioner doesn't have to know or not know if the assertion was tested or not. Just asking gets it out there and provides the openness that I think we'd like.

-Greg

The challenge is that 5 seriously vocal and active and belligerent and opinionated people of your description can destroy both the usefulness of a site like this and the s/n ratio

As the population grows, the S/N can only get worse (there will be more outliers in the distribution of signal strength and in the definition of "on topic" on any one topic). People come to sites like this carrying different bundles of needs and as the site population grows so does the fraction of people wanting among other things a sense of belonging, which suggests a predilection for groupthink on their part. The real problem is the changing weights in the population of the different kinds of needs. I suggest having a look at Clay Shirkey's essay at http://shirky.com/writings/group_enemy.html . It is about precisely this phenomenon.

"The likelihood that any unmoderated group will eventually get into a flame-war about whether or not to have a moderator approaches one as time increases."

The issue of the conventional wisdom is a problem, and I suspect that it is an insoluble one because so much of it is arguable and it has changed so much over the last few years. The changes in the conventional wisdom are in a sense smeared out over the population with different people holding different versions of it. That is just natural but it is also part of the motivation of this discussion. How to change nature? That would require changing the way that people engage with the content and I doubt that that is possible. Part of the problem here is that the goals of the site are relatively diffuse; there is an awful lot that is in scope for this site, and so quality control is hard, keeping the site on track is more likely to come down to subjective issues of quality rather than keeping to a defined area, and that is why I am pessimistic.

The site has changed a lot over the last few years. When it started it seemed to deal with a higher percentage of seriously high-end domestic hardware and exotic machines. It was very cool and refreshing, it was much more interesting than coffeegeek at the time. The site is still distinguished by a strong technical and quantitative focus, but in percentage of content terms that is a declining part. The content on the site comes from only a few kinds of sources, it would be interesting to see what percentage of non-meta content is originated by reviews, what by "old" members and what by "new" members, and how much in each category is problematic on some definition like Luca's. I know its not an analysis that anyone has time for, but I have some suspicions about the distribution.

It seems to come down to "how do you weigh or QA contributions?". Greg Scace's approach is one way. I am not sure that you can do that kind of fine tuning of contributions without some loss of vitality, either in the evolution of the conventional wisdom or in the participation of new people. The unfortunate side effect is a lower S/N.

Greg