Yes, this is really long. But if you actually care about game recommendations and want to support making them on-topic (or at least want to understand some part of our opposition in clearer detail), then please read it in full.
Since the second week of our existence, there has been a battle waged between allowing or disallowing game recommendation questions on the website. Today, I would like to highlight 2 of the points that I find to be part of the core problem in our support for what I call Repositories: Motivation for Quality and Values Dissonance. Our ability to handle these 2 problems is essential to allowing game recommendations to be on-topic. If we are unable to solve these, it is going to be an extremely uncomfortable ride and will reflect poorly on our image as we struggle to support recommendations. And if you do not think one or both of these is an actual issue, please show me how it is not.
Keep in mind our goal as a site - we want to be an authoritative Q&A website for people who play games.
Preface - What is a Repository?
I have been using this term often when I refer to game recommendations and similar questions. I use this term because I want to differentiate them from being just a generic "list" question, because they are a separate class of entity.
If we exclude the entirely off-topic content that we have received, our question base on the parent site consists of 3 classes of objective questions: problem solving, fact finding, and repositories.
Problem solving questions are the majority of questions we field: questions about particular strategies, how to get past certain puzzles, and how to attain a specific goal. This also includes the tech support and hardware questions we get. In essence, a problem exists and the answer solves it.
Fact finding questions are where the "problem" is more of a curiosity than a hardship. Plot and character questions to start, but also the inquiries we get on specific game mechanics qualify. Trivia is a class of fact finding as well. Basically, the question asks for information rather than a strict "solution", but that information still concludes the question. We get a lot of lists in this class of question, such as weapons lists or damage tables, which serve as a whole as a reference material.
Repositories are almost a subset of fact finding questions in that they seek information that is compiled into a list. However, they possess a number of traits which make them a whole different class than the other two. First, Repositories are often non-finite; many will continue to be updated as new information comes up. Building from that, Repositories also tend to be non-static; the other question classes have a far less frequent rate of change over time. And coming from that, Repositories are expected to be compiled gradually. If someone asks for the finite list of weapons for a game, the more likely scenario is that an expert will generate the full list before posting it; repositories instead are built by individual contributions of singular items over time. The final point is that Repositories tend to place greater importance on the individual items in the list over the compiled list itself. People referencing a chart are looking to find something in that chart or to calculate/compare something in that chart; people referencing a Repository tend to simply be looking for an item that qualifies for the repository rather than specifically care for the repository as a whole.
I'm going to take this a step further and equate the value of problem solving and fact finding questions together into one class "Questions", with a capital Q. While the goals of the two are not the same, their behavior and traits are pretty identical and they can be reasonably compared on the same scale. I can reasonably feel that an expert answer to "How does my grapple howitzer affect my scoring" is as useful as an equally voted expert answer to "How can I safely kill the Ancient Chaos Wyrm". I can also reasonably expect that those votes reflect how useful the material is. For traits, both tend to be finite with a complete conclusion, they tend to be static because updates to games which change them tend to be infrequent, and the answer tends to be compiled in one shot. These traits also make them fit very well in the Stack Exchange system, which is a fairly important point.
Henceforth in this question, I shall be referring to Repositories and Questions using the above definitions. Let us progress to the problems, then!
Motivation for Quality
If our mission as a site is to be an authoritative Q&A website, then we need to show it by providing high quality content. For Questions, this is very strongly indicated in the votes and the effort put into the contributions of our users. You see a passage that gives you great detail on what's going on, you see high quality. The high quality is expected to be reflected in solving the problems of people who have the same issue.
A high quality Repository, then, should reflect in addressing the needs of each person who wants something there. Because the value of a Repository is less in the list itself and more in the presence of a winning item, then a high quality Repository is one that will have the items that newcomers are seeking. This means that as many qualifying items should be added to the Repository as soon as possible, and there should be people who are directing their efforts to filling these Repositories as best they can. This is not happening.
Let us take an existing example: Games that feature procedural generation of content. In the first day, it received 19 answers before being closed on the following day. It was reopened shortly afterwards, and subsequently tzenes posts a comment listing about 15 qualifying games. Despite basically feeding the place a whole set of answers, none of that list is actually added over the course of a month; we only get 4 more answers. Elsewhere on the site, you can find games like POWDER and Elona in game recommendations. In the chat room, I also have heard many users mention a fair number of qualifying games such as a shoot-em-up and a platformer. And yet, all this wealth of information is not being added to the Repository that people are fighting to keep open.
There is no motivation to excel and be an expert in building Repositories here. Otherwise, these Repositories should be filled to the brim with pages upon pages of contributions. So we are really just providing a small subset of qualifying content, namely the subjectively and currently popular content. Including myself, there are many people who continue to state that game recommendations can be founded on knowledge and that there exists expertise, but these past two months have shown quite clearly that nobody seems to care about making a good quality repository that is founded on knowledge.
"Oh, but if the question author is satisfied, isn't that enough for a good answer?", you ask? To be blunt, it isn't enough because that is where the other half of the subjective equation comes into play. People argue that recommendations are founded on objective content, but if you are settling to satisfy only one individual then you are operating on the subjective need of that one individual instead of the larger need of what the question is actually asking. As more people come who want new games that are in that class, they may look at the list and not see something they want. Bounty comes in, and for a week we might pour in new answers that may satisfy that user. Respite comes only to back down when the next unsatisfied user walks in. The lazy approach is detrimental to quality, and also amplifies in effect when we introduce the eventual volume of Repositories that will exist.
How can we expect to be a good resource of information for Repositories if we continue this lazy and subjective approach? I don't know about you, but a recommendation service which only has a handful of qualifying games based on what people simply feel like adding whenever they want does not sound like a very appealing source of information.
There is expertise in addressing Repositories in the same way that there exists expertise in addressing Questions. I mentioned before that I was able to group problem solving and fact finding into one class because they have an equivalent type of value. I cannot do the same for Repositories, because they do not have the type of value. I am not saying that their value is worth less (or more) than that of Questions; they simply are not a comparable entity.
On the effort end, to simply mention Worms and take a screenshot is not nearly the amount of effort it would take to research the more obscure qualifying games and get enough information to post a quality analysis. If that game fails to take off and attract people to try it (which is highly likely once the list exceeds 30 items and it gets buried in the pages), then it is not going to get as many votes despite the greater effort.
Instead votes are accrued in a Repository based on the popularity of an entry. They mark what people simply like. And that is also what people who are seeking recommendations are going to want - they want something they are going to like, and knowing what people like is a stronger indicator than how well a game qualifies for the criteria. After all, the criteria is only a means to narrow down the scope of what is being examined.
This is where the dissonance begins. The way votes work on Questions is in-line with what Stack Exchange is meant to be, and what our mission of providing high quality is meant to be. But votes on Repositories do not reflect the usefulness, effort, or quality of the contributions to the list as they should be. Even though there is value in high quality Repositories, the votes are not likely to reflect that. A highly rated answer to a Repository is simply something that is highly liked. Which is not even consistent, as the timing of the exposure greatly determines how many votes an answer will get.
This is not about reputation; admittedly, though, if we consider an entire class of questions on-topic then they really should be worth something. The fact is, votes are actually important in indicating quality. There are aggregate statistics for highly voted items in both answers and questions, and there are people who use votes to help find what is useful. And the people who follow by these are not pleased when they have to deal with volumes of votes awarded to things seemingly effortless and subjective. It's rather demotivating - why put so much effort in solving problems and finding facts when just waiting for the next good Repository and tossing out the most popular qualifying game is all you need?
Simply put, trying to house this kind of popularity rating system in the same content rating system used for Questions is not a functional solution. But we only have one voting system to use here.
These are the two points. One is a flaw in motivation, that our users do not actually give the effort to produce quality Repositories. The other is a flaw in the Stack Exchange system's ability to handle these questions, as the vote system becomes overloaded and ends up favoring subjective popularity over good quality and effort.
How can we solve these? Or do you think they are not a real problem, in which case why should we not worry about them?