Yes, this is really long. But if you actually care about game recommendations and want to support making them on-topic (or at least want to understand some part of our opposition in clearer detail), then please read it in full.


Since the second week of our existence, there has been a battle waged between allowing or disallowing game recommendation questions on the website. Today, I would like to highlight 2 of the points that I find to be part of the core problem in our support for what I call Repositories: Motivation for Quality and Values Dissonance. Our ability to handle these 2 problems is essential to allowing game recommendations to be on-topic. If we are unable to solve these, it is going to be an extremely uncomfortable ride and will reflect poorly on our image as we struggle to support recommendations. And if you do not think one or both of these is an actual issue, please show me how it is not.

Keep in mind our goal as a site - we want to be an authoritative Q&A website for people who play games.

Preface - What is a Repository?

I have been using this term often when I refer to game recommendations and similar questions. I use this term because I want to differentiate them from being just a generic "list" question, because they are a separate class of entity.

If we exclude the entirely off-topic content that we have received, our question base on the parent site consists of 3 classes of objective questions: problem solving, fact finding, and repositories.

  • Problem solving questions are the majority of questions we field: questions about particular strategies, how to get past certain puzzles, and how to attain a specific goal. This also includes the tech support and hardware questions we get. In essence, a problem exists and the answer solves it.

  • Fact finding questions are where the "problem" is more of a curiosity than a hardship. Plot and character questions to start, but also the inquiries we get on specific game mechanics qualify. Trivia is a class of fact finding as well. Basically, the question asks for information rather than a strict "solution", but that information still concludes the question. We get a lot of lists in this class of question, such as weapons lists or damage tables, which serve as a whole as a reference material.

  • Repositories are almost a subset of fact finding questions in that they seek information that is compiled into a list. However, they possess a number of traits which make them a whole different class than the other two. First, Repositories are often non-finite; many will continue to be updated as new information comes up. Building from that, Repositories also tend to be non-static; the other question classes have a far less frequent rate of change over time. And coming from that, Repositories are expected to be compiled gradually. If someone asks for the finite list of weapons for a game, the more likely scenario is that an expert will generate the full list before posting it; repositories instead are built by individual contributions of singular items over time. The final point is that Repositories tend to place greater importance on the individual items in the list over the compiled list itself. People referencing a chart are looking to find something in that chart or to calculate/compare something in that chart; people referencing a Repository tend to simply be looking for an item that qualifies for the repository rather than specifically care for the repository as a whole.

I'm going to take this a step further and equate the value of problem solving and fact finding questions together into one class "Questions", with a capital Q. While the goals of the two are not the same, their behavior and traits are pretty identical and they can be reasonably compared on the same scale. I can reasonably feel that an expert answer to "How does my grapple howitzer affect my scoring" is as useful as an equally voted expert answer to "How can I safely kill the Ancient Chaos Wyrm". I can also reasonably expect that those votes reflect how useful the material is. For traits, both tend to be finite with a complete conclusion, they tend to be static because updates to games which change them tend to be infrequent, and the answer tends to be compiled in one shot. These traits also make them fit very well in the Stack Exchange system, which is a fairly important point.

Henceforth in this question, I shall be referring to Repositories and Questions using the above definitions. Let us progress to the problems, then!


Motivation for Quality

If our mission as a site is to be an authoritative Q&A website, then we need to show it by providing high quality content. For Questions, this is very strongly indicated in the votes and the effort put into the contributions of our users. You see a passage that gives you great detail on what's going on, you see high quality. The high quality is expected to be reflected in solving the problems of people who have the same issue.

A high quality Repository, then, should reflect in addressing the needs of each person who wants something there. Because the value of a Repository is less in the list itself and more in the presence of a winning item, then a high quality Repository is one that will have the items that newcomers are seeking. This means that as many qualifying items should be added to the Repository as soon as possible, and there should be people who are directing their efforts to filling these Repositories as best they can. This is not happening.

Let us take an existing example: Games that feature procedural generation of content. In the first day, it received 19 answers before being closed on the following day. It was reopened shortly afterwards, and subsequently tzenes posts a comment listing about 15 qualifying games. Despite basically feeding the place a whole set of answers, none of that list is actually added over the course of a month; we only get 4 more answers. Elsewhere on the site, you can find games like POWDER and Elona in game recommendations. In the chat room, I also have heard many users mention a fair number of qualifying games such as a shoot-em-up and a platformer. And yet, all this wealth of information is not being added to the Repository that people are fighting to keep open.

There is no motivation to excel and be an expert in building Repositories here. Otherwise, these Repositories should be filled to the brim with pages upon pages of contributions. So we are really just providing a small subset of qualifying content, namely the subjectively and currently popular content. Including myself, there are many people who continue to state that game recommendations can be founded on knowledge and that there exists expertise, but these past two months have shown quite clearly that nobody seems to care about making a good quality repository that is founded on knowledge.

"Oh, but if the question author is satisfied, isn't that enough for a good answer?", you ask? To be blunt, it isn't enough because that is where the other half of the subjective equation comes into play. People argue that recommendations are founded on objective content, but if you are settling to satisfy only one individual then you are operating on the subjective need of that one individual instead of the larger need of what the question is actually asking. As more people come who want new games that are in that class, they may look at the list and not see something they want. Bounty comes in, and for a week we might pour in new answers that may satisfy that user. Respite comes only to back down when the next unsatisfied user walks in. The lazy approach is detrimental to quality, and also amplifies in effect when we introduce the eventual volume of Repositories that will exist.

How can we expect to be a good resource of information for Repositories if we continue this lazy and subjective approach? I don't know about you, but a recommendation service which only has a handful of qualifying games based on what people simply feel like adding whenever they want does not sound like a very appealing source of information.


Values Dissonance

There is expertise in addressing Repositories in the same way that there exists expertise in addressing Questions. I mentioned before that I was able to group problem solving and fact finding into one class because they have an equivalent type of value. I cannot do the same for Repositories, because they do not have the type of value. I am not saying that their value is worth less (or more) than that of Questions; they simply are not a comparable entity.

Votes exist on these websites in order to mark clear and useful content. On answers, they will generally mark how well the post addresses and completes the question. More effort put to create a superb answer will yield more votes. This is not the case when it comes to Repositories. Looking at the procedural generation question, the top rated answer is Worms at 23. Comparatively Nethack has a 7. In terms of usefulness to addressing the question, both are identical. In fact, when it comes to what upvotes are supposed to be used for, every answer that satisfies the criteria of a Repository should get upvoted. It is not as if the level of procedural generation in Nethack is less applicable than that of Worms.

On the effort end, to simply mention Worms and take a screenshot is not nearly the amount of effort it would take to research the more obscure qualifying games and get enough information to post a quality analysis. If that game fails to take off and attract people to try it (which is highly likely once the list exceeds 30 items and it gets buried in the pages), then it is not going to get as many votes despite the greater effort.

Instead votes are accrued in a Repository based on the popularity of an entry. They mark what people simply like. And that is also what people who are seeking recommendations are going to want - they want something they are going to like, and knowing what people like is a stronger indicator than how well a game qualifies for the criteria. After all, the criteria is only a means to narrow down the scope of what is being examined.

This is where the dissonance begins. The way votes work on Questions is in-line with what Stack Exchange is meant to be, and what our mission of providing high quality is meant to be. But votes on Repositories do not reflect the usefulness, effort, or quality of the contributions to the list as they should be. Even though there is value in high quality Repositories, the votes are not likely to reflect that. A highly rated answer to a Repository is simply something that is highly liked. Which is not even consistent, as the timing of the exposure greatly determines how many votes an answer will get.

This is not about reputation; admittedly, though, if we consider an entire class of questions on-topic then they really should be worth something. The fact is, votes are actually important in indicating quality. There are aggregate statistics for highly voted items in both answers and questions, and there are people who use votes to help find what is useful. And the people who follow by these are not pleased when they have to deal with volumes of votes awarded to things seemingly effortless and subjective. It's rather demotivating - why put so much effort in solving problems and finding facts when just waiting for the next good Repository and tossing out the most popular qualifying game is all you need?

Simply put, trying to house this kind of popularity rating system in the same content rating system used for Questions is not a functional solution. But we only have one voting system to use here.


These are the two points. One is a flaw in motivation, that our users do not actually give the effort to produce quality Repositories. The other is a flaw in the Stack Exchange system's ability to handle these questions, as the vote system becomes overloaded and ends up favoring subjective popularity over good quality and effort.

How can we solve these? Or do you think they are not a real problem, in which case why should we not worry about them?

share
    
You're just Epic @Grace Note - votes are actually important in indicating quality - that says it all –  Ivo Flipse Sep 10 '10 at 15:25
5  
@Grace Holy crap! Wall-O-text hits you for OVER 9000! I kid, its a good read with lots of valid points. I'll be chipping in shortly. –  jblaske Sep 10 '10 at 15:36
    
@Grace More effort put to create a superb answer will yield more answers. I think you meant votes. –  C. Ross Sep 10 '10 at 17:42
1  
something in the rollback seems to be borking the markdown. –  tzenes Sep 11 '10 at 14:13
    
@tzenes: fixed. –  badp Sep 11 '10 at 15:04
    
The Jon Skeet for gaming! –  Mugen Dec 10 '10 at 18:02

6 Answers 6

up vote 8 down vote accepted

Honestly, I don't see a way to reconcile and fix these problems. Tzenes's post goes into pretty good detail about why it's not going to work. There is no way to make the voting objective because people will vote how they want to vote within the framework that we've given them, and the framework is set up to reward Questions, not Repositories.

I agree that, if we solve these two problems, game recommendation questions will be fair game. We will stop voting to close them then.

Until we solve these problems, however, all game recommendation questions should be closed and/or deleted as soon as possible. The current state of things, with handfuls of open and closed game recommendation questions with barely any differences between them, is quite sorry and haphazard. We must develop a policy now (ban these questions), and once we come up with a better one (that solves Grace's noted problems), we can overturn this one later.

share

Game recommendations are tricky, I myself would much rather play games that my peers deem as "good" than what random websites declare as "The top 10 must have games for the [platform]".

In that regard, the votes are exactly what I would want, I want to know what my peers (the users of the site) like the most. And if I'm really interested in answers, then its up to me to make sure to check back and sort by "Newest" occasionally to see what else is there that I might have missed.

I'm in the opinion that all repositories/game-rec questions should be forced into CW. each answer should be it's own game. Keeping everything nice and separate and clean and easy to vote.

A simple Heading link to the games page, and a short description would be perfect, no need for screen shots (as you can find that on their site anyway). This keeps effort down, but lets the "Cream rise to the top" so to say.

We as a community just need to moderate and make sure that a question isn't too general to have as a repository. Things to avoid would be requests for "The best" or "good" we should instead force the question to be specific enough and just ask for a list. Such as "Games that use the Gyroscope for iPhone/iPod Touch" or "Roguelikes for Android" would be acceptable repository CW questions.

Questions like "New Xbox 360 - What to play?" would be a bad question to let stick around as its WAAAYY to broad.

We need to be able to provide a "top 10 list" as to stay relevant as people are looking for that information. Why not give it to them from a community that knows what they are talking about. Instead of some blogger's arbitrary list?

I feel like I'm rambling...

share
3  
Community Wiki doesn't solve that the votes don't indicate quality, just popularity –  Ivo Flipse Sep 10 '10 at 16:03
    
Oh and I disagree on the images, since a picture says more than a 1000 words. Judging from the title I would have no idea Dwarf Fortress has an ASCII-art style –  Ivo Flipse Sep 10 '10 at 16:07
    
RE: Quality. Are we talking quality of answer or quality of game? Quality of answer is moot if we follow a template, quality of game would correspond with people who thought it was a good game, would it not? –  jblaske Sep 10 '10 at 16:13
1  
RE: Pictures. Click on the link to the game would quickly reveal that its an ascii game. And not having pictures in our lists would make them easier to browse and look thru, as well as more uniform. –  jblaske Sep 10 '10 at 16:13
    
@Ivo he specifically wants popularity. As far as he is concerned this is a good thing. –  tzenes Sep 10 '10 at 16:16
    
I think that the quality of the answer being moot is rather the core issue, actually. I understand quite fully that opinion helps a lot in recommendations - it's why I don't use generic top X lists when surfing for new games and instead just toss a message to the most appropriate circle I can contact. But I think that the conflict this generates when mixed with the rest of our content, which is measured by the quality of the answer, is very dangerous. And Community Wiki with templates still won't really solve that problem. –  Grace Note Sep 10 '10 at 16:21
    
@Grace Exactly, i think we need to make an exception to the rule, that when things are tagged as game-rec they are "popularity contest" instead of standard Q & A. I think people can realize the difference (i know that's a lot to ask the general internet public.) Otherwise i don't think game-rec questions really belong on the site, as they really aren't "Answerable" to a definite point, which would only hamper our usefulness to become the one stop shop for gaming answers. –  jblaske Sep 10 '10 at 16:25
2  
That exception is a source of conflict, though. There is a severe discomfort caused by the nature of these questions, and we have to be able to manage it in a way that reduces this discomfort, not simply pardon it off. And to be straight and honest, game recommendations not belonging here at all is entirely the conclusion I will draw if we are unable to solve the problems I presented in my post. Trying to be a one-stop-shop for all of gaming is too ambitious to the point that it is harmful if we are unable to properly manage it all. –  Grace Note Sep 10 '10 at 16:34

Following your train of thought every answer to a list question should be a hybrid between a Wikipedia article and a Metacritic/Gametrailers review.

If these answers would tell me:

  • what kind of game is it?
  • what the game is about?
  • why the game is so darn awesome?
  • how does it fit the special requirements of the question?

They should give me enough info, so that based on the answer, I can judge whether I want to give the game a shot or not.

I would be willing to stop voting to close, if we cherry pick some list questions and actually put effort in them to make them worthwhile. However, it should be deleted if it fails to do so. Any takers?

share

I have developed my new position on this topic on chat and it's high time I do it here.

A game recommendation answer must link to a webpage that does the actual listing of games.

Why don't we keep lists up to date here? Because it is a bad idea. Every edit pollutes the homepage, stealing visibility from "big-Q" questions and truly making the site a worse place to be.

By linking to lists, which can be objectively compared by their quality (comprehensiveness, up-to-dateness, descriptions, screenshots, ...), instead of games, which instead can only be subjectively compared, we become able to objectively answer a game-rec question.

Also, further updates must be done on the external site, not ours, which means the question gets just as much visibility as any other question.

This is the only way game-rec questions can fit under the model. No more value dissonance and no more motivational problem.

Does the asker need a cooperative game? co-optimus.com. Would you rather want the fps's that wrote the story of gaming? Wikipedia's article on FPSs has the answer. Want a mere list of roguelikes? Here we go.

There are many websites dedicated to such requests, we don't need to be yet another one.

Can't find a list matching your needs? Maybe make one on your personal blog, then link to it, I say.

share
2  
And what do you propose we do to the questions that fail to give a proper external listing –  Ivo Flipse Sep 11 '10 at 8:58
    
@Ivo: if an answer can't be found an answer isn't given. :) If there are multiple requirements, I think it's fair to list to multiple lists -- and only then maybe suggest one or two titles from the intersection. –  badp Sep 11 '10 at 9:03
    
I meant: what if users don't give a list, but instead fall back to one post per answer. Then shouldn't those be deleted? –  Ivo Flipse Sep 11 '10 at 9:12
1  
@Ivo: I guess then we should do the same thing we do for answers that really are comments :) –  badp Sep 11 '10 at 12:02
2  
Since such a list wouldn't need to actually include a recommendation I think this is an ideal solution. It's likely this isn't what the asker wants (because it doesn't include a recommendation) but it is beneficial to the community. It seems the perfect solution to a game recommendation is not to have a recommendation. –  tzenes Sep 11 '10 at 14:11
    
@tzenes: Well, I'd still add a recommendation -- but as a side remark, or even a comment. The actual answer would be the link. –  badp Sep 11 '10 at 14:58
1  
@badp I would definitely encourage "value added" information (such as: this list of FPS contains some shooters with third person view) in these or any other question. As long as the recommendation is in line with that (the top answer on metacritic is X) and isn't just "I happen to like Halo which is also on this list," I think that's a worthwhile effort. I could easily see this system be abuse in a manner such as: Here is a list but the only good FPS is Half-Life; but as long as that doesn't happen I see no harm in "value added" information. –  tzenes Sep 12 '10 at 1:18
2  
I'm still going to wait on commenting in detail for this. Ideally, this does solve both problems completely (in the same vein that removing game-rec as a whole does). But for now, I really fear that this solution is too utopian. It has too much expectation on all the users to follow a pattern that is counter to what many expect when they see this kind of question as well as our tradition of containing content. And we already have an image of Gaming having the strictest moderation on all of the Stack Exchange sites (allegedly), so enforcing this by moderation would worsen the situation. –  Grace Note Sep 13 '10 at 15:08
    
I don't see how removing game-rec completely doesn't make us any less strict :) –  badp Sep 13 '10 at 15:47
    
It's one thing to make a kind of question off-topic; we have done such with adult games and with the matter of piracy. But to enforce this proposal with moderation involves actually controlling the answers that people give, not a simple matter of content acceptance. Blocking illegal, spam, and offensive answers as well as ones that just aren't answers at all or don't address the question... those are understandable things to fix. But "I'm sorry, you gave the answer here instead of linking to an outside site that has the list" is downright perplexing, and I can already see the backlash. –  Grace Note Sep 13 '10 at 16:07
4  
I think it's easier for users to swallow "We cannot accept game repositories because the system cannot reasonably handle the required level of maintenance" than to swallow "We allow game recommendations, but unlike the other lists and walkthrough questions that people are going to address on the site, you can only receive links to content that exists elsewhere". At least for me, I think the discomfort for users is past the "why bother" point. –  Grace Note Sep 13 '10 at 16:12
    
@Grace Note, I agree we have reached a "why bother" point. However as the site becomes live and moves beyond beta, we can expect an influx of new users who have not reached this point yet. Furthermore not every random/new user will read the FAQ in detail, while we can close questions, it is not an ideal solution. –  Elpezmuerto Oct 5 '10 at 15:35
3  
I disagree with this. We already have established that we want to contain content, not link to it.. Additionally, an answer becomes a list of lists, which is just as subjective. I think that if a question can not fit on our site following accepted guidelines, it should not be accepted. –  user56 Oct 9 '10 at 20:08

I'm sure I don't have to state my solution to this problem as you can all probably guess it.

At its core attempts to accumulate this repository are going to be flawed because gaming is an intensely personal experience. If there was a repository for "Good Console Systems," you can bet people would vote with their hearts before their heads. Why else do we allow questions to remain open because Jeff Atwood commented, or some guy wants a game for his 4 year old kid, or about one of our favorite classic games. Why do iPhone users deserve to have their question, but OSX users don't? Because the iPhone is cooler. Our close votes are just as influenced by our personal response to the question as our answers are.

In addition to the problem of people being unable to objectively give recommendations, we also have the problem that askers want the popularity. To them this is the ideal repository: one where votes correspond to popularity. If you consider the members of the gaming.se community experts (and I hope you and others do), then finding out what the most popular games amongst experts is a desirable goal for someone seeking a new game to play. "You guys are smart, what do you think is cool?" Now, while I appreciate the compliment, ultimately we don't care about what is popular, because that is an opinion and anyone can give that. We care about expert questions, because as experts, those are interesting to us.

To quote @jblaske

i think we need to make an exception to the rule, that when things are tagged as game-rec they are "popularity contest" instead of standard Q & A.

So I would say we have three problems:

  • Users don't want to close/open based on objective answers
  • Askers don't want objective answers to these question
  • No one feels the need to be objective with their votes

Now I get that you're arguing: we can fix these things. I don't disagree we could create meaningful objective repositories. I don't think that's what the people who are asking these game-rec's want. They don't want a repository of games so they can make an enlightened decision. They want you to tell them what good games are. They want you to decide.

I would say that our motivations here are cross purpose and thus irreconcilable. But, again, I don't need to state my opinion, you all know it by now.

share
3  
What it really comes down to is do we want this site to be a place to come to for definitive answers about games only. Which has its merit, and I can stand by that. But if that's the case, we need to hold fast to whatever rule is decided and retroactively apply it, as to not confuse the userbase. –  jblaske Sep 10 '10 at 18:30
2  
I'm totally neutral to the issue by the way. I see merits and flaws in both directions. –  jblaske Sep 10 '10 at 18:32
    
To take the example question, I wasn't asking anything subjective or "intensely personal". I simply wanted to know games that featured procedural content generation. -1 because you mis-characterize some repositories. –  RCIX Sep 12 '10 at 5:46
1  
@RCIX You're misreading what I said. It was not that your question was intensely person but rather our response to it. We keep it open not because it is a "good repository" but rather because Jeff said he liked it. This has more to do with our feels for Jeff than any characteristic of the repository. –  tzenes Sep 12 '10 at 17:50
    
From your answer: Askers don't want objective answers to these question. I wanted objective responses. :/ –  RCIX Sep 13 '10 at 4:40
    
@RCIX I think you're sort of taking that out of context. I was saying that one of the problems is Askers not wanting objective answers. I think you'll agree this is a problem. I did not mean to imply that this was true for ALL askers or even you in particular. That said I did supply 15 games for your list which no one felt the need to add as answers. May I take this opportunity to ask why you didn't add them? –  tzenes Sep 13 '10 at 5:17
    
Well, 1, i was asking and not answering, and 2, didn't know many if any games that had procedural generation. –  RCIX Sep 13 '10 at 7:30
    
@RCIX Your plight here is actually rather indicative of why I consider this a problem with Repositories in general that is far beyond just the subjectiveness of recommendations. It only takes a small level of fine-tuning to transform a subjective request into the foundation an objective Repository. It is the answerers in charge of constructing that Repository, wherein the problem lies. –  Grace Note Sep 13 '10 at 14:12
    
@RCIX Consider this question, which asks for a list of games that is compiled using collective ratings and got exactly such as its first answer. It isn't even tagged [game-rec], yet in two days it accumulated 4 recommendations on what to play, which really don't even address the asked question! While we do have a problem with people who do seek subjective suggestions, there is an equal problem in the fact that users will treat the objective ones as the same as subjective ones, because they functionally are identical. –  Grace Note Sep 13 '10 at 14:15
    
@RCIX ok, so what I'm trying to get at here is: 15 new games came up in comments. You wanted an "Objective list," so why wouldn't you add those 15 games as answers? Does it not matter to you that those are in comments? Do you feel no responsibility to organize your own question? I don't mean to criticize, I'm curious as to why you didn't help construct the objective list you wanted. –  tzenes Sep 13 '10 at 14:30

I am not sure how I feel about this solution, but I want to throw it out for discussion. It is similar to what jblaske suggested but with a couple twists.


We can set up community wikis for genre such as RPG, FPS, RTS, etc. This would be special game recommendation wikis. When a user posts something in the wiki, they must list the game title, related games, where to buy, and impressions. Similar to how we have rules on what questions are valid, we need some enforceable template for these special game recommendation CW. The information provided would provide the basis for up or down votes. For example, if there is a FPS special CW, if one of the criteria was co-op options, games that included online co-op compared to just local would be voted up. This is an attempt to shift from what is popular to voting on hard facts such as features.

This solution will keep the game recommendations questions contained and it tries to avoid the CW becoming repositories. I would view them more as a catalog. I consider a repository as a simple game list such as List of Xbox360 Games. There is no ranking involved in a repository, a category however is more actively maintained by the community.

If we allow game recommendations, an element of popularity and preference will show up, there is no conceivable way we can do game recommendations and be totally objective. A problem with this approach is suppose we have a FPS special CW and suppose Halo:Reach and CoD:MW2 are the top two games. What happens when a "great" game released a year from now, will it be able to garner enough votes to properly place it on the list? I believe if we would do game recommendations in any form, the community must be active with that. One of the current advantages of the chat implementation is that it has an element of being current.

share
    
A catalog is a repository, and it doesn't solve the underlying issues. Votes remain in conflict - we will still end up measuring popularity on the same scale as with authoritative content. And time has proven quite well that popularity will overshadow authority on this scale; as a Q&A site I would rather that our top questions are questions like they currently are, not catalogs. –  Grace Note Oct 5 '10 at 15:50
    
I totally agree with the popularity issue, hence why I am suggesting some sort of special template for this and voting is based on the information provided which is an attempt to introduce it as authoritative content. –  Elpezmuerto Oct 5 '10 at 16:00
    
You close your suggestion with "Popular games would be voted up and this would create a consensus from the community that a particular game is 'good'.", and I'm not seeing the voting-on-information bit. –  Grace Note Oct 5 '10 at 16:11
    
I reworded it to better reflect the voting-on-information list. I did a poor job in my original response. –  Elpezmuerto Oct 5 '10 at 16:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .