|Posted on September 26, 2012 at 5:25 AM|
How Not to Do a "Best" List
Newsflash: I love videogame role playing games.
"Really!?" you might ask, that tongue-of-sarcasm in your cheek as you pretend to not be aware.
Yes, really. After decades of playing them, collecting them even, since the 1980s, I'm pretty well versed in the genre. Hell, I've written enough blogs here to go into that, so I don't need to describe it in detail. But let me just say I've played most of the major RPGs out there, but more importantly I'm familiar with the importance of RPGs that I still need to play and their place in history and what the RPG community views them as.
So I must step on a soapbox for a moment to discuss the latest "click here for list" lists of people putting things in some sort of numerical order. Of course, this is list of the "Top RPGs" and it was put out by IGN last week. You can read that article here, but you'd probably be wasting your time. To put it mildly, the entire thing is pretty much an example of what's wrong with videogame "journalism" (more accurately called "enthusiast" press).
It has nothing to do with my disagreements. Lists don't bother me. It's more about how IGN put the entire thing out there. In other words: it's how IGN tossed together a meaningless, shallow list of "top" games with really only the desire to generate more clicks on their website. That's why most sited do "lists" in the first place...but this one was unequivocally bad and shows how not to do such a list more than anything.
Let's break it down:
The First Problem:
The first problem is that we're, right away, asking more questions than anything. IGN doesn't really go into who is making this list. I have no idea who the writers are, what their experience with the role playing genre is or their qualifications for putting a list like this together. Questions like:
What are the qualifications for being on this list? How is quality determined? Is it the impact of the title on the industry? The popularity? The progression it did for the genre and influence it had on technology? What is and is not considered a role playing game? (Just FYI, even fans of the genre have a hard time determining that one)
Who is putting this list together? Did they play all these games? Do they have a good knowledge of the games and the history of RPGs or just a passing knowledge? Were interns involved?
How are entries determined? Is it a voting poll? Is it one guy in a van in the parking lot? A show of hands? A roundtable discussion? Is it like the BCS and some sort of formula based on votes and ranks on everyone's entries?
I don't know about you, but those are big issues that aren't made clear from the very beginning: who made the
The Second Problem:
Numbering. Yes, numbering. IGN wants to make this a heiarchy here. If they didn't, they wouldn't have bothered putting numbers in it. Now some people might say "oh, it's not really in any order, it's just a list of 100 great games." After all, that's kind of what IGN vaguely says in their introduction page.
If so, then why put a number entry at all? IGN realizes they're gearing up for a shit-storm of fans reading this list and all of them seeing those numbers put in what seems to be a specific order, right? Of course they do. Even if we, and probably even IGN itself, aren't entirely clear or sure why game X is above or below game Y, there is an appearance of an order here even if there isn't intended to be one.
The reasons to put a list like this together, and to have one entry per page I might add, is to do one thing: generate hits. It's entirely here for marketing because putting a number on these things will cause someone to take notice. Bottom-feeding film critics do the same. "Oh, you liked the Dark Knight...well so-and-so at this paper you never read before gave it a D-...better go and read his review and link to it on facebook and twitter."
Doing something even slightly "controversial" (and this isn't controversial, just generic website junk without really thinking) is just to get noticed. Especially with something that has a passionate fanbase like RPGers (and gamers in general, really). Putting a "#1" on something automatically makes that list a hierarchical list.
Another issue is that the RPG genre is incredibly expansive. This would have made a great feature series, maybe one a week, going into a great game similar to the way Roger Ebert does his "Great Movies" series (and other similar series that celebrate films, AO Scott has one too). A fan of one particular and well-loved game sits and writes a page or two going into detail about why they love it. Insert nice screenshots, boxart, make it an event. Have guest writers.
Hell, I think I just outdid IGN and a lot of gaming-enthusiast outlets on a brainstorm of gaming article ideas and I was just talking out my ass.
The Third Problem:
The entries. Each entry is given one little paragraph. That's it. And none really detail what qualities allowed it to be on the list and are pretty much general statements about the game. You want to celebrate this genre yet are completely unclear on how this list was put together and only give a couple of sentences to these games you wish to celebrate? I know these exist, tell me why they're important. That is celebrating the genre and loving it.
Now, let me make something clear once more: I don't care about any of these orders myself. I'm not writing this because I saw Suikoden II and Final Fantasy IX a lot lower than where I would place them on a personal list (of course, I would also make it clear it's a personal list first off, but I digress). I'm writing because everything about this feature from IGN, from its origins to its gestation to its release, is how you don't go around making Best Lists - especially when your website isn't fully regarded with great journalistic integrity to begin with.
By taking time to celebrate them beyond just a couple of sentences, it might have been something really great. The RPG genre is ripe for it. The fact that the entries aren't explained, just described with something not much better than what you'd read on the back of the box, shows a complete lack of dedication to the project to begin with.
The Fourth Problem:
Qualifications: how is the placement of the entries determined? You have them numbered, so there's some order going on here. Is it influence and importance in the grand history of RPGs? Is it popularity? This ties strongly in to the question of voting process.
The biggest issue here, though, is this: "what's an RPG?" Seriously, even fans of the genre argue over what is or isn't an RPG. Some say Secret of Mana is, but Zelda isn't. Some argue that Red Dead Redemption even is. I've heard pretty convincing arguments on all of these but nobody has come up with a certainty.
This has nothing to do with trying to define RPG, though. It just has to do with what is defined for this particular list. If you state right off "action-RPGs like Zelda or Mana are not included" then fine. If you say "strategy RPGs like Ogre Battle are included" then fine. But the rules need to be clear.
I touched on a few of these, but there's probably the best ways IGN could have approached it:
1) The genre is incredibly broad, even fans have a hard time determining what is or isn't considered a role playing game, so why not break it down into something smaller? Specify for individual eras, consoles and styles of role playing. You can do one, or all of these. It's hard to put Ultima on DOS next to Chrono Cross on the Playstation and try and figure out which is "better," but by looking narrower instead of wider, you can make a pretty good list overall for various categories.
2) Get rid of the numbers. This is the worst thing about the list if you're going to be doing "Top" anything. Numbering diminishes the focus. If you want to celebrate, then celebrate. In other words, don't bother with numbers. If you want to put together a list of great games, then do it any other way that doesn't involve putting a number in front of it. If you're putting numbers, then you're going to have to defend those number placements. Just be aware of that.
If you ever do a list with numbers that's meant to be an "objective" list, then give reasons for placements. This is probably IGN's list biggest shortcoming: we don't know why some games are ranked where they are (a combination of lack of explanation how rankings are assessed and what makes games "good"). They just summarize what the game is about. It must be made clear the process of voting and polling as well as reasons for them. But I digress, because IGN shouldn't have bothered with numbering to begin with it.
3) Expand the entry paragraphs, make it a few per week instead of ten at a time. Have individual people choose some of their favorite games at your offices and have them write up an editorial. I would have LOVED this. As it is now, it just comes across as half-assed...and there's more than enough of half-assed going around. Want to be a serious website, do a more serious take on something like this. For example, Gametrailers "celebrates" many games by doing great video retrospectives, like this fantastic multi-part Zelda retrospective.
4) Take the Sight and Sound or AFI Approach to their respective film lists. In these cases, it's not just one group, but just about everyone you can think of sending in ballots and nominations. Game developers, programmers, editors, critics - people that know this industry and are involved with it to make a comprehensive list. While people might still disagree with both of these examples, not as many people are up in arms and arguing because they make apparent who is voting and their qualifications.
5) If not that, and you want to keep it in-house, do the same voting process, but also throw in roundtable discussions about it. Hell, record the discussions. That would be fun. If the pool is going to be this much smaller, however, it must be made clear who is going to be putting the list together. The fact we have no idea who put this list together at all is the biggest red mark on the thing. If it's a bunch of interns reading Wikipedia, then that's a problem.
6) Last but not least, be upfront with Intentions. If this is a personal list, then note that. Those are, overall, pretty easy. You just list your favorites and shouldn't pass it off as some sort of quantifiable objective list. If you're trying to do a more objective list, know the approach you want to take. Note to the reader your experience, why you're qualified to put together a list.
At the End of the Day:
I can't blame IGN for trying. Sure, it's an obvious marketing ploy to get some hits, just putting one beloved game low and one unknown game high is enough to start a viral campaign, but at least they're trying to put something together beyond just the usual review or PR-mouthpiece most videogame sties are (seriously, most sites just do those and don't try anything else). Still, it was a missed opportunity. At the end of this life-cycle of gaming, it would have been nice for a great look back.
IGN puts it out there to try and market itself, not really contribute to the grand scheme of "celebrating" anything. Plus, who cares what IGN says anyway, especially if they do such a half-assed job in the first place? If they don't care...why should we? They aren't an authority. Then again...who would be an authority? There's no Academy of Art and Sciences or AFI version for videogames to put it all together, gather people up and start figuring it out. Instead, we just have websites doing half-ass jobs and passing it off as a definitive list of greatness.
Yeah, I know I'm a nerd for pointing out shortcomings of the gaming press yet again, but really this is kind of more for educational purposes in terms of all those "Top This" and "Best That" lists that are everywhere on the internet. Just because something is written down on a supposed-big website doesn't mean it is credible. Nor does it mean it should mean anything - you have your favorites, others have theirs, and nobody is going to take that away from you.