Quote from: obrynDon't get me wrong - I still find a lot of value in non-playtest reviews.  I like to know, for instance, if the production values are shoddy, the stat blocks suck, the rules are ill-thought, and the artwork sucks.

What I don't get is how either playtest or non-playtest reviews can be anything other that the reviewer's personal opinion and experience (apart from sheerly factual stuff like pagecount, stat blocks, and so on).  They're both editorials, and both contribute to my understanding of a potential purchase in complementary ways.

Absoloutely. There seems to be some kind of platonic ideal of reviews and reviewers in certain quarters that simply doesn't mesh with the process as I understand it. Of course they're subjective. Of course they're opinionated. That's kind of the point. It's only when a body of reviews are taken in aggregate that we even begin to approach a consensus on common emergent properties, and even then it's a fuzzy, semi-abstract phenomenon.

Quote from: SeanchaiThing is, most game products are bought, read, then put on a shelf. If your goal is to require reviews to conform to the reality of how the product will be used, then playtest reviews aren't the way to go.

Seanchai

That's why a "complete" review is more helpful. The act of participating in an RPG is a process that begins at purchase ( or maybe before, if you've done some research) and ends with a decent stretch of actual play. I think it would be enormously helpful if reviewers attempted to follow that process as faithfully as possible. That way a potential customer who intends to purchase for reading pleasure only would be presented with a clear point where the review ceases to be of any productive use, and can adjust their expectations accordingly. Those who are interested in actually playing would be catered for too.

Drew, let me kiss you.

Quote from: SettembriniDrew, let me kiss you.

Dalek nookie?! I don't swing that way, mate.

That fit, pteradactyl-punching Cyberwoman from Torchwood on the other hand...

EXTERMINATE!

Quote from: jrientsThe idea that I can't properly review the text of an rpg without playing it strikes me as akin to the suggestion that in order to write criticism of Shakespeare I have to produce/direct/perform all his plays.  

Actually, in unis a lot of discussion and essays go around pretty much that point. But it's reading the plays vs seeing them performed, rather than performing them yourself. Some take the middle position that you should at least read the words out, they're meant to be spoken aloud - like poetry is, too.

So the "akin" isn't as absurd as you might think.

A game is not merely its text. Roleplayers seem to be unique amongst people who play games in that they think that simply reading the rule manual is a fun thing to do and tells them something about the game. You almost never find that in players of board games or wargames or computer games. They say, "just play it to see what it's like and understand it properly."

It's because a game has emergent properties, things that only emerge in play. Just a glance at the rules and game board might make you think that Trivial Pursuit is just a variation on Snakes & Ladders. In play it's entirely different, but you can't know that until you play.

Quote from: jrientsThe idea that I can't properly review the text of an rpg without playing it strikes me as akin to the suggestion that in order to write criticism of Shakespeare I have to produce/direct/perform all his plays.  Performing Macbeth might very well yield valuable insight, but that doesn't prevent the review of the play as text.

An RPG might be a book, but it isn't a novel. It's not something you're supposed to experience solely by reading it, that's just the smaller part of the equation. You can very well review a Stephen King novel without being a writer, from the standpoint of a prospective buyer.

By the same token you can review a play without being involved as an artist, from the standpoint of a spectator. The primary purpose of Shakespeare's work is to be read/watched. You don't need to be an actor to watch, anymore than you need to be a cook to taste food.

You can't review a game extensively unless you play it. Or rather, you can, if your review targets people who pointlessly buy RPG books with the intent of reading them only.

A capsule review, while it can be helpful for a real gamer, will never be as helpful to a real gamer as a playtest review because that's what you are supposed to do with those books. They are being written with the express purpose of being used in a playing environment, with all sorts of unexpected interaction ensuing.

That being said, I have a small list of favorite reviewers and many of them do not playtest games. I prefer a capsule review to no review at all and I think an experienced gamer/reviewer can generally estimate how it's going to play out. I can't expect prolific reviewers to playtest extensively every product they review. It's just way too time consuming.

But I just love those Davenport reviews where he points out how the game reacts in play, the small details.

Quote from: jrientsAnd given the large number of rpgs on the market that are read but never played, one could construct an argument that insisting on playtesting actually disserves a large segment of the audience.

It's a segment that is actually irrelevant to this hobby and not needed for its long-term welfare. If people just want to read, they can find MUCH more talented writers outside of this hobby and buy much cheaper books without being drowned in typos.

And RPGs are probably one of the worse "read-only" investment one can make. The quality-for-the-buck ratio is fucking abyssmal. The only way to make up for that is to play them.

It puzzles me that people would buy pricey, often disorganized books full of derivative concepts written by average-at-best author-geeks with little to no editing, solely to enjoy reading them.

Despite the fluff our RPG books might contain, they really are nothing more than instruction manuals and some specs for something else: a gameplay experience. It is that gameplay experience that counts the most for me.

Quote from: Kyle AaronActually, in unis a lot of discussion and essays go around pretty much that point. But it's reading the plays vs seeing them performed, rather than performing them yourself. Some take the middle position that you should at least read the words out, they're meant to be spoken aloud - like poetry is, too.

Agreed, it is a poor analogy.  Reading Shakespeare is a completely different prospect compared with watching it being performed.  And actually performing Shakespeare yourself, is another layer altogether.
 
I don't see RPGs as being much different.
 
A review based on reading an RPG is nice.  A review based on thorough reading of the game and actual play, is even better still.  It should come with a deeper understanding of the game.

Quote from: SeanchaiLet me ask this: Is this something a person couldn't figure out from reading it or that you didn't? My understanding of these emergent qualities is that they don't really exist prior to play and thus no amount of intelligence or dilligence could uncover them.

As I've said, I, myself, have been pleasantly and unpleasantly surprised in play. But looking back, I never felt these were things I couldn't have discovered from reading and thinking about the rules, just things I didn't.

Seanchai

Fair question. I think it's conceivable that one person could sit down and mentally run through the course of the scenario, taking notes along the way, and figure out what would happen, but it's unlikely and by that ponit you'd be effectively playing the scenario by yourself.

The way it's presented really needs a group of people actually playing through it for the course of the story to become apparent. There are just too many bits of information that become relevant at later points but (and this is the key point) aren't flagged up as such when they are introduced, nor when they become necessary. This could be a serious problem if you skipped over something while GMing, thinking it unimportant.

Of course this is also not the ideal way to present a scenario - "trust us, it'll work!" - if you want people to buy it after reading through it in the shop.

Quote from: Consonant DudeDespite the fluff our RPG books might contain, they really are nothing more than instruction manuals and some specs for something else: a gameplay experience. It is that gameplay experience that counts the most for me.

That's cool.

But your gameplay experience is not my gameplay experience. You like things I don't like. I like things you don't like. You're expecting things I'm not, and vice versa. You're tolerant of things I'm not. And so on.

Seanchai

Quote from: CaudexFair question. I think it's conceivable that one person could sit down and mentally run through the course of the scenario, taking notes along the way, and figure out what would happen, but it's unlikely and by that ponit you'd be effectively playing the scenario by yourself.

Whether it's unlikely or not, this doesn't make it (based on my understanding of the subject) an emergent property.

Quote from: CaudexThe way it's presented really needs a group of people actually playing through it for the course of the story to become apparent. There are just too many bits of information that become relevant at later points but (and this is the key point) aren't flagged up as such when they are introduced, nor when they become necessary. This could be a serious problem if you skipped over something while GMing, thinking it unimportant.

Sure. But whether it'll become clear or not depends on the group, not actual play.

A more concrete example: The written scenario doesn't make it clear that groups who align themselves with a certain NPC will experience prejudice later on. As written, it's expected that the PCs will align themselves with said NPC.

The scenario is run for a group, they ally themselves with the NPC, and the connection is thereby discovered. But what happens when the scenario is run and the group doesn't, for whatever reason, ally themselves with the NPC?

In this instance, it's clear that actual play isn't universally helpful in discovering these kinds of hidden qualities.

Seanchai

Quote from: Tyberious FunkA review based on reading an RPG is nice.  A review based on thorough reading of the game and actual play, is even better still.  It should come with a deeper understanding of the game.

Here's my problem (and my basic point): It also comes with a more subjective understanding of the game.

Seanchai

Quote from: SeanchaiHere's my problem (and my basic point): It also comes with a more subjective understanding of the game.

Seanchai

How is this subjective experience minimized by not playtesting?

-O

Quote from: SeanchaiWhether it's unlikely or not, this doesn't make it (based on my understanding of the subject) an emergent property.



Sure. But whether it'll become clear or not depends on the group, not actual play.

A more concrete example: The written scenario doesn't make it clear that groups who align themselves with a certain NPC will experience prejudice later on. As written, it's expected that the PCs will align themselves with said NPC.

The scenario is run for a group, they ally themselves with the NPC, and the connection is thereby discovered. But what happens when the scenario is run and the group doesn't, for whatever reason, ally themselves with the NPC?

In this instance, it's clear that actual play isn't universally helpful in discovering these kinds of hidden qualities.

Seanchai

I see what you're getting at - I think I wasn't clear enough.
What I mean is that the actual progress of the scenario is not explained properly within the text itself. It only becomes clear when you play it. This is not, in my opinion, a matter of who plays it - though without reading the book yourself you'll just have to take my word for it.

I grant that this is a particularly rare thing in scenarios, in fact it's the only example I can think of. In general, if you can't see it on the page, it isn't there.

The problem with the very discussion of "emergent properties" is, I think, that if you dismiss 'very unlikely' as not good enough to qualify, then there is indeed no such thing. I can't think of an example of emergent qualities in anything at all where it isn't conceivably possible for someone to deduce what will happen ahead of time. Look at, for example, the development of carbon-fibre monocoque design in racing cars. The pioneers in its use didn't know how good it was going to turn out to be - they were expecting it to have a lower torsional strength than existing chassis design (for reasons to boring to go into here) and were pleasantly surprised. But in theory someone could have predicted it.

Quote from: SeanchaiHere's my problem (and my basic point): It also comes with a more subjective understanding of the game.

Seanchai

Of course it's fucking subjective, it's ALL fucking subjective, whether it's actual play or not.

Honestly my biggest pet peeve when it comes to reviews is when the author isn't subjective enough, because it tells me jack fucking shit.  I hate 99.9999% of the reviews on RPGnet simply because they're so utterly rote and mechanical, nothing more than recitations of the chapter contents.  I hate most video game reviews because they pussyfoot around so much trying to be "objective" in the interests of not pissing off the fanboys and the advertisers, that they fail to deliver any useful criticism.  

Which is exactly why I get all my damn opinions on games from message boards most of the time, because at least I can trust posters I know and whose tastes I click with to give me a decent, honest opinion.  

No it's not going to play out the same for every group.  No it's not going to appeal to every group.  But when I go looking for information about a game, I fully expect every last detail of how it worked out for them, and yes, how it went in actual play.

Was there shit that had to be glossed over because it slowed down the gameplay, like most of GURPS 4's combat system?  Was there a die mechanic that looked straight forward but turned out to be broken, like the oWoD botch rule?

THis shit does happen, and I want to know about it, even if it doesn't even turn out to be the case with me, it's still a hell of a lot more useful for me to know about the possibility of it being the case than it is to just shut off any commentary in the name of some imaginary goal of "objectivity".  I'm not an idiot or a child, I can usually tell when I'm being snowjobbed, years of RPGnet experience will give you that skill, so just give me whatever the fuck opinion you've got, and I'll decide for myself whether it's worth a squit of piss.