Quote from: deadDMwalking;732359I thought the article was funny because it sounds to me like they're consistently describing a 'bell' shape.  A bell isn't just a hump - there is a flange around the edge before you get to the steep middle portions.  

And of course, it seems to ignore the fact that the 'worst performers' aren't included.  If you were measuring everyone's ability to hit a baseball, there are some who would NEVER hit, but when you measure professional baseball players, you're well outside of the 'normal population'.  Once you eliminate everyone under a certain level of basic competency, you get rid of the bell shape completely.

If you want to read more, there's also a Forbes article including a diagram of the curve (see below - it's flipped around from what we normally see in RPGs). And you're correct, the right-hand portion of the curve is cut off because people below a certain performance level wouldn't even be included in the analysis. How sharply does the curve rise again when including the "losers"? Probably depends on the field.



Here's the Forbes article:

The Myth of the Bell Curve: Look for the Hyper Performers
http://www.forbes.com/sites/joshbersin/2014/02/19/the-myth-of-the-bell-curve-look-for-the-hyper-performers/

A fact often overlooked by gamers is that bell curves don't emerge organically from the laws of nature or business. An HR analyst takes all the performance data and manipulates it into a bell regardless of its actual distribution. The only reason a Gaussian curve was chosen for this purpose is because it encourages "performance stability", which at one time in western corporate history was considered the ideal. The bell curve is just an analytical tool that provides consistency from one fiscal year to the next, and the only reason it's stuck around so long is because of corporate inertia. People have been complaining about bell curves as a performance metric for a long time.

The power law distribution, or a close enough approximation, is common in Internet things. It might not hold here, but the poster with the most posts at, say, rpgnet, probably has about twice as many as the second most proliferate. Ditto the most looked image at Flickr, the person who spends the most time playing World of Warcraft, and so on.

Quote from: Phillip;732366The power law distribution, or a close enough approximation, is common in Internet things. It might not hold here, but the poster with the most posts at, say, rpgnet, probably has about twice as many as the second most proliferate. Ditto the most looked image at Flickr, the person who spends the most time playing World of Warcraft, and so on.

And of course, if you plot the Power Law curve to a "# of results" graph, the High Performers category actually takes a downswing to somewhat resemble a bell. What's important is the relative flatness of the curve until you reach the High Performer category, and the lack of a "lip" that peters off gradually.

Quote from: Arminius;732112Herr Arnulfe, the article you cite is a case of some people misapplying statistics, suddenly realizing their mistake, and then patting themselves on the back for their discovery.

QFT

Quote from: Herr Arnulfe;732363If you want to read more...

"You keep using that..." study "...I do not think it means what you think it means."
  - with respect to Mr. Montoya

Quote from: Brander;732375QFT

In many fields, they're finding performance patterns resemble something other than a bell curve. Instead of clustering around the middle before dropping steeply and then tapering off gradually, performance often tends to look more like a hockey stick (or a pair of hockey sticks placed end-to-end perhaps, if you completed the curve).

Quote from: Herr Arnulfe;732381In many fields, they're finding performance patterns resemble something other than a bell curve. Instead of clustering around the middle before dropping steeply and then tapering off gradually, performance often tends to look more like a hockey stick (or a pair of hockey sticks placed end-to-end perhaps, if you completed the curve).

No shit.

The point of the bell curve is that it measures a full sample. If you limit the sample only to professionals within a given field of expertise, of course the curve is gonna skew towards the high end.

If I take 100,000 random people and test their writing ability, it's probably gonna look kinda like a bell curve.

If I take 1,000 working professional writers, it sure as shit ain't, because those writers are probably all within the top 1% of my previous sample.

To put it in RPG terms, this is like comparing the DEX scores of all D&D characters vs. the ones who took Rogue.

Quote from: J Arcane;732384No shit.

The point of the bell curve is that it measures a full sample. If you limit the sample only to professionals within a given field of expertise, of course the curve is gonna skew towards the high end.

If I take 100,000 random people and test their writing ability, it's probably gonna look kinda like a bell curve.

If I take 1,000 working professional writers, it sure as shit ain't, because those writers are probably all within the top 1% of my previous sample.

To put it in RPG terms, this is like comparing the DEX scores of all D&D characters vs. the ones who took Rogue.

Well, we don't actually know what shape a "universal curve" (i.e. across the entire population) would look like for any given field, because they're only used within specific fields or industries. The closest thing we have are probably school classroom performance evaluations, which often do look like U-curves before the teacher mashes them together in the grading process to produce a "bell curve" of final marks.

Quote from: Herr Arnulfe;732381In many fields, they're finding performance patterns resemble something other than a bell curve. Instead of clustering around the middle before dropping steeply and then tapering off gradually, performance often tends to look more like a hockey stick (or a pair of hockey sticks placed end-to-end perhaps, if you completed the curve).

That's fine, but you are talking about fields where everyone involved exists on the right half of the bell curve.  It's not a surprising new discovery that the results don't look like a bell curve.  In fact if you take slices out of the bell curve, swap X and Y, look at them as a floor or ceiling, or other things, you get results that look nothing like a bell, even with the exact same data.

Quote from: Brander;732390That's fine, but you are talking about fields where everyone involved exists on the right half of the bell curve.  It's not a surprising new discovery that the results don't look like a bell curve.  In fact if you take slices out of the bell curve, swap X and Y, look at them as a floor or ceiling, or other things, you get results that look nothing like a bell, even with the exact same data.

It's a bit like starting PC proficiency effectively being 8+ in a 3d6 system right? Which creates a "half-bell", just as we discussed earlier. The point here is that, even if you chop off the left half of the curve, the shape of the remaining portion is often very different from a "half-bell" in reality. (e.g. instead of "diminishing returns" at the high end, the reality is often escalating returns instead).

Here's a baseball example to illustrate potentially opposite curve shapes, depending on the task being attempted:

Fail = Out
Partial Fail = 1st base
Average = 2nd base
Partial Success = 3rd base
Success = Home Run

A batter who's swinging for a double will probably follow bell-curve performance patterns, whereas a batter swinging for a home run will probably follow U-curve patterns instead.

Quote from: Herr Arnulfe;732391It's a bit like starting PC proficiency effectively being 8+ in a 3d6 system right? Which creates a "half-bell", just as we discussed earlier. The point here is that, even if you chop off the left half of the curve, the shape of the remaining portion is often very different from a "half-bell" in reality. (e.g. instead of "diminishing returns" at the high end, the reality is often escalating returns instead).

That's because they are measuring different things.  "How many people brought in $X" is a different thing than "which people brought in $X" and while they seem similar they are totally different graphs.  One person performing insanely beyond everyone else in the first will at most extend the tail to the right because 1 is still really really close to zero (in any significant population).  But, that same person, if you are measuring how much they performed, will result in a power curve, exactly like you were showing, because their performance sets the height of the curve.

Quote from: Brander;732400That's because they are measuring different things.  "How many people brought in $X" is a different thing than "which people brought in $X" and while they seem similar they are totally different graphs.  One person performing insanely beyond everyone else in the first will at most extend the tail to the right because 1 is still really really close to zero (in any significant population).  But, that same person, if you are measuring how much they performed, will result in a power curve, exactly like you were showing, because their performance sets the height of the curve.

To really understand the phenomenon, of course we'd require graphs plotting multiple, repeated performances from a single individual. However, I think what these new studies can show us is that the trajectory of performance improvement often doesn't follow a "diminishing returns" algorithm at the high end, but rather the opposite.

Quote from: Herr Arnulfe;732402To really understand the phenomenon, of course we'd require graphs plotting multiple, repeated performances from a single individual. However, I think what these new studies can show us is that the trajectory of performance improvement often doesn't follow a "diminishing returns" algorithm at the high end, but rather the opposite.

What you just wrote reads to me as:  "Because Apples aren't Oranges we should eat Strawberries."

You are conflating "chance of success" with "degree of success."  Each of which would be a differently shaped curve. Or not a curve in the case of pass/fail only.

Quote from: Brander;732409What you just wrote reads to me as:  "Because Apples aren't Oranges we should eat Strawberries."

You are conflating "chance of success" with "degree of success."  Each of which would be a differently shaped curve. Or not a curve in the case of pass/fail only.

If you're a star performer in your company, and the next guy above you is 50% more productive, and the guy above him is 150% more productive, do you think it makes sense that your next "skill level" should only give you a 2% increase?

Quote from: Herr Arnulfe;732397Here's a baseball example to illustrate potentially opposite curve shapes, depending on the task being attempted:

Fail = Out
Partial Fail = 1st base
Average = 2nd base
Partial Success = 3rd base
Success = Home Run

A batter who's swinging for a double will probably follow bell-curve performance patterns, whereas a batter swinging for a home run will probably follow U-curve patterns instead.

So what you're saying is, you know even less about baseball than you do statistics.