Oh My God it's Finally Happening!

[Ted]

Gentlemen, instead of my yearly rant on WAR and it's positional adjustments being absurd, I bring you THIS EXCELLENT PIECE by Russell Carleton of Baseball Prospectus where he demonstrates that the defensive spectrum is bunk. Comparing players to players playing other positions on the basis of their primary position is deeply flawed, and that we need to look at the whole thing again. Basically, if the conclusions he draws are valid, and the seem to be, WAR will need a major revamp.

[Ted]

The article is long and data driven, but excellent if you have the time. Here are some highlights from the conclusions portions.

Excerpts:

"But the hidden assumption in the WAR system, as it is currently constructed, is that players are infinitely modular and that we can compare a third baseman to a shortstop because moving back and forth between the two positions would have a penalty (or a bonus) of X runs. The data suggest that this is a seriously flawed assumption."


"The positional adjustments are based largely on the study of utility players. ....And even utility players appear to be a special class of player. Someone thought that it was a good idea to give them a reasonably large amount of playing time at both shortstop and second base. If a player only ever plays second, and never short, is his team indirectly trying to tell us something? Maybe. What we find in these analyses is that players are not infinitely modular. If you put a guy at a position he barely ever plays, he’s going to be really bad. You can’t just put a guy at third base and expect him to get by on sheer athleticism. There seems to be a lot of value in having practiced at a spot for years, even if you aren’t the greatest athlete. Not just anyone can be a utility player. It seems that this is a skill that one must aspire to and achieve."

"So, we confront a metric in WAR that assumes infinite modularity with the reality that players aren’t really all that modular. Even shortstops, who should “play up” as they move down the defensive spectrum, seem to have trouble when they get to their new address"

"The main lesson, though, is that sabermetrics has never really questioned the word “spectrum” when it comes to defensive abilities. When Bill James introduced the concept, it was to point out that we expect less of our shortstops offensively than our first basemen offensively, and that this was a commentary on the relative difficulty of the positions. Taken in isolation, this is a good point. But the word spectrum seems to have taken on a life of its own. It denotes, perhaps even subconsciously, baseball positions as existing on some sort of uni-dimensional sliding scale, as if they were all the same basic thing, just with differing difficulty levels.

The data suggest that they are not. It appears that each position has its own set of skills, and to play that position at an acceptable level requires at least some experience. Each position is its own box and the boxes are a lot less similar to each other than we might have thought. It’s possible to learn how to go between boxes, but that appears to be a skill unto itself. We need metrics that reflect that reality."

[RonCo]

I think the main thing that's gone wrong is this guy's definition of the defensive spectrum.

There's nothing inherent in the system that says a great shortstop who is suddenly put into a role at 3B won't struggle. The is about what skills will translate over time. If you want to teach that natural shortstop third base, he will eventually learn it (or is greatly more likely to learn it to a respectable level) than if you take the natural third baseman and make him a shortstop. But it doesn't suggest that that SS will become a gold glove 3B. That's just silly talk.

I mean, OF COURSE each position has its own skillset, and OF COURSE they don't all perfectly overlap.

And OF COURSE when you put any human being into a role they aren't experienced at (especially at the highest level of a sport) they will be awkward and less effective than a guy who has played the role often.

The defensive spectrum is about learning curves and likelihood of being eventually successful in a conversion. That's really about it. (Edit: But they do suggest a hierachy of scarcity of skillset).

[RonCo]

The next question is all about process...which at this point feels wrong, particularly his pure metrics (which seem not to be needed since actual BIP/BIZ are available) but which I admit I need to re-read the paper to really think about.

[RonCo]

WAR is supposed to answer to the question: “If Smith had disappeared prior to the season, what would we have gotten from a replacement-level player (bench/waivers/minors) who plays his same position?”

This does not seem correct to me. The answer to that question is always zero WAR. WAR answers the question (well or not) "how much advantage will I get bu adding Smith to my team rather than a generic Quad-A guy?"

These are two different questions.

[RonCo]

Still reading through the last section...

But the hidden assumption in the WAR system, as it is currently constructed, is that players are infinitely modular and that we can compare a third baseman to a shortstop because moving back and forth between the two positions would have a penalty (or a bonus) of X runs. The data suggest that this is a seriously flawed assumption.

The data does not seem to suggest that at all. WAR does, I suppose, make a general assumption that a player at the major league level has been trained at that position, so isn't a total imbecile. Therefore switching that player back and forth would have whatever value change a role served. But the data taken here is taken for players who are playing out of position by design. So it really has little bearing on the question of WAR--or at best strikes it a glancing blow.

[RonCo]

So, we confront a metric in WAR that assumes infinite modularity

This is not what WAR assumes at all.

with the reality that players aren’t really all that modular.

Duh.

Even shortstops, who should “play up” as they move down the defensive spectrum, seem to have trouble when they get to their new address.

Again, shortstops who don't play 3B regularly would never be expected to do well. And they won't. Their ACTUAL WAR at 3B will be bad while their defense remains poor. They will learn, however. If they play 3B the whole time, however, their WAR will go down because that position has less value due to the number of plays that come to 3B vs. SS (as well, arguably, as the degree of difficulty of the two...though the skills are different). The positional adjustment is about both the skillset required to play the role AND the number of plays a role is expected to deal with.

WAR is a counting stat. Two totally equal players from a Runs Saved Above Average will have create different values at 3B and SS merely because the SS will make 4x the number of plays.

[RonCo]

The only guy who’s going to replace a catcher is a catcher. The only guy who’s going to replace a shortstop is a shortstop. The same seems to go for second and third basemen. It’s not that it could never happen, it’s that the circumstances where it would make sense are rare. So, effectively, it’s not really an option. Even in the “easy” corner positions, you might shift an outfielder around or stick a guy at first base, but there is going to be a defensive penalty to be paid, even if the replacement is coming from higher up on the “defensive spectrum.”

The data this guy's taken is basically the team saying "we need a guy to play three innings at 3B, or three games, or whatever, in an emergency role. Rather than call up a AAA guy who can play the position for three days, we'll take a chance with the guy already up here who can't really play it.

Everyone who does that knows there's a penalty coming. That has nothing to do with WAR, though.

[Ted]

The defensive spectrum is about learning curves and likelihood of being eventually successful in a conversion. That's really about it. (Edit: But they do suggest a hierachy of scarcity of skillset).

Right, and this is I think a large part of his point. With WAR, the defensive spectrum is being used to flatly assign value to positions. You CAN"T just slide players up and down the defensive spectrum as traditionally defined and assuming they will get worse or better even AFTER whatever time is needed to adjust. It doesn't work that way, and the data collected here bears it out in some ways. Yes, the main problem is that he is only looking at "emergency players", but by simply selection process by managers most of those players are guys who could "handle it better" based on some judegment of their other fielding ability. The most important parts of this are how players moved to 1st are actually pretty darn bad at it, and players moving to center actually haven't done nearly as bad as one would expect. What is being shown is that competence at a "tougher" position does not translate well to "easier" positions. At least as well as advertised. Players just have to learn a whole new position. SOME of the skills transfer, and of course, center fielders and shortstop are generally better and more well rounded athletes, but they still have to learn a whole new position.

If that's the case, then simply saying "CF is harder so here's X more WAR than first base and X less than SS" is simply inadequate, and saying this bat is worth more WAR in X position compared to Y position based on the ability of utility players to move up and down the defensive spectrum is a bad model.

These are in fact, distinct positions. Players should only be compared in terms of the results they provide against their own position. IF you want to make WAR a once size fits all stat, the adjustment for positional difference should be based on an year by year analysis of position scarcity and the relative impact of that positions impact on defense overall, compared to some combination of a batting metric like wRC+ and whatever defensive metric we finally get fined turned enough to be reliable. NOT a flatly assigned value.

In terms of this logic passing the eye test, I've watched a good example of this all year. Jose Martinez has been atrocious at first base. He was a sub par corner OF, so yes, he's a bad defender to start with, but he's AWFUL at first. Yes, he didn't spend his entire career learning first, but he's not going to get much better. After watching him for 80 or so games at the position, I can just tell you this. I know that this isn't nearly enough games to learn the position, but it doesn't matter. He doesn't have the footwork for it. He doesn't have the body type. He's long and gangly and somehow clumsy (for an elite athlete) it will just never happen. These are distinct skill sets. They take different physical abilities and body types.

[RonCo]

It’s possible that I’m being too hard on the new guy at a position. I did pick guys who played very few innings at a spot as my sample. Perhaps if they really did stick around at a position over the course of a year, they’d get less bad. Maybe after 10 or 20 or 50 games in the spot, they’d be fine, or at least in line with what is now known as the positional adjustment chart. But that’s a very big “maybe,” and how many teams can afford to spend 20 games with a guy who’s an absolute sinkhole defensively because he’s learning on the job? That initial transaction friction is strong and probably prohibitive within the context of an actual season.

That's not a big maybe at all. That's the absolute truth of all position transitions and learning curves, and that's why it's almost never tried at the major league level by anyone competing. It's why teams use the off season and Spring Training to teach new roles (or the minors). It takes time to learn a new position, and the WAR adjustment for a player assumes that when you get to the majors you've learned it to whatever degree the league needs you to learn it.

[Ted]

Dammit Ron, stop picking everything apart and let me enjoy this. Yes, the methodology isn't prefect, and yes there are some assumptions or statements that aren't perfect, but is being argued is that defensive positions in baseball are distinct, take different skill sets, and the idea that you can just say compare stuff across positions because we "know" first base is easy and shortstop is hard and incorporate that into a stat is wrong minded.

Carleton is a good writer for an established baseball analytic powerhouse, and he's saying this. It's about damn time. Let me enjoy this.

[RonCo]

Right, and this is I think a large part of his point. With WAR, the defensive spectrum is being used to flatly assign value to positions. You CAN"T just slide players up and down the defensive spectrum as traditionally defined and assuming they will get worse or better even AFTER whatever time is needed to adjust.

This is, to my mind, an incorrect assessment of what the positional adjustment of WAR is doing.

The positional adjustment of WAR is taking into account both the difficulty of a role (which this seems to be trying to focus on) and the quantity of plays that happens at any one position. A guy playing CF will field twice the number of plays that a LF will. Feel free to argue the adjustments could (and maybe should be tweaked), but they are needed because WAR is a counting stat, not an overall rating.

[Ted]

It’s possible that I’m being too hard on the new guy at a position. I did pick guys who played very few innings at a spot as my sample. Perhaps if they really did stick around at a position over the course of a year, they’d get less bad. Maybe after 10 or 20 or 50 games in the spot, they’d be fine, or at least in line with what is now known as the positional adjustment chart. But that’s a very big “maybe,” and how many teams can afford to spend 20 games with a guy who’s an absolute sinkhole defensively because he’s learning on the job? That initial transaction friction is strong and probably prohibitive within the context of an actual season.

That's not a big maybe at all. That's the absolute truth of all position transitions and learning curves, and that's why it's almost never tried at the major league level by anyone competing. It's why teams use the off season and Spring Training to teach new roles (or the minors). It takes time to learn a new position, and the WAR adjustment for a player assumes that when you get to the majors you've learned it to whatever degree the league needs you to learn it.

Where exactly is this assumed by WAR? IS it the part where the values they result are adjusted because players often move around the diamond to less experienced positions all the time now? Or the asterisk by the totals in the leaderboards explaining this for each player and how much relative experience they have at each position? Oh wait, none of that is being done. CF's just get 1.5 more WAR than first basemen per 162 games played regardless of competence or experience.

[RonCo]

Dammit Ron, stop picking everything apart and let me enjoy this. Yes, the methodology isn't prefect, and yes there are some assumptions or statements that aren't perfect, but is being argued is that defensive positions in baseball are distinct, take different skill sets, and the idea that you can just say compare stuff across positions because we "know" first base is easy and shortstop is hard and incorporate that into a stat is wrong minded.

Carleton is a good writer for an established baseball analytic powerhouse, and he's saying this. It's about damn time. Let me enjoy this.

Enjoy it all you want. And take into account that I'm a no-good hack who isn't published on a baseball site. But as far as I can see the problem here is that none of the base assumptions he's making about what the defensive spectrum is and how it relates to WAR is right. If you don't have those assumptions/understandings right, then the conclusions are meaningless.

"We all know apples are different from oranges, so when we decide to make an orange pie it doesn't taste anything like apple pie...so we need to change our recipe for bread pudding."

Meh.

[RonCo]

Where exactly is this assumed by WAR? IS it the part where the values they result are adjusted because players often move around the diamond to less experienced positions all the time now? Or the asterisk by the totals in the leaderboards explaining this for each player and how much relative experience they have at each position? Oh wait, none of that is being done. CF's just get 1.5 more WAR than first basemen per 162 games played regardless of competence or experience.

One can argue if 1.5 is the right number or not. But they are getting that 1.5 because of two things:

1) Someone has determined he has the basic skillset required to play the position
2) A CF will make many more plays than a 1B will. The base/replacement value of those plays has been determined to be 1.5 WAR.

The volume of plays is considered part of the difficulty of workload of a player, and some mathematical adjustment is required in order to take this fact into account. A league average CF will add (arguably) 1.5 Wins more value than a league average 1B merely because he plays a position that gets more chances.

[RonCo]

https://www.fangraphs.com/library/misc/ ... djustment/

[Ted]

Right, and this is I think a large part of his point. With WAR, the defensive spectrum is being used to flatly assign value to positions. You CAN"T just slide players up and down the defensive spectrum as traditionally defined and assuming they will get worse or better even AFTER whatever time is needed to adjust.

This is, to my mind, an incorrect assessment of what the positional adjustment of WAR is doing.

The positional adjustment of WAR is taking into account both the difficulty of a role (which this seems to be trying to focus on) and the quantity of plays that happens at any one position. A guy playing CF will field twice the number of plays that a LF will. Feel free to argue the adjustments could (and maybe should be tweaked), but they are needed because WAR is a counting stat, not an overall rating.

If by adjusted you mean "completely revamped and calculated differently" then we agree. A flat positive or negative value is inappropriate.

Look at the example of Albert Pujols this year and last. He's terrible right? The ONLY significant difference between his stat line from last year is that his iso power is about 45 points higher. That's a big deal, but is it enough that last year he was a negative 2 win player, and this year he's replacement level? Seriously? Who buys that argument. Yes that's a good solid iso power jump, but two wins? Oh wait he'll end up with about half a win from playing shitty first base instead of being a DH. That's ridiculous. If anything, being a terrible first basemen instead of a DH should even MORE negatively impact his value. In reality, he's probably wiped out most of his gains from his iso power "surge" by being in the field.

[Ted]

Where exactly is this assumed by WAR? IS it the part where the values they result are adjusted because players often move around the diamond to less experienced positions all the time now? Or the asterisk by the totals in the leaderboards explaining this for each player and how much relative experience they have at each position? Oh wait, none of that is being done. CF's just get 1.5 more WAR than first basemen per 162 games played regardless of competence or experience.

One can argue if 1.5 is the right number or not. But they are getting that 1.5 because of two things:

1) Someone has determined he has the basic skillset required to play the position
2) A CF will make many more plays than a 1B will. The base/replacement value of those plays has been determined to be 1.5 WAR.

The volume of plays is considered part of the difficulty of workload of a player, and some mathematical adjustment is required in order to take this fact into account. A league average CF will add (arguably) 1.5 Wins more value than a league average 1B merely because he plays a position that gets more chances.

1B are involved in far more plays than any player in baseball either than pitchers and catchers. We're just assuming putouts are automatic. Have watch a stone fisted, lead footed clutz receive throws for most of season now, I can tell you they are not. I'm not arguing that the impact of a bad CF is not more than the impact of a bad 1B, but

A league average CF adds no more wins than a league average 1B. This is wrong. If they are both average, they have the same impact as each other. If the CF is bad or good defensively, he may have far more relative impact than a bad or good 1B, but that is then a problem that needs to be solved by measuring the difference made and scaling things appropriately. Also, the argument that an average CF is more hard to come by than an average 1B is ridiculous. That's not possible. He's literally defined as the average.

The idea that an average CF is worth 1.5 more wins than an average 1B is just flat out wrong. This is why the positional adjustments are WRONG. A CF that is 10% better than the average CF might be worth more wins than a 1B that is 10% better than the average 1B. In fact, he almost assuredly is. But again, the impact should then me measured and WAR scaled appropriately.

[RonCo]

Without actually even pretending to do math, I'll note Pujols is playing the field this year, so he's adding value that he didn't last year as a fulltime DH. So, yeah, at sub-zero WAR I could care more. WAR is noty meant to be precise...but it's meant as shorthand far relative value created in a season.

Again, counting stat.

Again, adjustment made for difficulty/workload of the role. A good shortstop is more valuable than a good 3B...not because he's a better player, but because he's going to field 3 or 4 times more opportunities.

[RonCo]

A league average CF adds no more wins than a league average 1B. This is wrong. If they are both average, they have the same impact as each other.

The average shooting guard will score more points than the average center. If you're trying to project this counting stat using variance from average, you'd best make a baseline adjustment for position.

Also, the argument that an average CF is more hard to come by than an average 1B is ridiculous. That's not possible. He's literally defined as the average.

The selection process takes care of this. The average nurse is not "as good" at being a doctor as the average doctor. But when pressed into service a nurse will often save your life.