One for Ted: Bill James on WAR

[RonCo]

A few of us have commented on having issues with WAR--most seeming to be centered on positional adjustments. I've personally never minded the positional adjustment, though I think it's something you have to keep in mind as you look at the number. Bill James, however, has dropped a thought bomb on the WAR stat that I've walways thought to be far more of a question...I thought you guys might like to see it.

[Ted]

I love it. Thought about this too. Brett probably won't read this, but he'd like a ton of it. Two takaway points I noticed.

1) Player WAR should be viewed in context of how many wins their team had. If the the team's players' WAR (or run differential or however you want to say they should have won a certain number of games) adds up to more than the team won, then an individual's WAR should be viewed skeptically (and vise versa)
2) Estimates (like WAR), that base a lot on expected values, rather than what ACTUALLY HAPPENED, are inherently flawed in that if what actually happened disagrees with them, they are flat out wrong. You can call it luck, or normal variation, or whatever, but after it happens the results are what they are. If a pitcher has an era of 2.30 when his FIP is 4.5, he still had an era of 2.30, and his WAR (calculated from FIP) can go screw itself.

[RonCo]

I think your point two is well-intended, but too strong. I think WAR/FIP are very valuable, but are not properly tethered...meaning that their goal is very good (and achievable), but their accuracy is suspect.

[Ted]

I think your point two is well-intended, but too strong. I think WAR/FIP are very valuable, but are not properly tethered...meaning that their goal is very good (and achievable), but their accuracy is suspect.

Possibly, but they are more valuable to future expectation. If you can show that they are inaccurate for any given completed event, which frankly isn't too hard in the case of pitchers (not the majority of cases but enough to cause concern), they lose a TON of value for that event.

[bschr682]

Why wouldn’t I have read it?! I read everything posted on these forums. This was great. If they can adjust WAR with this very arguement in mind, I’ll hate it a whole lot less.

[Ted]

The more I learn, the more WAR becomes a secondary or even tertiary stat for me. It's a nice number for reference. It's certainly better than some of the traditional stats, but it's fuzzy enough that it should not be used in a vacuum. Much of my anti WAR push has more to do with over-reliance on it as a single stat than the problems with the stat itself. If you are trying to evaluate a player, and you don't look at the rest of the team (their wins, their defense, the player's stats in the context of his teammates, etc), you're doing it wrong. As a single, all encompassing player eval metric, WAR may be the best thing we have, but it is really not that much better than OPS (at least not as much as it's touted to be).

I can tell you a lot more about a player if you just give me any one of wRC+, or OPS+, or OBA, or OPS and what position they play than I can with WAR. Throw in some sort of defensive metric and I can do even better. Give me two offensive stats and I'll tell you more. Most of us who love this love stats, so why are we always trying to dumb things down into single variable analysis?

[Ted]

Why wouldn’t I have read it?! I read everything posted on these forums. This was great. If they can adjust WAR with this very arguement in mind, I’ll hate it a whole lot less.

Sometimes I think you see a big post about sabermetrics and go, "F this shyte!"

[bschr682]

Oh. Yea I spose. Mostly for the very reasons you just posted above. The entire idea behind creating a super stat is flawed.

[Lane]

All due respect to Bill James for his past contributions, but I really don't think he or most of his current thoughts are relevant.

Baseball is an individual sport masquerading as a team sport. You can't take what a player did in a certain year and subtract from it just because his team didn't win as many games as another team. This argument sounds way too much like saying a player can't be the MVP because his team didn't make the playoffs.

[felipe]

How can you be the MVP if your team doesn't make the playoffs?

If someone hits 80 homeruns and their team wins 65 games, how can he possibly be MVP?

I guess there could be a second division MVP...

[bschr682]

Playing devils advocate here, what exactly is wrong with the arguement that if his team didn’t make the playoffs he can’t be the mvp?

[Ted]

All due respect to Bill James for his past contributions, but I really don't think he or most of his current thoughts are relevant.

Baseball is an individual sport masquerading as a team sport. You can't take what a player did in a certain year and subtract from it just because his team didn't win as many games as another team. This argument sounds way too much like saying a player can't be the MVP because his team didn't make the playoffs.

His point is not that the Yankees won fewer games than the Astros. He's saying that the run differential the Yankees posted should have had them winning 100 games rather than 90, implying that those WAR numbers are not reliable. So 8 WAR on a team who "should have" won 100, but only won 90 games, is less valuable than 8 WAR on a team who "should have" and actually did win 100 games. He furthers his argument by pointing out Judge performed considerably worse in high leverage situations than Altuve, which is not taken into account by WAR. WAR sees a solo homer in a 8 run blowout equally valuable to a solo homer that is the winning run in a 3-2 game. He argues that whether or not clutch or luck are measurable real things, after they happen, they DID in fact happen, so a "clutch" home run IS more valuable than the above mentioned meaningless solo shot. WAR does not take this into account. It is purely a counting stat. In essence the crux of the argument is that we have gone too far in discounting the OUTCOMES of randomness. Discounting randomness (or luck, or clutch, or whatever) for future projection may be fine, but you shouldn't be saying things players actually DID matter less because they were the result of randomness.

[Ted]

Also, thanks Ron. I freaking love this kind of stuff. 20 year ago me would have thought this was insane nerdery. 10 year ago me would have disagreed with present day me, and ten years from now I'll wonder how I could have been so dumb.

[RonCo]

WAR is a fascinating stat, really. I absolutely love it, as much because of its flaws as for its value. I think it would be a better stat if it was truly tied to wins, and if it resulted in the same "replacement value" for all teams. But, then would it? As Stephen says, a guy is sometimes only as valuable as the guys around him. On the other hand, a guy who blasts the ball in certain situations, but failed when the game was on the line was not as valuable as his numbers might say...because his team lost games they should have won if he was better at those time (shich is the Judge comment).

Neither is "wrong." The current calculation can be said to equate players on the average...James' approach would tie the player's value more directly to the team's success (but still leaves the question of whether the blame or praise was evenly distributed or not). Neither of them can possibly be an air-tight assessment of value.

[RonCo]

On the other hand, if you looked at both, you might get a better overall picture. Current WAR is a better predictor of future value, WAR based on Wins is, perhaps, a better indicator of actual value created. Dunno. Have to think about that a little.

That said, it's interesting to look at WAR across a player's career, as (with obvious noise) it does tend to match the basic ideas of a development curve overall.

[Lane]

Okay, so I'll post a response in my own words as well, but here's Dave Cameron saying it better than I ever could: https://www.fangraphs.com/blogs/putting ... ill-james/

Here's the key:

Off Topic

James strongly believes that the metric falls apart by building up from runs, rather than working backwards from wins, since the context-neutral nature of the metric means that what WAR estimates a group of players are worth won’t add up to how many wins their team actually won.

[Lane]

Playing devils advocate here, what exactly is wrong with the arguement that if his team didn’t make the playoffs he can’t be the mvp?

I shouldn't have opened this can of worms, but I think it should be awarded to the best player in the league regardless of how their team did.

Mike Trout, literally the best player of his generation has been to the playoffs, what once? Twice? One player can't take a crap team to the playoffs on his own, so why exclude him from the award just because his team didn't win enough games?

[Ted]

Okay, so I'll post a response in my own words as well, but here's Dave Cameron saying it better than I ever could: https://www.fangraphs.com/blogs/putting ... ill-james/

Here's the key:

Off Topic

James strongly believes that the metric falls apart by building up from runs, rather than working backwards from wins, since the context-neutral nature of the metric means that what WAR estimates a group of players are worth won’t add up to how many wins their team actually won.

I really like this piece. To me, it's a nice continuation of the discussion rather than a rebuttal. I'm not trying to entirely discount the usefulness of WAR. If it seems so, it is only because I am prone to hyperbole. I think THIS is the point in there that continues what I thought was the most important about the James argument

Off Topic

I think the answer is that it depends on how you’re using WAR. In the case of MVP voting, I do think there is a case to be made for looking at the circumstances under which a player performed, and I did use context-dependent metrics when I was an MVP voter. WAR is an imperfect tool, and it’s particularly imperfect for things like the MVP award, which is why even those of us who host sites that promote WAR fairly extensively suggest not relying solely on its results when filling out a ballot.

[Lane]

All due respect to Bill James for his past contributions, but I really don't think he or most of his current thoughts are relevant.

Baseball is an individual sport masquerading as a team sport. You can't take what a player did in a certain year and subtract from it just because his team didn't win as many games as another team. This argument sounds way too much like saying a player can't be the MVP because his team didn't make the playoffs.

His point is not that the Yankees won fewer games than the Astros. He's saying that the run differential the Yankees posted should have had them winning 100 games rather than 90, implying that those WAR numbers are not reliable. So 8 WAR on a team who "should have" won 100, but only won 90 games, is less valuable than 8 WAR on a team who "should have" and actually did win 100 games. He furthers his argument by pointing out Judge performed considerably worse in high leverage situations than Altuve, which is not taken into account by WAR. WAR sees a solo homer in a 8 run blowout equally valuable to a solo homer that is the winning run in a 3-2 game. He argues that whether or not clutch or luck are measurable real things, after they happen, they DID in fact happen, so a "clutch" home run IS more valuable than the above mentioned meaningless solo shot. WAR does not take this into account. It is purely a counting stat. In essence the crux of the argument is that we have gone too far in discounting the OUTCOMES of randomness. Discounting randomness (or luck, or clutch, or whatever) for future projection may be fine, but you shouldn't be saying things players actually DID matter less because they were the result of randomness.

WAR is what it is. Like you said, a counting stat, tied to runs created. It's not tied to wins. Aaron Judge shouldn't be tied to the Yankees wins any more than Altuve should be tied to the Astros.

Now, if you want to make an argument that Judge was less valuable because he was less clutch, fine, no problem with that.

WAR is doing exactly what it is supposed to be doing. It sounds like James wants a metric that is tied to actual wins, which, fine, but I don't see why he needs to rail on WAR to do so. I also don't see the point in a metric that ties players performance to actual wins. I thought that a major point of sabermetrics was to strip away randomness in order to determine true talents and make better projections of player performance.

[Ted]

In reality, the validity of any statistic depends on how you use it. My frustration with WAR is simply the over-reliance on it. There are times when a non-contextual stat is less worthwhile. There are times when contextual stats are meaningless.