Page 1 of 2

Why I Hate Overall Ratings, A Thread

Posted: Fri Nov 13, 2020 11:24 am
by RonCo
(Or, How I Stopped Giving a Crap and Learned to Love the Component)


If you’ve listened to any of the various podcasts I’ve done, you have, by now heard me say something in the range of “I hate overall ratings” enough times. Still, there’s always room for one more, I figured, so what the hell, right?

First, a little process junk: Last night, for giggles, I grabbed a view in OOTP and exported out all of our ratings (1-10 scale, plus a bunch of other stuff—specifically including overalls. Then, after deciding to focus on pitchers for this one, and after stripping out potential info, I was left with every pitcher in the league. This I cut into a couple different segments—specifically “all SP” and “all relievers with 10 Stuff” (note the “10 Stuff” was a random thing). At that point, I then sorted each into ratings blocks by Stu/MOV/CON pairings—then these parings were sorted by OVR.

In other words, I ended up with the ability to put equally rated STU/MOV/CON pitchers together and see how their OVR varied.

I’ll add the two splits as a stand-alone spreadsheet at the end of this rock so you can play with the process yourself. And most likely I’ll be posting follow on cuts in different ways over the next few days.

That said, I’ll do a few cuts here.

For example:


Let’s Look at some 10/5/7 Relievers

I chose this block for a could reasons. First, there are no YS9 pitchers in here, so I can look at them “objectively.” In addition, it was the first grouping of pitchers that had enough sample size that it spanned FOUR rating points. (There are pitchers in this segment who score 35, 40, 45, and 50 in OVERALL Rating)

Here are the pitchers:

OVR-RP-10-5-7.PNG

Can you see why I hate Overall Ratings here? I mean…

Compare Aurelio Castillo (35) with Mitch Freeman (50), and you see two power pitchers with three pitches that are all pretty danged good. The only difference is that Freeman throws two MPH faster, and Castillo is an over the top fly ball pitcher. I note, though, that Castillo controls the running game at an elite level, while the barn door is always open for Freeman. Is the two MPH difference (from 99-101?) worth 15 overall points? Is it the FB thing? If so, other than Castillo’s very brief stint in the bigs, there’s nothing in his numbers to say his “5” movement is any worse than Freeman’s.

And, if you say, yeah, two MPH is worth 15 rating points in OVERALL, well, whot do you make when you compare Castillo to Portland’s Faical Engeitado? Engitaido tops out at 93 MPH and his repertoire would seem to pale in comparison to Castillo’s. Engeitado is a lefty sidearm thrower with good splits. Perhaps that’s bump-worthy, but three notches? 15 points?

Play the same game with Charlotte’s Robert Hobbs—who blazes at 101 MPH and has a four-pitch arsenal.

Bottom line: if all these guys were within a rating of each other, I could see it. But they get spread over four total ratings…which makes me scratch my head.

Perhaps it’s more interesting to look at the common elements here—the three guys in the list who are all rated 45s? I’ll do that later, too—or even go further and collect up all 45s (or whatever) and see what that pile looks like in total. Or you can. Feel free…


Now Let’s Look at a Couple Starters

Personally, I find looking at starters to be more interesting by comparing across different rating segments, and I’ll probably do that a little later, but while we’re on this process, let’s look at an interesting pair.

OVR-SP-2.PNG

Madison’s 25-yo Yorikane Miyamoto (55) matches with New Orleans’ Cristian Garcia (65). If you looked at their ERAs you’d probably say that makes sense—but compare FIPs and WARs and you get a different feel. Until this past season, when Miyamoto seemed to implode, the two were quite similar.

Which makes sense when I scan the components.

I mean…both are young, both throw pretty hard and have better than 3-pitch repertoires. Both deal with the running game. Both go medium deep into games. Does the velocity bump account for the overall rating delta? I dunno, but I don’t think it should. I mean, looking at these two pitcher’s I’d value them about evenly all other things considered. Put them behind the same defenses and in the same ballparks and I’d expect a similar performance. But the game says one is 55 and the other 65. Am I the crazy one? Possibly.

Anyway, my morning time is up, so I’ll stop here…but more to come … maybe. (grin)

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 9:25 am
by bcslouck
You know, thinking about this a coupe days after checking this out, I may be more on board with getting rid of them. I think a lot of people, including myself, tend to hang on to players because of overalls instead of looking at the ratings of the player. Or seeing if they actually fit what you need. I think it would take a little bit, but you may see the trade market open up more if you got rid of them of the overall potential and current. Keep all other ratings as is.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 9:38 am
by Ted
bcslouck wrote:
Tue Nov 17, 2020 9:25 am
You know, thinking about this a coupe days after checking this out, I may be more on board with getting rid of them. I think a lot of people, including myself, tend to hang on to players because of overalls instead of looking at the ratings of the player. Or seeing if they actually fit what you need. I think it would take a little bit, but you may see the trade market open up more if you got rid of them of the overall potential and current. Keep all other ratings as is.
I see your point. My concern is that this is an added degree of difficulty for less experienced people. Less importantly, I personally like to have more data. Even if the overall ratings doesn't make sense, it's telling you something. I've been able to figure some things out about players before because of it that I wouldn't have been able to without it.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 10:49 am
by aaronweiner
You didn't figure out that Garcia was a 65 because he's a lefty?

And I'd clearly rate Neumann above the others.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 11:06 am
by Lane
I don't think they're perfect, but I really appreciate having them as a quick point of reference.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 11:07 am
by RonCo
Yeah, I don't think they need to be nixed, but they are quite wonky to my eyes in many cases.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 11:40 am
by bcslouck
Ted wrote:
Tue Nov 17, 2020 9:38 am
I see your point. My concern is that this is an added degree of difficulty for less experienced people. Less importantly, I personally like to have more data. Even if the overall ratings doesn't make sense, it's telling you something. I've been able to figure some things out about players before because of it that I wouldn't have been able to without it.
I feel like having actual ratings with potentials is plenty enough. I feel like fog of war is something most people want, but the BBA is about as far from it as you can be.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 11:55 am
by niles08
I think I have been pretty vocal about getting rid of overall/potential ratings for some time. Often enough they don't make tons of sense as Ron showcases above. I will be the first to admit even when I argue we should get rid of them, I have found myself hanging onto players that I shouldn't have because of their "overall rating". I would even be in favor of half removing these and just leaving the all mighty "potential" rating and hiding the overall. That way it's not total darkness on this number which doesn't really mean much anyhow.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 12:17 pm
by Ted
aaronweiner wrote:
Tue Nov 17, 2020 10:49 am
You didn't figure out that Garcia was a 65 because he's a lefty?

And I'd clearly rate Neumann above the others.
I'm pretty sure you're messing around, but for the sake of the less informed, handedness does not impact overall/potential ratings.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 12:19 pm
by RonCo
Ted wrote:
Tue Nov 17, 2020 12:17 pm
aaronweiner wrote:
Tue Nov 17, 2020 10:49 am
You didn't figure out that Garcia was a 65 because he's a lefty?

And I'd clearly rate Neumann above the others.
I'm pretty sure you're messing around, but for the sake of the less informed, handedness does not impact overall/potential ratings.
Technically, this is true in that you can change handedness and it won't affect the overall ratings. But handedness does come into actual splits, which the game engine uses (and I assume the overall rating system uses). Splits are created upon player creation, and will also not change if you alter player handedness.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 12:21 pm
by Ted
Ted wrote:
Tue Nov 17, 2020 12:17 pm
aaronweiner wrote:
Tue Nov 17, 2020 10:49 am
You didn't figure out that Garcia was a 65 because he's a lefty?

And I'd clearly rate Neumann above the others.
I'm pretty sure you're messing around, but for the sake of the less informed, handedness does not impact overall/potential ratings.
Hmm, except in one possible way. Our league is set so that if overall performance exceeds potential, it will diplay overall for both overall and potential. For example, if say a developed 30 ovr/pot player puts up the stats the game would expect from an 80 ovr/POT player, it will show 80 OVR 80 POT. (The other option is to have it say 80/30. We do not have it set up that way).

So, in theory, if handedness gave a player a huge advantage and that player massively exceeded the games ovr/pot expectation, you could see handedness "affecting" the displayed ovr/pot. But it's not affecting the internal ratings, just what we see. This could be contributing to Ron's confusion in the initial post.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 1:14 pm
by RonCo
Yea, that is true. If we alter the weightings to 100% ratings, rather than being influenced by performance than I suppose things would be clearer.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 1:41 pm
by RonCo
Here's a little test snippet of what I mean (and I think Aaron & Ted mean). Overall ratings can be influenced by both actual ratings (with different splits vR and vL in the editor), and by performance.

I'm pretty sure ours are done by the standard set (but am not perfectly sure!), so:

Ratings: 30%
Current Year Stats: 50%
Last Year Stats: 15%
Two Years Ago Stats: 5%

If I create a test player in a standard test league I've got, and go play around with all the factors (batter handedness, batter splits vR/vL, and whether stats are included in the overall ratings), I get a chart that looks like this:

OVR-BATTING-SPLITS-HANDEDNESS.PNG

So, batter (or pitcher) handedness drives hitter splits upon player creation, but changing handedness does not do anything unless you also go into the editor and change the core ratings.

This means that Aaron is not exactly "right" when he says Player A is rated better than Player B because he is left=handed, but it is _likely_ true that his splits represent a split more normally attributed to a left-handed player. Alternately, though, a player's performance could have been up or down so far that the influence on stats is enough to make a difference. You just don't know.

All of this really goes to my point. It's not that overall ratings are "bad" or even wrong, but to Brandon's point above, I think we get complacent in using them as if they are real--or at least as if they are more solid than they really are.

At the end of the day, I don't actually 'hate" overall ratings. I like them as generic markers for my players. But when I read an overall rating I guess what I'm doing inside my mind is adding a "plus or minus 15 points" to the rating--and that sometimes, that +/- 15 can be pretty important.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 1:57 pm
by Jwalk100
Let's go stats only!

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 2:47 pm
by niles08
Jwalk100 wrote:
Tue Nov 17, 2020 1:57 pm
Let's go stats only!
now you going and saying something like that is going to end any discussion on any changes period lol.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 2:52 pm
by RonCo
Stats-only is fun, but takes a different kind of GM mindset.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 4:08 pm
by aaronweiner
Even MLB doesn't go stats-only. Why should we?

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 4:29 pm
by sjshaw
I’ve been in leagues without OVR/POT ratings and it was a nightmare. They suck, but help you quickly separate wheat from chaff.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 4:29 pm
by bcslouck
aaronweiner wrote:
Tue Nov 17, 2020 4:08 pm
Even MLB doesn't go stats-only. Why should we?
That has always been my mindset on stats-only.

Re: Why I Hate Overall Ratings, A Thread

Posted: Tue Nov 17, 2020 4:30 pm
by bcslouck
sjshaw wrote:
Tue Nov 17, 2020 4:29 pm
They suck, but help you quickly separate wheat from chaff.
I disagree. This is the only league I've been in with overalls and the others were/are perfectly fine.