OOTP Scouting

[Ted]

So, I was watching a game today, thinking about Kolten Wong's draft day scouting report and the player he's become, and I think I stumbled on the major reason why scouting in OOTP is not widely liked.

As I remember, the report on Wong was that he was an advanced hitter who should be an above average bat in the middle infield, but will only be an adequate glove. Instead, he's a very averagish bat with an outstanding glove. Scouting baseball prospects is hard.

But here's the thing, everyone said the same thing. The scouts these days are largely in agreement. Maybe this is a flaw. Maybe they are relying too much on other people's opinions, or no one want s to rock the boat, but that's the way it is. Scouting is no longer a "my guy vs your guy". It's an industry consensus.

So, OOTP's model of everyone having a scouting department individual to a team, and rating players differently, just isn't how it works. And that's maybe why none of us can get ourselves to use it. We'd be okay with it being wrong, but not okay when one guys is right and another is wrong. Or when there are huge differences in opinion.

Now, I haven't used it in many versions, so maybe it works better now.

But if I was writing a baseball simulator, mine would have 90+ percent agreement by scouts, with the differences being driven by "we think we can develop this guy better so we'll rate him higher", the overwhelming majority of the rest of the difference being half grades (5 points, not 10 on th 20-80) and only very rare big differences in actual scout eval.

Then you would make a lot of variance in potentials during ages 16-23. Lots of "lumping and bumping". Waaaay more than we have. Because real scouts are very good at telling you what a player is now, but not at how they will develop, even if that's what they are trying to to. It's just too hard.

Thoughts?

[niles08]

I agree. There was a long discussion on scouting a while back on slack. Not to dive back into it, but I proposed giving every team the exact same scout with the same ratings and turning on team scouting. You'd then have to request scouting reports from your scout for players in order to get a "high or "very high" accuracy. Otherwise rely on the OOTP scouting which could be tempered down to average or even "high" on accuracy. Doesn't give anybody a disadvantage but let's me target the players I want when I want and know more about what I have compared to what others think I have. Adds to the fog of war type situation.

I was love to see more lumping and bumping from 16-23 year olds though. And then once they reach 24 have it level off a bit.

[Ted]

I agree. There was a long discussion on scouting a while back on slack. Not to dive back into it, but I proposed giving every team the exact same scout with the same ratings and turning on team scouting. You'd then have to request scouting reports from your scout for players in order to get a "high or "very high" accuracy. Otherwise rely on the OOTP scouting which could be tempered down to average or even "high" on accuracy. Doesn't give anybody a disadvantage but let's me target the players I want when I want and know more about what I have compared to what others think I have. Adds to the fog of war type situation.

I was love to see more lumping and bumping from 16-23 year olds though. And then once they reach 24 have it level off a bit.

Whelp, sorry I missed that discussion. Maybe I just spat out a bunch of crap you guys already figured out.

[niles08]

I agree. There was a long discussion on scouting a while back on slack. Not to dive back into it, but I proposed giving every team the exact same scout with the same ratings and turning on team scouting. You'd then have to request scouting reports from your scout for players in order to get a "high or "very high" accuracy. Otherwise rely on the OOTP scouting which could be tempered down to average or even "high" on accuracy. Doesn't give anybody a disadvantage but let's me target the players I want when I want and know more about what I have compared to what others think I have. Adds to the fog of war type situation.

I was love to see more lumping and bumping from 16-23 year olds though. And then once they reach 24 have it level off a bit.

Whelp, sorry I missed that discussion. Maybe I just spat out a bunch of crap you guys already figured out.

Nope, nothing was missed. Nothing was figured out. When I meant there was a discussion, there was a discussion among me and me and me. Lol. Then there was the paddle on my cold hairy ass for bringing up scouting. It still brings me nightmares.

[Lane]

Yeah, that makes sense.

[sjshaw]

It does not work better now. The scouting changes in v21 are horrendous. Basically it's "all the things you hate about OOTP scouting, TIMES TEN."

[Ted]

It does not work better now. The scouting changes in v21 are horrendous. Basically it's "all the things you hate about OOTP scouting, TIMES TEN."

Ugh. I swear this game only seems to go backwards.

[RonCo]

On the whole I much agree with the base concept Ted's talking about. Bottom line, OOTP scouts are horrible, and will always be horrible until Markus and Matt Arnold stop listening to people who talk what I think is mumbo-jumbo about how scouting works. The entire point of OOTP scouting is, IMHO, specifically written to provide the feel the "my scouts are greater than your scouts" or visa versa. And maybe there was a big point to that back in the old barnstorming days of baseball. But, yeah. OOTP scouts are horrific.

I've spent hundreds of hours on scouting back in the day. It's just not going to get better.

That said, "No Scouts" with semi-low resolution ratings and a high TCR comes close to mimicking the environment Ted is talking about.

[RonCo]

It does not work better now. The scouting changes in v21 are horrendous. Basically it's "all the things you hate about OOTP scouting, TIMES TEN."

Ugh. I swear this game only seems to go backwards.

The game has made a LOT of strides forward over the years, but scouting is hamstrung because the dev team is unable to think about scouting in a different direction--and to be perhaps too honest, keeps getting "Advice" from people on the beta team who talk very good games but don't seem to get the idea of gameplay.

[Ted]

It does not work better now. The scouting changes in v21 are horrendous. Basically it's "all the things you hate about OOTP scouting, TIMES TEN."

Ugh. I swear this game only seems to go backwards.

The game has made a LOT of strides forward over the years, but scouting is hamstrung because the dev team is unable to think about scouting in a different direction--and to be perhaps too honest, keeps getting "Advice" from people on the beta team who talk very good games but don't seem to get the idea of gameplay.

It's easy to get hung up on the flaws and forget about improvements. I'll admit that. I think a big contributor to my growing dissatisfaction is that new analysis tools in baseball have fundamentally changed the way wee think about players, and OOTP is simply an outdated model. Much like how sim engines from the late 90's didn't even have EYE ratings, or differentiate different types of power for hitters, OOTP no longer really represents the way baseball players are analyzed. And it can't. As we've talked about before, it would need to completely replace most of its engine. It would have to become pitch based, rather than at bat based. It would need to match a pitch up against a particular swing. Rather than sets of ratings against each other. Sure, the ratings kind of represent the stats a batter might produce, but the closer you look, the more you see the artifice.

To bring this back the the original topic, I think I could deal with the artifice, but the devs simply don't place a lot of priority on the part of the game I enjoy most. I don't give a crap about big league competition outside of using the W-L record to test whether my system is working. I want to develop players and a system. I want to put draft and dev theories to the test and see what kind of teams I can build. Find the best way to compete. And given how scouting is terrible, the draft pools the game makes aren't representative of real life talent and uncertainty, and the player development isn't uncertain enough, it's just hard for me to get into anymore. (I do agree with your idea above about how to best force OOTP to make something resembling a real environment). For me the the fun of OOTP was that it let me be the GM. Instead of trying to convince all the idiots who blog about how the Cards should trade multiple top prospects for Nolan Arenado and his huge, stupid contract that doing so would be a huge mistake, I can prove that trading for old, expensive, established stars is stupid by actually having superior outcomes when not doing it.

OOTP was fun for awhile. I got better. The game didn't grow with me. It's less interesting now.

[RonCo]

I should do some more testing on how the dev engine works while set to "Mega-high" TCR. It's been a long time since I did that. It's an interesting conundrum. I think it's human nature that most GMs get so pissed when their guys lump that they can't enjoy when they bump.

[RonCo]

I have these huge scripts that I used to run that calculated career arcs for thousands of players in hundred year test runs, and at one time could give you "real" odds on development of skills and whatnot of players by age and position and ....

[sjshaw]

I'm right with you, Ted. Actual games are not my primary interest in OOTP. It's all about asset management for me.

[RonCo]

I enjoy both, but this actually a game wherein you are a GM first.

[Ted]

I should do some more testing on how the dev engine works while set to "Mega-high" TCR. It's been a long time since I did that. It's an interesting conundrum. I think it's human nature that most GMs get so pissed when their guys lump that they can't enjoy when they bump.

Sorry to keep reviving this thread, but I just wanted to mention I both agree with and am frustrated by your point about most GMs reacting to lumping so negatively.

It does suck, and frankly I think OOTP has something wrong with its model where once a guy lumps he just keeps lumping and vice versa with bumping, although the lumping to death, especially with control, seems more prevalent. Like a 7/7/7 pitcher loses one control to 7/7/6 and there seems to be a 50% or better chance he'll end up 7/6/2 or something like that. I just don't know how many times I've drafted a guy, and by the time rookie ball starts he's lost 2 control. And by the next summer he's a 20 pot 2 control non prospect. And that doesn't seem realistic at all. MLB teams aren't giving up on 19 year olds that they drafted the year before thinking that they will never get their command. Lack of control may be the big difference maker for a lot of players, but this isn't the way it plays out in real life.

I'd love to see more up and down. Or some ratings up and others down. Repeated re-eval, etc. Or a guy shows something and impresses scouts but it turns out to be a mirage.

But I'm just as frustrated by most GMs reactions to lumping as I am to the "this draft sucks" argument. I know it's a but fuzzy and we've gone back and forth on it, but there is a general trend for the game to try to control its talent pool. So if you are super lumping adverse, and set the game that way, you'll likely not get much bumping. And that's just so freaking boring to me. Couple that with people wanting to get high potential guys into the second round of the draft, and you've basically set yourself up for a "majority lumping" league, which makes managing the minors really dull.

I hindsight, I think this was a contributing reason to me leaving the BBA. The minors just got boring. In my first decade, I had tons of pleasant surprises and combed the minor league FA's for projects. The drafts were "weaker" back then. Towards the end of my second decade, we had fairly highly talented drafts (not even including the late 30's bonkers drafts), and you'd pick up 3-5 potential big leaguers every year in the draft and just hoped one or two didn't lump. That wasn't much fun.

[RonCo]

No problem to me. It’s a fun thread. I’ve got a few thoughts to ramble on about, so I’ll probably use a couple different posts to carry them.

First, let me say that I share your (Ted) view of where the fun lies. I love developing minors, and focusing on development. It’s probably why I can never really play in more than one league at a time. When I’m on my game, I know exactly what is happening with most players at every level of my organization. Since I’m already running essentially six teams (or eight if you include INT and FPL), I don’t have energy for any other league.

Anyway, given that, I’ve focused a lot of my OOTP life (and testing thereof) to looking at development specifically because I find it to be the entire core of all other things about the game—or any game, really. I also find player development algorithms interesting as a nerd because everything about them sits smack dab in the middle of the whole game design and philosophical issue of scouting vs. “real life.”

For instance, in my ideal world there would be no “Talent/Potential” in each stat category (because that’s what our OOTP ratings are—projections of stats in a neutral context). Instead, there would be a “Current” and a whole bunch of things about a player (call then skills or physical traits or whatever) that influence future direction of that skill’s progression over time.

I think, if we were sitting at a bar or in front of a white board and markers, we could have a grand time describing how this would work, and it would be amazing.

Alas, that model is not in OOTP, nor as best as I can tell, in any other game.

In OOTP, as in almost every other sports game I’ve ever seen, Ratings represent rates of stat accumulation, and Ratings move toward Talents (which therefor represent a player’s peak ability to accumulate stats). This is the game’s beauty and it’s downfall.

[RonCo]

What Does That Mean For Scouting?

If you really think about it awhile, you can see why this is a “problem.”

While it makes some sense that the game would use a Ratings-to-Talent system to decide when to move ratings, and while mechanically it “works” (*), in reality no one except the powers that made the universe itself can say whether that approach is “right.” I mean, it’s certainly possible that we all have an ACTUAL Talent level for everything, and that God or whatever random fate is in charge of us moves it up and down for giggles. I tend to doubt that, but who can tell? And, really, it doesn’t matter for gameplay, except….

Off Topic

* Aside: I say it "works" because this is the model that almost every sports game I'm aware of uses—it’s an obvious answer for a developer making a system where players need to grow to a certain level. Set a level, and make players grow to it. Done!)

… except when the human players of the game (and hence the designers) want their scouts to be finding those “Talent” bars. Think about that a bit. Go to the OOTP boards. Think about yourself. Most humans who play the game take if as natural that OOTP scouts are there to properly identify those talents. To them it’s, “Tell me what Talent this guy has, and I’m happy.”

There are two problems here, the most indirect but important is that is almost certainly not how development of human beings in the real-world works. In that light, no scout in the world is really assessing a true talent. I know that’s hard to get brains around. But it’s true, and this truth is important. What a scout is really doing is looking at physical packages, and emotional maturities, and current environments, and then projecting how much better this guy is going to get. Is he a major leaguer? Could he be a regular? A star?

You can try to say that “right, but in the end it boils down to pointing to a chart and saying '60 Power' and that relates to 35 HR." I mean, sure, they break it into various categories for various teams, but no one expects the details to be really right, and the fact is that this “35 HR” power is just an affect. No one knows what the environment the kid is going to play in is going to look like in seven years. No. The details only matter in that they support the overall assessment of whether a kid is going to possibly play. That’s the real question on the table.

Bottom Line: It’s my opinion that Scouting will not work very well in OOTP until they break the idea that the purpose of scouts is to assess their True Talents. Because at the end of the day, that’s what OOTP scouts are doing. The best are better at guessing those talents, the worst a little further off. There’s some adjustments based on “Tools” and “Skills,” but that’s different, too. Then they add some fog of war for Time Since Last Assessment. Meh.

I’ve put this forward for years and years, and—since too many people are embedded in this incorrect idea, and because for so many people scouting in game is made fun by the “my scout is better than yours” concept—I don’t see it ever changing, so to me the idea of using scouts is a now a non-starter for me.

I’ll get to the second “problem” I noted above with this approach in just a moment…

[RonCo]

BOTTOM LINE, THOUGH, THERE IS NO “TALENT” BAR FOR HUMAN BEINGS. (he says, pompously)

There are physical traits, mental maturities, and personal interests that limit or enable us, and then there are a lot of environmental elements that play into our ability to become better or cause us to become worse.

So any game built on “Talent” will be, by definition, Gamey.

[RonCo]

And another thing before I get to that second problem …

The Dev Engine and Draft Classes

As I think you know, Ted, I tend to agree with you on the view of draft classes—that it’s more fun to have weaker classes, and more positive development. I don’t think it’s quite right to say that bigger draft classes result in NO prospect bumps, but it’s certainly my opinion that big draft classes make the development field more harsh, hence you get fewer “finds.”

Because, as you note, (and I believe, anyway) the game does attempt to keep its average ratings in a zone, so artificially pumping players up (or shutting them down) will make a difference in league-wide development. It's not a tidal wave variance. But it's a variance. It's my experience, however, that most human GMs are not like me, and do not find great joy in drafting 5/5/6/4/5 hitters and seeing if they pan out. Sad to me, but true. I fully admit that is one of my personal joys, as manifested by the fact that I'm usually that guy sitting at the keyboard and making selections by hand in rounds 15-20.

Our BBA is weird because it’s been around for such a long time. With Matt bumping parameters here and there, for probably good reasons, that has created a weird little warpage of talents in various classes. So we've got a moderately unique situation there, too. Some classes huge, others smaller.

[RonCo]

Why Does That Matter?

Well…it doesn’t. Except when it does.

By this I mean that as long as you can break the barrier of “reality” and play the game for the fun of it—which basically means being okay with seeing how the OOTP God moves talent up and down to affect ratings progression—then it’s all good. If I’m thinking about it “well,” I am better at remembering that "Talent” in OOTP does not mean a “Rating Ceiling” so much as it means what direction the players ratings should move and how fast.

Off Topic

For example: both a 20/80 and 60/80 guy are probably going to get better in a single season, but the 20/80 guy will probably grow more raw rating points in a single year than a 60/80 guy will.

The second problem is this: Since a Scout’s job is to accurately identify whatever that player’s Talent bar is (rather than project how ratings should grow in a normal human being of whatever skills and traits that player has), and since OOTP’s “God” uses Talent Bars changes to create rating progression, then the Scouting algorithm is designed to see exactly when a Talent Bar (ratings vector) changes.

You are completely correct. We shouldn’t know our 17 year old is effectively washed up at age 19 because, to be honest, he is not—even if his Talent bar says he is. OOTP Gods move the talent bar up and down all the time. That 19 year old who took a hit will often bounce back to at least some degree. But OOTP Scouts are not designed to see it that way (and when we play without scouts because OOTP scouts suck anyway) then we see it in its rawest truth.

The “second problem” is a fallout of the first problem. OOTP TALENT is not actually Talent in the form of a rating ceiling. OOTP Talent is a vector for ratings growth. As such, though it doesn’t change rapidly very often, it is only truly relevant in the moment the dev engine runs (*). Every sim has the ability to change that vector a little, and those changes appear to be mostly mathematical rather than based on some other trait (physical or whatever) of the player.

Off Topic

(*) I should note here that I do not actually know how the dev engine does its thing, but I’ve been testing this thing for a very long time, and this is the framework that seems best to view it under.

I can think of one other thing that I should probably blather on about even more…will get to it if I can remember to…

Note to self: The OOTP Gods and TCR….