14 January 2005

The Scouts vs. The Sabes...



Alan Schwarz recently hosted a roundtable discussion on the debate between the more statistically inclined generation of baseball people- represented here by the genius Voros McCracken (inventor of DIPS, consultant with the Red Sox) and Gary Huckabay (leading analyst at Baseball Prospectus, stat consultant for the A's)- and the old-line scouts- represented by Gary Hughes (30 year scout, Cubs assistant GM) and Eddie Bane (Angels scouting director).


Left to right: Gary Hughes, Voros McCracken,
Eddie Bane, Gary Huckabay


My first comment is, you'd be able to tell just from the pictures who the stat guys are, and who the scout guys are. Kinda funny.

Anyway, I'm far more inclined to come down on the side of the stat side, mostly because it is an imminently more effective way to judge and evaluate baseball players than subjective observation. Bane and Hughes may be very good at what they do- and all accounts say they are- but what they do is, by it's very nature, an imperfect science. Scouting is simply an educated gamble every time out- to rely on it solely is absolute lunacy for a mutli-million dollar franchise.

But, to dismiss their resources out of hand isn't worthwhile either. The entire "intangibles" ferrago is a bit of a joke for grown adults playing baseball- not only is it a worthless concept at the major league level, even if it was, it isn't something any of us can really quantify, making it a bit of a fool's errand to apply to any sort of analysis. At the very young ages these kids come up though, scouts can be something of a group of social scientists- the whole "this kid has what it takes" actually carries considerable merit from someone with so much experience speaking on a kid that may exhibit a more considerable ability to adapt to professional sports.

Anyway, at the end of the day, stats are (when interpreted correctly), statements of fact that portray what a player can and cannot do. Scouting is an observation in a closed set of circumstances over a frightfully small set of sample, based solely on opinion. One should drive MLB development decisions, the other should only aid it. The Red Sox, for instance, work in this way, with the Bill James and Voros McCrackens reporting with consultation from Bill Lajoies and other scouting types.

Let's look at some highlights from the discussion-

First, here's Schwarz posing to the scouts- "what's the nature of the relationship between the two sides?" Hughes gives this very sensible answer-

GARY HUGHES: I think the longtime scouting guys, probably the initial reaction is to get their back up and try to defend their position. They feel somewhat threatened by it because of all the publicity that's come out. It's probably become somewhat of an adversarial thing for maybe not a lot of well thought-out reasons. A lot of what scouts feel they do is based on gut instinct, their history of being in the game. Their experience. They have a hard time quantifying it. And they see all these things in charts and graphs and things, and maybe when you don't understand something, you feel a little challenged by it.


Hughes makes a very fair point- there is very definitely a concern that sabermetrics and Moneyball will put many scouts and scouting culture out of work, render it irrelevant. This actually may even be true, to an extent, so any animosity likely stems from that fear. To wit, Eddie Bane plays into this fear in his answer-

EDDIE BANE: It is adversarial right now. Our guys, the so-called old-school guys, the thought is out there that we don't know how to handle a computer and we wouldn't know how to use that stuff. I'm very comfortable with a computer. Our people are very comfortable with a computer. We do have to drag some of our old-time guys through it. But the main adversarial thing is that some of our old-time guys are losing jobs that we didn't feel they should be losing. It was due to cutbacks. Maybe the cutbacks were due to money or whatever. But we correlate it to the fact that some of the computer stuff is causing that. And we resent it.


Emphasis mine. Some of that is non-sensical- what does it matter if a scout can use a computer?- but the rest is pretty standard. "We resent them because they do our job better and more efficiently. It was a boy's club, they're breaking it up. It sucks."

Right away, they jump into the college players vs. high school players debate...

GARY HUCKABAY: And it costs a lot in terms of upside. You can talk about high school and college players all you want, but the reality is some of these high school kids are gonna be superstars, and if a club's not looking at them, that's a serious opportunity cost.


VOROS McCRACKEN: The lower-revenue teams are in a bit of a bind when it comes to high school prospects because they are more of an unknown. It becomes difficult for a team that's not bringing in that much in terms of revenue to take a big-money chance ...


GARY HUGHES: Why are they an unknown? I don't understand. Because of the data?


VOROS McCRACKEN: Because a player who is 21 is simply closer to his peak abilities than a player who's 18, for starters.


This is a pretty decent breakdown of the differences on the subject. Drafting high school players is a higher risk- but potentially higher reward- move, and requires the input of scouts because of a lack of relevant stats to evaluate. College players are far more clearly evaluated because they're reaching their ability level when you're looking at them, and can take a quicker path to the big leagues that can be more accurately forecasted.

EDDIE BANE: My point would be that the reason to have at least as many scouts, if not more, is when you're drafting Marquis Grissom, as Gary Hughes did with Montreal from Florida A&M, he doesn't cost $100,000 anymore, he costs a million maybe. And his stats at Florida A&M can be thrown out the window. Because you need to see him in the two games a year that he plays against a pitcher that might have any ability whatsoever. That would be my reasoning to have more evaluators see this guy, because the bonus money is going to be astronomical on a guy like that if you have the guts to take him that high. Gary didn't care what his stats were. A player at UConn, his stats, compared to a guy that I'm watching in the Pac-10, mean almost nothing to me. I'm in the middle of a negotiation right now (with Jered Weaver) where a guy wants to compare our first-round pick's stats to Mark Prior's. And to me, there's no correlation whatsoever.


VOROS MCCRACKEN: My response to that would be that those sorts of things, say the difference between playing at Cal and playing at Florida A&M or UConn, you can study those sorts of things and find out what do the stats mean at UConn, what do they mean at Florida A&M, what do they mean at Cal? It's not as if we treat a guy like Rickie Weeks, his stats at Southern -- he had ridiculous stats at Southern, in a weak conference -- the same as if he was playing for USC or Arizona State. Those kinds of things are studied. You can find out information.


Obviously, I don't think it's useful to draft players simply based on their stats. The issue I would bring up is that for all of these issues -- level of play, the type of pitchers, his raw abilities like his speed, his strength, his size -- these are all things that can be, to an extent, measured. Six-foot-1 is a measurement. Five-foot-7 is a measurement. Hitters who are 6-1, do they turn out better than hitters who are 5-7, with similar stats at similar schools? These are the sorts of things that people can analyze, and I think it could provide useful information.


GARY HUGHES: All your statistics are going to tell you is what a guy has done. Somebody has got to make the decision on what the guy's gonna do.


VOROS MCCRACKEN: I have no idea what the guy's gonna do. But my point would be, the scouts also have only a limited idea of what the guy's gonna do. He might do this, he might do that, he might be somewhere in the middle. What you're trying to do is you're trying to take the guys who you think have the best chance. I fully admit that you can't tell the future via stats. My point is that scouting has that equal amount of unpredictability. You can only know so much. You're scouts, you're not fortune tellers.


You can already sense a bit of resentment from Hughes here towards McCracken- the "your" statistics is clearly a flippant characterization of stat guys as some hybrid race of people there to take their jobs- sort of like a McDonald's to their Mom-and-Pop-Diner.

Obviously though, McCracken makes the stronger point here. Anything that can be looked at can ostensibly be analyzed- there's no reason to say, "well, they play in different divisions. Guess I can't understand their performance." McCracken, and sabermetricians, don't pretend to be able to predict, which is something that's often misunderstood. They are, on the other hand, able to better understand what the player is capable of based on available evidence crossed with history. Huckabay actually says as much in his next quote-

GARY HUCKABAY: I think it's important to understand that a lot of people have overclaimed what you can do by statistical analysis. It's a tool. A car is a tool as well -- you can use it to drive to the store, or you can use it to drive into a tree. I think there's more of a dichotomy between good statistical analysis and bad statistical analysis. But all the information you can get your hands on -- as long as you understand what it's good for, and what its quality is -- is always a good thing. We're all after the same thing here: We're out to build a great baseball team. As long as you have X number of pieces of information, whether it's performance data -- a term I prefer to use rather than statistics, because these things are records of what happened on the field -- and then also, if you've got people who have tremendous insight who are well trained, they know how to scout a guy, give me that information too. I want both of it. What I don't want is someone going, "I want this guy because he had 120 RBIs."


Huckabay's being a lot nicer.

Voros on the Red Sox philosophy of melding the two sides-

VOROS MCCRACKEN: Certainly, we in Boston are not antagonistic to the concepts in "Moneyball" either. Obviously they hired me as a consultant. When they promoted Theo, basically the idea was he was going to try to meld the two approaches and get them to where they were not only getting along, but are complementing one another. The stats can help the scouts zero in on the guys they should be zeroing in on. And the scouts, once the stats are sorting things through, can tell you who exactly are the best guys to go after. The success of that can obviously be overblown because a World Series championship is a big thing, big news. How much it had to do with stats, how much it had to do with improved scouting . . . I think the point is that Boston has at least tried to reconcile the two positions.


Here's a little tete-a-tete between McCracken and Hughes on the relevance of "tools," which is just a semantic excuse for them to jab at each other, it would seem-

ALAN SCHWARZ: Let's talk tools for a moment. Are the five tools (hitting for average and power, running, fielding, arm) as relevant today as they've always been? Or given what we've learned over the past 10 or so years, should something like plate discipline be made the sixth tool?


GARY HUGHES: I think the five tools are five physical tools. I don't see where plate discipline becomes a tool.


VOROS MCCRACKEN: I don't know how much hitting for average is a physical tool, either. There are a lot more gifted athletes out there than Wade Boggs, who had the single ability to hit for a high average. Plate discipline is such a critical complement to the physical abilities. If you can take the physical abilities and combine them with plate discipline at a young enough age, I think you can work through whatever problems the hitter has at the plate, and he can become a good hitter. Everything that I've seen, from college baseball stats to minor league baseball stats, time and time again, walks and strikeout ratios, it just seems to keep coming up as very, very important.


GARY HUGHES: You're absolutely right. It's very important. I just don't think it's a tool with a capital T. And you don't need computers and stat guys to see it. You show up at a game and the first thing you get is a stat sheet and you look at it. This has only been for the last 30 or 35 years that I've been doing this. But guess what? The guy with the best strikeout-to-walk ratio is usually the best player. Wow! This didn't just happen in the past five years.


Listen, Hughes is probably right- plate discipline isn't a "tool," because it likely isn't a despcription of a primitive physical ability, which is what "tools" are supposed to represent at such a young age. The problem is, though, at two levels-
a) if plate discipline is not a physical tool, then neither is hitting for average, which isn't nearly as important as plate discipline anyway.
b) what the fuck is the point of holding yourself to some arbitrary guidelines in "tools," especially if they can't account for a player's ability to know the strike zone?

Hughes goes on to make the classic fallacy regarding performance analysis, which is to say that the McCracken's of the world are somehow claiming that what they're discussing only came to be when they decided to start discussing it.

It's moronic. No kidding the best players have always been the ones with the best strikeout-to-walk ratio, Gary- that's the point! You seem not to want to rely on that fact!

Here's an interesting exchange that seems to outline a large part of the issue with scouting-

ALAN SCHWARZ: OK, so it's the trading deadline, and you want to evaluate another team's Double-A right-field prospect. Everyone agrees that he has considerable skills, and you're going to scout him for three games. How will you evaluate what kind of asset he might be for your big league club a few years from now?


GARY HUGHES: You'll have a history coming in, but you'll evaluate his five tools. You'll compare what you have on your own club. You'll think about what your immediate needs are and what your long-term needs are. And you'll make your decision based on your feeling.


Emphasis mine. You're making a decision that will affect the organization for years down the road based on three days of observation and feeling? That doesn't strike these guys as a tremendously inefficient way of doing things?

Huckabay makes a great distinction on the difference between what is likely to be used in evaluation and what isn't, coupled with the philosophy of evaluating players- specifically, in this case, pitchers.

ALAN SCHWARZ: Let's talk about a major league pitcher. We're in the offseason, and your club needs to sign a No. 2 or No. 3 type starter, a good but not great veteran guy. How will you evaluate this pitcher?


GARY HUCKABAY: The first thing I'm going to do is make a distinction between the statistics that describe what a player has already done versus those that do a better job of predicting the future. For example, ERA, year to year, is kind of iffy from time to time. What I'm going to look for instead is someone with a big strikeout rate. I'm going to like someone who doesn't give up a lot of hits. I'm going to like someone who has not been abused. I am more of a hardliner on that than just about anybody. If a guy has thrown a ton of pitches per game, I'm gonna take a look and try to figure out, based on the actuarial curves and other stuff that I've done, how likely he is to get hurt.


Interesting how hardcore he is about pitcher abuse and injury actuaries. This, along with intensive defense analysis, seems to be the newest wave of saber-research.

This is a really funny exchange-

ALAN SCHWARZ: One thing that Eddie and Gary, you might not be aware of, is that a few years ago Voros came up with something called Defense Independent Pitching Stats, which ...


EDDIE BANE: Alan, you said, "You guys may not be aware." That's one of the things we're battling. We are aware. I read these guys' stuff all the time.


ALAN SCHWARZ: I said, "May not be aware." Gary, have you ever heard of DIPS?


GARY HUGHES: No.


ALAN SCHWARZ: OK then! (Laughter)


EDDIE BANE: But I'm going to read everything I can, and on top of that have Gary Hughes in the ballpark to see what the guy does. We're trying to dispel these things. It's not like when we're drafting we spit tobacco at the board, and whatever name we hit is the guy we take. I've read this stuff.


GARY HUGHES: Is that what DIPS is? Tobacco? (Laughter)


DIPS is, of course, "Defense Independent Pitching Statistic," which calculates only things the pitcher controls to accurately assess his success even better than ERA can. Voros came up with it a few years ago, and is currently one of the more used pitching metrics in evaluating a pitcher's performance.

ALAN SCHWARZ: That gets us to this question -- do you guys think Triple-A stats can predict player performance in the majors?


GARY HUGHES: I don't know. I can't answer that. That's not my thing.


VOROS McCRACKEN: I think to the extent that that's your answer, that you don't really know . . .


GARY HUGHES: I don't think you know.


VOROS McCRACKEN: I don't know. But I do have an idea. I have looked at stats for tons of Triple-A players, and what they've done in the major leagues, and I think with this sort of information, I don't think that "I don't know" should be the final answer. I think, "I don't know, and I would like to find out" would be the better approach. I'm not sure that's always been the approach. I would say that you know almost as much about what a guy's gonna do in the big leagues from his Triple-A stats as you do from his major league stats.


GARY HUCKABAY: I'll go further and say exactly as much.


EDDIE BANE: That doesn't surprise me, but I don't believe it. I won 15 games in Triple-A two years in a row. I won seven games total in the major leagues. The level of play is completely different. We weren't into DIPS in '73 but I led the league in ERA both years. I wasn't good enough to pitch in the major leagues. You get up there and you lose the confidence level. David Newhan bounced around, he finally got an opportunity to play, he's all right. But where was he going to play for the Anaheim Angels other than on the bench? When it comes to the stats, I want to know who he's playing against, where he's playing at and who's he's hitting these balls against. I want Moose Stubing to find Brendan Donnelly (in the minors) because of how he saw Brendan Donnelly throw, not because of the statistical edge he might have had.


VOROS McCRACKEN: His statistics were excellent.


GARY HUCKABAY: Donnelly and Newhan were people we were screaming about for years.


EDDIE BANE: But the thing I'd like to hear -- I know you guys work for two clubs -- but it's easy to bring up Newhan and Donnelly. Scream about someone who's gonna do it next year, or that we should be on. Right now. Because I'd like to know. Write it down and give it to Alan, and we'll look at it a year from now.


VOROS McCRACKEN: It's funny, I can't say the guy's name, but someone was just claimed from your organization that I was very interested in.


EDDIE BANE: Steven Andrade by the Seattle organization -- we'll see how that works.


VOROS McCRACKEN: His stats are great stats. They're flat-out great stats. I've never even seen him pitch. And even if I had, I'm not a scout. I wouldn't know what to look for. All I know is he's got great stats that very few other relief pitchers in the minor leaguers have.


This was really a great look by Alan Schwarz. Go read the article in full.

************





I don't like football at all, but like anything else, when there's something significant at stake, it can be fun. I'll actually probably watch two games this weekend- the Patriots vs. Colts game and the Falcons game (not sure who they're playing). Mostly because these games have one of two things I like in a football game-
a) lots of passing, especially deep passing.
b) Michael Vick.

Vick's a lefty! And he runs, he throws far, he has a cool jersey (what am I, 8?)- what's not to like?

I hope the Patriots and Falcons win, too. That'd be a cool Super Bowl.

************





As you may have noticed from his comments to the post from January 13th, the Broseph did, in fact, pass, biatches.

Reminds me of my driver's test story. It sucks.

I got this really surly middle aged driving tester, and as I got into the test car, I realized that my legs were a little long for that particular car, and that I couldn't figure out how to adjust the seat. As a result, when I would try to move my foot from gas to brake, my knee would slow the transfer up, making it kinda iffy from the get-go.

Most of the test was easy, save for the fact that I couldn't understand his mumbling very well, and had to say "what?" every time he said something. It even caused me to take a wrong turn at one point, which clearly ticked him off. He wasn't too ecstatic about Bullshit Memorial by any means.

Anyway, so we go to a parking lot to test my ability to back into a parking spot- something I was great at. I was looking forward to nailing this and heading back to the school triumphantly. Instead, when we go to the parking lot, I noticed none of the blacktop had defined space lines. In other words, there seemed to be no place for me to "back in to." Instead, he decided instruct me, in very confusing fashion, to back into a space defined by two arbitrary road signs about ten feet away from the edge of the blacktop we were on. In other words, use them as guideposts for the "space" you were backing into. Being so far from the edge of the blacktop, I was convinced there was something I was doing wrong, and very uneasily started to just kind of back in- not really knowing what I was doing, and knowing the guy would blow up if I said "what?" again.

I fucked up. He looked at me, sort of like, "what the fuck was that?", and I claimed I wasn't 100% sure what he wanted me to do. He reached over, turned off the car. "Get out of the car," he said.

We both got out, and in pure frustration, the instructor paced around the edge of the car. This, I thought, couldn't be good for my passing. He started to yell at me- "You listen to me the first time I say things, understand?" He proceeded to try and explain what he wanted me to do, again, and in terms I truly didn't understand, again. "Well," I told myself, "just fucking try it. Nothing much to lose."

What followed was easily the most unreal, stunningly clutch performance of my entire life. Not really knowing WHAT I was doing (I still don't even totally remember it- a lot lik Will Ferrell in the debate in Old School- "What happened, I blacked out..."), sure I was going to fuck up and fail, I eased it into what I guessed he was calling the "spot." I just sort of clunked it into park, and sat, staring forward.

"Perfect," he grumbled. "Let's go back."

When we got back, he took out his evaluation and started the berating again- telling me I didn't do so well, and that I had trouble listening to people (a funny assumption to make knowing someone for five minutes). After a commendable scolding, and paused, and said- "I'm passing you- barely."

Barely! Like I gave a shit! God, he was a real cocksucker.

______________________________ |