Prep Star or College

[JMUalumni]

One of the bigger draft debates through the years has been whether you take a prep player with your first round pick or go for the supposedly safer college player.  I have always thought in recent years that there are some college players worth taking in the first round, but overall many of them are not first round talent.  Prep stars are harder to project and take longer to develop, but I think they are much more valuable in the long run.  Marc Hulet took a look at the first round picks from 2001-2003 to see if there were any patterns.  His conclusions are based on only three years and the test for determining if players pass/fail is based on "if they met expectations" but if nothing else it is a categorized list of players taken in the first rounds for those year.  Even if you don't agree with the evaluations of the author, look at the list and judge the players taken for yourself.  Take it FWIW:

For those of you who have been reading this site for a while, you probably know that one of my favorite things to write about is Major League Baseball's amateur draft. The 2009 draft is about a month away (June 9-10) so I though it might be a good time to take a look at one of the more popular debates around baseball, as well as the Internet.

There is a belief amongst some people that it is "safer" to pick a college player in the first round of the MLB amateur draft than it is to select a prep player. This belief was spawned - or at least made popular - by the Moneyball era. But is there really any such thing as a "safe" draft pick in baseball, given the nature of the beast? Baseball, unlike most other pro sports, generally requires top-ranked amateurs to spend many years honing their skills in the minor leagues before they are ready to play amongst the best athletes in the world in their chosen sport. The skill-level gap between Major League Baseball and amateur baseball is much larger than with football or basketball. And we haven't even mentioned the risk of injuries.

So let's take a look at the first rounds of the draft from 2001-2003 and see if the above belief has held true or not. We'll also break it down by position to see if college shortstops are more likely than, say, college catchers to meet expectations (We'll make the assumption that clubs are expecting a first-round pick to be at least a league-average regular at their position). There is a certain amount of subjectivity to deciding if a player has met expectations so you may disagree slightly with my opinions. As well, this type of study is difficult because players' stocks can fluctuate from year-to-year, but let's see how things play out. Players will be assigned either a (Pass) or (Fail) for meeting expectations. There are a couple (Undecided) as well.

* I was going to include 2004 and 2005 as well, but there were just too many players that had futures that were still too much up in the air.

2001 MLB Draft
College Hitters
Mark Teixeira, 1B (Pass)
Chris Burke, SS (Fail)
Jake Gautreau, 3B (Fail)
Gabe Gross, OF (Fail)
Mike Fontenot, 2B (Pass)
John-Ford Griffin, OF (Fail)
Bobby Crosby, SS (Pass)

High School Hitters:
Joe Mauer, C (Pass)
Casey Kotchman, 1B (Pass)
Josh Burres, SS (Fail)

College Success Rate: 3/7 (43%)
High School Success Rate: 2/3 (67%)

College Pitchers
Mark Prior, RHP (Fail)
Dewon Brazelton, RHP (Fail)
Josh Karp, RHP (Fail)
Chris Smith, LHP (Fail)
John VanBenschoten, RHP (Fail)
Kenny Baugh, RHP (Fail)
Aaron Heilman, RHP (Pass)
Brad Hennessey, RHP (Fail)
Jason Bulger, RHP (Fail)
Justin Pope, RHP (Fail)
Noah Lowry, LHP (Undecided)

High School Pitchers
Jeremy Sowers, LHP (Did Not Sign)
Gavin Floyd, RHP (Pass)
Colt Griffin, RHP (Fail)
Mike Jones, RHP (Fail)
Kris Honel, RHP (Fail)
Dan Denham, RHP (Fail)
Macay McBride, LHP (Fail)
Jeremy Bonderman, RHP (Pass)
Alan Horne, RHP (Did Not Sign)

College Success Rate: 1/10 (10%)
High School Success Rate: 2/7 (29%)

2002 MLB Draft
College Hitters:
Khalil Greene, SS (Pass)
Drew Meyer, SS (Fail)
Russ Adams, SS (Fail)
Nick Swisher, OF (Pass)
Josh McCurdy, SS (Fail)

High School Hitters:
B.J. Upton, SS (Pass)
Prince Fielder, 1B (Pass)
Scott Moore, SS (Fail)
Jeremy Hermida, OF (Pass)
James Loney, 1B (Pass)
Denard Span, OF (Pass)
Jeff Francoeur, OF (Pass)
Sergio Santos, SS (Fail)
John Mayberry, 1B (Did Not Sign)

College Success Rate: 2/5 (40%)
High School Success Rate: 6/8 (75%)

College Pitchers:
Bryan Bullington, RHP (Fail)
Jeff Francis, LHP (Pass)
Joe Saunders, LHP (Pass)
Royce Ring, LHP (Fail)
Bobby Brownlie, RHP (Fail)
Jeremy Guthrie, RHP (Pass)
Joe Blanton, RHP (Pass)
Ben Fritz, RHP (Fail)

High School Pitchers:
Matt Cain, RHP (Pass)
Cole Hamels, LHP (Pass)
Chris Gruler, RHP (Fail)
Adam Loewen, LHP (Fail)
Clint Evert, RHP (Fail)
Zack Greinke, RHP (Pass)
Scott Kazmir, LHP (Pass)

College Success Rate: 4/8 (50%)
High School Success Rate: 4/7 (57%)
Notes: Derick Grigsby, RHP, who was drafted out of community college, was not considered.

2003 MLB Draft
College Hitters:
Rickie Weeks, 2B (Pass)
Michael Aubrey, 1B (Fail)
Aaron Hill, SS (Pass)
Brian Anderson, OF (Fail)
David Murphy, OF (Pass)
Brad Snyder, OF (Fail)
Conor Jackson, 3B (Pass)
Brian Snyder, 3B (Fail)
Carlos Quentin, OF (Pass)
Mitch Maier, C (Fail)

High School Hitters:
Delmon Young, OF (Pass)
Chris Lubanski, OF (Fail)
Ryan Harvey, OF (Fail)
Ian Stewart, 3B (Pass)
Lastings Milledge, OF (Undecided)
Matt Moses, 3B (Fail)
Brandon Wood, SS (Undecided)
Eric Duncan, 3B (Fail)
Daric Barton, C (Undecided)

College Success Rate: 5/10 (50%)
High School Success Rate: 2/6 (33%)

College Pitchers:
Kyle Sleeth, RHP (Fail)
Tim Stauffer, RHP (Fail)
Paul Maholm, LHP (Pass)
Ryan Wagner, RHP (Fail)
Chad Cordero, RHP (Pass)
David Aardsma, RHP (Fail)
Brad Sullivan, RHP (Fail)

High School Pitchers:
Chad Billingsley, RHP (Pass)
Jeff Allison, RHP (Fail)
John Danks, LHP (Pass)

College Success Rate: 2/7 (29%)
High School Success Rate: 2/3 (67%)
Notes: Nick Markakis, OF, who was drafted out of junior college, was not considered.

* * *
So let's total things up:
College Hitters Success Rate: 10/22 (45%)
High School Hitters Success Rate: 10/17 (59% - three undecided)
College Pitchers Success Rate: 7/25 (28% - one undecided)
High School Pitchers Success Rate: 8/17 (47%)
Conclusions: Maybe clubs like Toronto need to re-think their reluctance to draft prep pitchers due to their "unpredictability." Drafting pitching in the first round was not a safe bet (less than 50%) regardless of picking a college or prep pitcher between 2001-03, but the college success rate was dismal. It would be interesting to see if this trend would hold true if we were to research the draft back another 10 years. Surprisingly, prep hitters also performed better than their college counterparts.

College Success Rates by:
Catcher: 0/1 (0%)
First Base: 1/2 (50%)
Second Base: 2/2 (100%)
Third Base: 1/3 (33%)
Shortstop: 3/7 (43%)
Outfield: 3/7 (43%)
Right-Handed Pitcher: 4/19 (21%)
Left-Handed Pitcher: 3/5 (60%)

Notes: Wow. College right-handers were really not the best choice, although they were by far the most popular. Yikes.

High School Success Rates by:
Catcher: 1/1 (100%)
First Base: 3/3 (100%)
Second Base: ---
Third Base: 1/3 (33%)
Shortstop: 1/4 (25%)
Outfield: 4/6 (67%)
Right-Handed Pitcher: 5/12 (42%)
Left-Handed Pitcher: 3/5 (60%)

Notes: Obviously high school second basemen are not a hot commodity, which is not overly surprising, considering a large number of MLB keystone players probably started out as shortstops in the minors. The prep first basemen that were chosen have performed pretty well.

Overall Conclusion (sort of):
So, is there any such thing as a safe pick? Not really. But interestingly enough, prep hitters were the more successful choice between 2001 and 2003, followed by... prep pitchers. Teams that chose prep prospects, in general, had a 53% success rate. Teams that chose a college prospect had a success rate of just 36%. Collectively, the three years studied is a pretty small sample size in the grand scheme of things, so we cannot really read too much into the numbers above, but what it does is provide some food for thought. It could also serve as a great starting point (or hypothesis) for a much larger study on the successes and failures of the Major League Baseball amateur draft.

http://baseballanalysts.com/archives/2009/05/the_mlb_draft_c.php

[DPMOmaha]

My opinion is that you take the best player available, regardless of whether they're college or HS players.  There are exceptions where you might draft for need but those seem to be the exception rather than the rule.

[NFA Brian]

The one talent pool I am really hesitant about wading into is high school pitchers. There is way too much variability there.

The problem with all of these analyses (mine included) is that each draft is an independent sample. It does not adjust for the overall strength/weakness of the class as a whole.

For example, this year, the hitting talent in the early part of the draft is down. Compared to last year's strong class, this year is really bad. So, making an assessment of drafting a bat here would most likely produce poor results in retrospect while the class of 2008 would more likely be positive. It does not accurately paint a picture of some unknown draft in the future.

[JMUalumni]

My opinion is that you take the best player available, regardless of whether they're college or HS players.  There are exceptions where you might draft for need but those seem to be the exception rather than the rule.

Well, I think that is the crux of the problem.  What is the 'best' available player?  Most college players are going to be better players at the time of the draft than almost any high school player, but a lot of of prep players are going to project to be better players than many college players.

The problem with all of these analyses (mine included) is that each draft is an independent sample. It does not adjust for the overall strength/weakness of the class as a whole.

For example, this year, the hitting talent in the early part of the draft is down. Compared to last year's strong class, this year is really bad. So, making an assessment of drafting a bat here would most likely produce poor results in retrospect while the class of 2008 would more likely be positive. It does not accurately paint a picture of some unknown draft in the future.

This is a good point.  Overall, I am against thinking along the lines of we should shy away from prep prospects because they have a lower success rate than college players or vice versa.  Context and situations play a large part in the draft process.  But, hypothetically if you determined that the prep pool and college pool were roughly equal in talent, I would prefer to take one of the top prep guys over one of the top college guys.  Sure, there are exceptions to this line of thinking and it depends on how your specific team views the draft, but I personally believe that early round prep stars are much more valuable to teams than the early round college picks.

[blue911]

The one talent pool I am really hesitant about wading into is high school pitchers. There is way too much variability there.

But isn't that where the long shots pay off? Three years ago, any team could have signed Steven Strasburg for $1M.

[hammondsnats]

I'm of the mind take the best player available, but I would really prefer a college bat/arm with the 10th pick. 

[NFA Brian]

But isn't that where the long shots pay off? Three years ago, any team could have signed Steven Strasburg for $1M.

Yes. With the increased risk comes increased reward. My hesitation is based more on the fact over the draft's history, prep arms slide. Whether it's Jack McGeary or Tim Melville, you'll find intriguing prep arms in the 2nd, 3th, 4th rounds. I see the value in grabbing the HS pitchers who slide like that in the 3rd round and focus on the other areas earlier in the draft.

It's not an exact science and the cardinal rule should remain BPA, it's just something I've observed over time.

[blue911]

Yes. With the increased risk comes increased reward. My hesitation is based more on the fact over the draft's history, prep arms slide. Whether it's Jack McGeary or Tim Melville, you'll find intriguing prep arms in the 2nd, 3th, 4th rounds. I see the value in grabbing the HS pitchers who slide like that in the 3rd round and focus on the other areas earlier in the draft.

It's not an exact science and the cardinal rule should remain BPA, it's just something I've observed over time.

I think people are way to critical of McGeary. From what I understand, the kid is basically a finesse pitcher. Given the few innings he's actually amassed people are expecting more from him than is justified.

Granted that many high school pitchers flame out. But what is the overall percentage of pitchers drafted every year? I'd bet it's close to 40%

[JMUalumni]

I think people are way to critical of McGeary. From what I understand, the kid is basically a finesse pitcher. Given the few innings he's actually amassed people are expecting more from him than is justified.

McGeary has shown some good flashes.  His main pitch is the curveball, but he has a power fastball too.  I think the high expectations can be traced to none other than "Desmond is the next Jeter" Bowden and the fact that they paid way beyond slot to get McGeary.  I think McGeary will be fine and will one day be a good trading chip or a #2 or #3 guy in the rotation.

[HalfSmokes]

This is a pretty small sample size. BP did a similiar/larger study and found the best bets to be College bats. I can't remember whether college arm or high school bat was next, but high school arm was the worst. I let my subscription expire, so someone else will have to post it if anyone wants it.

[JMUalumni]

This is a pretty small sample size. BP did a similiar/larger study and found the best bets to be College bats. I can't remember whether college arm or high school bat was next, but high school arm was the worst. I let my subscription expire, so someone else will have to post it if anyone wants it.

I remember that study vaguely, but I believe it may have been something like the first 5 or 10 rounds of the draft.  I was directing my specific comments to the first round only.  It is my personal belief that there quite a large drop off from first round talent to the next round.  Many college bats/arms are more worthy of the middle rounds IMO.  The article above is flawed, as I mentioned in my original comments, but just looking at the names it is clear that the better players are in the high school category for almost every year (with Texiera being the only exception).  The scouting field has also drastically expanded and improved over the last 5 or so years, I believe this will affect positively the selection of players in the first round in the last few drafts and the drafts to come.

[sportsfan882]

from what everyone has said, I think it is best we take two college arms with our top two picks.

[NatsDad14]

How did Mark Prior fail and Joe Blanton pass?

[NFA Brian]

How did Mark Prior fail and Joe Blanton pass?

His parameter is "meeting expectation". In the case of Prior, since he was expected to be Walter Johnson and Cy Young rolled up into one pitcher ... and wasn't, he failed. While Blanton's expectations were likely to make the majors.

Arbitrary? Yes.

(Prior should be given a "Pass" or the parameters need to be less nebulous)

[JCA-CrystalCity]

Actually, the evaluation criterion should be, "how likely is it that the draft choice will turn into a quality player" rather than meet expectations or not.  While he's bombed since his trade, Jon Rauch would be a "fail" were the test to include his expectations, but he was a very effective set up guy and occasional closer for 3 years. 

As Brian suggests, anything short of Dwight Gooden start to Strasburg's career would lead Hulet to give him a "fail."  If he gives us 12 wins next year and a 3.90 ERA, that would be a good major leaguer, potential ROY, but a "fail" due to expectations.

This analysis really does not tell us much about who to draft with our top 2 picks and the 2d rounder.

(note - slight edit for clarity)

[NFA Brian]

This analysis is more along the lines of what I find valuable. Beyond the Boxscore looked at the 1994, 1995, and 1996 drafts using WAR (Wins Above Replacement) during their first six years.

There are some things to quibble with. I don't see how a team should be penalized for having a player make the majors and perform less than a replacement level player while a team whose choice flames out earns a '0'. Things can get skewed by one dominant player (see Nomar in 1994). But, I agree most with this comment the writer made

not every draft is equal. Each draft has it's strengths and weaknesses

[Obed_Marsh]

Why even bother developing a pass/fail metric? Just count the number of service years above replacement player level and average out the cost per draft to evaluate teams effectiveness. Now that's a evaluation I'd like to see.

[Lintyfresh85]

Ok... I find this list to be totally flawed.

For example, guys like Royce Ring have made the bigs and have stuck for a little while... seeing as the draft is such a crap shoot, I think any player that actually makes it to the bigs and is here for more than a cup of coffee should be considered a pass.

If we use that criteria just on the class of 2001 for pitchers it would look like this.

Mark Prior, RHP (Fail) (PASS)
Dewon Brazelton, RHP (Fail) (PASS)
Josh Karp, RHP (Fail)
Chris Smith, LHP (Fail)
John VanBenschoten, RHP (Fail)
Kenny Baugh, RHP (Fail)
Aaron Heilman, RHP (Pass)
Brad Hennessey, RHP (Fail) (PASS)
Jason Bulger, RHP (Fail)
Justin Pope, RHP (Fail)
Noah Lowry, LHP (Undecided) (PASS)

That's 45% which isn't bad.

Also, any list that says Mark Prior was a failure because he was not all he was hyped up to be is just not credible in my eyes. Prior had an amazing year or two... he made the bigs and he dominated, that's a pass in my eyes every day of the week.

[JCA-CrystalCity]

Bulger actually is sticking in the LAA bullpen this year.  Also, I think Chris Smith might have had a cup of coffee last year.  Perhaps > 50 IP, or whatever the rookie cutoff is.  Bulger I think might make it because he was a call up last year.

[JMUalumni]

Again, the rating system is flawed and based off of no metric, I just posted for discussion.

As for considering players who just made the bigs for a few years as a 'Pass' would be a mistake as well.  Having just one average year and then a bunch of rotten years like Brazelton doesn't qualify as value to me and in the end should be considered a failure as a draft pick at the end of the road.  A guy like Lowry has showed some flashes, but has mostly failed to live up to the buzz around him.  His career is now hampered by injuries.  This also brings up the issue of injuries which cannot be separated from the value of a prospect.  A guy that has one good year and then is followed by injuries for the rest of his career can be taxing to a team, who many times will hold onto the guy longer than they should hoping that he works his way through the injuries and returns to the form he flashed for that one year (Shawn Hill anyone?).  Ultimately, in order to evaluate the success of draft picks, a statistical measure needs to be set forth.  I like the idea of using WAR, but combined with some additional parameters in other statistical categories (maybe IP/WHIP/ERA+ for pitches and PA/OPS+ for hitters).

After reading a bunch of different studies on draft picks since posting this article, I am able to conclude that evaluating whether to take prep vs college involves entirely too many variables for evidence to point to one or the other.  As others have mentioned, the best policy will always be BPA as no player taken in the rule 4 draft is guaranteed to be a sure thing.

[CALSGR8]

Didn't Tom Boswell write an article on Draft Pick Pitchers?  I'll see if I can find the link.

http://voices.washingtonpost.com/nationalsjournal/2009/05/final_word_on_top-10_pick_pitc.html