UZR ratings are out

[JMUalumni]

Johnny Damon having a high UZR thus far indicates that he has played great defense in a small amount of games, just like Pudge hitting over .400 thus far indicates that he has been hitting very well thus far in the season.  Just because his UZR number is high now, though, doesn't mean that he is a good defender, just that he has been playing good defense.  Meaningless?  Not fully.  The stats tell a story and you can't deny it if you want, but it doesn't mean they aren't indicating something that is true.  I don't expect Damon to continue to play at the same level, just like I expect Zimmerman's numbers to trend upward as the season continues.  But using the UZR numbers to make a case that Zimmerman's hamstring issues caused his defense to dip a little bit in the early part of the season is quite valid and not very meaningless in my opinion.

EDIT:  Well said JCA, you beat me to the punch.

[PANatsFan]

The other point is that UZR is vs an average player. There have only been 12 games this year, but lots of games in history to develop the numbers for the average player.

[JMUalumni]

There are some changes in this year's UZR.  Fangraphs will also be adding splits this year, which is pretty cool.

The first UZR updates of the 2010 season are in, and from here on out they’ll be updated every Sunday night.

There have been a few improvements made to UZR this year, which will also be reflected in prior year’s UZR data. The changes do impact a few players, but for the most part, each player’s UZR has remained unchanged or is within a couple runs of what a player was rated before the improvements. Mitchel Lichtman, the man behind UZR, outlines the changes below:

Park factors have been improved, especially for “quirky parks and portions of parks,” such as LF and CF at Fenway, LF in Houston, RF in the Metrodome, and the entire OF in Coors Field. Of course, park factors in general are updated every year, as we get more data in each park, and as new parks come into existence and old parks make material (to fielding) changes.

In the forthcoming UZR splits section, we will also be presenting UZR home and road splits, as a sanity check for those of you who are skeptical of park factors. Please keep in mind that regardless of the quality of the park adjustments, there can and will be substantial random fluctuations in the difference between home and away UZRs and it is best to evaluate a fielder based on as much data as possible (e.g., using home and road stats combined), as we do with most metrics and statistics.

Adjustments have been added to account for the power of the batter as a proxy for outfielder positioning, so that, for example, if an outfielder happened to have “faced” a disproportionate percentage of batters with less than or more than average power, the UZR calculations will make the appropriate adjustments (as best as it can). Obviously, these kinds of adjustments are more important for smaller samples of data than for larger samples, since, in larger samples, these kinds of anomalies (in terms of opponents faced) tend to “even out.”

For infielders, similar adjustments are made for the speed of the batter, as a proxy for infielder positioning and how quickly the infielders have to field and release the ball, as well as the speed of the throw.

When a “shift” is on in the infield, according to the BIS stringers, if the play was affected by the shift, the UZR engine ignores the play. As well, if an air ball hits the outfield wall and in the judgment of the BIS stringers, no outfielder could have caught the ball, the play is similarly ignored.

Also keep in mind that UZR does not include first basemen “scoops” or the ability of the first baseman to influence hits and errors caused by errant throws from the other infielders. According to my (MGL) research, yearly “scoops” numbers are generally in the 1-4 run range, which means that the true talent range of most first basemen with respect to “scoops” is probably in the plus or minus 2 runs per year range – i.e., not much.

http://www.fangraphs.com/blogs/index.php/uzr-updates

[JCA-CrystalCity]

BTW - Jason Bay went from one of the worst LFs last year to a plus 2 after a change taking out uncatchable balls off the Fenway walls.  Ellsbury also improved a lot.

One of the things I like about Dunn at first is the big target and his apparent scooping ability.  these will still not show in Fangraphs UZR but I think are real aspects of his first base D.  I think the big target gives confidence to FotF and Desmond just to throw it rather than aim it like our IFs had to do with Belliard, Young, or even Nick in his injured state the past few years. This should be reflected in Desmond's and RZ's UZR.

[blue911]

BTW - Jason Bay went from one of the worst LFs last year to a plus 2 after a change taking out uncatchable balls off the Fenway walls.  Ellsbury also improved a lot.

One of the things I like about Dunn at first is the big target and his apparent scooping ability.  these will still not show in Fangraphs UZR but I think are real aspects of his first base D.  I think the big target gives confidence to FotF and Desmond just to throw it rather than aim it like our IFs had to do with Belliard, Young, or even Nick in his injured state the past few years. This should be reflected in Desmond's and RZ's UZR.

I told you UZR was a bunch of crap, but would you listen to me?

[JMUalumni]

I told you UZR was a bunch of crap, but would you listen to me?

From all the articles I've read on the subject, most people agree that UZR is better for OFers while the Plus/Minus is much better for infielders.  In the absence of a better metric, it is a raw and dirty way to get a quick read on a player.

[blue911]

From all the articles I've read on the subject, most people agree that UZR is better for OFers while the Plus/Minus is much better for infielders.  In the absence of a better metric, it is a raw and dirty way to get a quick read on a player.

Better than what? To use bad data doesn't give you an accurate picture what-so-ever.

[KnorrForYourMoney]

I told you UZR was a bunch of crap, but would you listen to me?

I basically pay no attention to fielding stats, and thus this hot new "WAR" thing.

[JMUalumni]

Better than what? To use bad data doesn't give you an accurate picture what-so-ever.

I disagree that it is bad first of all, but will admit that it is extremely raw still (but improving each year).  I didn't say accurate at all, though, I just said quick read.  Is it completely wrong?  Absolutely not.  Using it as the crux of an argument is a different story.

[blue911]

I disagree that it is bad first of all, but will admit that it is extremely raw still (but improving each year).  I didn't say accurate at all, though, I just said quick read.  Is it completely wrong?  Absolutely not.  Using it as the crux of an argument is a different story.

I think we are now talking semantics. Raw vs Bad. It doesn't include all of the factors you would use to rate a player if you saw them on a daily basis. The idea of statistical analysis is to provide the reader with as good a picture as possible. UZR admits that it doesn't include parts of the game (arm strength notably) because they can't figure out how to work it into their formula.

[JMUalumni]

I think we are now talking semantics. Raw vs Bad. It doesn't include all of the factors you would use to rate a player if you saw them on a daily basis. The idea of statistical analysis is to provide the reader with as good a picture as possible. UZR admits that it doesn't include parts of the game (arm strength notably) because they can't figure out how to work it into their formula.

Ok, fair enough.  I'm like the idea of UZR, but have come to recognize its flaws over the last year or so.  But, the development of good statistics takes time and discussion in the public forum.  They improve the statistic every year and I think within the next decade they could have a pretty useful stat on their hands.  Completely writing it off would do no good, though, and we would be stuck with lame ass crap that we already have.

Part of using any statistic is recognizing what it leaves out and what it is telling you.  Unfortunately, most people misuse nearly every statistic to make skewed arguments.  But, I do think UZR has the most promise out of any of the fielding statistics out there right now.

[blue911]

Ok, fair enough.  I'm like the idea of UZR, but have come to recognize its flaws over the last year or so.  But, the development of good statistics takes time and discussion in the public forum.  They improve the statistic every year and I think within the next decade they could have a pretty useful stat on their hands.  Completely writing it off would do no good, though, and we would be stuck with lame ass crap that we already have.

Part of using any statistic is recognizing what it leaves out and what it is telling you.  Unfortunately, most people misuse nearly every statistic to make skewed arguments.  But, I do think UZR has the most promise out of any of the fielding statistics out there right now.

I think most of the newer stats are better than the old crap. But I've never been that big on any stat, maybe that's why I tend to watch a lot of teams. So I can see the players and not have to go solely on some arbitrary stat. 

[JMUalumni]

I think most of the newer stats are better than the old crap. But I've never been that big on any stat, maybe that's why I tend to watch a lot of teams. So I can see the players and not have to go solely on some arbitrary stat. 

Good statistics are no substitute for good observation, no doubt.

[blue911]

Good statistics are no substitute for good observation, no doubt.

I guess my main nag about UZR is that it wont incorporate an outfielders arm. Everybody should know that Johnny Damon can still run but his arm is terrible. How many times a year is his better range going to overcome his lack of throwing ability? Not very often would be my answer as he'll have 400 times where he'll need to get a live ball back into the infield but only 20 times where his range will come into play.

[JCA-CrystalCity]

I told you UZR was a bunch of crap, but would you listen to me?

Funny, but your point is the similar to what a stat geek over on SoSH's reaction to this news.  He was happy because he thought UZR was being used too much, without understanding, in the press and online.  for what it is worth, very few numbers changed much, except for fielders in weird ballparks.

I think I've gotten a heck of a lot more cautious about using it over the past year or so as I've gotten to understand what it leaves out, what it reflects, and what it takes to be meaningful.  For example, I think I questioned a lot of the "Dunn can't field at first because of his UZR/150" stuff that has been out there, precisely because of the scoop / target / small sample size problem with UZR, observation of his play there once his move was permanent, and the trend in his UZR once the move was permanent.

[JCA-CrystalCity]

Blue - it does incorporate an arm measure.  Look at the far left of his "advanced fielding" stats.
http://www.fangraphs.com/statss.aspx?playerid=185&position=OF

[blue911]

Funny, but your point is the similar to what a stat geek over on SoSH's reaction to this news.  He was happy because he thought UZR was being used too much, without understanding, in the press and online.  for what it is worth, very few numbers changed much, except for fielders in weird ballparks.

I think I've gotten a heck of a lot more cautious about using it over the past year or so as I've gotten to understand what it leaves out, what it reflects, and what it takes to be meaningful.  For example, I think I questioned a lot of the "Dunn can't field at first because of his UZR/150" stuff that has been out there, precisely because of the scoop / target / small sample size problem with UZR, observation of his play there once his move was permanent, and the trend in his UZR once the move was permanent.

After watching Giambi play first, Dunn looks like Keith Hernandez to me.

[blue911]

Blue - it does incorporate an arm measure.  Look at the far left of his "advanced fielding" stats.
http://www.fangraphs.com/statss.aspx?playerid=185&position=OF

The ARM rating looks like an attempt by fangraphs, but isn't listed as one of the improvements to UZR. I would like to see the methodology for the arm rating.

[Nick the Pig]

Saying a dozen games of UZR does not reflect what a guy has done in that dozen games is a bit like saying a home run total for a dozen games does not reflect the guys home runs in that period.  You might be crazy if you said Alex Gonzalez is a better power hitter than Adam Dunn because he has 4 home runs over a dozen game stretch while Dunn has one, but if the question is who has more home runs over his first dozen games, the correct answer Alex Gonzalez.  Substitute Zimmerman for Dunn and Damon for Gonzalez, and you have the same point.  Zimmerman has not made plays that an average 3b would make over this stretch, while Damon has made plays that an average LF would not make (given the historical records of fielding).

That's part of my disagreement, I think.  Zimmerman, to my eyes, has been very good defensively this year.  Maybe not as good as he's capable of being, nursing the hammy, but still well above average.

I'm no expert on UZR.  But it seems to me that a lot of it is based on chances.  I other words, if opponents have been hitting a lot more balls over the outfielders heads this year that anywhere that Zimm could possibly get a glove on it (which seems to be the case) then his UZR might be low through no fault of his own.

As a side note, can anyone who follows UZR closely answer a question:

How did Zimmerman's 2009 UZR (and UZR/150) change from the ~18 range to ~13 ?  WHen I looked at it in the offseason, it was 18.  Now it's 13. 

[JCA-CrystalCity]

Nick - there was a change in the methodology but i don't think it affected infielders.  They do recrunch the numbers at the end of the year. 

I am not sure, but I believe the recrunch is based on totaling up fielding for the past year (2009) across the majors rather than basing it in comparison to how fielders did in the prior year (years).  I recall reading a number that UZR went down in the offseason recrunch because fielding improved across the board in the majors (more teams emphasizing gloves) so what an average player did in 2009 was better than before, but that seems like a big move.

Balls going over outfielders heads would not affect RZ because those plays would be outside the zone of play for which a 3d baseman is responsible.  What I am guessing happened with RZ is that a few misplays / missed chances, even one, have / has a lot of weight at this point in the season for UZR/150 because he has had so few chances.  I do remember him not getting to a DP ball in one of the slaughter games that I thought he would ordinarily get.  If that was a ball an average 3d baseman would have come up with, that may be some of the difference right there.  he had a great DP either last night or the night before, but that was his first classic RZ play since he came back from the hammy.

[Nick the Pig]

Nick - there was a change in the methodology but i don't think it affected infielders.  They do recrunch the numbers at the end of the year. 

I am not sure, but I believe the recrunch is based on totaling up fielding for the past year (2009) across the majors rather than basing it in comparison to how fielders did in the prior year (years).  I recall reading a number that UZR went down in the offseason recrunch because fielding improved across the board in the majors (more teams emphasizing gloves) so what an average player did in 2009 was better than before, but that seems like a big move.

Balls going over outfielders heads would not affect RZ because those plays would be outside the zone of play for which a 3d baseman is responsible.  What I am guessing happened with RZ is that a few misplays / missed chances, even one, have / has a lot of weight at this point in the season for UZR/150 because he has had so few chances.  I do remember him not getting to a DP ball in one of the slaughter games that I thought he would ordinarily get.  If that was a ball an average 3d baseman would have come up with, that may be some of the difference right there.  he had a great DP either last night or the night before, but that was his first classic RZ play since he came back from the hammy.

So if an infielder, in theory, never had a ball hit into his designated zones, his UZR over the course of a season would be what - 0.00 ?

Something's definately fishy with Zimmerman's 2009 UZR.  At the end of last season, it was ~18.  He was right there neck & neck with Longoria.  I remember it well because I had a debate with someone on another board as to which one of them was the better fielder.

A couple of months (maybe even weeks) ago, it was still ~18.  When I look at it today, it's now 13 for the 2009 season.  If there was an across the board adjustment, you'd think that Longoria's 2009 #'s would have changed, too.  But they are still at 18, same as they were during the offseason.

[ronnynat]

I've always wondered how often during the course of a season fielders have to shift away from their normal spots on the field. 10% of the time? More?

[JCA-CrystalCity]

Nick - I posted a question to Mitchel Lichtman (MGL) on this Fangraphs thread. I hope he responds.
http://www.fangraphs.com/blogs/index.php/uzr-updates

Some folks on a Nats message board picked up a big drop in Ryan Zimmerman’s UZR relative to Evan Longoria. People’s recollection is that he was +18 or so at the end of last year and comparable to Longoria. Now he is +13 and 5 runs worse. Is that right? What would drive that sort of change? Did Zimmerman field chances from a lot of slow hitters?

[Nick the Pig]

Nick - I posted a question to Mitchel Lichtman (MGL) on this Fangraphs thread. I hope he responds.
http://www.fangraphs.com/blogs/index.php/uzr-updates

Thanks, JCA.

I read their reply over there.  While you were doing that, I had emailed them about the same thing.  Here's the exchange:

David Appelman: "Please see this post here:  http://www.fangraphs.com/blogs/index.php/uzr-updates


Also, please see this post here if you're interested in how much things changed (which was not much)


http://www.hardballtimes.com/main/blog_article/out-with-the-old-uzr-in-with-the-new/"

Me: "Thanks, David.  I'll check them out.
 
I will be looking with a skeptical eye, as an adjustment that takes away 5 defensive runs from Zimmerman doesn't pass the smell test.
 
The comparison between Zimmerman and Longoria's 2009 defense was discussed on FanGraphs during the offseason:
 
http://www.fangraphs.com/blogs/index.php/is-ryan-zimmerman-probably-better-defensively-than-evan-longoria

Note Zimmerman's UZR for 2009 at that point.  And the fact that Dewan had him rated higher in plus/minus."

David Appelman: "Well, that's why we display multiple metrics.  UZR still thinks Zimmerman since 2008 is one of the four best defensive third basemen in the league.  DRS thinks he's been one of the top 3 defensive thirdbaseman in the league.


It's not like UZR changed its opinion of Zimmerman and if you really think you can tell by watching a player whether he's a +13 or a +18 player, well then kudos to you, but when it comes to UZR they're both considered very good players.  Which one is actually better at this point I think is pretty unclear."

Both of those responses basically say "13 is still really good.  Don't worry about it"

And that might be true.  But a 30% change for a player in a particular metric deserves a better explaination.

[JCA-CrystalCity]

I guess the point here is that +/-5 runs is mostly noise in UZR. I've read that, too, over there, so I'm not thinking Appelman is making something up just for this case. I think what this shows is that while maybe it is one of the best systems out there in the public domain, and an upgrade over officials scorers (e.g., "if he did not touch it, he can't get an error," "no one else would have reached it, but because he reached it and made a bad throw, he gets the error"), it should be treated as being bit fuzzier than how it is generally reported, especially over a week, month, or season.