I attended tonight's game (unfortunately). I saw something that spoke volumes about how Milledge's defense has been so far this season.
In the process of Milledge making a catch relatively early in the game, a glove fell out of his back pocket. Afterward, as he walked back into position, Pena picked up the glove at tossed it to him. Milledge dropped it. To me, that pretty much typifies his season, as far as his defense is concerned.