You may be onto something there - some past Fourth of July trips home from Nats matches have been real ordeals, especially if the opponent is a big draw like the Cubs (as has specifically happened).
I've got no problem with the 11.05 first-pitch myself - I rather enjoy day baseball.
For me the worst July 4 Metrorail trip was coming home from the fireworks in 1994 (I remember the year because the World Cup was here). The train in front of us broke down when we were under the river approaching Rosslyn and we sat for around 90 minutes.
As I said before, I doubt concern is so much with crowding per se as it is with assisting law enforcement in MANAGING the crowds. It's difficult enough as it is on July 4, especially when you factor in that there are a lot of people who seldom (or never) ride the Metrorail, coupled with stop closures (Smithsonian) and train re-routings and the like. Adding the baseball crowd in late afternoon, especially when a lot of the baseball crowd will want to change trains downtown at the same stops where fireworks-goers are trying to exit, surely can't help matters (and let's be realistic, most Metrorail riders are not savvy enough to go to, say, Archives to change to a Virginia-bound Yellow Line train instead of jamming in at L'Enfant).