nice post JCA. It seems to me RE24 takes into account the value of "situational hitting" and no other stat really does (except WPA, but WPA seems to be too dominated by extremely high leverage situations...)
From a comment on the fangraphs article you mentioned: "I grabbed the 778 batters with 5000+ PA from 1974 to 2012; here are the leaders and trailers in RE24/600PA – wRAA/600PA:
Jose Cruz +12.4
Terry Pendleton +11.2
Ryan Klesko +11.0
Edgardo Alfonzo +10.4
Steve Garvey +10.1
Carl Everett +9.7
Darryl Strawberry +9.6
Mike Piazza +9.4
Tony Gwynn +9.3
Derek Bell +9.2
…
Brook Jacoby -7.0
Chet Lemon -7.1
Tom Brunansky -7.3
Ivan Rodriguez -7.4
Lance Parrish -7.4
Rick Dempsey -7.6
Neifi Perez -7.6
Jason Varitek -7.9
Jim Rice -9.6
Rick Burleson -9.7"
This seems to suggest that RE24 is capturing something important other stats (most notably RC or WAR) are missing out on.
Like any counting stat you can always start by looking at RE24/PA to improve comparisons. But you also make an excellent point (one that I'd thought of before) that position in batting order and quality of players both before and after you also matters. If your coming up with 2 outs nobody on all the time there isn't much you can do to add a lot of RE.
BTW for relievers I do like RE24 quite a lot. For starters it will very closely mirror runs allowed (earned and unearned) if I understand the stat correctly (I believe it just gives full credit to any outcome to the batter, meaning errors on the defense, or extremely good/bad baserunning by the guys already on base all gets credited to the batter). So ya I think it's a great stat for relievers, for starting pitching I'm thinking it's not all that useful. (Altho I would comment that if situational batting exists, one would assume so does situational pitching and hence some guys will always out/under-perform some of the advanced pitching stats that don't take into account situation)
