Baseball Reference WAR: A Response
For some reason I don't understand, my friend Sean Foreman's response to my Baseball Reference WAR piece went into spam. Sean is president and founding partner of Baseball Reference and someone who has been hugely helpful to me through the years. And he makes his expected excellent points here.
I have pulled his response out here for a full reading.
* * *
I would have been happy to comment on the record for this post. I’m on vacation, so this is a bit rushed.
I’m not sure what you are asking of us here. We state clearly that we don’t find differences of 1-2 wins to be definitive. We also break out each and every component of the stat so users can take that into account. Should we require user training before viewing our stats?
You are correct that the only real difference in WAR between the two pitchers in WAR is on the defensive adjustment.
Then the question becomes the following. While JoeP believes that the Red Sox were a “much, much, much” better defensive team than the Tigers overall, Joe believes that the Tigers were then also way better for Verlander than for their other pitchers.
Maybe it’s true that the Tigers were above average fielders when Verlander was on the mound, maybe not, but keep in mind BIS had the Sox at +59 and the Tigers at -49 for the season. How on earth does a team that’s the 3rd worst defensive team transform itself into an above average defensive team for the 228 innings JV was on the mound given they’d then have to be EVEN worse the other 1200 innings to get to -49?
If we assume for the moment the Tigers were in fact “excellent” behind Verlander, then the question becomes how to handle this. We apply the team’s DRS to each pitcher based on the percentage of the team’s balls in play which in probably 95% of the cases is a good way to do this and may still be in Verlander’s case. If you start to dice things up by the pitcher on the mound you then run into very small samples where something like Mookie Betts pulling back a home run and getting a double play has a dramatic impact on the pitcher’s WAR. I don’t think you’d like the alternative as you run the risk of conflating random variance with real performance differences.
Joe seems to agree that defense aside that Porcello and Verlander were equals. I think it is a huge leap to then say that they were still equal when considering the likely impact of the team defense given what we know. I’m very comfortable saying that Verlander’s performance was more valuable than Porcello’s, though AS ALWAYS the amount of difference is up for debate.
If we find there is an issue with our defensive numbers, we’ll look at making a change. We are always trying to improve.