Detroit Tigers Analysis

Saves are bunk, but better than losses

Are relievers struggling this year, or are saves nothing more than random variation?

By John A McGeehan May 30, 2013, 9:40am EDT

Melissa linked to an article by Tom Verducci at SI.com which makes the case that while many relievers are struggling this year, this is normal. Let's dig a little deeper into the numbers.

No I do not believe that saves are a worthwhile statistic. But the "game" does. Verducci makes the point that it is strange how every manager anoints a closer to start the 9^th inning of games with a save situation. The Red Sox have Bill James in the front office and tried a different approach a few years ago. But when they had a few blown saves, they reverted to having a closer. Even Joe Maddon consistently uses Fernando Rodney as his closer, with 10 saves and 5 blown saves this year. And Joe Maddon lives to go against the grain. When we judge Jose Valverde, is not part of the discussion that he is six for seven in save opportunities this year?

In the past five years, teams have averaged 40 saves and 19 blown saves. Both numbers came as a surprise to me. Teams average 81 wins, so in half of the wins there is a save situation. And 32% of save situations do not result in a save. Would you have guessed that almost one third of saves are blown? Fernando Rodney has blown 33% of his saves this year and is considered to be struggling, but is posting average results.

If these numbers scare you, jump to the conclusion right now. Because it is time for more math. I created a spreadsheet with 30 teams of 60 games each, representing a full season of save situations. I assumed every team had the same "average" closer, successful in 68% of save situations. Admittedly the average closer may be better than this, as occasionally another reliever is used in a save situation. On the other hand, the closers may not perform better as a group than any other reliever. There are many more caveats, but back to the spreadsheet. I had each of the games result in a save or a blown save based on nothing but a random result, always compared to this fictional "average" closer. Think of this like rolling a six-sided die and a result of one, two, three, or four being recorded as a save, while a five or six is recorded as a blown save. Needing 1800 rolls of the die to simulate a season, I am glad we have computers. This model ignores all real life variations - the closer's command is a bit off, his fastball a bit slower, he is dealing with personal issues. In real life the variables are endless.

I ran a full season for every team, and what did I find? On the top end, the best the average closer did was to save 80% of the chances, or 48 out of 60. Remember, this is strictly a random event. Perhaps the computer was thinking that he was used more often with three run leads than with one run leads and a runner on second base. Perhaps that season balls were hit right at fielders more often than normal. We are only talking about 60 innings here, and many pitchers can be good for 60 innings. The worst result? One poor guy blew 26 saves, more than twice the best guy. Except that in the model, the best and the worst closer have exactly the same talent level. The one who blew 26 saves in the computer would never have 60 opportunities in real life, as he would lose his closer role before then. But he has the same inherent ability as the one with the best result, who would land a big contract.

In this model, the best result was 80% success. Clearly there are closers who regularly exceed this. Mariano Rivera has 626 saves in his career, and only 40 blown saves, for a 94% success rate. He is an elite talent, not just an average reliever who was lucky for a stretch.

I changed the closer in the model to one well above average, with an 85% success rate, and simulated him pitching a season for each of the 30 teams. In one instance the closer blew five saves in a row! All we are talking about here is something like rolling a one on a six-sided die five times in a row. Yahtzee! If you play enough games, it happens. And in another instance, this same closer with a "true talent" of 85% success saved 59 out of 60 games. He would be labeled "elite" and score a huge payday. Or maybe he was Jose Valverde in 2011.

The real game is not random, and players constantly make adjustments. See Jeff Sullivan's article at FanGraphsabout Rick Porcello to appreciate adjustments that pitchers make. This vastly simplified model supports Verducci's claim that many closers can look great one year and struggle the next. The saves statistic is a lousy way to decide who should pitch at the end of close games. The sample size is too small, and random variation can provide a wide range of results. Teams need to look to other measures to judge who should preserve the win.