How is Brad Ausmus managing the Tigers' bullpen?

We can evaluate a manager's bullpen management with two main statistics: Win Probability Added (WPA) and Leverage Index (LI). Win Probability Added is rather intuitive, and it is best explained with a simple example. Let's say Joba Chamberlain is pitching, and the Tigers currently have a 50-percent chance of winning. Then, suppose Chamberlain gives up a homerun, and the team subsequently has a 35-percent chance of winning. For that event, Chamberlain's actions led to the Tigers having a 15-percent less chance at winning, so he is given -0.15 WPA.

Then, let's say Al Alburquerque is summoned from the bullpen, and he strikes out the next batter. If the Tigers then have a 37-percent chance of winning, Alburquerque gets credited with 0.02 WPA.

In other words, a player's WPA depends on the situations his manager puts him in. For regular starters, there isn't much variance here since they play at the beginning of the game, in tight games, and in blowouts.

But for relief pitchers, that's not true. A manager decides whether to use his long reliever in a 16-0 blowout, his best pitcher in a one-run game, or his closer with a three-run lead.

Leverage Index tries to capture how important any given moment in a game is. Intuitively, a 16-0 blowout in the bottom of the ninth has a much lower leverage than a tie game in the ninth. Leverage index is related to WPA in that it measures the possible change in win expectancy at any moment.

Then, we subtract context-neutral WPA in order to just leave the context from WPA. (Yes, this is where it gets confusing.) We arrive at context-neutral WPA by dividing WPA by pLI, the average leverage index for the player.

Let's take a look at the data, sorted by WPA/LI. Keep in mind that mean leverage index is defined to be 1.0, but the median LI will actually be below 1.0.

There’s a number of insights we can take from this dataset:

• Alex Wilson has been very good in low-leverage situations. He deserves a chance for more meaningful innings.
• Unsurprisingly, Joakim Soria has pitched in the highest leverage innings. He deserves those innings.
• Blaine Hardy, like Wilson, has excelled in spots where excellence was not exactly necessary.
• Despite the pristine 1.13 ERA, Joba Chamberlain has overall lowered his club’s chances of winning after he exits the eighth inning that has been handed to him.
• Angel Nesbitt has been the Tigers’ worst reliever, and he can use the quiet car ride to Toledo as a time to sit and think about what he’s done.

Brad Ausmus has been bad at managing his bullpen.

Why? Take a look at the far right column. There are only two values that are significantly greater than zero: Soria and Krol. Soria has been very good in his closer role, but any manager would know that Soria is the club’s best reliever. Krol, meanwhile, has been bad in unimportant innings. By assigning Krol low-leverage innings instead of high-leverage innings, Ausmus has won his team 0.7 wins, despite the fact that Krol has been worth -0.2 fWAR on the year.

The rest of the values are either around zero or negative. Wilson, Hardy, and Alburquerque are all around -0.3. Since Ausmus has used these good relievers in low-leverage situations, he has cost his team around one win in total just in his usage of these three players. Among all relievers used this year, Ausmus has accumulated 0.5 WPA.

Now, keep in mind that zero WPA here is not league average, nor even replacement-level managing. Zero is the equivalent of picking a name out of a hat, throwing a dart at a wall, or asking a magic 8-ball to determine who will be summoned from the bullpen.

Most managers accumulate around 2.0 WPA with their bullpen management over the course of the season. Some managers, such as Bob Melvin of the Oakland A’s and Davey Johnson of the Nationals, contributed around five WPA for their teams in 2012. Those two teams surprised the nation and made the playoffs that year, a feat they both likely would not have achieved if not for their managers’ efforts.

The Tigers have only played 60 games in 2015, so if Ausmus keeps his current pace, he’ll finish with 1.35 WPA with his bullpen management. That’s below average, but not shockingly-so.

But if you ignore the 0.87 WPA Ausmus’s closer has given him, Ausmus sits at -0.37 WPA from the rest of the bullpen. All of the sudden, the artificial construct and manager’s crutch that is the closer’s position doesn’t seem so confusing – when Ausmus hasn’t been handing the ball to his closer, he would have been better off using a random number generator to determine who to bring into the game.

I’m not calling for Ausmus’s job. He’s highly intelligent, even-keeled, and young. Ausmus simply needs to realize that Chamberlain and Nesbitt shouldn’t be entrusted with the eighth inning. His second, third, and fourth best pitchers are instead Alex Wilson, Blaine Hardy, and Al Alburquerque.