Measuring Situational Hitting

Situational Hitting

Last week, I gave a talk about situational effects in baseball and the most interesting or most talked about situation is “clutch”.  That is, does a player have the ability to perform better in clutch situations in baseball?  I’m not as interested in the answer as I am in the methods that one might use to explore this question.

Anyway, Nate Silver in his baseball analysis days wrote an interesting article in the Baseball Behind the Numbers book that was reprinted in ESPN that explored this issue, focusing on David Ortiz’s 2005 season where he had a number of clutch performances.  My intent here is to describe an alternative simulation-based method of detecting clutch ability, or as Nate would say “smart situational hitting.”

Assessing Situational Hitting by a Permutation Test

I illustrate this method for Ortiz’s 2005 season.  In this particular season, Ortiz had 713 plate appearances.  If we remove some situation-dependent events such as sacrifices, we have 708 plate appearances that consists of walks/hit-by-pitches, singles, doubles, triples, home runs, outs.  These 708 PA’s happened during different runners on base and outs situations.  The question is — is there a relationship between the PA outcomes and the game situations?  That is, is there evidence that Ortiz was able to get his hits in situations that would produce more runs?

Here’s the setup to my method:

1.  First I created a spreadsheet listing all of the runner/outs situations (there are 24 of these) and all of the possible PA outcomes (the 6 listed above) — there are 144 = 24 times 6 rows in the table.
2. For each situation (variable STATE) and PA outcome, I created a new situation variable (called NEW.STATE) based on the outcome.  I used simple runner advancement rules here where a single would advance all runners two bases and a double would advance runners three bases.  I could be more accurate here using actual runner advancement distributions, but I don’t think these assumptions will have a big impact on the results.
3. Using the season’s run expectancy table, I found the runs expectancy before and the runs expectancy value after the PA and computed the runs value of the outcome using the usual “RUNS.VALUE = runs after – runs after + runs scored” recipe.Here is a snippet of this table — I add a runs value column to this table using the season runs expectancy matrix.

   

4. Using this runs value table, I can compute the total runs value of Ortiz’s 708 PA’s in the 2005 season  — I got a value of 68.86 runs.  Remember this is not quite the actual runs created since I am ignoring the actual “after” situations and basing this calculation on my runner advancement model.

If the situation has nothing to do with the outcome, then the 708 factorial ways of permuting the outcome arrangements are all equally likely.  This motivates the following simulation (essentially it is a so-called permutation test).

1.  Randomly mix up the sequence of 708 PA outcomes (using the R sample() function).
2. Use the runner advancement model, compute the runs values of these 708 outcomes.
3. Repeat this simulation 1000 iterations, collecting the (simulated) season run values.

What Do We Find?

Below I show a histogram of these simulated runs and show Ortiz’s value of 68.86 as a vertical line.

So Ortiz was clutch in the sense that he produced more runs than one would expect if the situations were independent of the outcomes.  The right-tail probability (p-value) here is about 0.04 which indicates that he was “significantly” clutch for the 2005 season.  Nate Silver also concluded that Ortiz was unusually clutch this season.

But the obvious follow-up question is “Was Ortiz consistently clutch in his career?”  I went ahead and repeated this work for each of the seasons 2000 through 2016.  For each season, I computed two numbers:  R, the actual number of runs Ortiz produced, and M, the mean number of runs using the “situation is independent of outcome” model.  Below I show the values of R in black and the values of M in red.  Generally both the black points and red points show the same pattern, showing Ortiz’s great rise as a hitter up to the 2005 season.

The next graph focuses on the clutch issue by plotting R – M against season, where R – M is the number of additional runs Ortiz got by clever situational hitting.

We see that Ortiz was most clutch in 2005, getting 13 additional runs, but this was unusual.  Note that the values of R – M fluctuate around zero indicating that there is little evidence that Ortiz was consistently clutch through his career.

Where to Go From Here?

By the way, last summer I talked about this same general approach in measuring “cluster luck” in scoring runs.

In his article, Nate Silver speculates that good “situational” hitters can adjust their hitting to the situation, namely the hitters with good bat control who rarely walk.  (I don’t think David Ortiz fits in this category.)  It would be interesting to do a study that more thoroughly  explores this conjecture.

R Work?

I have a R markdown file that implements this permutation test procedure for David Ortiz and Derek Jeter.  (Spoiler alert:  There is little evidence that Jeter had clutch hitting ability using this method.)  Here I’m using a single data frame that contains 17 seasons of Retrosheet play-by-play data.