Visualizing Variability in Performance Outcomes with HR Rate

Bryce Harper had a good season. No, that doesn’t need to be said, but it lays the foundation of thinking about the following question: What sort of variability in home runs might we expect from Bryce Harper if we assume the observed HR production is his true talent level?

This is, of course, an overly simplistic question. And this will be an overly simplistic blog post. But it begins to lay a foundation for developing projection systems, and just understanding uncertainty and variability in general in performance outcomes. Today, I’ll exhibit some simple concepts about variability in performance, even when true talent is assumed to be known. Ben showed how to do projections like Marcel a while back, and here we will take a step back from that and just think about the uncertainty inherent in the game that lays the foundation in projection systems that we see at the beginning of each year.

So let’s begin with a simple, and very likely incorrect, proposition: we knew that Bryce Harper’s true ability in hitting home runs after seeing his 2012 numbers, and that this is fixed. In this proposition, we implicitly say that the probability that Bryce Harper hits a home run in any plate appearance is his home runs divided by his number of plate appearances in 2012. Then, we can draw from a binomial distribution using this probability over 700 trials. We do this for 10,000 seasons, and take a look at how likely it would be for Harper to hit is 42 home runs in 654 plate appearances if Harper’s talent level was truly his 2012 home run rate.

###define true home run rate as 2012 rate
pHR12 <- (22/597)
dist12 <- rbinom(10000, 654, pHR12)
length(subset(dist12, dist12 < 41))
[1] 4

We can see here that out of 10,000 seasons, Harper would hit 42 or more home runs in 654 plate appearances only 4 times, and 3 of those would be exactly 42 home runs. Now let’s look at the full distribution of his possible outcomes, given this true talent level.

setwd("c:/Users/Mills/Dropbox/Baseball With R")
png(file="2012Talent.png", height=500, width=600)
plot(density(dist12), lwd=2, col="darkgreen", main="Bryce Harper 654 PA Home Run Density", 
    xlab="Home Runs", ylab="Probability", xlim=c(0, 70), ylim=c(0, .1))
abline(v=pHR12*650, col="darkgreen", lty="dashed")


Given all of this, it seems rather likely that Harper’s true ability to hit home runs has changed since the 2012 season, or at least that the output we observed wasn’t his true talent level, but a draw from some unknown distribution that included hitting 22 home runs in 597 plate appearances. This seems somewhat obvious, but it’s not often we see a breakout like Harper’s. And we’re of course ignoring improvements in strikeout and walk rates. But let’s keep focused on home run rate.

We have another possibility: Harper’s true talent level is 42 home runs in 654 at bats. If we assume that this is his true home run talent, we can then calculate the distribution of expected home runs based on this number. With that, we take a look at the probability that he has seasons with as few home runs as 2012, 2013, and 2014–again, taking plate appearances as given–despite such a high true home run talent level.

###home runs per game
pHR15 <- (42/654)
dist12b <- rbinom(10000, 597, pHR15)
length(subset(dist12b, dist12b < 25))
[1] 25
length(subset(dist12b, dist12b < 25))/10000
[1] 0.0025

dist13b <- rbinom(10000, 497, pHR15)
length(subset(dist13b, dist13b < 21))
[1] 120
length(subset(dist13b, dist13b < 21))/10000
[1] 0.012

dist14b <- rbinom(10000, 395, pHR15)
length(subset(dist14b, dist14b < 15))
[1] 64
length(subset(dist14b, dist14b < 15))/10000
[1] 0.0064

png(file="2015Talent.png", height=500, width=600)
plot(density(dist12b), lwd=2, col="darkgreen", main="Bryce Harper Yearly Home Run Density (2015 True Talent)", 
    xlab="Home Runs", ylab="Probability", xlim=c(0, 70), ylim=c(0, .1))
abline(v=pHR15*597, col="darkgreen", lty="dashed")
lines(density(dist13b), lwd=2, col="darkred")
abline(v=pHR15*497, col="darkred", lty="dashed")
lines(density(dist14b), lwd=2, col="darkblue")
abline(v=pHR15*395, col="darkblue", lty="dashed")
legend(0, 0.1, c("2012 PA", "2013 PA", "2014 PA"), col=c("darkgreen", "darkred", "darkblue"), lty=c("solid", "solid", "solid"), lwd=c(2, 2, 2), bty="n", cex=1) 


Here, we can see that the probability of him hitting 22 or less home runs in 597 plate appearances as he did in 2012, assuming that his 2015 home run rate is his true talent level, is only 0.25%. There is also a rather low probability of Harper hitting 20 home runs in 497 plate appearances in 2013, or 13 home runs in 395 plate appearances in 2014, assuming his 2015 output is his true talent. This helps us think about whether we think his talent is changing, or we’re well within some expected distribution of output, given his fixed skill.

Given his age, it seems likely that Harper’s talent has changed over time. There are a few ways to think about this. First, it could be simply that his true talent changed and is as observed in 2015. Alternatively, it could be that it both changed, and he had a lucky year. Lastly, there is the possibility that Harper’s HR talent hasn’t changed at all. While unlikely, we could take his entire career home run per plate appearance rate and see how likely it would be for him to hit 42 home runs in 654 plate appearances again.

###average of all 4 years, probability of 42 home runs in 2015
pHRall <- (20+22+13+42)/(597+497+395+654)
dist12c <- rbinom(10000, 597, pHRall)
dist13c <- rbinom(10000, 497, pHRall)
dist14c <- rbinom(10000, 395, pHRall)
dist15c <- rbinom(10000, 654, pHRall)

length(subset(dist15c, dist15c > 41))/10000
[1] 0.0148

png(file="AvgTalent.png", height=500, width=600)
plot(density(dist12c), lwd=2, col="darkgreen", main="Bryce Harper Yearly Home Run Density (Averaged Talent)", 
    xlab="Home Runs", ylab="Probability", xlim=c(0, 70), ylim=c(0, .1))
abline(v=pHRall*597, col="darkgreen", lty="dashed")
lines(density(dist13c), lwd=2, col="darkred")
abline(v=pHRall*497, col="darkred", lty="dashed")
lines(density(dist14c), lwd=2, col="darkblue")
abline(v=pHRall*395, col="darkblue", lty="dashed")
lines(density(dist15c), lwd=2, col="gold")
abline(v=pHRall*654, col="gold", lty="dashed")
legend(0, 0.1, c("2012 PA", "2013 PA", "2014 PA", "2015 PA"), col=c("darkgreen", "darkred", "darkblue", "gold"), 
    lty=c("solid", "solid", "solid", "solid"), lwd=c(2, 2, 2, 2), bty="n", cex=1)


So, it seems that even if we assume his true talent in 2015 was a draw from his average fixed home run rate over the course of his career, the probability he would put up 42 home runs in 654 plate appearances is below 1.5%. So, it could be that he’s the same guy, and he got very, very lucky this year. Or his talent could have changed. Given all the other indicators we have in his game this season, it seems like the latter is the most likely choice. But there is still some minuscule chance that he’s the same guy in the power department. Ultimately, we might weight more recent output more heavily in an estimate of true talent (something that Marcel and many other projection systems will do), rather than just using plate appearances as our weights. But, again, this is just to show what performance outcomes are possible, even with a known talent level.

As I’ve noted throughout, these are all overly simplistic examples. We’ve still ignored every other part of Harper’s development here. We’ve ignored the general aging effects, as we might expect players to improve with more experience, especially early in their respective careers. We also haven’t regressed Harper’s observed talent back to some MLB population distribution, either, which is what most projection systems do.

But hopefully this post has made clear the point that, even with a known true talent level in some skill, there is substantial variability across seasons. We see this both with players and with teams, and the variability is often much larger than standard intuition might lead us to believe. Even if Harper really is as good as he showed this year at hitting home runs, there’s a reasonable chance that he could hit less than 30 of them in 2016 in a full season. He could be even better, and he had an unlucky season. Or, his true talent could be somewhere between 2012 and 2015 output. Or, it could change again next season as he continues to mature and develop his game.

So, if you get anything from this post, understand that even if sabermetrics manages to estimate true talent levels exactly as they are, there’s plenty of uncertainty over the events that take place in any season. Embrace that uncertainty and know that while statistical analysis can try and get close to predicting the future, there will always be substantial deviation even in a grueling 162 game season.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: