Hall of Fame Classification Using randomForest
Last week, Ben exhibited how to put together a classifier for the Hall of Fame using rpart. Interestingly enough, I had planned on doing the same thing without knowledge of Ben’s plan for that post, but with a different package: randomForest. In fact, a co-author and I wrote a paper on this at the Journal of Quantitative Analysis in Sports back in 2011.
I’m going to use some of the basic code from that paper here, but with the data that Ben used last week for comparison’s sake (and so you can get it yourself through the Lahman package). One thing to note here is that I’m not trying to predict the best players, but trying to identify the voting patterns within the voting process. These are different goals, particularly if we assume the BBWAA doesn’t to a very good job at identifying the “best” players as measured by baseball talent.
First, let’s talk a bit about the relationship between rpart and randomForest. The Random Forest method was developed by Leo Breiman. The method is closely related to the classification trees that Ben used last week, but apply some additional methods (bagging) in order to ensure that we don’t over-fit the data. Rather than build a single tree to classify Hall of Fame players, this method builds numerous trees, with each tree randomly choosing only a fraction of the input variables. At each tree split, one of these variables is chosen at random and split to optimize the classification in each node. The individual classification trees are created to ensure perfectly classified nodes from the training data. If a player in the holdout sample–called the out-of-bag sample–ends up in a pure HOF node, they get a vote of Yes. If they end up in a pure non-HOF node from the given tree built on the training group, they get a vote of No. The final classification is based on a “majority vote” in a holdout sample for each tree.
The first thing I’ll do is get the data prepared just as Ben did last week. I have re-posted the modified code below, with some changes to have some additional variables in our data set. I also include only players that retired in 1950 or later, since All-Star games started in 1933 and those retiring earlier than this would have this variable biased downward. I also separate batters and pitchers, since some trees would only include pitching statistics, while others will only include batting statistics. This isn’t as trivial as I expected, and I try to do so by implementing a minimum IP and AB for pitchers and batters, respectively.
##set working directory
setwd("c:/...")
##load necessary packages
library(mosaic)
library(Lahman)
library(randomForest)
##get statistics, inductee information, awards information, and all-star games and merge for pitchers and batters separately
inductees <-
HallOfFame %>%
group_by(playerID) %>%
filter(votedBy %in% c("BBWAA", "Special Election") & category == "Player") %>%
summarise(yearsOnBallot = n(), inducted = sum(inducted == "Y"), best = max(votes/ballots)) %>%
arrange(desc(best))
batting <-
Batting %>%
group_by(playerID) %>%
summarise(numSeasons = length(unique(yearID)), lastSeason = max(yearID), tAB = sum(AB), tH = sum(H), tHR = sum(HR), tR = sum(R), tSB = sum(SB), tRBI = sum(RBI), tBA = round(sum(H)/sum(AB), 3)) %>%
arrange(desc(tH))
pitching <-
Pitching %>%
group_by(playerID) %>%
summarise(numSeasons = length(unique(yearID)), lastSeason = max(yearID), tIP = round(sum(IPouts)/3, 2), tW = sum(W), tSO = sum(SO), tSV = sum(SV), tERA = round((9*sum(ER))/(sum(IPouts)/3), 3), tWHIP = round((sum(BB)+sum(H))/(sum(IPouts)/3), 3)) %>%
arrange(desc(tW))
awards <-
AwardsPlayers %>%
group_by(playerID) %>%
summarise(mvp = sum(awardID == "Most Valuable Player"), gg = sum(awardID == "Gold Glove"), cy = sum(awardID == "Cy Young Award"))
allstar <-
AllstarFull %>%
group_by(playerID) %>%
summarise(ASgame = sum(GP))
candidatesBat <- merge(batting, awards, by="playerID", all.x=T)
candidatesBat <- merge(candidatesBat, allstar, by="playerID", all.x=T)
candidatesBat <- merge(candidatesBat, inductees, by="playerID", all.x=T)
candidatesPitch <- merge(pitching, awards, by="playerID", all.x=T)
candidatesPitch <- merge(candidatesPitch, allstar, by="playerID", all.x=T)
candidatesPitch <- merge(candidatesPitch, inductees, by="playerID", all.x=T)
candidatesBat[is.na(candidatesBat)] <- 0
candidatesPitch[is.na(candidatesPitch)] <- 0
##get player names
Pnames <- Master[,c("playerID", "nameLast", "nameFirst")]
candidatesBat <- merge(candidatesBat, Pnames, by="playerID", all.x=T)
candidatesPitch <- merge(candidatesPitch, Pnames, by="playerID", all.x=T)
##subset data into training and test data separately for batters and pitchers using minimums for AB and IP
trainBat <- subset(candidatesBat, candidatesBat$numSeasons >= 10 & candidatesBat$lastSeason < 2009 & candidatesBat$lastSeason > 1949 & candidatesBat$tAB > 2500)
testBat <- subset(candidatesBat, candidatesBat$lastSeason >= 2009 & candidatesBat$tAB > 500)
trainPitch <- subset(candidatesPitch, candidatesPitch$numSeasons >= 10 & candidatesPitch$lastSeason < 2009 & candidatesPitch$lastSeason > 1949 & candidatesPitch$tIP > 200)
testPitch <- subset(candidatesPitch, candidatesPitch$lastSeason >= 2009 & candidatesPitch$tIP > 50)
head(trainBat)
playerID numSeasons lastSeason tAB tH tHR tR tSB tRBI tBA mvp gg cy ASgame yearsOnBallot inducted best nameLast nameFirst
2 aaronha01 23 1976 12364 3771 755 2174 240 2297 0.305 1 3 0 24 1 1 0.978313253 Aaron Hank
54 adairje01 13 1970 4019 1022 57 378 29 366 0.254 0 0 0 0 0 0 0.000000000 Adair Jerry
61 adamsbo03 14 1959 4019 1082 37 591 67 303 0.269 0 0 0 0 1 0 0.003311258 Adams Bobby
89 adcocjo01 17 1966 6606 1832 336 823 20 1122 0.277 0 0 0 2 0 0 0.000000000 Adcock Joe
109 ageeto01 12 1973 3912 999 130 558 167 433 0.255 0 2 0 2 1 0 0.000000000 Agee Tommie
168 alfoned01 12 2006 5385 1532 146 777 53 744 0.284 0 0 0 1 0 0 0.000000000 Alfonzo Edgardo
head(trainPitch)
playerID numSeasons lastSeason tIP tW tSO tSV tERA tWHIP mvp gg cy ASgame yearsOnBallot inducted best nameLast nameFirst
2 aasedo01 13 1990 1109.33 66 641 82 3.797 1.390 0 0 0 1 0 0 0.0000000 Aase Don
7 abbotgl01 11 1984 1286.00 62 484 0 4.388 1.366 0 0 0 0 0 0 0.0000000 Abbott Glenn
8 abbotji01 10 1999 1674.00 87 888 0 4.253 1.433 0 0 0 0 1 0 0.0251938 Abbott Jim
10 abbotpa01 11 2004 720.67 43 496 0 4.920 1.492 0 0 0 0 0 0 0.0000000 Abbott Paul
14 abernte02 14 1972 1147.67 63 765 148 3.458 1.396 0 0 0 0 0 0 0.0000000 Abernathy Ted
25 ackerji01 10 1992 904.33 33 482 30 3.971 1.379 0 0 0 0 0 0 0.0000000 Acker Jim
Now that the data are ready to go, we’ll have to decide a few things. First, we need to identify which variables will be worth using for building our random forest. Next, we need to decide how many variables that each classification tree in the forest will use. Luckily, the randomForest package includes a tuning function that tells us the optimal number of variables for each tree. The code below allows us to choose the number of variables that minimizes our out-of-bag error: the rate of incorrect classifications for those left out of each tree. I have 1,000 trees in the forest, but this is an arbitrary choice by me here. And the tuneRF function can also automatically plot the differences in out-of-bag error for each of the variable number choices. I don’t post the plots here, but they should pop up in your R window with the following code:
##tune the random forest
set.seed(12345)
mtryBat <- tuneRF(trainBat[,c("tH", "tHR", "tR", "tSB", "tRBI", "tBA", "mvp", "gg", "ASgame")], factor(trainBat[,c("inducted")]), ntreeTry=1000, data=trainBat, plot=T)
mtryBat
mtry OOBError
2.OOB 2 0.02963776
3.OOB 3 0.03073546
6.OOB 6 0.03512623
mtryPitch <- tuneRF(trainPitch[,c("tW", "tSO", "tSV", "tERA", "tWHIP", "gg", "cy", "ASgame")], factor(trainPitch[,c("inducted")]), ntreeTry=1000, data=trainPitch, plot=T)
mtryPitch
mtry OOBError
1.OOB 1 0.02119205
2.OOB 2 0.01324503
4.OOB 4 0.01854305
As you can see, using two variables for each tree—chosen at random by randomForest—is the best choice for both batters and pitchers. Therefore, we include this as our mtry in the fitting of the forest. Additionally, we note that we want our sampling stratified by Hall of Fame status when building each tree, because the large discrepancy in sample size across the two groups (HOF and non-HOF) could result in some trees including no Hall of Fame players. The rest of the commands allow us to use the forest to predict future Hall of Fame induction (keep.forest=TRUE), exhibit the importance of each variable in our classification (importance=TRUE), exhibit the importance of each variable in the classification of specific individuals (localImp=TRUE), and visualize the proximity of players based on how often they end up in the same node throughout the entire forest using a multi-dimensional scaling approach (proximity=TRUE).
##fit separate random forests for pitchers and batters forestBat <- randomForest(factor(inducted) ~ tH + tHR + tR + tSB + tRBI + tBA + mvp + gg + ASgame, data=trainBat, ntree=1000, replace=TRUE, strata=factor(trainBat$inducted), keep.forest=TRUE, importance=TRUE, localImp=TRUE, proximity=TRUE, mtry=2) print(forestBat) Call: randomForest(formula = factor(inducted) ~ tH + tHR + tR + tSB + tRBI + tBA + mvp + gg + ASgame, data = trainBat, ntree = 1000, replace = TRUE, strata = factor(trainBat$inducted), keep.forest = TRUE, importance = TRUE, localImp = TRUE, proximity = TRUE, mtry = 2) Type of random forest: classification Number of trees: 1000 No. of variables tried at each split: 2 OOB estimate of error rate: 3.29% Confusion matrix: 0 1 class.error 0 851 10 0.0116144 1 20 30 0.4000000 forestPitch <- randomForest(factor(inducted) ~ tW + tSO + tSV + tERA + tWHIP + cy + gg + ASgame, data=trainPitch, ntree=1000, replace=TRUE, strata=factor(trainPitch$inducted), keep.forest=TRUE, importance=TRUE, localImp=TRUE, proximity=TRUE, mtry=2) print(forestPitch) Call: randomForest(formula = factor(inducted) ~ tW + tSO + tSV + tERA + tWHIP + cy + gg + ASgame, data = trainPitch, ntree = 1000, replace = TRUE, strata = factor(trainPitch$inducted), keep.forest = TRUE, importance = TRUE, localImp = TRUE, proximity = TRUE, mtry = 2) Type of random forest: classification Number of trees: 1000 No. of variables tried at each split: 2 OOB estimate of error rate: 1.46% Confusion matrix: 0 1 class.error 0 726 3 0.004115226 1 8 18 0.307692308 ##get predicted classes and votes for individual players and attach to training data set trainBat$predict <- forestBat$predicted trainBat$votes <- forestBat$vote head(trainBat) playerID numSeasons lastSeason tAB tH tHR tR tSB tRBI tBA mvp gg cy ASgame yearsOnBallot inducted best nameLast nameFirst predict votes.0 votes.1 2 aaronha01 23 1976 12364 3771 755 2174 240 2297 0.305 1 3 0 24 1 1 0.978313253 Aaron Hank 1 0.424242424 0.575757576 54 adairje01 13 1970 4019 1022 57 378 29 366 0.254 0 0 0 0 0 0 0.000000000 Adair Jerry 0 1.000000000 0.000000000 61 adamsbo03 14 1959 4019 1082 37 591 67 303 0.269 0 0 0 0 1 0 0.003311258 Adams Bobby 0 1.000000000 0.000000000 89 adcocjo01 17 1966 6606 1832 336 823 20 1122 0.277 0 0 0 2 0 0 0.000000000 Adcock Joe 0 0.995024876 0.004975124 109 ageeto01 12 1973 3912 999 130 558 167 433 0.255 0 2 0 2 1 0 0.000000000 Agee Tommie 0 1.000000000 0.000000000 168 alfoned01 12 2006 5385 1532 146 777 53 744 0.284 0 0 0 1 0 0 0.000000000 Alfonzo Edgardo 0 1.000000000 0.000000000 trainPitch$predict <- forestPitch$predicted trainPitch$votes <- forestPitch$vote head(trainPitch) playerID numSeasons lastSeason tIP tW tSO tSV tERA tWHIP mvp gg cy ASgame yearsOnBallot inducted best nameLast nameFirst predict votes.0 votes.1 2 aasedo01 13 1990 1109.33 66 641 82 3.797 1.390 0 0 0 1 0 0 0.0000000 Aase Don 0 1 0 7 abbotgl01 11 1984 1286.00 62 484 0 4.388 1.366 0 0 0 0 0 0 0.0000000 Abbott Glenn 0 1 0 8 abbotji01 10 1999 1674.00 87 888 0 4.253 1.433 0 0 0 0 1 0 0.0251938 Abbott Jim 0 1 0 10 abbotpa01 11 2004 720.67 43 496 0 4.920 1.492 0 0 0 0 0 0 0.0000000 Abbott Paul 0 1 0 14 abernte02 14 1972 1147.67 63 765 148 3.458 1.396 0 0 0 0 0 0 0.0000000 Abernathy Ted 0 1 0 25 ackerji01 10 1992 904.33 33 482 30 3.971 1.379 0 0 0 0 0 0 0.0000000 Acker Jim 0 1 0
Here we have our out-off-bag error rates, as well as our class errors in the confusion matrices for each forest. Note that the error for the non-HOF class is much better than those within the HOF. This is because there are so many more observations, and there are so many sure non-Hall of Fame players here. I encourage you to look around within the data to see which players in the Hall of Fame were predicted to not be inducted, and those that are not in the Hall of Fame that were predicted to be inducted. Also note some of the vote counts are surprising, which could be a data issue (I’ll get to that later with our predictions of future Hall of Fame players). Notably, we see that Hank Aaron was classified as a HOF player in only about 58% of the trees in our forest.
Now that we’ve built our forests, we can use R to tell us which of the variables were the most important in classifying players as Hall of Famers. This is useful in shedding some light on the “Black Box” approach of the randomForest function, and is one of the nice advantages to using this over some other machine learning techniques. While I won’t get into the calculations of this measure, I again encourage you to look at the background information on this method to get an understanding of how things like the Mean Decrease in Accuracy are calculated. Note that in the plots below, the dots further to the right indicate more important variables for deciding whether or not players should be inducted.
##variable importance plot png(file="varImp.png", height=800, width=600) par(mfrow=c(2,1)) varImpPlot(forestBat, pch=19, type=1, main="Variable Importance for Batters") varImpPlot(forestPitch, pch=19, type=1, main="Variable Importance for Pitchers") dev.off()
As you can see, All-Star Appearances seem to be picking up a large portion of the variation in Hall of Fame induction for Hitters, while Stolen Bases don’t have much predictive ability. For pitchers, Wins and Strikeouts dominate, whereas Gold Gloves are essentially useless in predicting Hall of Fame induction. Saves aren’t particularly useful, but we know it is a relatively new phenomenon for players to rack up lots of Saves. This approach may be more useful by splitting up starting pitchers and relief pitchers in our training of the forest.
So why are Runs so and RBI important, while Home Runs lag so far behind? In our JQAS paper, my co-author and I posited that this is because of the dependence of a large number of R and RBI on the HR that are hit by each player. To remedy this, we created a metric of “HR-Independent Runs” and “HR-Independent RBI” (note, this is imprecise language, but it gets the point across) by simply subtracting HR from the total R and total RBI. I won’t do that here, but I can tell you that it seemed to improve the variable importance plot—without affecting the error rate much—making the importance of Home Runs much closer to what we would expect.
While it’s nice to see the importance of variables for classification as a whole, we can also see which variables were the most important factors in any individual players’ induction with our “local” importance matrix. This can be done for the training model or the test data set, but I’m not going to go through this portion of the code, since I already have an inordinately long post. I’ll just note that you can find the local importance information under the following objects.
forestBat$localImp forestPitch$localImp
Finally, there is a nice multidimensional scaling approach that we can use from the random forest output, which can visualize a distance measure for individual players in a two-dimensional representation. We can visualize both the current inductees, as well as our predictions for the future. While there is a direct plotting approach using the MDSplot function, this makes it a bit difficult to see which player is which. Therefore, I’ve used some of my own code, which uses the players’ last names in the plot.
##create HOF variable for color coding in MDS plot
trainBat$newhof <- trainBat$inducted + 1
trainPitch$newhof <- trainPitch$inducted + 1
##run 3-dimension MDS and plot using player classes
cmdB <- cmdscale(1 - forestBat$proximity, k=3)
trainBat$x <- cmdB[,1]
trainBat$y <- cmdB[,2]
trainBat$z <- cmdB[,3]
cmdP <- cmdscale(1 - forestPitch$proximity, k=3)
trainPitch$x <- cmdP[,1]
trainPitch$y <- cmdP[,2]
trainPitch$z <- cmdP[,3]
png(file="MDSviz.png", height=1000, width=800)
par(mfrow=c(2,1))
plot(trainBat$x, trainBat$y, type="n", xlab="Dim 1", ylab="Dim 2", main="Batters Proximity Visualization")
text(trainBat$x, trainBat$y, trainBat$nameLast, cex=.75, col=trainBat$newhof)
legend(-0.18, 0.4, c("Member", "Non-Member"), fill=c("red", "black"), bty="n", cex=1.2)
plot(trainPitch$x, trainPitch$y, type="n", xlab="Dim 1", ylab="Dim 2", main="Pitchers Proximity Visualization")
text(trainPitch$x, trainPitch$y, trainPitch$nameLast, cex=.75, col=trainPitch$newhof)
legend(0.65, 0.4, c("Member", "Non-Member"), fill=c("red", "black"), bty="n", cex=1.2)
dev.off()
Here, you can see the closeness of Jackie Robinson and Roy Campanella, which makes sense given that they both had shorter MLB careers, were pioneers of integration, and were excellent players in their shorter stints. Similarly, we see Mark McGwire and Tim Raines right on the edge of where the clusters of most Hall of Fame hitters lie in the visual, alongside some Veteran’s Committee inductions like Orlando Cepeda. Also note that we see the same Tom Glavine mistake as in Ben’s post (go there to see what happened). And of course, we know why Roger Clemens is in that elite cluster on the right of the pitcher visualization but isn’t in red font.
Finally, and perhaps the most fun part of the Hall of Fame prediction model exercise is predicting future Hall of Fame induction of current or recently retired players. However, remember that this is a naïve model: we don’t predict the career trajectories of current players. We simply ask: if they retired now, and we relax the 10-year minimum requirement, would their statistical output qualify them for the Hall of Fame based on what we’ve seen voters do in the past?
##Predict Future Hall of Fame Induction ##get predicted classes for test data futureBat <- predict(forestBat, testBat, type="response") futurePitch <- predict(forestPitch, testPitch, type="response") ##get vote percentages for test data futureBat.vote <- predict(forestBat, testBat, type="vote", norm.votes=TRUE) futurePitch.vote <- predict(forestPitch, testPitch, type="vote", norm.votes=TRUE) ##attach predicted class and vote percentages from MODEL 1 testBat$HOF <- as.numeric(futureBat) - 1 testBat$votes <- futureBat.vote[,2] testPitch$HOF <- as.numeric(futurePitch) - 1 testPitch$votes <- futurePitch.vote[,2] batterHOF <- subset(testBat[,c(1,18,19,20,21)], testBat$votes > 0.1) pitcherHOF <- subset(testPitch[,c(1,17,18,19,20)], testPitch$votes > 0.1) batterHOF[order(-batterHOF$votes),] playerID nameLast nameFirst HOF votes 14896 sheffga01 Sheffield Gary 1 0.823 13206 pujolal01 Pujols Albert 1 0.797 13934 rodriiv01 Rodriguez Ivan 1 0.797 6334 griffke02 Griffey Ken 1 0.790 13353 ramirma02 Ramirez Manny 1 0.775 15956 suzukic01 Suzuki Ichiro 1 0.741 6424 guerrvl01 Guerrero Vladimir 1 0.729 13914 rodrial01 Rodriguez Alex 1 0.592 8195 jonesch06 Jones Chipper 1 0.577 8018 jeterde01 Jeter Derek 1 0.541 16278 thomeji01 Thome Jim 0 0.445 16879 vizquom01 Vizquel Omar 0 0.381 10348 mauerjo01 Mauer Joe 0 0.369 3971 delgaca01 Delgado Carlos 0 0.361 2268 cabremi01 Cabrera Miguel 0 0.354 7034 heltoto01 Helton Todd 0 0.350 12277 ortizda01 Ortiz David 0 0.333 3712 damonjo01 Damon Johnny 0 0.299 5828 giambja01 Giambi Jason 0 0.277 1055 beltrca01 Beltran Carlos 0 0.210 7465 hollima01 Holliday Matt 0 0.199 8847 konerpa01 Konerko Paul 0 0.187 1718 braunry02 Braun Ryan 0 0.181 6623 hamiljo03 Hamilton Josh 0 0.170 8176 jonesan01 Jones Andruw 0 0.149 5649 garcino01 Garciaparra Nomar 0 0.148 16899 vottojo01 Votto Joey 0 0.142 34 abreubo01 Abreu Bobby 0 0.108 13082 poseybu01 Posey Buster 0 0.105 pitcherHOF[order(-pitcherHOF$votes),] playerID nameLast nameFirst HOF votes 3951 johnsra05 Johnson Randy 1 0.733 8204 wagnebi02 Wagner Billy 1 0.611 4962 martipe02 Martinez Pedro 1 0.610 5699 nathajo01 Nathan Joe 1 0.601 3596 hoffmtr01 Hoffman Trevor 1 0.599 7412 smoltjo01 Smoltz John 1 0.506 6022 papeljo01 Papelbon Jonathan 0 0.475 3173 hallaro01 Halladay Roy 0 0.400 6680 riverma01 Rivera Mariano 0 0.359 4156 kershcl01 Kershaw Clayton 0 0.332 7222 shellst01 Shell Steven 0 0.273 1306 chapmar01 Chapman Aroldis 0 0.268 6739 rodrifr03 Rodriguez Francisco 0 0.268 4188 kimbrcr01 Kimbrel Craig 0 0.266 3613 hollagr01 Holland Greg 0 0.260 42 adamsmi03 Adams Mike 0 0.255 4706 loupaa01 Loup Aaron 0 0.254 6750 rodrist02 Rodriguez Paco 0 0.254 8165 vinceni01 Vincent Nick 0 0.253 231 avilalu01 Avilan Luis 0 0.252 2418 fernajo02 Fernandez Jose 0 0.252 3322 harvema01 Harvey Matt 0 0.252 6691 roarkta01 Roark Tanner 0 0.252 7960 torreal01 Torres Alex 0 0.252 6976 santajo01 Santana Johan 0 0.242 8036 ueharko01 Uehara Koji 0 0.240 6793 romose01 Romo Sergio 0 0.235 3863 janseke01 Jansen Kenley 0 0.234 6924 saitota01 Saito Takashi 0 0.234 7442 soriajo01 Soria Joakim 0 0.232 8570 wilsoju10 Wilson Justin 0 0.200 1377 cishest01 Cishek Steve 0 0.190 1552 cookry01 Cook Ryan 0 0.184 5965 oteroda01 Otero Dan 0 0.177 1607 cosarja01 Cosart Jarred 0 0.175 8136 ventejo01 Venters Jonny 0 0.173 4877 manesse01 Maness Seth 0 0.172 8795 zieglbr01 Ziegler Brad 0 0.169 3018 grayso01 Gray Sonny 0 0.120 3653 hoovejj01 Hoover J. J. 0 0.120 5857 odayda01 O'Day Darren 0 0.120 6816 rosentr01 Rosenthal Trevor 0 0.102
Here we can see the proportion of trees that each player is predicted to be inducted. Sheffield, Pujols, Pudge, Griffey, Manny, Ichiro, and Vlad are all easily predicted to be Hall of Famers, with A-Rod, Jones and Jeter all predicted as inductees. But this approach doesn’t account for steroid suspicions, so that would likely be an important omitted variable in predicting BBWAA voting behavior.
A quick note: it’s interesting to see A-Rod so low on this list, and Mariano Rivera listed as a non-inductee. As it turns out, both Alex Rodriguez and Mariano Rivera are recoded as having zero All-Star appearances. So it looks like it will be important to clean the data before making any final conclusions here, though I don’t have the time to go back through all the possible omissions.
Clearly, there are also a lot of names on this list that shouldn’t be getting any votes, like Nick Vincent and other relievers. Our forest is likely getting false positives due to low ERA and WHIP numbers for some relievers. Maybe splitting up starters and relievers, and requiring some larger minimum number of innings pitched would be helpful for our predictions here. But our Top 6 (plus Rivera), all make sense as being above the 50% forest vote threshold for classification as Hall of Fame pitchers.
Well, that’s it for today. Note that while I’ve shown a number of things that can be done with the randomForest package, there are a number of other things that we can investigate with the models. For example, we can look at the average tree size in each forest, track the error rates as the forest is built, and look at outliers in our data. Lastly, there is the possibility of smoothing the classification choice by using vote percentage with a logistic regression. However, I simply want to highlight the use of the randomForest package here. I’ll leave further investigation up to you to see if you can improve on the rates presented here.
Exploring Baseball Data with R 
Recent Comments