Thursday, 23 July 2015

Corsi Plus-Minus: Individual Player Value Accounting for Teammates

***Reminder, if you are just interested in looking at the data/visualizations you can check out the separate page here instead of scrolling through this entire article.

Hockey stats have existed for about as long as the game itself. Simple boxscore stats such as goals and assists can be traced back almost a full century now. These stats have helped informed fans/coaches/managers of the value held by some players. Around the 1950’s the Montreal Canadiens began to track a player’s plus-minus. This stemmed from the idea that simple boxscore stats fail to capture many important elements of a game. Plus-minus was a good start to tracking impact that is not realized in traditional boxscore stats. Unfortunately, Plus-Minus has been recently shown to be quite incomplete and lacking by modern evaluation standards. 

The most famous stat to come out of the modern hockey analytics movement is probably Corsi (or just simply all shots). Corsi has numerous benefits over goal based metrics. It accumulates faster leading to more reliable information and is actually more predictive of who will be better in the future. Raw Corsi% (or CF%) tracks the share of shots (all shots, not just shots-on-goal) directed at the oppositions net versus how many are directed at a player’s own net when said player is on the ice, the higher a player’s CF% the better. This metric suffers from a key drawback however, each player’s CF% is heavily dependent upon the quality of his on-ice teammates. This is how a depth player on a good team (ex. Shawn Horcoff) typically has a higher CF% rating than a star player on a bad team (ex. Taylor Hall).

Issues with Current Metrics

Above, I have outlined some current issues with standard CF%, but these are not new issues. People have been aware of the effect playing on a good/bad team can have on a player for a while now, so next progression in the history of hockey stats resulted in Corsi%Relative. To calculate the CF%Rel of a Player Y on Team X is done as follows, Team X’s CF% while Player Y is on the ice minus Team X’s CF% while Player Y is not on the ice. This still runs into issues of comparing players on vastly different teams. Consider the example below:

Whose stat line is more impressive? It seems as though Malkin makes a pretty good team into a great team when he is on the ice while Hudler turns his team from awful to just bad. Which is more valuable? Or are they equally valuable? This is an improvement upon raw CF% but still shares many of the same problems. 

dCorsi fairly prevalent metric constructed by Stephen Burtch. dCorsi is calculated for offense as, Player X’s Actual Corsi For minus Player X’s Expected Corsi For. The real trick here is calculating what a player’s expected Corsi For is. I won’t dive too deep into this stat since it is not mine and you can read up more on it here or reach out to Stephen Burtch via Twitter here. Expected Corsi is calculated using a multivariate regression using five independent variables:
  • A dummy variable for each Team/Season
  • Time on Ice per game 
  • Team total time on ice that the player in question wasn’t on the ice
  • Offensive Zone Start%
  • Neutral Zone Start%
dCorsi moves the right direction here, especially with the dummy variable accounting for which team a player was on (if you don't know what a dummy variable is, just try to hold on until the Methodology section where I do my best to explain it). Unfortunately this doesn’t seem to help address the largest issue with CF% which is how heavily dependent an individual player’s CF% is on their teammates. 

The next evolution of CF%Rel is CF%RelTM, which stands for Corsi For % Relative to Teammates and is calculated by subtracting a player's average teammate CF% (calculated by weighting a player's teammates' CF% without him by their TOI spent with him) from his observable CF%. This sounds good but runs into issues of collinearity. Collinearity occurs in hockey data because of the way coaches use their players. 

The most famous example is probably with the Sedin twins on Vancouver. During the 2014-2015 regular season, of all 1100 minutes of 5v5 ice-time each the twins played in that season, 92% of their minutes were played with each other. This can greatly boost their teammate relative stats because their CF% is being boosted by playing with there also talented brother and is being compared to when one is instead playing with their lesser teammates. When Daniel wasn’t playing with Henrik last season his CF% was 29% but that was only about 77 minutes of ice time (or 8% of his seasonal total), unfortunately CF%RelTM will then weight this 29% heavily because of how much Daniel and Henrik play together. These rare instances where one player plays without the other can have disproportionately large effects on both player’s ratings.

This issue also occurs between players who are never on the ice together such as Jarret Stoll and Anze Kopitar on Los Angeles. Stoll played 905 minutes at 5v5 last season, how many of those were with Anze Kopitar? 1 minute and 30 seconds or 0.16% of Stoll’s total ice-time. When we look at Jarret Stoll’s most common teammates (the ones who will be weighted highest by CF%RelTM) we find that when they aren’t playing with Stoll they are playing with Kopitar. This unfairly punishes Stoll for playing the same position on the same team as one of the top centres in the game. Kopitar boosts Stoll’s teammates CF% when Stoll is on the bench while never providing the same boost to Stoll’s own CF% simply because they are never on the ice together.

This finally brings me to explain Corsi Plus-Minus, how it is calculated and why I believe it is a better metric for isolating a player’s contributions independent of the strength of that player’s teammates.

Methodology

Using the play-by-play data provided by NHL.com I was able to look at every even-strength shift that took place from 2007-2015. From this data we set up a multivariate regression where our dependent variable is rate at which a Corsi event took place, our independent variables are which players were on the ice during that shift and each shift is weighted by how long it was. This model does not account for a player's zone starts for a variety for reasons (see. here here) and it also does not account for the strength of a player's opponents (see. here). Simply, there has been a lot of research to show that both of those components might not be as relevant in helping determine a player's value. 

For those of you that might not be familiar with multivariate regressions here is a relatively simple and hopefully helpful example to help you grasp the concept. 

y=β01X12X23X3
  • y - Dependent Variable - How well you will do on your test? Measured in points
  • x - Independent Variables
    • X1 - How long did you study for? Measured in hours
    • X2 - How long did you play video games? Measured in hours
    • X3 - Did you go to the study session? Yes/No?
      • This is an example of a dummy variable 
  • B - Coefficients - The value of each independent variable 

So if you had a sheet of data filled out with how well everyone did on their test (dependent variable) along with the information on the 3 independent variables and then ran that data as a multivariate regression you would get something that looks like this (reminder I made all these numbers up): 

y = 75 + 2X- 3X+ 6X3

For the purposes of my methodology, the regressions coefficients are what we will be focusing on. Looking at coefficient β1, for every extra hour you studied (holding all other independent variables the equal) you can expect to score 2 points better on the test. Looking at the dummy variable, X3, it can either be a 1 (yes you did go to the study session) or 0 (no you did not go to the study session). If you go to the study session (holding all other independent variables the equal), you can expect to do 6 points better on your test.

Now back to how this relates to my methodology. Picture every player as a dummy variable in our data, that player was either on the ice during a shift or they weren’t. Running our regression will then give us coefficients that tell us (holding all other players equal) how much of an impact Player X has on his team’s CF%, a value I have coined Corsi Plus-Minus (CPM). Two of these regressions are run for each season, one for offence and one for defence. 

However, due to the collinearity nature of our data, as was touched upon earlier, we would be better served to use a ridge regression (also known as Tikhonov Regularization) to help account for this collinearity. This method adds a penalty factor to the regression for results being far away from the mean. This penalty factor, called lambda, is chosen based on a 10-fold cross-validation. This helps remove a lot of the noise accompanied with such a process. Players falling in the bottom 25% of the league’s playing time (measured by games played instead of TOI to not bias against forwards) are group together and treated as a single player to help reduce volatility in the results that would be caused by their extreme values. While randomness can still have an effect, the damage is less so due to this regularization. 

Repeatability and Predictiveness 

I ran some basic correlation tests to see how repetitive of a skill Corsi Plus-Minus (CPM), Offence CPM (OCPM) and Defence CPM (DCPM) is from year-to-year, players must have played at least 20 games in back-to-back years to be included in the correlation. I compared CPM to the other metrics (CF%RelTM and dCorsi) I mentioned earlier just as a means of reference. A correlation (Pearson R) rangers from -1 to 1. The closer to either -1 or 1 the stronger the relationship is, the closer to 0 the weaker the relationship.


I also ran some correlations to see how well CPM does at prediction future GF%. Reminder, negative correlations are a good thing for DCPM/dCA, so focus on the numerical values and not necessarily positive or minus signs.

  • OCPM -> GF/60
  • DCPM -> GA/60
  • CPM -> GF%




Reliability vs. Validity 

As I said above, having all 3 different metrics here is strictly for comparative purposes. Just because CF%TMRel is higher in most categories doesn't automatically qualify it as a better metric. I will summarize key parts from this Columbia article with regards to reliability and validity. 
“Reliability refers to a condition where a measurement process yields consistent scores (given an unchanged measured phenomenon) over repeat measurements.”  

This is a measure that quantifies how much random fluctuations can interfere with getting consistent results. As we have seen above, the collinearity of CPM samples decreases CPM's reliability. Compared to single-year With-Or-Without-You stats, CPM seems to be quite reliable.
“Validity refers to the extent we are measuring what we hope to measure (and what we think we are measuring).”  

CPM is completely valid, because it is directly measuring the result. Stats such as CF%, CF%Rel and CF%TMRel are not as valid, because they measure a proxy rather than the subject directly. So even while they may seem reliable and predictive they are not hitting the target of determining individual contributions.

Final Notes

Corsi Plus-Minus is just the next step in accurately determining a player's true value. There are many different versions of this methodology that could still be applied instead of Corsi, including goals, shots on net, Fenwick and xG. There is also a Bayesian version where instead of assuming every player starts the season with a rating of zero, we can tell the model what rating a player had the prior season so that it can more accurately estimate that player's true value. This version wouldn't help describe what happened during a given season quite so accurately but it could help provide an even great idea of a player's value. I have looked pretty heavily into this version of CPM and hopefully might release it shortly. In a follow-up post I hope to explore further correlations (ex. Y -> Y+3) as well as how CPM relates to time-on-ice and salary. 

The data is posted below in both as a spreadsheet and in the form of some Tableau visualizations. I also have noticed an error with the Comparison tab in my original Tableau that I am unable (for some  unknown reason) to update, so I created a new Tableau (posted below the spreadsheet) that now includes player salary data.
  • OCPM/DCPM/CPM are rate stats (per 60 minutes of ice-time)
  • The Impact version of these stats apply a player's ice-time to determine their actual impact in a given season.

Thanks to C. Tompkins for helping me with the Tableaus, as well as War-On-Ice and Puckalytics for the data. If you are interested in reading about topics like this in further depth, I will direct you to previous work done by Brian McDonald (here, here & here) that was an instrumental guide in my work.

Please let me know if you have any thoughts, questions, concerns or suggestions. You can comment below or reach me via email me here: DTMAboutHeart@gmail.com or via Twitter here: @DTMAboutHeart