A word about my statistics

In reading an article today, I came across an offensive efficiency statistic for a team that was different than the value for that team that is posted on the website from which I took the defensive efficiency statistics that I have been using in my evaluations. Looking more closely at the site I've been using, I don't see any information telling me when the information was last updated, so it's possible it stopped being updated at some point during the past season. Eyeballing the data, it looks similar enough to what I see elsewhere that I don't think any discrepancies will make a large difference in my evaluation, but this problem obviously creates some worries. I guess the first step in redoing my regression analysis will be to make sure I have completely accurate data, not actually relearning how to do a regression!

What about the Sixers defense?

Last season, the Sixers ranked 18th in the NBA in defensive efficiency with a mark of 107.73. No real surprise that the Sixers weren't a great (or even good) defensive team last year, but I was a little surprised to see how they reached that ranking.

Most surprisingly, the Sixers were actually ranked 4th in opponent's turnover rate (DTOR). Despite forcing a high number of turnovers, the Sixers were a slightly below average defensive team because of their average defensive true shooting percentage (TS% = 16) and piss poor defensive rebound rate (DRR = 27). What explains this wide range of ability on our three major defensive statistics? Living in Boston, I didn't have many opportunities to see the Sixers play last year (to put it charitably, they weren't on national TV many times), so I can't really make an evaluation with a high degree of certainty. However, my best guess is that the Sixers defensive story begins and ends with Samuel Dalembert.

If you recall my post from yesterday, you probably remember that having mobile big men seems to play a big role in whether or not a team forces a number of turnovers. Dalembert is, if nothing else, extremely mobile for a big man. The top defensive teams that force a high number of turnovers are able to do so because their big men are mobile enough to help force the turnovers and to still recover quickly enough to be in a good position to help out defensively. If I remember correctly, one of the reasons Coach Jim O'Brien and Samuel Dalembert didn't get along well during O'Brien's stint with the Sixers was because O'Brien thought Dalembert was slow on his help defense rotations. These defensive numbers indicate to me that Dalembert is still slow on his rotations--he's mobile enough to help force turnovers and recover, but he doesn't read the game quickly enough to actually recover effectively. He still gets his blocks because of his athleticism, but he doesn't get enough of them to make up for his late rotations (thus the average DTS%) and his late rotations put everyone out of position for getting rebounds (thus the putrid DRR).

This story sounds believable to me, but like I said, I didn't actually get a chance to watch many Sixers games this past year. I'm being hard on Dalembert, but maybe that's being unfair to him. I tried to look at +/- stats from 82games.com to see if they shed any further light on the subject, but they weren't particularly helpful. The Sixers were worse with Dalembert not on the floor, but mostly that was because the offense was much worse when he was not playing (the defense was also worse, but the difference wasn't nearly as big). Assuming my intuition is correct, playing Dalembert might be the Sixers best option defensively (and looking at their roster this isn't hard to believe), but that option doesn't seem likely to lead to very good results unless something changes dramatically. In watching the Sixers play this year, we'll just need to make sure and pay special attention to the big man rotations on the defensive end of the court to see what is really going on.

Sixers-Celtics and Tim Donaghy

I have nothing insightful to add to this post, other than to point out that the Sixers have now become a tangential part of the Tim Donaghy story. When I first saw this story, I also thought I had been at the game in question, but then I re-read it and realized this was the game in Philadelphia. I attended the game in Boston between the Sixers and Celtics that took place the week after this one.

Via Truehoop:

Shira Springer of the Boston Globe: "The events surrounding a Dec. 13, 2006, game between the Celtics and 76ers in Philadelphia raised the suspicion of investigators. According to the indictment, Donaghy 'spoke with a coconspirator by telephone regarding his pick for an NBA game' Dec. 13. The next day, Donaghy met with coconspirators in Pennsylvania and 'received a cash payment.' Donaghy received payments for as much as $5,000 for correct picks and nothing for incorrect picks. Donaghy officiated the game between Boston and Philadelphia along with Matt Boland and Derrick Stafford. It was not a game that would automatically raise suspicion, though Donaghy admitted relaying information regarding the officiating crews assigned games, the interaction between certain officials and players, and the health of players. According to the Associated Press, the pointspread moved 2 points before the game was taken off the board as Boston went from a 1 1/2-point to a 3 1/2-point favorite. The Celtics defeated Philadelphia, 101-81. The over-under line was 194 points. The blowout came with Allen Iverson sidelined as the Sixers pursued a trade for him. Danny Ainge, the Celtics' executive director of basketball operations, declined to comment on the possibility of a Boston game being affected by the scandal." In Brian Mahoney's account of the proceedings for the Associated Press, Mahoney writes: "A person close to the investigation, speaking on condition of anonymity because the case is ongoing, said the payment was for a successful tip on the 76ers-Celtics game." Here is the play-by-play of that game.

Defensive Efficiency: Looking at the league leaders

[This post builds off of yesterday's post, so go read that one first if you haven't read it already.]

I had hoped to start off my post today by talking to you about the relative importance of opponent (defensive) turnover rate (DTOR), opponent (defensive) true shooting percentage (DTS%), and defensive rebound rate (DRR) based on the results of the fancy-schmancy regression analysis I was working on (evaluating the impact of those three variables on defensive efficiency). However, when I ran the regression, the coefficients I ended up with struck me as a little bizarre. Since I last took a statistics course four years ago, I've decided that I need to go back and relearn my regression analysis techniques to find out what (if anything) I was doing incorrectly. With that said, I will point out that my R-squared value was 96%, so it seems like the statistics I was looking at do account for most of the variation in defensive efficiency. (In case that sentence wasn't clear: the R-squared value basically tells you how much of the variation in the dependent variable--defensive efficiency in this case--can be explained by the variation in the independent variables tested--DTOR, DTS%, and DRR in this case)

Foiled in my attempt to do fancy statistics, I will just limit myself to discussing what we can learn about what accounts for the success of the league's defensive leaders by eyeballing the data.

Before looking at the data, my initial assumption was that our three key defensive statistics needed to be evaluated together because they are likely to have interactive effects. As an example, a team might play a gambling style of defense. As a result, they might get a number of steals (thus allowing the other team a fewer total number of shots), but when they miss the steal opportunity the resulting shot for the other team might be more wide open (thus resulting in fewer missed shots per shot attempt). In the same way, a team might be so aggressive in contesting shots (forcing more missed shots per shot attempt) that they do not rebound effectively, thus giving the other team more chances for a second shot each possession. For each team, there is probably some optimum trade-off to be made between these different styles of defensive play, but figuring out the optimal trade-off is difficult.

The top 5 teams in the NBA in defensive efficiency in 2007 were the Chicago Bulls, San Antonio Spurs, Cleveland Cavaliers, Houston Rockets, and Detroit Pistons. In looking at the the league-wide ranks of these teams with respect to our three categories, two distinct observations can be made. First, the order of importance of our categories appears to be DTS%, DRR, and then DTOR. Keeping in mind that a "perfect" average ranking in these categories by the top five teams would be a "3" (an average rank of three would mean that all five top defensive teams would be in the top five in that category), the average ranking of these teams was DTS% = 3 (perfect!), DRR = 8, and DTOR = 14 respectively. This order of importance seems to hold true even as we go further down the list of top defensive teams: the average ranking of the top ten defensive teams in these categories was DTS% = 7, DRR = 10, DTOR = 16 (remember, an average rank of "5" would be perfect in this case).

This order of importance also makes sense when you give it some thought. The Golden State Warriors have the best DTOR in the league at about 18%, so even for the Warriors that leaves 82 out of 100 possessions for which DTOR does not help their defensive efficiency. DTS% matters on every single one of these 82 possessions. With DTS% being between 50-57% for every team in the league (the Spurs have the league's best DTS% at 51.06%), DRR matters for about 40 possessions out of every 100. The best defensive teams do the best on the defensive measures that impact the greatest number of possessions.

The second observation is that, while having a good DTS% is obviously key, there is no "correct" trade-off between our three key defensive measures for being a top defensive team. In fact, we can see three distinct paths towards being a top defensive team among our top five. The first path is to be very good (in the top ten) in all three categories. The Chicago Bulls (DTOR = 2, TS% = 3, and DRR = 10) and Cleveland Cavaliers (DTOR = 9, TS% = 5, DRR = 2) reached success going in this direction. I think the key is that both of these teams played fairly mobile big men last season (Ben Wallace and PJ Brown for the Bulls, with an occassional cameo at power forward spot by Andres Nocioni; Drew Gooden and Anderson Verajao for the Cavaliers). Mobile big men can help trap guards (causing more turnovers), while still being able to recover and guard the paint. These big men are also good at rotating quickly for help defense, allowing them to be in position to contest shots and still have good rebounding position. (The Cavaliers also played Zydrunas Ilgauskas, who is not exactly known for being fleet of foot. If the information was readily available, looking at the Cavaliers DTOR with and without Ilgauskas on the floor would be a good way to test my intuition about the role mobile big men play.)

The second path is to be very, very good at DTS% and DRR, but not very good at DTOR. The San Antonio Spurs (DTOR = 20, TS% = 1, DRR = 4) and Houston Rockets (DTOR = 25, TS% = 2, DRR = 1) reached success by following this strategy. For the Rockets, neither Yao Ming nor Dikembe Mutombo was going to successfully trap a guard on the perimeter, but they serve as a strong deterrent in the lane to complement the position defense Jeff Van Gundy has used to great success since his days in New York (it's hard to believe that those Knicks teams were so good on defense despite being so seemingly unathletic). For the Spurs, Tim Duncan is no longer as mobile as he was when he first came into the league, but that sort of big man pressure isn't really the San Antonio way. The Spurs guards chase the shooters off of the 3-point line and into the waiting, shot-blocking arms of a well-positioned Duncan. Solid positioning keeps San Antonio and Houston defenders in place to challenge shots and gobble up the rebounds...and to spots as two of the top defenses in the league.

The third path is to be very good at DTS%, but not particularly good at either DTOR or DRR. The Detroit Pistons (DTOR = 12, TS% = 4, DRR = 24) took this path. In truth, the Pistons are a full step below the other top defensive teams. The Rockets rank fourth in defensive efficiency at 102.18, while the Pistons sit in fifth at 104.68 (for comparison, the Bulls are first at 100.75). More than anything, the Pistons high rank is a testament to the importance of DTS%. Of the top 15 ranked defensive teams, only two of them (the Toronto Raptors, ranked 13, and the Utah Jazz, ranked 15) do not have DTS% within the top 15. The above average mobility of the Pistons big men (Rasheed Wallace and Antonio McDyess) allows them to help cause more turnovers and still recover to contest shots, but they don't rotate and recover quite quickly enough to also grab defensive rebounds effectively.

So what's the moral of the story: Teams can make different trade-offs between our three major defensive categories and still be a top defensive team, but if the trade-offs they make hurt their DTS% then it probably isn't one that will lead to very much defensive success! And without mobile big men, you're unlikely to be able to both force turnovers and defend shots effectively.

More stat talk: Defensive Efficiency

[I know that nothing in this post is particularly new, but I wanted to give an explanation of my ongoing thought process as I figure out what statistics I want to look at when evaluating players.]

If we're going to use statistics to evaluate a player's value to a team, then making sure we're looking at the "right" statistics is obviously the first priority.

Well, what statistics are the right ones? Based on my personal preference, I decided to start my exploration of this issue by looking at the defensive side of the ball. And before you can evaluate an individual players contribution to defensive success, you need to first figure out what factors contribute to a team's defensive success.

The first question I asked was very straightforward--"how can you tell that a team plays good defense?" The answer was also fairly straightforward (and, I assume, not shocking to anyone reading this blog). You want to look at how many points the other team scores each time they have the ball. Defensive Efficiency (also called points per possession) measures how many points a team gives up per 100 possessions. Using Defensive Efficiency as the measure of defensive ability makes it possible to compare the defensive ability of teams that play at vastly different tempos in a way that using points per game, the most common method for evaluating defenses, does not.

Next, I asked, "How can a team decrease the number of points it gives up per possession?" I think that there are three basic ways that a defense can limit an opponents ability to score per possession: (1) preventing the offensive team from ever getting off a shot, (2) forcing the offensive team to miss a shot, and (3) grabbing the ball after an attempted shot (normally by a rebound) to end the offensive team's possession. If these are the ways a defense can limit an opponent's ability to score, then is it possible to find statistics that measure how a defense performs in each of these three categories? At least at the team level (the level with which we are currently concerned), the answer appears to be "yes."

A defense can prevent the other team from getting off a shot by forcing a turnover. Types of turnovers include getting a steal, forcing a bad pass (that results in the ball going out of bounds), forcing some other offensive violation (traveling, 3-seconds, moving screen, charge, etc.), forcing a player out of bounds, or a 24 second violation. From the perspective of evaluating a team’s defense, all of these turnovers are equally valuable. Since we want to be able to compare teams that play at different tempos, the important statistic is turnover rate (the percent of an opponent's possessions that result in a turnover).

A defense can force a team to miss a shot in two principal ways. First, the team can play solid position defense and force the opponent to take an off-balance or contested shot or a shot from further away (or a different position) than the offensive player would prefer. Second, the team can block the attempted shot, preventing it from going in the basket. From the standpoint of the team, either method of forcing a missed shot is equivalent. While the most common way to measure this form of defensive performance is to look at field goal percentage defense, this measure is flawed because it does not take into account the difference between 2-point and 3-point field goal attempts. Effective field goal percentage (eFG%) accounts for the different values of these shots, but still is flawed because it does not take into account fouls committed while the other team is shooting or the defensive team is in the penalty or when defensive three seconds is called (all of which lead to free throws, and thus to points for the other team). True shooting percentage (TS%) accounts for free throws, in addition to 2-pointers and 3-pointers, and therefore gives a more accurate indicator of the opposing team's likelihood of scoring when they have a chance to shoot the ball. In its simplest terms, TS% tells you the percentage that the opposing team would have needed to shoot to score its points if they had only attempted 2-point field goals (i.e. no 3-pointers and no foul shots). For a more detailed comparison of TS% to other measures of offensive efficiency, you can read this article by Kevin Pelton.

After an offensive team has attempted a shot that does not go in, a defensive team can end the offensive team's possession by grabbing the basketball. In most cases, grabbing the basketball will be accomplished by getting the rebound of the missed shot. Since we're interested in comparing the defensive efficiency of team's that play at different tempos, the defensive team's ability to obtain a rebound can best be measured by defensive rebound rate, the percent of available rebounds that the defensive team corrals. In a few cases, the defensive team will grab possession of the ball after a shot has been blocked. Grabbing possession of the ball in this way is considered a rebound (I hadn't known this before, but I just received email confirmation on this fact from the webmaster of sixers.com), so defensive rebound rate accurately reflects a defensive team's full ability to obtain possession of the ball after the other team attempts a shot.

If my thought process is accurate, then we can identify what accounts for a particular team's Defensive Efficiency by looking at the opponent's turnover rate, the opponent's true shooting percentage, and that team's defensive rebound rate. Tomorrow I'll evaluate whether the numbers support my thought process, and I'll also look at what these measurements tell us about what accounts for the defensive efficiency of the NBA's top defensive teams.

More on the assist-to-turnover ratio

If I think the intuition behind using assist-to-turnover ratio to evaluate point guards is that it can be a proxy for the PG's ability to drive and dish, then why don't I think this ratio is particularly useful?

First, just because something can be a proxy doesn't mean that it is going to be a GOOD proxy. A point guard gets credit for assists (and the blame for turnovers) that occur off of set plays and other situations that do not involve him driving and dishing. Turnovers are still bad and assists are still good in these situations, but when these statistics are included in an assist-to-turnover ratio, then that ratio can no longer serve as a useful proxy for evaluating a PG's ability to drive and dish. If the powers-that-be decide to compile an assist-to-turnover ratio that only includes drives into the lane, then we'd have a useful proxy.

The second reason I think assist-to-turnover ratio is not particularly useful is because it treats all passes the same. A higher assist-to-turnover ratio is always considered better, but that just isn't the case. Let's say Steve Nash runs the pick-and-roll with Amare Stoudemire ten times, and all ten times he tries to thread the needle to Stoudemire. Six times he manages to get the pass to Amare (and from 3 feet away Amare almost always dunks it, so we can assume Amare hits all six of his shots after getting the passes from Nash), but four times his passes get intercepted. The Suns have had ten possessions, they've scored twelve points, and Nash has a 3:2 assist-to-turnover ratio. Now let's say Steve Nash runs a high pick-and-roll ten times with Kurt Thomas, gets into the lane each time, and then kicks the ball back out to one of the Suns' shooters when the defense collapses. There are a lot of long arms in the way, so two of his passes are picked-off, but he still gets eight of them to his shooters. Since the shooters are open, they knock down their shots at a 75% clip (six of eight). The Suns have had ten possessions, they've scored twelve points, and Nash has a 3:1 assist-to-turnover ratio. Does Nash's improved assist-to-turnover ratio in the second scenario make him a better point guard? From the standpoint of the Suns offense, the answer is clearly "No."

Why not? Despite what assist-to-turnover ratio tells us, not all passes are the same. The types of shots to which those passes lead matter a great deal--more specifically, the likelihood of players making the shots to which those passes lead makes a big difference. The higher percentage the shot created by a point guard's pass, the worse a PG's assist-to-turnover ratio can be and still be helpful to a team's chances to win a game.

A "perfect" PG could theoretically compute the chance of scoring based on each potential pass (evaluating both the likelihood of the pass being intercepted and the likelihood of the shot resulting from the pass being made if the pass is not intercepted) and always choose the pass with the highest expected scoring value. Making these decisions is complicated enough under our simple model, but becomes even more complicated once you take into account the fact that different teams are better at defending different shots. For instance, Amare Stoudemire might normally make 100% of his shots from in close, but the Spurs interior defense gives him problems so that he "only" makes 2/3 of his shots from in close. Now, it might no longer make sense for Nash to try and thread his passes inside to Stoudemire on the pick-and-roll. In practice, point guards make the decision where to pass based on "feel"--an intuitive, split-second thought process, based on years of playing, that helps them identify the pass that has "best" chance of resulting in a positive play. This feeling may or may not be right (after all, most point guards have been coached to protect the ball at all costs so they might be very risk-averse), but assist-to-turnover ratio by itself does not help us properly evaluate that decision in any useful way.

The situation gets complicated even further when we recognize that we are also generally making two further assumptions. First, we are assuming that the shot taken by the "open" man is likely to be made at a higher percentage than the shot the point guard could have taken once he had penetrated the lane, and that the open man's shot will be made at a higher enough percentage that the potential turnovers are worthwhile. Going back to our previous examples, if Steve Nash can hit his shots in the lane at a 60% clip, then him shooting the ball is just as good as either of the other two options. And with no assists and no turnovers, he won't have an assist-to-turnover ratio at all!

Second, and related to the previous point, we're assuming that we want an offense based around a point guard (or other player) penetrating and passing in the first place. Once the possibility of turnovers on passes is taken into account, a team's best offensive strategy might actually be to just let players create shots individually. A team's shooting percentage might be lower, but it could end up scoring more points because the lack of turnovers means it is taking more shots. The Dallas Mavericks are an example of a team that has gone this route. Using a steady diet of foul-line isolations (Dirk on a smaller guard after a switch is the favorite, but they also run this play for other other players), the Dallas Mavericks were 25th in assists per game in 2007, but were 2nd in offensive efficiency. (I couldn't find team assists adjusted for pace, but I think my point is still clear despite using these mismatched statistics)

Wow. My discussion of assist-to-turnover ratio certainly ballooned into something a bit more expansive, but I think it was all related. Wade through it all and (I think) my basic point is this: assist-to-turnover ratio can give us some useful information, but it falls short as a statistical measure because it unfairly punishes high-risk/high-reward passes that can be just as helpful to a team over the course of an entire game as the lower-risk/lower-reward passes that often result in a "better" assist-to-turnover ratio.

Assist to Turnover ratio: Fool's gold?

In response to my last post, commenter Louis singles out the assist to turnover ratio as a statistic that is overvalued. I'm mostly in agreement with the point he makes, but I do want to quibble with it a little bit.

Louis says:

I would rather know a player’s steals-to-turnover ratio because stealing the ball is the opposite of giving it away.

I think he makes a good point here, and one that a number of statistical evaluation systems have taken into account. By comparing steals-to-turnovers you can see how many possessions a player gains for his team (i.e. steals) versus how many the player loses for his team (i.e. turnovers). Steals, like defensive rebounds, take a possession away from the other team and give it to your team. If anything, I think I'd want to broaden the category of "steals" to include other forced turnovers that do not get recorded as steals, such as charges taken and "over-the-backs" (is there a technical name for that violation?) drawn. A turnovers forced to turnovers committed ratio would identify the players that help a team win by creating additional possessions for their team. That said, I would point out that apart from their relationship to the number of possessions created/lost for a team, turnovers forced and turnovers committed have very little to do with one another. These measures involve very different skill-sets, and are unlikely (I would guess) to have much relationship to one another.

My quibble with Louis' point comes in here. Unlike Louis, I think that turnovers and assists are actually related to one another. Assist-to-turnover ratio is normally brought up when point guards are being evaluated, and I think there is a good reason for this relationship. The intuition is that the defense collapses when a point guard penetrates. The better the point guard, the more likely the point guard is going to be able to find the man that has been left open by the sagging defender. The worse the point guard, the more likely that his pass to the "open" man will be intercepted (either because he makes a physically bad pass or he was incorrect in believing that the man he was passing to was open). With this bit of intuition in hand, we can see that assist-to-turnover ratio can function as a proxy for a point guard's ability to drive and dish successfully.

Despite its ability to serve as a proxy for the ability to drive and dish, I still agree with Louis that assist-to-turnover ratio isn't a particularly useful statistic as currently used. I'll elaborate on my reasons for taking this position tomorrow. In the mean time, let me know what you think of my intuition regarding the assist-to-turnover ratio.