# Chapter 9 Break Down for Additive Variable Attributions

In the Section 7 we introduced a method for assessment of local variable importance based on Ceteris Paribus Profiles. But the main disadvantage of this method is that importance scores do not sum up to final model predictions.

In this chapter we introduce Break Down Plots which solve this problem. Note that the described method is also similar to the EXPLAIN algorithm introduced in (Robnik-Šikonja and Kononenko 2008) and implemented in (Robnik-Sikonja 2018) package.

## 9.1 Intuition

For any model we may repeat the intuition presented in the Section 30 to calculate variable contribution as shifts in expected model response after conditioning over consecutive variables. This intuition is presented in Figure 9.1.

Panel A shows distribution of model responses. The row `all data`

shows the model response of the validation dataset. The red dot stands for average model response and it is an estimate of expected model response \(E [f(x)]\).

Since we want to calculate effects of particular values of selected variables we then condition over these variables in a sequential manner.
The next row in panel A corresponds to average model prediction for observations with variable `class`

fixed to value `1st`

. The next for corresponds to average model prediction with variables `class`

set to `1st`

and `age`

set to `0`

, and so on. The last row corresponds to model response for \(x_*\).

Black lines in the panel A show how prediction for a single point changes after coordinate \(j\) is replaced by the \(x_*^j\), so they span between \(f(x)\) and \(f(x^{j|=x_*^j})\). But finally we are not interested in particular changes, not even in distributions but only in averages - expected model responses.

The most minimal form that shows important information is presented in the panel C. Positive values are presented with green bars while negative differences are marked with red bar. They sum up to final model prediction, which is denoted by a violet bar in this example.

## 9.2 Method

Again, as in previous chapter, let \(v(j, x_*)\) stands for the contribution of variable \(j\) to prediction of model \(f\) in point \(x_*\).

We expect that such contribution will sum up to the model prediction in a given point (property called *local accuracy*), so
\[
f(x_*) = v_0 + \sum_{j=1}^p v(j, x_*)
\]
where \(v_0\) stands for average model response (it may be different for different models).

Note that the equation above may be rewritten as

\[ E [f(X)|X^1 = x^1_*, \ldots, X^p = x^p_*] = E[f(X)] + \sum_{j=1}^p v(j, x_*) \] what leads to quite natural proposition for \(v(j, x_*)\), such as

\[ v(j, x_*) = E [f(X) | X^1 = x^1_*, \ldots, X^j = x^j_*] - E [f(X) | X^1 = x^1_*, \ldots, X^{j-1} = x^{j-1}_*] \] In other words the contribution of variable \(j\) is the difference between expected model response conditioned on first \(i\) variables minus the model response conditioned on first \(j-1\) variables.

To simplify notation, let’s define a symbol \(\Delta^{i|J}\) as \[ \Delta^{i|J} = E [f(X) | X^{J \cup \{i\}} = x^{J \cup \{i\}}_*] - E [f(X) | X^{J} = x^{J}_*], \] So \(\Delta^{i|J}\) is the change between expectation over variables from the set \(J \cup \{i\}\) minus expectation over variables from the set \(J\).

Then

\[ v(j, x_*) = \Delta^{j|\{1, ..., j-1\}}. \]

Such proposition fulfills the *local accuracy* condition.

Unfortunately, for non-additive models, variable contributions depend on the ordering of variables. See for example Figure 9.2. In the first ordering the contribution of variable `age`

is calculated as 0.01, while in the second the contribution is calculated as 0.13. Such differences are related to the lack of additivness of the model \(f()\).

There are different attempts to solve the problem with the ordering.

A. choose an ordering in which variables with largest contributions are first. In this chapter we will describe a heuristic behind this approach. B. identify interactions that causes difference in attributions for different orderings and show these interactions. In the chapter 10 we will describe a heuristic behind this idea. C. calculate average across all possible orderings. There is \(p!\) possible orderings, be the may quite accurately approximate the average. This approach will be presented in the chapter 11.

So, let’s start with approach A. The easiest way to solve this problem is to use two-step procedure. In the first step variables are ordered and in the second step the consecutive conditioning is applied to ordered variables.

First step of this algorithm is to determine the order of variables for conditioning. It seems to be reasonable to include first variables that are likely to be most important, leaving the noise variables at the end. This leads to order based on following scores

\[ \Delta^j = \left| E [f(X)] - E [f(X)|X^j = x_*^j] \right| \] Note, that the absolute value is needed as variable contributions can be both positive and negative.

Once the ordering is determined in the second step variable contributions are calculated as

\[ v(j, x_*) = E [f(X) | X^{J \cup \{i\}} = x^{J \cup \{i\}}_*] - E [f(X) | X^{J} = x^{J}_*] = \Delta ^{i|J} \] where \(J\) is the set of variables that have scores \(\Delta^i\) smaller than score for variable \(i\).

\[ J = \{j: \Delta^{j} < \Delta^{i}\} \]

The time complexity of the first step is \(O(p)\) where \(p\) is the number of variables and the time complexity of the second step is also \(O(p)\).

## 9.3 Example: Titanic data

Let us consider a random forest model `titanic_rf_v6`

and the passenger `johny_d`

as defined in section 4.1.7.

Average model response for all passengers for this model is \(v_0 = 0.2356585\). For each coordinate of `johny_d`

we can calculate scores \(\Delta^j\) and they are presented in the table 9.1. These scores determine the order in which we do the conditioning.

variable \(j\) | \(E[f(X : x^j = x^j_*)]\) | \(\Delta^j\) |
---|---|---|

age | 0.7407795 | 0.5051210 |

class | 0.6561034 | 0.4204449 |

fare | 0.6141968 | 0.3785383 |

sibsp | 0.4786182 | 0.2429597 |

parch | 0.4679240 | 0.2322655 |

embarked | 0.4602620 | 0.2246035 |

gender | 0.3459458 | 0.1102873 |

Once the order is determined we can calculate sequential contributions \(\Delta^{i|J}\). For `johny_d`

and the model unde consideration, these coefficients are listed in Table 9.2.

variable \(i\) | \(E[f(X : x^I = x^I_*)]\) | \(\Delta^{i|J}\) |
---|---|---|

intercept | 0.2353095 | 0.2353095 |

age = 8 | 0.5051210 | 0.2698115 |

class = 1st | 0.5906969 | 0.0855759 |

fare = 72 | 0.5443561 | -0.0463407 |

gender = male | 0.4611518 | -0.0832043 |

embarked = Southampton | 0.4584422 | -0.0027096 |

sibsp = 0 | 0.4523398 | -0.0061024 |

parch = 0 | 0.4220000 | -0.0303398 |

prediction | 0.4220000 | 0.4220000 |

These results can be visually presented with waterfall plot as in the Figure 9.3.

## 9.4 Pros and cons

Break Down approach is model agnostic, can be applied to any predictive model that returns a single number. It leads to additive variable attribution. Below we summarize key strengths and weaknesses of this approach.

**Pros**

- Break Down Plots are easy to understand and decipher.
- Break Down Plots are compact; many variables may be presented in a small space.
- Break Down Plots are model agnostic yet they reduce to intuitive interpretation for linear Gaussian and generalized models.
- Complexity of Break Down Algorithm is linear in respect to the number of variables.

**Cons**

- If the model is non-additive then showing only additive contributions may be misleading.
- Selection of the ordering based on scores is subjective. Different orderings may lead to different contributions.
- For large number of variables the Break Down Plot may be messy with many variables having small contributions.

## 9.5 Code snippets for R

In this section we present key features of the `iBreakDown`

package for R (Gosiewska and Biecek 2019a) which is a part of `DrWhy.AI`

universe. This package covers all features presented in this chapter. It is available on CRAN and GitHub. Find more examples at the website of this package `https://modeloriented.github.io/iBreakDown/`

.

In this section, we use a random forest classification model developed in the chapter 4.1, namely the `titanic_rf_v6`

model. It is trained to predict probability of survival from sinking of Titanic. Instance level explanations are calculated for a single observation `johny_d`

- 8 years old passenger that travels 1st class.

`DALEX`

explainers for both models and the Henry data are retrieved via `archivist`

hooks as listed in Chapter 4.1.7.

```
library("randomForest")
explain_rf_v6 <- archivist::aread("pbiecek/models/9b971")
library("DALEX")
johny_d <- archivist::aread("pbiecek/models/e3596")
johny_d
```

### 9.5.1 Basic usage for the `break_down`

function

The `iBreakDown::break_down()`

function calculates Break Down contributions for a selected model around a selected observation.

The result from `break_down()`

function is a data frame with additive attributions for selected observation.

The simplest use case is to set only the arguments - model explainers and observation of interest.

Note that the table below recreates values presented in Table 9.2.

```
## contribution
## Random Forest v6: intercept 0.235
## Random Forest v6: age = 8 0.270
## Random Forest v6: class = 1st 0.086
## Random Forest v6: fare = 72 -0.046
## Random Forest v6: gender = male -0.083
## Random Forest v6: embarked = Southampton -0.003
## Random Forest v6: sibsp = 0 -0.006
## Random Forest v6: parch = 0 -0.030
## Random Forest v6: prediction 0.422
```

The generic `plot()`

function creates Break Down plots.

Note that the plot below recreates Figure 9.3.

### 9.5.2 Advanced usage for the `break_down`

function

The function `break_down()`

can take more arguments. The most commonly used are:

`x`

a wrapper over a model created with function`DALEX::explain()`

,`new_observation`

an observation to be explained is should be a data frame with structure that matches the training data,`order`

if specified then it can be a vector of characters (column names) or integers (column indexes) that specify order of variable conditioning. If not specified (default) then a one-step heuristic is used to determine the order,`keep_distributions`

logical value.

if`TRUE`

, then additional diagnostic information is about conditional distributions is stored and can be plotted with the generic`plot()`

function.

Let’s see these additional arguments in action.

First we will specify order. You can use integer indexes or variable names. Note that the second option is in most cases better because of higher readability. Additionally, to reduce clutter in the plot we set `max_features = 3`

argument in the `plot()`

function.

```
library("iBreakDown")
bd_rf_order <- break_down(explain_rf_v6,
johny_d,
order = c("class", "age", "gender", "fare", "parch", "sibsp", "embarked"))
plot(bd_rf_order, max_features = 3)
```

The `plot_distributions = TRUE`

argument of `break_down()`

function enriches model response with additional information about conditional distribution.

It can be presented after setting `plot_distributions = TRUE`

in the `plot()`

function. Conditional distributions are presented as vioplots. Red dots stand for conditional average model response. Thin black lines between vioplots correspond to predictions for individual observations. With them we can trace how model predictions change after consecutive conditioning.

### References

Robnik-Šikonja, Marco, and Igor Kononenko. 2008. “Explaining Classifications for Individual Instances.” *IEEE Transactions on Knowledge and Data Engineering* 20 (5): 589–600. https://doi.org/10.1109/TKDE.2007.190734.

Robnik-Sikonja, Marko. 2018. *ExplainPrediction: Explanation of Predictions for Classification and Regression Models*. https://CRAN.R-project.org/package=ExplainPrediction.

Gosiewska, Alicja, and Przemyslaw Biecek. 2019a. “iBreakDown: Uncertainty of Model Explanations for Non-additive Predictive Models.” https://arxiv.org/abs/1903.11420v1.