Chapter 19 Conditional Dependency Profiles

One of the largest advantages of the Partial Dependency Profiles is that they are easy to explain, as they are just an average across Ceteris Paribus profiles. But one of the largest disadvantages lies in expectation over marginal distribution which implies that \(x^j\) is independent from \(x^{-j}\). In many applications this assumption is violated. For example, for the apartments dataset one can expect that features like \(surface\) and \(number.or.rooms\) are strongly correlated as apartments with larger number of rooms usually have larger surface. It may makes no sense to consider an apartment with 10 rooms and 20 square meters, so it may be misleading to change \(x^{surface}\) independently from \(x^{number.of.rooms}\). In the titanic dataset we shall expect correlation between fare and passanger class as tickets in the 1st class are the most expensive.

There are several attempts to fix this problem. Here we introduce Local Dependency Profiles presented in the (Apley 2018b) under the name M-profiles.

The general idea is to use conditional distribution instead of marginal distribution to accomodate for the dependency between \(x^j\) and \(x^{-j}\).

19.1 Definition

Conditional Dependency Profile for a model \(f\) and a variable \(x^j\) is defined as

\[ g_{CD}^{f, j}(z) = E[f(X^j, X^{-j})|X^j = z]. \]

So it’s an expected value over conditional distribution \((X^j,X^{-j})|X^j=z\).


Let \(f = x_1 + x_2\) and distribuion of \((x_1, x_2)\) is given by \(x_1 \sim U[0,1]\) and \(x_2=x_1\).

Calculate \(g_{CD}^{f, 1}(z)\).

Answer \(g_{CD}^{f, 1}(z) = 2*z\).

19.2 Estimation

Partial Dependency Profiles are defined as an expected value from Ceteris Paribus Profiles.

\[ g^{PD}_i(z) = E_{X_{-i}}[ f(x|^i = z, x^{-i}) ]. \] And can be estimated as average from CP profiles.

\[ \hat g^{PD}_i(z) = \frac{1}{n} \sum_{j=1}^{n} f(x|^i = z, x_j^{-i}). \]

As it was said, if \(X_i\) and \(X_{-i}\) are related it may have no sense to average CP profiles over marginal \(X_{-i}\). Instead, an intuitive approach would to use a conditional distribution \(X_{-i}|X_i=x_i\).

\[ g^{M}_i(z) = E_{X_{-i}|X_i=x_i}[ f(x|^i = z, x^{-i}) ]. \]

19.3 Example

See Figure ?? for illustration of difference between marginal and conditional distribution. Such profiles are called Conditional Dependency Profiles and are estimated as

\[ \hat g^{M}_i(z) = \frac{1}{|N_i|} \sum_{j\in N_i} f(x|^i = z, x_j^{-i}). \] where \(N_i\) is the set of observations with \(x_i\) close to \(z\).


Figure 19.1: (fig:accumulatedCor)

As it is justified in (Apley 2018b), there is a serious problem with this approach, illustrated by a following observation. If \(y\) depends on \(x_2\) but not \(x_1\) then the correlation between \(x_1\) and \(x_2\) will produce a false relation in the Marginal profiles for feature \(x_1\). This problem is also illustrated in the Figure ??.

Conditional Dependency Profile for 100 observations

(#fig:mp_part_1)Conditional Dependency Profile for 100 observations


Apley, Dan. 2018b. ALEPlot: Accumulated Local Effects (Ale) Plots and Partial Dependence (Pd) Plots.