General idea
Below graphics show general idea used in xspliner model 
Black box variable response
When black box model is already built, you may want to check how each predictor variable affects the final response. This is quite easy when you use a linear model or have low dimensional data (up to 2, 3 dimensions). One of the ideas for testing the predictor impact in more complicated model is to check an average model response of the selected variable.
One of such approaches called Partial Dependence Plots is implemented in the pdp package (Brandon M. Greenwell (2017). Pdp: The R Journal, 9 (1), 421-436.), or the ALEPlot package (Dan Apley (2017). ALEPlot: Accumulated Local Effects Plots and Partial Dependence Plots.) using Accumulated Local Effects Plots.
In each case, we get a single variable function, which should explain the impact of the predictor on the response variable.
The following pictures show the ptratio impact on cmedv in some random forest model based on the Boston Housing Data. Below curves are obtained by the approach of the pdp and ale methods respectively.
As we can see, above functions are irregular, making it difficult to interpret explained effect.
If above functions had linear character, one would be tempted to approximate them with linear function. As a result it could be easy to interpret how it affects the variable explained in the black box model. What if the function is irregular, as above? Could we approximate it with polynomials?
Splines
Due to the large errors that occur with approximation of functions with polynomials, the approach using spline approximation is the most common solution. Splines (functions that are piecewise polynomials) have good approximating properties, in addition their form is overt so we can thus interpret the resulting function.
The following graphics show spline approximations of pdp and ale curves:
But even with approximated PDP, how could we use it to interpret black box model?
Linear models based on response function and splines
The general idea of how to use the response function and splines to build an interpretable model that could be used as black box explainer is as follows.
- For each variable used in the black box model, create a response function based on one of the known methods, let’s mark it \(f_{x}\)
- Approximate \(f_{x}\) using spline - the result is \(\widetilde{f}_{x}\)
- Build linear model, in which each predictor \(x_{i}\) is transformed with approximated response function: \(\widetilde{f}_{x_{i}}(x_{i})\)
Shortly, using black box formula \[y \sim x_{1} + \cdots + x_{n}\] use \[y \sim \widetilde{f}_{x_{1}}(x_{1}) + \cdots + \widetilde{f}_{x_{n}}(x_{n})\] in linear one.
The resulting model uses a part of the information that was extracted while building black box model. If linear model performance is similar to black box one, it could be used as it’s good interpretation.
