Skip to contents

Visualizing a compboost model

visualizations.Rmd

Compboost comes with a variety of function to gain deeper insights into a fitted model. Using these function allows to get different views on the model.

Fit compboost

The data set we use is mpg:

mpg cyl disp hp drat wt qsec vs am gear carb
Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4
Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4
Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1
Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1
Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0 3 2
Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0 3 1

We want to model the miles per gallon (mpg). As features we include the linear and centered spline of hp, wt, and qsec. Additionally, we add a categorical base learner for the number of cylinders cyl:

mtcars$cyl = as.factor(mtcars$cyl)

set.seed(31415)
cboost = Compboost$new(data = mtcars, target = "mpg", learning_rate = 0.02, oob_fraction = 0.2)

cboost$addComponents("hp", df = 3)
cboost$addComponents("wt", df = 3)
cboost$addComponents("qsec", df = 3)
cboost$addBaselearner("cyl", "ridge", BaselearnerCategoricalRidge, df = 3)

cboost$train(500L, trace = 100L)
#>   1/500   risk = 17  oob_risk = 17   
#> 100/500   risk = 3  oob_risk = 2.8   
#> 200/500   risk = 2.4  oob_risk = 3.2   
#> 300/500   risk = 2.3  oob_risk = 3.2   
#> 400/500   risk = 2.3  oob_risk = 3.1   
#> 500/500   risk = 2.2  oob_risk = 3.1   
#> 
#> 
#> Train 500 iterations in 0 Seconds.
#> Final risk based on the train set: 2.2

Visualize risk, feature importance, and selection traces

A starting point when analyzing a component-wise boosting model is to take a look at the train and validation risk:

plotRisk(cboost)

As we can see, the best validation risk is at iteration 98. Hence, we should set the model to this iteration:

m_optimal = which.min(cboost$getLoggerData()[["oob_risk"]])
cboost$train(m_optimal)

Next, we are interested in the most important base learners/features:

The last thing we can do to get a more better overview of the model is to have a look how the features/base learners were included into the model:

Visualize base learner and partial effects

Next, we want to deep dive into the effects of individual features, i.e, the effect of the base learners. To that end, we plot the partial effects of the most important feature wt:

plotPEUni(cboost, "wt")

We observe a clear negative trend, meaning that an increasing weight indicates lower mpg. Additionally, we can visualize individual base learners. For example the only categorical feature cyl:

plotBaselearner(cboost, "cyl_ridge")

Here, we observe that 4 cylinder indicates a positive contribution to mpg while 6 and 8 cylinder are reducing it.

Visualizing individual predictions

During predictions we also want to get an idea about the specific contribution of each feature to the predicted score. Therefore, we take a look at the first observation in the validation data set:

plotIndividualContribution(cboost, newdata = cboost$data_oob[1, ])

As we can see, the prediction is dominated by the offset. To remove it from the figure we set offset = FALSE:

plotIndividualContribution(cboost, newdata = cboost$data_oob[1, ], offset = FALSE)

The wt and hp have a positive contribution to the predicted score. This means the car requires less fuel while the 6 cylinder slightly increases the mpg prediction.

Visualizing tensor products

The last visualization convenience wrapper is to illustrate interactions included as tensors. Therefore, we have to add tensors into the model:

mtcars$vs = as.factor(mtcars$vs)
mtcars$gear = as.factor(mtcars$gear)

set.seed(31415)
cboost = Compboost$new(data = mtcars, target = "mpg", oob_fraction = 0.2)

cboost$addTensor("wt", "qsec", df = 2)
cboost$addTensor("hp", "cyl", df = 2)
cboost$addTensor("gear", "vs", df = 2)

cboost$train(500L, trace = 100L)
#>   1/500   risk = 16  oob_risk = 16   
#> 100/500   risk = 2.4  oob_risk = 4   
#> 200/500   risk = 2.2  oob_risk = 4.4   
#> 300/500   risk = 2.1  oob_risk = 4.4   
#> 400/500   risk = 2.1  oob_risk = 4.5   
#> 500/500   risk = 2  oob_risk = 4.5   
#> 
#> 
#> Train 500 iterations in 0 Seconds.
#> Final risk based on the train set: 2
table(cboost$getSelectedBaselearner())
#> 
#> gear_vs_tensor  hp_cyl_tensor wt_qsec_tensor 
#>            258            181             61

Depending on the feature combination (numeric - numeric, numeric - categorical, categorical - categorical) a different visualization technique is used:

library(ggplot2)

gg1 = plotTensor(cboost, "wt_qsec_tensor") + ggtitle("Num - Num")
gg2 = plotTensor(cboost, "hp_cyl_tensor") + ggtitle("Num - Cat")
gg3 = plotTensor(cboost, "gear_vs_tensor") + ggtitle("Cat - Cat")

library(patchwork)

gg1 | gg2 | gg3