Shapley Values and SHAP

Slides on Shapley Values & SHAP: Link to the slides

1 Lab Python: Shapley values

• Google colab: https://colab.research.google.com/drive/1nqxoOUASndbRnN1G6CMsZ8z4aV6vVKqW

2 Lab Python: SHAP

• Google colab: https://colab.research.google.com/drive/1aGcF9oHqh1qj13J2D9WNRpDuGUE7ccQ2

3 Shapley values in R

There is various packages. Here we rely on DALEX and shapviz packages. For computing shap values we additionally use the kernelshap package.

4 Lab R: Shapley values

• Source: Biecek and Burzykowski (2021), Chapter 8

4.1 Prepare data and train model

First, we’ll run the setup code to train our Random Forest model.

# Load necessary libraries
pacman::p_unload(pacman::p_loaded(), character.only = TRUE)
library(conflicted)
library(knitr)
library(kableExtra)
library(tidyverse)
library(tidymodels)
library(DALEX)
conflicts_prefer(DALEX::explain)


# Load and prepare data
#data <- read_csv(url(sprintf("https://docs.google.com/uc?id=%s&export=download",
#                         "1dnCK79T45Qa7RZrDg1qv6-EdGBoxCPjv")))
data <- read_csv("data/data_acspubliccoverage.csv")
data <- data %>% mutate(across(where(is.character), as.factor)) %>%
  sample_n(2000)

# Split the data
set.seed(123)
data_split <- initial_split(data, prop = 0.80, strata = public_coverage)
data_train <- training(data_split)
data_test  <- testing(data_split)

# Define the recipe and model spec
recipe_rf <- recipe(public_coverage ~ ., data = data_train)%>%
  step_impute_median(all_numeric_predictors()) %>% # Impute numeric NAs
  step_impute_mode(all_nominal_predictors())      # Impute categorical NAs

model_rf <- rand_forest(mode = "classification") %>% 
  set_engine("ranger") |> 
  set_mode("classification")

# Create and fit the workflow
workflow_rf <- workflow() %>%
  add_recipe(recipe_rf) %>%
  add_model(model_rf)
fit_rf <- fit(workflow_rf, data = data_train)

4.2 Create explainer

Then we construct the explainer for the model by using function explain() from the DALEX package.

# The predict function MUST match the outcome levels ("Yes", "No")
p_fun <- function(object, newdata) {
  predict(object, new_data = newdata, type = "prob")$.pred_Yes
}

# Create the explainer
explainer <- explain(
  model = fit_rf,
  data = select(data_test, -public_coverage),
  y = as.numeric(data_test$public_coverage == "Yes"),
  predict_function = p_fun,
  label = "Random Forest",
  type = "classification"
)

Preparation of a new explainer is initiated
  -> model label       :  Random Forest 
  -> data              :  400  rows  18  cols 
  -> data              :  tibble converted into a data.frame 
  -> target variable   :  400  values 
  -> predict function  :  p_fun 
  -> predicted values  :  No value for predict function target column. (  default  )
  -> model_info        :  package Model of class: workflow package unrecognized , ver. Unknown , task regression (  default  ) 
  -> model_info        :  type set to  classification 
  -> predicted values  :  numerical, min =  0.02397653 , mean =  0.3682996 , max =  0.9778514  
  -> residual function :  difference between y and yhat (  default  )
  -> residuals         :  numerical, min =  -0.7695264 , mean =  -0.008299591 , max =  0.9066654  
  A new explainer has been created!  

4.3 Pick a person and get prediction

We pick out one person. Here we simply pick the first person in the test dataset data_test and store that person’s data in the object person1.

person1 <- data_test %>% slice(100)

# View(person1)
kable(person1) # See the data of the person

public_coverage	income	age	education	marital_status	sex	disability	parent_employment	citizenship	mobility	military_service	ancestry	nativity	hearing_difficulty	vision_difficulty	cognitive_difficulty	employment	gave_birth	race
Yes	14800	39	No HS Diploma	Separated	Female	Without disability	NA	Not a citizen	Same house	Never served	Single	Foreign born	No	No	No	Not in labor force	No	White

And we can also get our models prediction for that person using the generic predict() function that works for explainer objects.

predict(explainer, person1)

[1] 0.5789295

To compute Shapley values for person1, we apply the function predict_parts() to the explainer-object explainer and the data frame for person1.

• type="shap" argument: we indicate that we want to compute Shapley values.
• B = 25 argument: select 25 random orderings (= coalations = feature permutations) of explanatory variables for which Shapley values are to be computed (note that B = 25 is the default but it is very low - pick a higher value)

The resulting object shap_person1 is a data frame with variable-specific attributions computed for every ordering.

shap_person1 <- predict_parts(explainer = explainer, 
                      new_observation = person1, 
                                 type = "shap",
                                    B = 25)

Printing out the object provides various summary statistics (below we will see the full table) of the attributions including the mean. age = 30 indicated the feature name as well as the value for that particular person.

#shap_person1

4.4 Understanding the SHAP Values Table

This table shows how each feature influenced the prediction. Let’s understand each column:

• Row names & variable: Contain the names of the features (e.g., education, mobility, race)
• contribution: The SHAP value - how much this feature changed the prediction
• sign: Direction of influence (1.0 = pushes UP, -1.0 = pushes DOWN)

# View(shap_person1)
kable(shap_person1) %>%
  #kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive")) %>%
  scroll_box(width ="600px", height = "300px")

	variable	contribution	variable_name	variable_value	sign	label	B
age	age = 26	-0.0052771	age	26	-1	Random Forest	0
ancestry	ancestry = Multiple	-0.0226531	ancestry	Multiple	-1	Random Forest	0
citizenship	citizenship = Citizen by birth (US)	0.0066929	citizenship	Citizen by birth (US)	1	Random Forest	0
cognitive_difficulty	cognitive_difficulty = Yes	0.0615974	cognitive_difficulty	Yes	1	Random Forest	0
disability	disability = With disability	0.0776482	disability	With disability	1	Random Forest	0
education	education = High School or GED	0.0319113	education	High School or GED	1	Random Forest	0
employment	employment = Unemployed	0.0555451	employment	Unemployed	1	Random Forest	0
gave_birth	gave_birth = NA	-0.0004880	gave_birth	NA	-1	Random Forest	0
hearing_difficulty	hearing_difficulty = No	0.0004981	hearing_difficulty	No	1	Random Forest	0
income	income = 0	-0.0372371	income	0	-1	Random Forest	0
marital_status	marital_status = Never married	0.0140615	marital_status	Never married	1	Random Forest	0
military_service	military_service = Never served	-0.0011935	military_service	Never served	-1	Random Forest	0
mobility	mobility = Same house	-0.0004345	mobility	Same house	-1	Random Forest	0
nativity	nativity = Native	0.0036920	nativity	Native	1	Random Forest	0
parent_employment	parent_employment = NA	-0.0020347	parent_employment	NA	-1	Random Forest	0
race	race = Some Other Race	0.0082222	race	Some Other Race	1	Random Forest	0
sex	sex = Male	0.0174977	sex	Male	1	Random Forest	0
vision_difficulty	vision_difficulty = Yes	0.0025816	vision_difficulty	Yes	1	Random Forest	0
disability1	disability = With disability	0.1433866	disability	With disability	1	Random Forest	1
vision_difficulty1	vision_difficulty = Yes	-0.0198850	vision_difficulty	Yes	-1	Random Forest	1
sex1	sex = Male	0.0737361	sex	Male	1	Random Forest	1
age1	age = 26	-0.0235503	age	26	-1	Random Forest	1
ancestry1	ancestry = Multiple	-0.0457524	ancestry	Multiple	-1	Random Forest	1
mobility1	mobility = Same house	-0.0005982	mobility	Same house	-1	Random Forest	1
marital_status1	marital_status = Never married	-0.0009053	marital_status	Never married	-1	Random Forest	1
employment1	employment = Unemployed	0.0690661	employment	Unemployed	1	Random Forest	1
income1	income = 0	-0.1083327	income	0	-1	Random Forest	1
military_service1	military_service = Never served	-0.0020356	military_service	Never served	-1	Random Forest	1
cognitive_difficulty1	cognitive_difficulty = Yes	0.0559523	cognitive_difficulty	Yes	1	Random Forest	1
citizenship1	citizenship = Citizen by birth (US)	0.0107551	citizenship	Citizen by birth (US)	1	Random Forest	1
nativity1	nativity = Native	0.0030299	nativity	Native	1	Random Forest	1
education1	education = High School or GED	0.0539288	education	High School or GED	1	Random Forest	1
hearing_difficulty1	hearing_difficulty = No	0.0003548	hearing_difficulty	No	1	Random Forest	1
parent_employment1	parent_employment = NA	-0.0008103	parent_employment	NA	-1	Random Forest	1
gave_birth1	gave_birth = NA	-0.0002279	gave_birth	NA	-1	Random Forest	1
race1	race = Some Other Race	0.0025179	race	Some Other Race	1	Random Forest	1
sex2	sex = Male	0.0263508	sex	Male	1	Random Forest	2
ancestry2	ancestry = Multiple	-0.0217455	ancestry	Multiple	-1	Random Forest	2
nativity2	nativity = Native	-0.0020847	nativity	Native	-1	Random Forest	2
disability2	disability = With disability	0.1550759	disability	With disability	1	Random Forest	2
education2	education = High School or GED	0.0403252	education	High School or GED	1	Random Forest	2
income2	income = 0	-0.0363254	income	0	-1	Random Forest	2
employment2	employment = Unemployed	0.0286255	employment	Unemployed	1	Random Forest	2
parent_employment2	parent_employment = NA	-0.0010559	parent_employment	NA	-1	Random Forest	2
hearing_difficulty2	hearing_difficulty = No	0.0002020	hearing_difficulty	No	1	Random Forest	2
age2	age = 26	-0.0348551	age	26	-1	Random Forest	2
vision_difficulty2	vision_difficulty = Yes	-0.0030717	vision_difficulty	Yes	-1	Random Forest	2
military_service2	military_service = Never served	-0.0012421	military_service	Never served	-1	Random Forest	2
citizenship2	citizenship = Citizen by birth (US)	0.0082154	citizenship	Citizen by birth (US)	1	Random Forest	2
marital_status2	marital_status = Never married	-0.0108917	marital_status	Never married	-1	Random Forest	2
gave_birth2	gave_birth = NA	-0.0003826	gave_birth	NA	-1	Random Forest	2
cognitive_difficulty2	cognitive_difficulty = Yes	0.0597258	cognitive_difficulty	Yes	1	Random Forest	2
race2	race = Some Other Race	0.0021637	race	Some Other Race	1	Random Forest	2
mobility2	mobility = Same house	0.0016003	mobility	Same house	1	Random Forest	2
vision_difficulty3	vision_difficulty = Yes	0.0098084	vision_difficulty	Yes	1	Random Forest	3
cognitive_difficulty3	cognitive_difficulty = Yes	0.1009293	cognitive_difficulty	Yes	1	Random Forest	3
gave_birth3	gave_birth = NA	-0.0005277	gave_birth	NA	-1	Random Forest	3
income3	income = 0	-0.0201955	income	0	-1	Random Forest	3
education3	education = High School or GED	0.0295420	education	High School or GED	1	Random Forest	3
age3	age = 26	0.0042447	age	26	1	Random Forest	3
ancestry3	ancestry = Multiple	-0.0072177	ancestry	Multiple	-1	Random Forest	3
sex3	sex = Male	0.0122144	sex	Male	1	Random Forest	3
hearing_difficulty3	hearing_difficulty = No	0.0006159	hearing_difficulty	No	1	Random Forest	3
military_service3	military_service = Never served	-0.0009969	military_service	Never served	-1	Random Forest	3
mobility3	mobility = Same house	-0.0016117	mobility	Same house	-1	Random Forest	3
disability3	disability = With disability	0.0739246	disability	With disability	1	Random Forest	3
parent_employment3	parent_employment = NA	-0.0006991	parent_employment	NA	-1	Random Forest	3
employment3	employment = Unemployed	-0.0063142	employment	Unemployed	-1	Random Forest	3
race3	race = Some Other Race	0.0013125	race	Some Other Race	1	Random Forest	3
nativity3	nativity = Native	0.0052908	nativity	Native	1	Random Forest	3
marital_status3	marital_status = Never married	0.0002221	marital_status	Never married	1	Random Forest	3
citizenship3	citizenship = Citizen by birth (US)	0.0100878	citizenship	Citizen by birth (US)	1	Random Forest	3
vision_difficulty4	vision_difficulty = Yes	0.0098084	vision_difficulty	Yes	1	Random Forest	4
mobility4	mobility = Same house	-0.0016871	mobility	Same house	-1	Random Forest	4
education4	education = High School or GED	0.0238065	education	High School or GED	1	Random Forest	4
marital_status4	marital_status = Never married	0.0228920	marital_status	Never married	1	Random Forest	4
income4	income = 0	0.0162395	income	0	1	Random Forest	4
race4	race = Some Other Race	0.0098214	race	Some Other Race	1	Random Forest	4
disability4	disability = With disability	0.0873688	disability	With disability	1	Random Forest	4
military_service4	military_service = Never served	-0.0006757	military_service	Never served	-1	Random Forest	4
parent_employment4	parent_employment = NA	-0.0017315	parent_employment	NA	-1	Random Forest	4
ancestry4	ancestry = Multiple	-0.0410374	ancestry	Multiple	-1	Random Forest	4
age4	age = 26	-0.0031937	age	26	-1	Random Forest	4
citizenship4	citizenship = Citizen by birth (US)	0.0083764	citizenship	Citizen by birth (US)	1	Random Forest	4
sex4	sex = Male	0.0159198	sex	Male	1	Random Forest	4
nativity4	nativity = Native	0.0044472	nativity	Native	1	Random Forest	4
cognitive_difficulty4	cognitive_difficulty = Yes	0.0617139	cognitive_difficulty	Yes	1	Random Forest	4
hearing_difficulty4	hearing_difficulty = No	0.0006633	hearing_difficulty	No	1	Random Forest	4
employment4	employment = Unemployed	-0.0018840	employment	Unemployed	-1	Random Forest	4
gave_birth4	gave_birth = NA	-0.0002179	gave_birth	NA	-1	Random Forest	4
ancestry5	ancestry = Multiple	-0.0217635	ancestry	Multiple	-1	Random Forest	5
citizenship5	citizenship = Citizen by birth (US)	0.0007429	citizenship	Citizen by birth (US)	1	Random Forest	5
hearing_difficulty5	hearing_difficulty = No	0.0006709	hearing_difficulty	No	1	Random Forest	5
age5	age = 26	0.0287538	age	26	1	Random Forest	5
nativity5	nativity = Native	-0.0014787	nativity	Native	-1	Random Forest	5
parent_employment5	parent_employment = NA	-0.0016670	parent_employment	NA	-1	Random Forest	5
sex5	sex = Male	0.0145216	sex	Male	1	Random Forest	5
marital_status5	marital_status = Never married	0.0048368	marital_status	Never married	1	Random Forest	5
mobility5	mobility = Same house	-0.0012446	mobility	Same house	-1	Random Forest	5
cognitive_difficulty5	cognitive_difficulty = Yes	0.1160661	cognitive_difficulty	Yes	1	Random Forest	5
race5	race = Some Other Race	-0.0088012	race	Some Other Race	-1	Random Forest	5
gave_birth5	gave_birth = NA	-0.0004512	gave_birth	NA	-1	Random Forest	5
vision_difficulty5	vision_difficulty = Yes	0.0092065	vision_difficulty	Yes	1	Random Forest	5
income5	income = 0	-0.0248225	income	0	-1	Random Forest	5
military_service5	military_service = Never served	-0.0012679	military_service	Never served	-1	Random Forest	5
education5	education = High School or GED	0.0222474	education	High School or GED	1	Random Forest	5
employment5	employment = Unemployed	0.0607254	employment	Unemployed	1	Random Forest	5
disability5	disability = With disability	0.0143552	disability	With disability	1	Random Forest	5
hearing_difficulty6	hearing_difficulty = No	0.0006418	hearing_difficulty	No	1	Random Forest	6
parent_employment6	parent_employment = NA	-0.0046185	parent_employment	NA	-1	Random Forest	6
marital_status6	marital_status = Never married	0.0211836	marital_status	Never married	1	Random Forest	6
mobility6	mobility = Same house	-0.0023150	mobility	Same house	-1	Random Forest	6
disability6	disability = With disability	0.1395734	disability	With disability	1	Random Forest	6
cognitive_difficulty6	cognitive_difficulty = Yes	0.0739893	cognitive_difficulty	Yes	1	Random Forest	6
age6	age = 26	-0.0098483	age	26	-1	Random Forest	6
race6	race = Some Other Race	-0.0037033	race	Some Other Race	-1	Random Forest	6
ancestry6	ancestry = Multiple	-0.0126673	ancestry	Multiple	-1	Random Forest	6
nativity6	nativity = Native	0.0090264	nativity	Native	1	Random Forest	6
gave_birth6	gave_birth = NA	-0.0002292	gave_birth	NA	-1	Random Forest	6
income6	income = 0	-0.0436813	income	0	-1	Random Forest	6
military_service6	military_service = Never served	-0.0010029	military_service	Never served	-1	Random Forest	6
citizenship6	citizenship = Citizen by birth (US)	0.0114182	citizenship	Citizen by birth (US)	1	Random Forest	6
employment6	employment = Unemployed	0.0132535	employment	Unemployed	1	Random Forest	6
vision_difficulty6	vision_difficulty = Yes	-0.0260356	vision_difficulty	Yes	-1	Random Forest	6
education6	education = High School or GED	0.0460830	education	High School or GED	1	Random Forest	6
sex6	sex = Male	-0.0004377	sex	Male	-1	Random Forest	6
marital_status7	marital_status = Never married	0.0212072	marital_status	Never married	1	Random Forest	7
military_service7	military_service = Never served	-0.0014619	military_service	Never served	-1	Random Forest	7
education7	education = High School or GED	0.0191879	education	High School or GED	1	Random Forest	7
disability7	disability = With disability	0.1449901	disability	With disability	1	Random Forest	7
sex7	sex = Male	0.0486253	sex	Male	1	Random Forest	7
race7	race = Some Other Race	-0.0109097	race	Some Other Race	-1	Random Forest	7
citizenship7	citizenship = Citizen by birth (US)	0.0100889	citizenship	Citizen by birth (US)	1	Random Forest	7
hearing_difficulty7	hearing_difficulty = No	0.0009617	hearing_difficulty	No	1	Random Forest	7
parent_employment7	parent_employment = NA	-0.0015854	parent_employment	NA	-1	Random Forest	7
income7	income = 0	-0.0135138	income	0	-1	Random Forest	7
nativity7	nativity = Native	0.0016712	nativity	Native	1	Random Forest	7
age7	age = 26	-0.0331352	age	26	-1	Random Forest	7
vision_difficulty7	vision_difficulty = Yes	0.0201077	vision_difficulty	Yes	1	Random Forest	7
employment7	employment = Unemployed	0.0237033	employment	Unemployed	1	Random Forest	7
ancestry7	ancestry = Multiple	-0.0657932	ancestry	Multiple	-1	Random Forest	7
mobility7	mobility = Same house	0.0011795	mobility	Same house	1	Random Forest	7
gave_birth7	gave_birth = NA	-0.0003322	gave_birth	NA	-1	Random Forest	7
cognitive_difficulty7	cognitive_difficulty = Yes	0.0456384	cognitive_difficulty	Yes	1	Random Forest	7
race8	race = Some Other Race	0.0202641	race	Some Other Race	1	Random Forest	8
employment8	employment = Unemployed	0.0960865	employment	Unemployed	1	Random Forest	8
citizenship8	citizenship = Citizen by birth (US)	0.0088523	citizenship	Citizen by birth (US)	1	Random Forest	8
income8	income = 0	-0.0574603	income	0	-1	Random Forest	8
disability8	disability = With disability	0.0997166	disability	With disability	1	Random Forest	8
gave_birth8	gave_birth = NA	-0.0001071	gave_birth	NA	-1	Random Forest	8
education8	education = High School or GED	0.0412960	education	High School or GED	1	Random Forest	8
vision_difficulty8	vision_difficulty = Yes	-0.0110643	vision_difficulty	Yes	-1	Random Forest	8
ancestry8	ancestry = Multiple	-0.0393342	ancestry	Multiple	-1	Random Forest	8
age8	age = 26	-0.0056844	age	26	-1	Random Forest	8
military_service8	military_service = Never served	-0.0009530	military_service	Never served	-1	Random Forest	8
nativity8	nativity = Native	0.0028338	nativity	Native	1	Random Forest	8
hearing_difficulty8	hearing_difficulty = No	0.0002022	hearing_difficulty	No	1	Random Forest	8
sex8	sex = Male	0.0181862	sex	Male	1	Random Forest	8
cognitive_difficulty8	cognitive_difficulty = Yes	0.0377156	cognitive_difficulty	Yes	1	Random Forest	8
mobility8	mobility = Same house	0.0014289	mobility	Same house	1	Random Forest	8
parent_employment8	parent_employment = NA	-0.0006920	parent_employment	NA	-1	Random Forest	8
marital_status8	marital_status = Never married	-0.0006572	marital_status	Never married	-1	Random Forest	8
ancestry9	ancestry = Multiple	-0.0217635	ancestry	Multiple	-1	Random Forest	9
marital_status9	marital_status = Never married	0.0187756	marital_status	Never married	1	Random Forest	9
mobility9	mobility = Same house	-0.0029678	mobility	Same house	-1	Random Forest	9
age9	age = 26	0.0128605	age	26	1	Random Forest	9
income9	income = 0	-0.0033521	income	0	-1	Random Forest	9
nativity9	nativity = Native	0.0003339	nativity	Native	1	Random Forest	9
employment9	employment = Unemployed	0.0650590	employment	Unemployed	1	Random Forest	9
parent_employment9	parent_employment = NA	-0.0022973	parent_employment	NA	-1	Random Forest	9
disability9	disability = With disability	0.0478241	disability	With disability	1	Random Forest	9
education9	education = High School or GED	0.0586585	education	High School or GED	1	Random Forest	9
race9	race = Some Other Race	0.0120157	race	Some Other Race	1	Random Forest	9
hearing_difficulty9	hearing_difficulty = No	0.0003388	hearing_difficulty	No	1	Random Forest	9
gave_birth9	gave_birth = NA	-0.0003948	gave_birth	NA	-1	Random Forest	9
military_service9	military_service = Never served	-0.0011591	military_service	Never served	-1	Random Forest	9
sex9	sex = Male	-0.0290765	sex	Male	-1	Random Forest	9
vision_difficulty9	vision_difficulty = Yes	0.0020573	vision_difficulty	Yes	1	Random Forest	9
citizenship9	citizenship = Citizen by birth (US)	0.0080792	citizenship	Citizen by birth (US)	1	Random Forest	9
cognitive_difficulty9	cognitive_difficulty = Yes	0.0456384	cognitive_difficulty	Yes	1	Random Forest	9
cognitive_difficulty10	cognitive_difficulty = Yes	0.1074487	cognitive_difficulty	Yes	1	Random Forest	10
parent_employment10	parent_employment = NA	-0.0029618	parent_employment	NA	-1	Random Forest	10
nativity10	nativity = Native	0.0014308	nativity	Native	1	Random Forest	10
sex10	sex = Male	0.0244686	sex	Male	1	Random Forest	10
citizenship10	citizenship = Citizen by birth (US)	0.0057037	citizenship	Citizen by birth (US)	1	Random Forest	10
vision_difficulty10	vision_difficulty = Yes	0.0109846	vision_difficulty	Yes	1	Random Forest	10
education10	education = High School or GED	0.0250177	education	High School or GED	1	Random Forest	10
military_service10	military_service = Never served	-0.0009012	military_service	Never served	-1	Random Forest	10
income10	income = 0	-0.0138919	income	0	-1	Random Forest	10
marital_status10	marital_status = Never married	0.0252898	marital_status	Never married	1	Random Forest	10
mobility10	mobility = Same house	-0.0033553	mobility	Same house	-1	Random Forest	10
employment10	employment = Unemployed	0.0460430	employment	Unemployed	1	Random Forest	10
hearing_difficulty10	hearing_difficulty = No	0.0003240	hearing_difficulty	No	1	Random Forest	10
ancestry10	ancestry = Multiple	0.0007846	ancestry	Multiple	1	Random Forest	10
race10	race = Some Other Race	0.0008841	race	Some Other Race	1	Random Forest	10
gave_birth10	gave_birth = NA	-0.0004231	gave_birth	NA	-1	Random Forest	10
disability10	disability = With disability	0.0344966	disability	With disability	1	Random Forest	10
age10	age = 26	-0.0507131	age	26	-1	Random Forest	10
race11	race = Some Other Race	0.0202641	race	Some Other Race	1	Random Forest	11
vision_difficulty11	vision_difficulty = Yes	0.0102899	vision_difficulty	Yes	1	Random Forest	11
hearing_difficulty11	hearing_difficulty = No	0.0005358	hearing_difficulty	No	1	Random Forest	11
parent_employment11	parent_employment = NA	-0.0045686	parent_employment	NA	-1	Random Forest	11
employment11	employment = Unemployed	0.0933059	employment	Unemployed	1	Random Forest	11
education11	education = High School or GED	0.0286554	education	High School or GED	1	Random Forest	11
age11	age = 26	0.0127293	age	26	1	Random Forest	11
mobility11	mobility = Same house	0.0035458	mobility	Same house	1	Random Forest	11
military_service11	military_service = Never served	-0.0004095	military_service	Never served	-1	Random Forest	11
ancestry11	ancestry = Multiple	-0.0429882	ancestry	Multiple	-1	Random Forest	11
citizenship11	citizenship = Citizen by birth (US)	0.0118774	citizenship	Citizen by birth (US)	1	Random Forest	11
income11	income = 0	-0.0429093	income	0	-1	Random Forest	11
marital_status11	marital_status = Never married	0.0176385	marital_status	Never married	1	Random Forest	11
gave_birth11	gave_birth = NA	-0.0013510	gave_birth	NA	-1	Random Forest	11
nativity11	nativity = Native	0.0079594	nativity	Native	1	Random Forest	11
sex11	sex = Male	0.0110838	sex	Male	1	Random Forest	11
disability11	disability = With disability	0.0393327	disability	With disability	1	Random Forest	11
cognitive_difficulty11	cognitive_difficulty = Yes	0.0456384	cognitive_difficulty	Yes	1	Random Forest	11
citizenship12	citizenship = Citizen by birth (US)	-0.0042873	citizenship	Citizen by birth (US)	-1	Random Forest	12
vision_difficulty12	vision_difficulty = Yes	0.0105327	vision_difficulty	Yes	1	Random Forest	12
employment12	employment = Unemployed	0.1038638	employment	Unemployed	1	Random Forest	12
ancestry12	ancestry = Multiple	-0.0368505	ancestry	Multiple	-1	Random Forest	12
education12	education = High School or GED	0.0347834	education	High School or GED	1	Random Forest	12
military_service12	military_service = Never served	-0.0013272	military_service	Never served	-1	Random Forest	12
income12	income = 0	-0.0468553	income	0	-1	Random Forest	12
nativity12	nativity = Native	-0.0017761	nativity	Native	-1	Random Forest	12
marital_status12	marital_status = Never married	0.0404297	marital_status	Never married	1	Random Forest	12
age12	age = 26	0.0087716	age	26	1	Random Forest	12
race12	race = Some Other Race	0.0143256	race	Some Other Race	1	Random Forest	12
disability12	disability = With disability	0.0372638	disability	With disability	1	Random Forest	12
sex12	sex = Male	0.0060752	sex	Male	1	Random Forest	12
cognitive_difficulty12	cognitive_difficulty = Yes	0.0445373	cognitive_difficulty	Yes	1	Random Forest	12
hearing_difficulty12	hearing_difficulty = No	0.0004560	hearing_difficulty	No	1	Random Forest	12
gave_birth12	gave_birth = NA	-0.0002211	gave_birth	NA	-1	Random Forest	12
parent_employment12	parent_employment = NA	-0.0006920	parent_employment	NA	-1	Random Forest	12
mobility12	mobility = Same house	0.0016003	mobility	Same house	1	Random Forest	12
gave_birth13	gave_birth = NA	-0.0006940	gave_birth	NA	-1	Random Forest	13
mobility13	mobility = Same house	-0.0020484	mobility	Same house	-1	Random Forest	13
parent_employment13	parent_employment = NA	-0.0046185	parent_employment	NA	-1	Random Forest	13
marital_status13	marital_status = Never married	0.0207495	marital_status	Never married	1	Random Forest	13
employment13	employment = Unemployed	0.1076054	employment	Unemployed	1	Random Forest	13
age13	age = 26	0.0099195	age	26	1	Random Forest	13
military_service13	military_service = Never served	-0.0010358	military_service	Never served	-1	Random Forest	13
cognitive_difficulty13	cognitive_difficulty = Yes	0.0707077	cognitive_difficulty	Yes	1	Random Forest	13
ancestry13	ancestry = Multiple	-0.0152428	ancestry	Multiple	-1	Random Forest	13
income13	income = 0	-0.0528172	income	0	-1	Random Forest	13
citizenship13	citizenship = Citizen by birth (US)	0.0136655	citizenship	Citizen by birth (US)	1	Random Forest	13
vision_difficulty13	vision_difficulty = Yes	-0.0017585	vision_difficulty	Yes	-1	Random Forest	13
hearing_difficulty13	hearing_difficulty = No	0.0002076	hearing_difficulty	No	1	Random Forest	13
sex13	sex = Male	0.0218483	sex	Male	1	Random Forest	13
education13	education = High School or GED	0.0299722	education	High School or GED	1	Random Forest	13
disability13	disability = With disability	0.0082475	disability	With disability	1	Random Forest	13
race13	race = Some Other Race	0.0014979	race	Some Other Race	1	Random Forest	13
nativity13	nativity = Native	0.0044238	nativity	Native	1	Random Forest	13
citizenship14	citizenship = Citizen by birth (US)	-0.0042873	citizenship	Citizen by birth (US)	-1	Random Forest	14
age14	age = 26	0.0181375	age	26	1	Random Forest	14
cognitive_difficulty14	cognitive_difficulty = Yes	0.1006342	cognitive_difficulty	Yes	1	Random Forest	14
nativity14	nativity = Native	0.0023481	nativity	Native	1	Random Forest	14
income14	income = 0	-0.0326994	income	0	-1	Random Forest	14
education14	education = High School or GED	0.0322386	education	High School or GED	1	Random Forest	14
gave_birth14	gave_birth = NA	-0.0006717	gave_birth	NA	-1	Random Forest	14
ancestry14	ancestry = Multiple	0.0036727	ancestry	Multiple	1	Random Forest	14
vision_difficulty14	vision_difficulty = Yes	0.0052499	vision_difficulty	Yes	1	Random Forest	14
hearing_difficulty14	hearing_difficulty = No	0.0005032	hearing_difficulty	No	1	Random Forest	14
disability14	disability = With disability	0.0737388	disability	With disability	1	Random Forest	14
employment14	employment = Unemployed	0.0010245	employment	Unemployed	1	Random Forest	14
mobility14	mobility = Same house	-0.0000620	mobility	Same house	-1	Random Forest	14
military_service14	military_service = Never served	-0.0009872	military_service	Never served	-1	Random Forest	14
race14	race = Some Other Race	0.0054081	race	Some Other Race	1	Random Forest	14
marital_status14	marital_status = Never married	0.0077939	marital_status	Never married	1	Random Forest	14
parent_employment14	parent_employment = NA	-0.0009742	parent_employment	NA	-1	Random Forest	14
sex14	sex = Male	-0.0004377	sex	Male	-1	Random Forest	14
ancestry15	ancestry = Multiple	-0.0217635	ancestry	Multiple	-1	Random Forest	15
race15	race = Some Other Race	0.0136920	race	Some Other Race	1	Random Forest	15
marital_status15	marital_status = Never married	0.0142005	marital_status	Never married	1	Random Forest	15
parent_employment15	parent_employment = NA	-0.0041065	parent_employment	NA	-1	Random Forest	15
vision_difficulty15	vision_difficulty = Yes	0.0070578	vision_difficulty	Yes	1	Random Forest	15
disability15	disability = With disability	0.0870878	disability	With disability	1	Random Forest	15
citizenship15	citizenship = Citizen by birth (US)	0.0094836	citizenship	Citizen by birth (US)	1	Random Forest	15
hearing_difficulty15	hearing_difficulty = No	0.0004914	hearing_difficulty	No	1	Random Forest	15
mobility15	mobility = Same house	-0.0023997	mobility	Same house	-1	Random Forest	15
education15	education = High School or GED	0.0317984	education	High School or GED	1	Random Forest	15
gave_birth15	gave_birth = NA	-0.0004269	gave_birth	NA	-1	Random Forest	15
income15	income = 0	-0.0007879	income	0	-1	Random Forest	15
military_service15	military_service = Never served	-0.0009858	military_service	Never served	-1	Random Forest	15
age15	age = 26	-0.0033660	age	26	-1	Random Forest	15
sex15	sex = Male	0.0159356	sex	Male	1	Random Forest	15
cognitive_difficulty15	cognitive_difficulty = Yes	0.0609709	cognitive_difficulty	Yes	1	Random Forest	15
employment15	employment = Unemployed	-0.0006756	employment	Unemployed	-1	Random Forest	15
nativity15	nativity = Native	0.0044238	nativity	Native	1	Random Forest	15
income16	income = 0	-0.0089310	income	0	-1	Random Forest	16
sex16	sex = Male	0.0309565	sex	Male	1	Random Forest	16
race16	race = Some Other Race	0.0284091	race	Some Other Race	1	Random Forest	16
employment16	employment = Unemployed	0.0699514	employment	Unemployed	1	Random Forest	16
education16	education = High School or GED	0.0342261	education	High School or GED	1	Random Forest	16
age16	age = 26	-0.0345696	age	26	-1	Random Forest	16
mobility16	mobility = Same house	-0.0008124	mobility	Same house	-1	Random Forest	16
cognitive_difficulty16	cognitive_difficulty = Yes	0.0216467	cognitive_difficulty	Yes	1	Random Forest	16
nativity16	nativity = Native	0.0127623	nativity	Native	1	Random Forest	16
hearing_difficulty16	hearing_difficulty = No	0.0006180	hearing_difficulty	No	1	Random Forest	16
ancestry16	ancestry = Multiple	0.0160296	ancestry	Multiple	1	Random Forest	16
disability16	disability = With disability	0.0235007	disability	With disability	1	Random Forest	16
military_service16	military_service = Never served	-0.0008678	military_service	Never served	-1	Random Forest	16
gave_birth16	gave_birth = NA	-0.0001574	gave_birth	NA	-1	Random Forest	16
marital_status16	marital_status = Never married	0.0000950	marital_status	Never married	1	Random Forest	16
parent_employment16	parent_employment = NA	-0.0007080	parent_employment	NA	-1	Random Forest	16
vision_difficulty16	vision_difficulty = Yes	0.0083928	vision_difficulty	Yes	1	Random Forest	16
citizenship16	citizenship = Citizen by birth (US)	0.0100878	citizenship	Citizen by birth (US)	1	Random Forest	16
vision_difficulty17	vision_difficulty = Yes	0.0098084	vision_difficulty	Yes	1	Random Forest	17
military_service17	military_service = Never served	-0.0017212	military_service	Never served	-1	Random Forest	17
marital_status17	marital_status = Never married	0.0213515	marital_status	Never married	1	Random Forest	17
citizenship17	citizenship = Citizen by birth (US)	0.0017047	citizenship	Citizen by birth (US)	1	Random Forest	17
gave_birth17	gave_birth = NA	-0.0006145	gave_birth	NA	-1	Random Forest	17
age17	age = 26	-0.0082977	age	26	-1	Random Forest	17
mobility17	mobility = Same house	-0.0018444	mobility	Same house	-1	Random Forest	17
race17	race = Some Other Race	0.0217204	race	Some Other Race	1	Random Forest	17
sex17	sex = Male	0.0051742	sex	Male	1	Random Forest	17
cognitive_difficulty17	cognitive_difficulty = Yes	0.0911113	cognitive_difficulty	Yes	1	Random Forest	17
parent_employment17	parent_employment = NA	-0.0014183	parent_employment	NA	-1	Random Forest	17
income17	income = 0	-0.0265021	income	0	-1	Random Forest	17
education17	education = High School or GED	0.0260205	education	High School or GED	1	Random Forest	17
hearing_difficulty17	hearing_difficulty = No	0.0007781	hearing_difficulty	No	1	Random Forest	17
ancestry17	ancestry = Multiple	-0.0045050	ancestry	Multiple	-1	Random Forest	17
employment17	employment = Unemployed	0.0571197	employment	Unemployed	1	Random Forest	17
nativity17	nativity = Native	0.0063891	nativity	Native	1	Random Forest	17
disability17	disability = With disability	0.0143552	disability	With disability	1	Random Forest	17
nativity18	nativity = Native	-0.0031464	nativity	Native	-1	Random Forest	18
marital_status18	marital_status = Never married	0.0258969	marital_status	Never married	1	Random Forest	18
education18	education = High School or GED	0.0224522	education	High School or GED	1	Random Forest	18
income18	income = 0	0.0126377	income	0	1	Random Forest	18
disability18	disability = With disability	0.1114597	disability	With disability	1	Random Forest	18
age18	age = 26	-0.0019674	age	26	-1	Random Forest	18
vision_difficulty18	vision_difficulty = Yes	-0.0144643	vision_difficulty	Yes	-1	Random Forest	18
citizenship18	citizenship = Citizen by birth (US)	0.0050718	citizenship	Citizen by birth (US)	1	Random Forest	18
employment18	employment = Unemployed	0.0296501	employment	Unemployed	1	Random Forest	18
parent_employment18	parent_employment = NA	-0.0016537	parent_employment	NA	-1	Random Forest	18
ancestry18	ancestry = Multiple	-0.0417476	ancestry	Multiple	-1	Random Forest	18
race18	race = Some Other Race	0.0143859	race	Some Other Race	1	Random Forest	18
hearing_difficulty18	hearing_difficulty = No	0.0001479	hearing_difficulty	No	1	Random Forest	18
gave_birth18	gave_birth = NA	-0.0003822	gave_birth	NA	-1	Random Forest	18
military_service18	military_service = Never served	-0.0010296	military_service	Never served	-1	Random Forest	18
mobility18	mobility = Same house	0.0004148	mobility	Same house	1	Random Forest	18
sex18	sex = Male	0.0072657	sex	Male	1	Random Forest	18
cognitive_difficulty18	cognitive_difficulty = Yes	0.0456384	cognitive_difficulty	Yes	1	Random Forest	18
military_service19	military_service = Never served	-0.0017331	military_service	Never served	-1	Random Forest	19
sex19	sex = Male	0.0263508	sex	Male	1	Random Forest	19
education19	education = High School or GED	0.0196153	education	High School or GED	1	Random Forest	19
gave_birth19	gave_birth = NA	-0.0009281	gave_birth	NA	-1	Random Forest	19
disability19	disability = With disability	0.1874995	disability	With disability	1	Random Forest	19
marital_status19	marital_status = Never married	0.0014586	marital_status	Never married	1	Random Forest	19
employment19	employment = Unemployed	0.0887690	employment	Unemployed	1	Random Forest	19
parent_employment19	parent_employment = NA	-0.0015920	parent_employment	NA	-1	Random Forest	19
mobility19	mobility = Same house	0.0011699	mobility	Same house	1	Random Forest	19
citizenship19	citizenship = Citizen by birth (US)	0.0162912	citizenship	Citizen by birth (US)	1	Random Forest	19
vision_difficulty19	vision_difficulty = Yes	-0.0119916	vision_difficulty	Yes	-1	Random Forest	19
age19	age = 26	-0.0291846	age	26	-1	Random Forest	19
nativity19	nativity = Native	0.0047162	nativity	Native	1	Random Forest	19
hearing_difficulty19	hearing_difficulty = No	0.0003573	hearing_difficulty	No	1	Random Forest	19
ancestry19	ancestry = Multiple	-0.0499370	ancestry	Multiple	-1	Random Forest	19
cognitive_difficulty19	cognitive_difficulty = Yes	0.0806026	cognitive_difficulty	Yes	1	Random Forest	19
race19	race = Some Other Race	-0.0073269	race	Some Other Race	-1	Random Forest	19
income19	income = 0	-0.1135072	income	0	-1	Random Forest	19
cognitive_difficulty20	cognitive_difficulty = Yes	0.1074487	cognitive_difficulty	Yes	1	Random Forest	20
ancestry20	ancestry = Multiple	-0.0111611	ancestry	Multiple	-1	Random Forest	20
age20	age = 26	0.0100752	age	26	1	Random Forest	20
income20	income = 0	-0.0173969	income	0	-1	Random Forest	20
education20	education = High School or GED	0.0245138	education	High School or GED	1	Random Forest	20
hearing_difficulty20	hearing_difficulty = No	0.0007625	hearing_difficulty	No	1	Random Forest	20
citizenship20	citizenship = Citizen by birth (US)	0.0043359	citizenship	Citizen by birth (US)	1	Random Forest	20
marital_status20	marital_status = Never married	0.0201305	marital_status	Never married	1	Random Forest	20
race20	race = Some Other Race	-0.0052250	race	Some Other Race	-1	Random Forest	20
military_service20	military_service = Never served	-0.0006907	military_service	Never served	-1	Random Forest	20
sex20	sex = Male	-0.0101416	sex	Male	-1	Random Forest	20
nativity20	nativity = Native	0.0023013	nativity	Native	1	Random Forest	20
vision_difficulty20	vision_difficulty = Yes	0.0177259	vision_difficulty	Yes	1	Random Forest	20
gave_birth20	gave_birth = NA	-0.0005816	gave_birth	NA	-1	Random Forest	20
disability20	disability = With disability	0.0698491	disability	With disability	1	Random Forest	20
employment20	employment = Unemployed	-0.0022246	employment	Unemployed	-1	Random Forest	20
mobility20	mobility = Same house	0.0016003	mobility	Same house	1	Random Forest	20
parent_employment20	parent_employment = NA	-0.0006920	parent_employment	NA	-1	Random Forest	20
citizenship21	citizenship = Citizen by birth (US)	-0.0042873	citizenship	Citizen by birth (US)	-1	Random Forest	21
vision_difficulty21	vision_difficulty = Yes	0.0105327	vision_difficulty	Yes	1	Random Forest	21
age21	age = 26	0.0152146	age	26	1	Random Forest	21
military_service21	military_service = Never served	-0.0014159	military_service	Never served	-1	Random Forest	21
nativity21	nativity = Native	-0.0015031	nativity	Native	-1	Random Forest	21
education21	education = High School or GED	0.0250236	education	High School or GED	1	Random Forest	21
gave_birth21	gave_birth = NA	-0.0009338	gave_birth	NA	-1	Random Forest	21
mobility21	mobility = Same house	0.0026759	mobility	Same house	1	Random Forest	21
disability21	disability = With disability	0.1188577	disability	With disability	1	Random Forest	21
employment21	employment = Unemployed	0.0874649	employment	Unemployed	1	Random Forest	21
hearing_difficulty21	hearing_difficulty = No	0.0002744	hearing_difficulty	No	1	Random Forest	21
race21	race = Some Other Race	0.0083555	race	Some Other Race	1	Random Forest	21
cognitive_difficulty21	cognitive_difficulty = Yes	0.0341394	cognitive_difficulty	Yes	1	Random Forest	21
marital_status21	marital_status = Never married	0.0128666	marital_status	Never married	1	Random Forest	21
parent_employment21	parent_employment = NA	-0.0012612	parent_employment	NA	-1	Random Forest	21
ancestry21	ancestry = Multiple	-0.0099386	ancestry	Multiple	-1	Random Forest	21
sex21	sex = Male	0.0280720	sex	Male	1	Random Forest	21
income21	income = 0	-0.1135072	income	0	-1	Random Forest	21
sex22	sex = Male	0.0263508	sex	Male	1	Random Forest	22
hearing_difficulty22	hearing_difficulty = No	0.0007906	hearing_difficulty	No	1	Random Forest	22
mobility22	mobility = Same house	-0.0020835	mobility	Same house	-1	Random Forest	22
vision_difficulty22	vision_difficulty = Yes	0.0165648	vision_difficulty	Yes	1	Random Forest	22
gave_birth22	gave_birth = NA	-0.0004983	gave_birth	NA	-1	Random Forest	22
parent_employment22	parent_employment = NA	-0.0032004	parent_employment	NA	-1	Random Forest	22
age22	age = 26	-0.0062110	age	26	-1	Random Forest	22
ancestry22	ancestry = Multiple	-0.0043867	ancestry	Multiple	-1	Random Forest	22
military_service22	military_service = Never served	-0.0015710	military_service	Never served	-1	Random Forest	22
employment22	employment = Unemployed	0.0966591	employment	Unemployed	1	Random Forest	22
nativity22	nativity = Native	0.0049935	nativity	Native	1	Random Forest	22
citizenship22	citizenship = Citizen by birth (US)	0.0104710	citizenship	Citizen by birth (US)	1	Random Forest	22
disability22	disability = With disability	0.0741502	disability	With disability	1	Random Forest	22
income22	income = 0	-0.1101051	income	0	-1	Random Forest	22
cognitive_difficulty22	cognitive_difficulty = Yes	0.0529976	cognitive_difficulty	Yes	1	Random Forest	22
marital_status22	marital_status = Never married	-0.0003438	marital_status	Never married	-1	Random Forest	22
education22	education = High School or GED	0.0535343	education	High School or GED	1	Random Forest	22
race22	race = Some Other Race	0.0025179	race	Some Other Race	1	Random Forest	22
income23	income = 0	-0.0089310	income	0	-1	Random Forest	23
age23	age = 26	0.0233924	age	26	1	Random Forest	23
hearing_difficulty23	hearing_difficulty = No	0.0003288	hearing_difficulty	No	1	Random Forest	23
employment23	employment = Unemployed	0.0777433	employment	Unemployed	1	Random Forest	23
education23	education = High School or GED	0.0259773	education	High School or GED	1	Random Forest	23
ancestry23	ancestry = Multiple	-0.0428663	ancestry	Multiple	-1	Random Forest	23
citizenship23	citizenship = Citizen by birth (US)	0.0032272	citizenship	Citizen by birth (US)	1	Random Forest	23
race23	race = Some Other Race	0.0115894	race	Some Other Race	1	Random Forest	23
vision_difficulty23	vision_difficulty = Yes	0.0067066	vision_difficulty	Yes	1	Random Forest	23
mobility23	mobility = Same house	-0.0015420	mobility	Same house	-1	Random Forest	23
marital_status23	marital_status = Never married	0.0175590	marital_status	Never married	1	Random Forest	23
nativity23	nativity = Native	0.0077241	nativity	Native	1	Random Forest	23
military_service23	military_service = Never served	-0.0014230	military_service	Never served	-1	Random Forest	23
disability23	disability = With disability	0.0398975	disability	With disability	1	Random Forest	23
parent_employment23	parent_employment = NA	-0.0013240	parent_employment	NA	-1	Random Forest	23
cognitive_difficulty23	cognitive_difficulty = Yes	0.0532332	cognitive_difficulty	Yes	1	Random Forest	23
gave_birth23	gave_birth = NA	-0.0002250	gave_birth	NA	-1	Random Forest	23
sex23	sex = Male	-0.0004377	sex	Male	-1	Random Forest	23
employment24	employment = Unemployed	0.0976472	employment	Unemployed	1	Random Forest	24
military_service24	military_service = Never served	-0.0014905	military_service	Never served	-1	Random Forest	24
nativity24	nativity = Native	0.0023309	nativity	Native	1	Random Forest	24
hearing_difficulty24	hearing_difficulty = No	0.0005262	hearing_difficulty	No	1	Random Forest	24
gave_birth24	gave_birth = NA	-0.0004448	gave_birth	NA	-1	Random Forest	24
vision_difficulty24	vision_difficulty = Yes	0.0062978	vision_difficulty	Yes	1	Random Forest	24
race24	race = Some Other Race	0.0221885	race	Some Other Race	1	Random Forest	24
education24	education = High School or GED	0.0311534	education	High School or GED	1	Random Forest	24
ancestry24	ancestry = Multiple	-0.0390680	ancestry	Multiple	-1	Random Forest	24
parent_employment24	parent_employment = NA	-0.0043487	parent_employment	NA	-1	Random Forest	24
citizenship24	citizenship = Citizen by birth (US)	0.0073300	citizenship	Citizen by birth (US)	1	Random Forest	24
marital_status24	marital_status = Never married	0.0270953	marital_status	Never married	1	Random Forest	24
sex24	sex = Male	0.0561713	sex	Male	1	Random Forest	24
disability24	disability = With disability	0.0735360	disability	With disability	1	Random Forest	24
income24	income = 0	-0.0738385	income	0	-1	Random Forest	24
mobility24	mobility = Same house	0.0003666	mobility	Same house	1	Random Forest	24
age24	age = 26	-0.0404612	age	26	-1	Random Forest	24
cognitive_difficulty24	cognitive_difficulty = Yes	0.0456384	cognitive_difficulty	Yes	1	Random Forest	24
education25	education = High School or GED	0.0177250	education	High School or GED	1	Random Forest	25
military_service25	military_service = Never served	-0.0014538	military_service	Never served	-1	Random Forest	25
marital_status25	marital_status = Never married	0.0226621	marital_status	Never married	1	Random Forest	25
citizenship25	citizenship = Citizen by birth (US)	0.0043196	citizenship	Citizen by birth (US)	1	Random Forest	25
hearing_difficulty25	hearing_difficulty = No	0.0006998	hearing_difficulty	No	1	Random Forest	25
income25	income = 0	0.0105590	income	0	1	Random Forest	25
age25	age = 26	0.0090097	age	26	1	Random Forest	25
employment25	employment = Unemployed	0.0863580	employment	Unemployed	1	Random Forest	25
gave_birth25	gave_birth = NA	-0.0007767	gave_birth	NA	-1	Random Forest	25
nativity25	nativity = Native	0.0138535	nativity	Native	1	Random Forest	25
race25	race = Some Other Race	0.0281872	race	Some Other Race	1	Random Forest	25
disability25	disability = With disability	0.0417174	disability	With disability	1	Random Forest	25
parent_employment25	parent_employment = NA	-0.0015896	parent_employment	NA	-1	Random Forest	25
cognitive_difficulty25	cognitive_difficulty = Yes	-0.0198279	cognitive_difficulty	Yes	-1	Random Forest	25
mobility25	mobility = Same house	-0.0018723	mobility	Same house	-1	Random Forest	25
vision_difficulty25	vision_difficulty = Yes	-0.0183214	vision_difficulty	Yes	-1	Random Forest	25
sex25	sex = Male	0.0086653	sex	Male	1	Random Forest	25
ancestry25	ancestry = Multiple	0.0107148	ancestry	Multiple	1	Random Forest	25

• variable column: feature = XX, e.g., age = 30 indicates feature name and feature value for person/instance (see also variable_name and variable_value)

• contribution column: shows contribution of feature + feature value, e.g., 0.012 for age means feature age(value) increases probability of coverage by 0.012

• Q: How would we interprete rows 1-3?

Important: The contributions may seem small (0.01, 0.009), but they add up! All these small pushes and pulls combine to move the prediction away from the baseline to the final prediction.

4.5 Visualizing Shapley Values

By applying the generic function plot() to the shap_person1 object, we obtain a graphical illustration of the results in Figure 1. There are two ways (predict_parts(..., type = "shap") vs. predict_parts(..., type = "break_down")) explained in the table below.

Feature	Plot 1 (Figure 1): SHAP contribution (type = "shap")	Plot 2 (Figure 2): Break Down profile (type = "break_down")
Calculation Method	Averaged & Permutational	Greedy & Sequential
What it Shows	The average (“fair”) contribution of each feature across many (B) random paths.	A single, specific path from the average prediction (intercept) to the final prediction.
Feature Ordering	By mean absolute contribution. The most impactful features are shown at the top for readability.	By algorithmic path. The order is the result of the greedy algorithm, not sorted by impact.
Visualization	Bar plot (or Boxplot). Bars show the mean contribution; they are not sequential and do not add up.	Waterfall plot. Bars are sequential and add up exactly from the intercept to the final prediction.
Uncertainty	Yes. The whiskers (error bars) show the variability of the contribution across the B random paths.	No. It shows one fixed path.
Concrete Interpretation	Read as a summary of facts: “On average, Feature A contributed +0.1 to the final prediction, and Feature B contributed +0.05…”	Read sequentially: “Starting at the average, Feature A added +0.1, then Feature B added +0.05…”

• Figure 1 shows individual contributions of 10 features
  • x-axis is centered at 0.0 to show how much each feature pushes the prediction up (positive - green) or down (negative -red) from the average (which is not indicated in the plot).

plot(shap_person1, max_vars = 18)

Figure 1: Visualizing contributions of features (Source: Own illustration)

Another option is the socalled waterfall plot in Figure 2 probably the most intuitive way to understand Shapley Values. It shows how the prediction was built step-by-step, starting from the baseline (average prediction) and adding each feature’s contribution. The plot starts at the baseline (average) and each bar shows how a feature pushes the prediction up or down. Reading from top to bottom, you can trace the journey from baseline to final prediction for that person/instance!

shap_person1_waterfall <- predict_parts(explainer = explainer, 
                          new_observation = person1, 
                          type = "break_down",
                          k = 18,
                          max_vars = 18)

# plot Break Down
plot(shap_person1_waterfall)

Figure 2: Waterfall plot for Shapley values for the random forest model and data of person 1

How to read this Figure 2:

• intercept: average prediction for all instances/individuals
• green/red bars: positive/negative contributions to probability based on respective features
  • e.g. employment = Not in labor force: +0.08 (this employment category increases coverage probability)
• + all other factors: contribution of the rest of the features summarized
• prediction (at the bottom): prediction for person1 and blue bar highlights difference to the average prediction

Most important features: The longest bars matter most!

• Q: What are the most important features here?

5 Lab R: SHAP

• Source(s)
  • Explain that tidymodels black box

5.1 Prepare data and train model

First, we’ll run the setup code to train our Random Forest model.

library(conflicted)
library(tidyverse)
library(tidymodels)
#library(hstats)
library(kernelshap)
library(shapviz)
library(patchwork)
conflicts_prefer(dplyr::filter)


# Load and prepare data
#data <- read_csv(url(sprintf("https://docs.google.com/uc?id=%s&export=download",
#                         "1dnCK79T45Qa7RZrDg1qv6-EdGBoxCPjv")))
data <- read_csv("data/data_acspubliccoverage.csv")
data <- data %>% mutate(across(where(is.character), as.factor)) %>%
  sample_n(2000) %>%
  select(public_coverage, income, age, education, marital_status, sex, disability, parent_employment, race)

# Split the data
set.seed(123)
data_split <- initial_split(data, prop = 0.80, strata = public_coverage)
data_train <- training(data_split)
data_test  <- testing(data_split)

# Define the recipe and model spec
recipe_rf <- recipe(public_coverage ~ ., data = data_train)%>%
  step_impute_median(all_numeric_predictors()) %>% # Impute numeric NAs
  step_impute_mode(all_nominal_predictors())      # Impute categorical NAs

model_rf <- rand_forest(mode = "classification") %>% 
  set_engine("ranger") |> 
  set_mode("classification")

# Create and fit the workflow
workflow_rf <- workflow() %>%
  add_recipe(recipe_rf) %>%
  add_model(model_rf)
fit_rf <- fit(workflow_rf, data = data_train)

5.2 Computing the SHAP values

We start by computing our SHAP values. To do so we also need to add a predict function p_fun() that outputs only the probability of the “Yes” class.

set.seed(1)

# Prediction function: extracts "Yes" probabilities
p_fun <- function(m, X) {
  predict(m, X, type = "prob")$.pred_Yes
}

# Check how that works internally
  # predict(fit_rf, head(data_test), type = "prob")

system.time(  # 12 minutes
  shap_values <- kernelshap(fit_rf, # kernelshap or permshap
                          X = data_test %>% select(-public_coverage), 
                          pred_fun = p_fun,
                          verbose = TRUE)
)

Then we use the shapviz() function to create an object of class shapviz from the matrix of SHAP values that are stored in in the object shap_values. Because computing the shap values can take a long time it’s often a good idea to save those values (in the new shapviz format) as an .rds file.

shap_values <- shapviz(shap_values)
saveRDS(shap_values, file = "data/shap_values.rds")

shap_values <- readRDS("data/shap_values.rds")

#shap_values <- readRDS(url(sprintf("https://docs.google.com/uc?id=%s&export=download", "1qPclnrp8h3NSLY2infMt0GNwOoTBytOk")))

shap_values  # 'shapviz' object representing SHAP matrix

'shapviz' object representing 401 x 8 SHAP matrix. Top lines:

           income         age   education marital_status          sex
[1,] -0.075507697 -0.01136358 -0.09416798   0.0006363036 -0.005020083
[2,]  0.008572405 -0.08671395  0.06591462   0.0110453234  0.004450835
      disability parent_employment        race
[1,] -0.05036509      -0.003400496  0.01782150
[2,] -0.05284541      -0.005271158 -0.01566334

5.3 Finding interesting instances to explain

Below we filter the dataset to identify interesting persons. Here, we want to investigate people for whom the prediction was wrong given their true label (their true status of public insurance coverage). And we further filter people for whom the predicted probability was far from the threshold of 0.5 (the model really got it wrong!).

persons_of_interest <- data_test %>%
  rowid_to_column("row_id") %>%
  augment(x = fit_rf) %>%
  select(row_id, .pred_class, public_coverage, .pred_Yes) %>%
  mutate(wrong_prediction = .pred_class != public_coverage,
         bad_predicted_probability = .pred_Yes >= 0.6,
         wrong_and_bad_probability = wrong_prediction & bad_predicted_probability) %>%
  arrange(desc(wrong_and_bad_probability))

5.4 Local: Waterfall plot

EXPLANATION: A waterfall plot shows how each feature contributes to pushing the model’s prediction from the base value to the final prediction for a SINGLE instance/individual.

How to read it:

• Starts with the base value (\(E[f(x)]\) = average prediction)
• Each bar shows a feature’s contribution (positive [yellow] pushes UP, negative [purple] pushes DOWN)
• Features are ordered by importance for this specific prediction (conditional on feature before)
• The final value is the actual model prediction

Use case: “Why did the model predict this person HAS public coverage?”

sv_waterfall(shap_values, 
             row_id = 386,
             max_display = "Inf") +
  ggtitle("Waterfall plot for second prediction")

Figure 3: Waterfall plot for second prediction

5.5 Local: Force plot

EXPLANATION: A force plot shows the same information as a waterfall plot but in a more compact horizontal format. It’s particularly useful for comparing multiple predictions.

How to read it:

• YELLOW features push the prediction HIGHER (toward positive class)
• PURPLE features push the prediction LOWER (toward negative class)
• The width of each bar shows the magnitude of the effect
• Output value \(f(x)\) shows the final prediction

Use case: “What are the main drivers pushing this prediction up or down?”

sv_force(shap_values, row_id = 386) +
  ggtitle("Force plot for second prediction")

Figure 4: Force plot for second prediction

5.6 Global: Feature importance (barplot)

EXPLANATION: This plot shows the mean absolute SHAP value for each feature across ALL predictions in the test set. It answers: “Which features are most important for the model overall?”

How to read it:

• Features are ranked from most to least important
• The bar length shows average magnitude of impact on predictions
• This tells you which features the model relies on most

Advantages over traditional feature importance:

• More theoretically rigorous
• Measured in the same units as predictions
• Shows actual impact on predictions, not just statistical contribution

Use case: “What are the most important factors determining public coverage?”

sv_importance(shap_values, show_numbers = TRUE) +
  ggtitle("Feature Importance: Mean |SHAP value|")

Figure 5: Feature Importance: Mean |SHAP value|

5.7 Global: Beeswarm plot

EXPLANATION: The most information-rich SHAP visualization! It combines feature importance with feature effects across all predictions.

How to read it:

• Y-axis: Features ranked by importance (most important at top)
• X-axis: SHAP value (impact on prediction)
• COLOR: Feature value (yellow = high, purple = low)
• Each dot is one prediction from the test set
• Horizontal spread shows the range of effects

Key insights:

• RELATIONSHIP: Does high feature value increase or decrease predictions?
  • Example: If yellow dots (high values) are on the right, high values increase predictions
• MAGNITUDE: How far dots spread horizontally shows strength of effect
• DISTRIBUTION: Density of dots shows how common different effects are

Use case: “How do different features affect predictions across all individuals?”

sv_importance(shap_values, 
              kind = "bee",
              max_features = 18) +
  ggtitle("Beeswarm Summary Plot: Feature Effects Across All Predictions")

Figure 6: Beeswarm Summary Plot: Feature Effects Across All Predictions

5.8 Global: Dependence plot(s)

EXPLANATION: Shows the relationship between a feature’s values and its SHAP values (impact on predictions). This reveals the exact form of the relationship.

How to read it:

• X-axis: Feature values
• Y-axis: SHAP value (impact on prediction)
• Each dot is one instance
• TREND: Shows if relationship is linear, non-linear, threshold-based, etc.
• VERTICAL SPREAD: Shows interaction effects with other features

Use case: “How exactly does income affect the prediction? Is it linear?”

# Picks strongest interacting feature
sv_dependence(shap_values, v = "age", color_var = "income")

sv_dependence(shap_values, v = "age", color_var = "sex")

sv_dependence(shap_values, v = "income", color_var = "sex")

sv_dependence(shap_values, v = "race", color_var = "sex")

sv_dependence(shap_values, v = "marital_status", color_var = "age")

xvars1 <- c("income", "age", "education", 
           "marital_status")
sv_dependence(shap_values, v = xvars1, share_y = TRUE)

5.9 Exercise

1. Please run the code above to explore whether it works locally.
2. For the local plots (e.g., waterfall plot, force plot) try to pick an interesting individual (or several). This could be individuals that have been misclassified, are close to the threshold or are high-confidence predictions (pred. probability close to 0 and 1).
3. Please explore more dependency plots using sv_dependence().

References

Biecek, Przemyslaw, and Tomasz Burzykowski. 2021. Explanatory Model Analysis: Explore, Explain, and Examine Predictive Models. Routledge Cavendish.