Understanding Factors Influencing US Household Income

We consider quantifying the importance of various factors in determining whether a US resident’s income exceeds $50k/year, based on the 1994 census data, as per the UCI machine learning repository: https://archive.ics.uci.edu/ml/datasets/census+income.

In [1]:
%load_ext autoreload
%autoreload 2

import pandas as pd
import numpy as np
import kxy

df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data', \
                 names=['Age', 'Workclass', 'Fnlwgt', 'Education',\
                        'Education Num', 'Marital Status',\
                        'Occupation', 'Relationship', 'Race', 'Sex', \
                        'Capital Gain', 'Capital Loss', 'Hours Per Week', \
                        'Native Country', 'Income'])

The Data

In [2]:
df.kxy.describe()

-----------
Column: Age
-----------
Type:   Continuous
Max:    90.0000
Mean:   38.5816
Median: 37.0000
Min:    17.0000

--------------------
Column: Capital Gain
--------------------
Type:   Continuous
Max:    99999.0000
Mean:   1077.6488
Median: 0.0000
Min:    0.0000

--------------------
Column: Capital Loss
--------------------
Type:   Continuous
Max:    4356.0000
Mean:   87.3038
Median: 0.0000
Min:    0.0000

-----------------
Column: Education
-----------------
Type:      Categorical
Frequency: 32.25%, Label:  HS-grad
Frequency: 22.39%, Label:  Some-college
Frequency: 16.45%, Label:  Bachelors
Frequency:  5.29%, Label:  Masters
Frequency:  4.24%, Label:  Assoc-voc
Frequency:  3.61%, Label:  11th
Frequency:  3.28%, Label:  Assoc-acdm
Frequency:  2.87%, Label:  10th
Other Labels: 9.63%

---------------------
Column: Education Num
---------------------
Type:   Continuous
Max:    16.0000
Mean:   10.0807
Median: 10.0000
Min:    1.0000

--------------
Column: Fnlwgt
--------------
Type:   Continuous
Max:    1484705.0000
Mean:   189778.3665
Median: 178356.0000
Min:    12285.0000

----------------------
Column: Hours Per Week
----------------------
Type:   Continuous
Max:    99.0000
Mean:   40.4375
Median: 40.0000
Min:    1.0000

--------------
Column: Income
--------------
Type:      Categorical
Frequency: 75.92%, Label:  <=50K
Frequency: 24.08%, Label:  >50K

----------------------
Column: Marital Status
----------------------
Type:      Categorical
Frequency: 45.99%, Label:  Married-civ-spouse
Frequency: 32.81%, Label:  Never-married
Frequency: 13.65%, Label:  Divorced
Other Labels: 7.55%

----------------------
Column: Native Country
----------------------
Type:      Categorical
Frequency: 89.59%, Label:  United-States
Frequency:  1.97%, Label:  Mexico
Other Labels: 8.44%

------------------
Column: Occupation
------------------
Type:      Categorical
Frequency: 12.71%, Label:  Prof-specialty
Frequency: 12.59%, Label:  Craft-repair
Frequency: 12.49%, Label:  Exec-managerial
Frequency: 11.58%, Label:  Adm-clerical
Frequency: 11.21%, Label:  Sales
Frequency: 10.12%, Label:  Other-service
Frequency:  6.15%, Label:  Machine-op-inspct
Frequency:  5.66%, Label:  ?
Frequency:  4.90%, Label:  Transport-moving
Frequency:  4.21%, Label:  Handlers-cleaners
Other Labels: 8.38%

------------
Column: Race
------------
Type:      Categorical
Frequency: 85.43%, Label:  White
Frequency:  9.59%, Label:  Black
Other Labels: 4.98%

--------------------
Column: Relationship
--------------------
Type:      Categorical
Frequency: 40.52%, Label:  Husband
Frequency: 25.51%, Label:  Not-in-family
Frequency: 15.56%, Label:  Own-child
Frequency: 10.58%, Label:  Unmarried
Other Labels: 7.83%

-----------
Column: Sex
-----------
Type:      Categorical
Frequency: 66.92%, Label:  Male
Frequency: 33.08%, Label:  Female

-----------------
Column: Workclass
-----------------
Type:      Categorical
Frequency: 69.70%, Label:  Private
Frequency:  7.80%, Label:  Self-emp-not-inc
Frequency:  6.43%, Label:  Local-gov
Frequency:  5.64%, Label:  ?
Frequency:  3.99%, Label:  State-gov
Other Labels: 6.44%

Quantifying Factor Importance

To quantify the importance of each factor for determining whether a US resident’s income exceeds $50k/year, we may simply run our variable selection analysis.

In [3]:
df.kxy.variable_selection_analysis('Income')
Out[3]:
Selection Order Univariate Achievable R^2 Maximum Marginal R^2 Increase Running Achievable R^2 Running Achievable R^2 (%) Univariate Achievable Accuracy Maximum Marginal Accuracy Increase Running Achievable Accuracy Univariate Mutual Information (nats) Conditional Mutual Information (nats) Univariate Achievable True Log-Likelihood Per Sample Maximum Marginal True Log-Likelihood Per Sample Increase Running Achievable True Log-Likelihood Per Sample Running Mutual Information (nats) Running Mutual Information (%)
Variable
Marital Status 1 0.166102 0.166102 0.166102 37.8598 0.827 0.068 0.827 0.090822 0.090822 -0.461189 0.090822 -0.461189 0.090822 31.4507
Education Num 2 0.107763 0.0946444 0.260746 59.4321 0.804 0.035 0.862 0.057012 0.060235 -0.494999 0.060235 -0.400954 0.151057 52.3094
Capital Gain 3 0.086237 0.0564584 0.317205 72.3007 0.795 0.021 0.883 0.045092 0.039723 -0.506919 0.039723 -0.361231 0.19078 66.065
Education 4 0.0220121 0.044731 0.361936 82.4963 0.769 0.016 0.899 0.011129 0.033878 -0.540882 0.033878 -0.327353 0.224658 77.7966
Hours Per Week 5 0.0715289 0.0254576 0.387393 88.2989 0.789 0.009 0.908 0.037108 0.020358 -0.514903 0.020358 -0.306995 0.245016 84.8464
Sex 6 0.0461683 0.00946442 0.396858 90.4561 0.779 0.003 0.911 0.023634 0.007785 -0.528377 0.007785 -0.29921 0.252801 87.5422
Occupation 7 0.0119322 0.0043414 0.401199 91.4457 0.764 0.002 0.913 0.006002 0.003612 -0.546009 0.003612 -0.295598 0.256413 88.793
Age 8 0.0739583 0.000276582 0.401476 91.5087 0.79 0 0.913 0.038418 0.000231 -0.513593 0.000231 -0.295367 0.256644 88.873
Capital Loss 9 0.020352 0.0123512 0.413827 94.3239 0.768 0.004 0.917 0.010281 0.010426 -0.54173 0.010426 -0.284941 0.26707 92.4834
Workclass 10 0.0156089 0.0101135 0.42394 96.6291 0.766 0.004 0.921 0.007866 0.008702 -0.544145 0.008702 -0.276239 0.275772 95.4969
Relationship 11 0.0767599 0.00708538 0.431026 98.2441 0.791 0.003 0.924 0.039933 0.006188 -0.512078 0.006188 -0.270051 0.28196 97.6397
Race 12 0.00907259 0.0054101 0.436436 99.4772 0.763 0.001 0.925 0.004557 0.004777 -0.547454 0.004777 -0.265274 0.286737 99.2939
Native Country 13 0.00179239 0.0013678 0.437804 99.789 0.76 0.001 0.926 0.000897 0.001215 -0.551114 0.001215 -0.264059 0.287952 99.7147
Fnlwgt 14 0.000263965 0.000925737 0.438729 100 0.759 0 0.926 0.000132 0.000824 -0.551879 0.000824 -0.263235 0.288776 100

The first row (marial-status) corresponds to the most important factor. When used in isolation in a classification model, an accuracy of up to 82.7% can be achieved (see the Univariate Achievable Accuracy column). This is 6.8% above the accuracy that can be achieved by the naive strategy consisting of always predicting the most frequent outcome, namely <=50K, (see the Maximum Marginal Accuracy Increase column).

The second row (education-num) corresponds to the factor that complements the marial-status factor the most. It can bring about an additional 3.5% classification accuracy (see the Maximum Marginal Accuracy Increase column).

More generally, the \((i+1)\)-th row corresponds to the factor that complements the first \(i\) factors selected the most.

Interestingly, both the length of a resident’s education and her marital status play a more important role in determining whether she earns more than USD 50,000 per year than the amount of capital gain or capital loss she’s had.

Comparison With Model-Based Factor Importance

The kxy package is the only Python package capable of ex-ante and model-free factor importance analysis.

Other tools adopt a model-based approach consisting of quantifying the extent to which a trained model relies on each input, as opposed to the extent to which inputs are intrinsically useful for solving the problem at hand. Thus, their analyses may vary depending on the model used and as such they are subjective.

This dataset was previously studied using the tree-based SHAP method in this notebook.

Shapley value based studies are inherently model-based as they explain the predictions made by a specific model. Additionally, they are intrinsically unrelated to accuracy or any other performance metric. Indeed, the approach merely explains model predictions (of any model, although it scales better on tree-based models), with no regard on how well said model performs.

Our approach on the other hand quantifies the importance of each variable for effectively solving the problem at hand, in a model-free fashion.