0 cumulative citations
View corpus contextAutomated credit models under-detect defaults among high-income borrowers: a 16.9 percentage-point recall gap that halves when income is withheld and narrows further when loan interest rates are removed, yet a residual disparity persists as loan size and home ownership reconstruct omitted signals.
Beyond Aggregate Calibration: Decomposing Income-Conditional Recall Disparities in Automated Credit Default Prediction
On 1.34M LendingClub loans, default-prediction models show a 16.86 percentage-point lower true positive rate for high-income defaulters versus low-income defaulters, and decomposing this gap attributes roughly half to direct income use, another portion to inherited information in origination interest rates, and a smaller residual (≈2.6–3.6 pp) to proxies like loan amount and home ownership.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Data-centric curation pipelines frequently rely on model confidence scores to flag and filter noisy or mislabeled training instances. Evaluating this filtering convention on a large-scale consumer lending sample (LendingClub, N = 1,344,936) uncovers an underlying demographic asymmetry: high-income defaulters are disproportionately classified as label noise relative to low-income defaulters (Cramer's V approximately 0.03-0.07). Re-examining this behavior through the lens of equal opportunity [Hardt et al., 2016] reveals a far more severe discrepancy: a 16.86 percentage point gap in true positive rate (recall) between high- and low-income borrowers who ultimately defaulted. Implementing a sequential feature-blinding methodology allows us to isolate the drivers of this disparity across three distinct mechanisms: (1) direct reliance on self-reported applicant income; (2) algorithmic absorption of upstream institutional bias encoded within origination interest rates; and (3) a residual disparity (3.55 percentage points in cross-validation; 2.56 percentage points on a held-out test partition, Z = -4.04, p < 0.0001) that remains even after purging both income and interest rates from the model. Out-of-sample signed SHAP valuations demonstrate that this residual gap is maintained by structural proxies, most notably loan amount and home ownership status. These empirical findings show that simply blinding an algorithm to sensitive attributes fails to ensure fairness when institutional pricing decisions and behavioral proxy variables collectively reconstruct the omitted signals. We outline the practical implications of these findings for auditing data-centric AI workflows within regulated financial institutions.
Summary
Main Finding
Confidence-based dataset cleaning and naive attribute blinding conceal and can perpetuate income-conditional disparities in credit-default prediction. On a 1.34M‑record LendingClub sample (2007–2018) the author decomposes a large recall (true positive rate) gap between income groups into three mechanisms: (1) direct use of self-reported income; (2) algorithmic absorption of upstream institutional signals (origination interest rates); and (3) a residual proxy-driven effect (loan amount, home ownership) that persists after blinding both income and interest rate.
Key Points
- Data and sample: LendingClub accepted-loans N = 1,344,936; target = charged-off (default). Income groups (post-hoc): Low < $50k, Middle $50–100k, High > $100k.
- Motivating audit (confidence-based label filtering):
- Using a 0.15 predicted-probability threshold, actual defaulters flagged as “noise” were discarded at different rates: High 2.40%, Middle 0.99%, Low 0.38% (χ2 significant; Cramér’s V ≈ 0.0666). This shows data-cleaning heuristics are income-skewed.
- Equal-opportunity (TPR among actual defaulters) results (expanded feature XGBoost):
- Full model TPRs: Low = 72.84%, Middle = 66.69%, High = 55.98%. High–Low recall gap = 16.86 percentage points.
- Income-blind model (remove annual_inc): gap compresses to 7.49 ppt (High = 61.35%, Low = 68.84%) — roughly half the original gap is due to direct income usage.
- Double-blind model (remove annual_inc and int_rate): gap further compresses to 3.55 ppt in CV and 2.56 ppt on held-out test (Z = −4.04, p < 0.0001; 95% CI [1.31%, 3.80%]).
- Mechanism attribution (out-of-sample SHAP on holdout):
- Interest rate acts as the dominant upstream signal when income is hidden: mean SHAP contribution for int_rate among high-income false negatives ≈ −0.355 (much larger than next feature).
- Residual proxy drivers when both income and int_rate are removed: loan_amnt and home_ownership (loan_amnt correlates with income: Spearman ρ ≈ 0.481). These features sustain the remaining TPR gap.
- Robustness:
- Results replicate across XGBoost and regularized logistic regression (double-blind TPR pattern persists).
- CV and strict 80/20 holdout agree within ≈1 percentage point; double-blind model retains discriminative power (holdout AUC ≈ 0.69).
- Limitations noted by author:
- Redacted borrower IDs prevent checking repeat-borrower leakage.
- Public dataset lacks protected-class labels (race, gender), so findings are specific to income and unsecured consumer loans.
- The discard-rate numbers depend on the chosen confidence threshold—presented as motivating, not primary, evidence.
Data & Methods
- Dataset: LendingClub accepted-loans (2007–Q4 2018), filtered to terminal outcomes; N = 1,344,936.
- Feature sets:
- Minimal baseline: loan_amnt, annual_inc, int_rate, dti.
- Expanded set: adds fico range low, revol_util, pub_rec_bankruptcies, emp_length, home_ownership, purpose, term, etc. Missingness encoded via sentinels + indicator flags.
- Modeling:
- Primary: XGBoost with scale_pos_weight ≈ 4:1; 5-fold stratified cross-validation (stratified on default status and income bracket).
- Baseline comparison: regularized logistic regression.
- Holdout: strict 80/20 split; SHAP explanations computed on held-out test partition using TreeExplainer.
- Sequential feature-blinding protocol:
- Full model (all features, annual_inc included).
- Income-blind (exclude annual_inc).
- Double-blind (exclude annual_inc and int_rate).
- Metrics and statistics:
- Primary fairness metric: equal opportunity (group-conditional TPR among actual defaulters).
- Statistical tests: two-proportion z-tests, chi-square tests, Cramér’s V, Spearman correlations; deterministic seed used for reproducibility.
- Mechanism analysis:
- Out-of-sample, signed SHAP values to identify which features push predictions up/down for false negatives and to quantify contributions.
Implications for AI Economics
- Data-curation effects matter for distributional risk: Confidence-threshold cleaning is not neutral — it can systematically exclude or downweight minority-class, high-income defaulters and therefore bias datasets used to train downstream models. Economists studying algorithmic credit outcomes should treat data-cleaning steps as policy-relevant interventions.
- Institutional decisions propagate into learned models: Human-originated pricing (interest rates) encodes prior underwriting judgments that algorithms re‑use as predictive signals when income is unavailable. This creates a channel where institutional decisions (and their biases) are amplified algorithmically, affecting market outcomes and allocation of credit.
- Blinding protected attributes is insufficient: Removing explicit income does not remove income information — correlated transactional and contract variables (loan size, home ownership, etc.) reconstruct omitted signals. Policy or regulatory approaches that rely on attribute omission will likely fail unless they also address proxy features and upstream signals.
- Measurement and audit design: Fairness audits in financial markets must go beyond aggregate calibration metrics and include subgroup recall (equal opportunity) analyses, decomposition of mechanisms (e.g., via sequential blinding and SHAP), and strict out-of-sample validation to detect persistent proxy effects.
- Remedies and research directions:
- For practitioners: combine feature-level audits, targeted collection of protected attributes where legally and ethically feasible, and explicit mitigation (reweighing, constrained optimization for equal opportunity, adversarial debiasing) that accounts for upstream institutional variables.
- For regulators and economists: monitor how pricing and underwriting practices create signals that algorithms absorb; consider transparency requirements for pricing rules and audit trails for data-cleaning steps.
- For researchers: pursue causal and counterfactual approaches to identify and block proxy channels; quantify welfare consequences of recall disparities (credit access, pricing distortion, adverse selection) and evaluate mitigation costs vs. benefits.
- Broader economic consequence: Misclassification concentrated in certain income strata can change who gets offered credit or monitoring, potentially altering default dynamics, adverse selection, and access to financial services. The study highlights that algorithmic risk models are endogenous to historical institutional behavior and transactional geometry; policy must therefore address both model training pipelines and the institutional inputs feeding them.
Assessment
Paper Typecorrelational
Evidence Strengthmedium — Large (N≈1.34M) real-world dataset, careful train/holdout splitting, model-agnostic robustness checks, and explicit statistical testing provide clear, reproducible descriptive evidence of income-conditional model behavior; however the study is observational and relies on feature removal and SHAP attributions for mechanism claims, so causal claims about how upstream institutional decisions produce disparities remain inferential rather than identified via exogenous variation.
Methods Rigormedium — The authors follow strong ML practice (stratified CV, untouched holdout, pipeline-encapsulated preprocessing, multiple model families, reproducibility notes, and appropriate statistical tests). Identification of mechanisms is implemented transparently via sequential blinding and out-of-sample SHAP, but such techniques cannot fully rule out confounding or alternative proxy pathways; reliance on a single public dataset with redacted identifiers and self-reported income also limits claims about general causal channels.
SamplePublic LendingClub accepted-loans dataset, origination years 2007–Q4 2018, N = 1,344,936 loans with terminal outcomes (Fully Paid or Charged Off); binary target = charged-off (default); borrowers partitioned post-hoc into Low (<$50k), Middle ($50–100k), High (>$100k) income brackets; features include loan amount, annual income (sometimes withheld in blinded specs), origination interest rate, dti, FICO, revol_util, pub_rec_bankruptcies, emp_length, home_ownership, purpose, term. Missing borrower IDs were redacted and income is self-reported.
Themesinequality governance
IdentificationObservational decomposition using sequential feature-blinding (remove income, then remove income+interest rate) combined with stratified cross-validation and a strict held-out test; feature-attribution via out-of-sample SHAP values; group-conditional performance compared with two-proportion z-tests and chi-square tests to attribute portions of an observed recall gap to direct use of income, inherited institutional signals (interest rate), and downstream proxies (loan amount, home ownership). No exogenous variation or causal instrumenting is used.
GeneralizabilityOnly U.S. unsecured consumer loans on LendingClub (2007–2018); results may not generalize to mortgages, other credit products, geographies, or later macroeconomic regimes., Sample restricted to accepted loans (no rejected applications), so selection on acceptance may bias observed relationships., Income is self-reported and coarse; protected attributes like race and gender are unavailable, so intersectional or demographic generalization is limited., Redacted borrower identifiers prevent testing for repeat-borrower leakage; dataset artifacts may affect estimates., Findings depend on feature set and origination practices of this platform; different underwriting or pricing processes could exhibit different upstream signals.
Claims (8)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Confidence-based filtering discarded true default labels at higher rates for high-income borrowers than for low-income borrowers: 2.40% versus 0.38%, respectively. Automation Exposure | negative | Income-conditional rate at which actual default labels were discarded as suspected noise |
Reading fidelity
high
Study strength
medium
|
n=1344936
2.02 percentage point absolute gap; roughly 6x in relative terms
|
| The full expanded-feature model had substantially lower default recall for high-income borrowers than for low-income borrowers: 55.98% versus 72.84%, a 16.86 percentage point gap. Decision Quality | negative | True positive rate (recall) among borrowers who defaulted |
Reading fidelity
high
Study strength
medium
|
n=1344936
16.86 percentage point recall gap
|
| Removing self-reported annual income from the model reduced the high-versus-low-income recall gap from 16.86 percentage points to 7.49 percentage points. Decision Quality | positive | High-versus-low-income gap in default recall |
Reading fidelity
high
Study strength
medium
|
n=1344936
Gap reduction of 9.37 percentage points, from 16.86 to 7.49 percentage points
|
| Removing both annual income and origination interest rate left a residual high-versus-low-income recall gap of 3.55 percentage points in cross-validation and 2.56 percentage points on the held-out test set. Decision Quality | negative | Residual difference in default recall between high- and low-income borrowers |
Reading fidelity
high
Study strength
high
|
n=1344936
3.55 percentage points in cross-validation; 2.56 percentage points on held-out test partition
|
| The double-blind model retained meaningful predictive discrimination, with a held-out AUC of approximately 0.69, so the residual recall disparity was not attributed to model collapse. Decision Quality | positive | Out-of-sample area under the ROC curve after feature blinding |
Reading fidelity
high
Study strength
medium
|
n=1344936
holdout AUC ≈0.69
|
| Origination interest rate acted as an upstream signal that suppressed predicted default risk for high-income false negatives in the income-blind model. Decision Quality | negative | Predicted default-risk score and resulting recall for high-income defaulters |
Reading fidelity
high
Study strength
medium
|
n=1344936
Mean SHAP contribution of -0.355 for interest rate; median rates 14.49% versus 15.31%
|
| After income and interest rate were blinded, loan amount was the dominant residual proxy associated with the recall disparity, followed by mortgage home-ownership status. Decision Quality | negative | Feature contributions to false-negative default predictions and income-proxy relationships |
Reading fidelity
high
Study strength
medium
|
n=1344936
Mean SHAP difference ≈0.065; Spearman correlation = 0.481
|
| The residual recall disparity was not specific to gradient-boosted trees: a regularized logistic regression model using the double-blind feature set also produced lower recall for high-income borrowers than for middle- and low-income borrowers. Decision Quality | negative | Income-group-specific default recall under an alternative model architecture |
Reading fidelity
high
Study strength
medium
|
n=1344936
3.64 percentage point high-versus-low recall gap
|