Mitigating Bias in Income Prediction using the UCI Adult Income Dataset

Abstract

This research investigates bias in machine learning models applied to income prediction, a domain with significant societal implications. Utilizing the widely-used UCI Adult Income dataset, we train a baseline classification model to predict whether an individual’s income exceeds $50K. The study employs fairness evaluation techniques from the Fairlearn library to assess disparate impact across sensitive attributes, specifically focusing on “sex” as identified in the dataset. We quantify metrics such as accuracy, selection rate, true positive rate, and false positive rate for different demographic groups to highlight performance disparities. Subsequently, we explore and apply a fairness mitigation technique, Exponentiated Gradient, to the trained model to reduce observed biases. The impact of mitigation on both overall model performance and group-specific fairness metrics is analyzed. This work demonstrates a practical workflow for identifying, quantifying, and mitigating bias, emphasizing the importance of fairness considerations in the development and deployment of machine learning systems.

1. Introduction

Machine learning models are increasingly deployed in high-stakes decision-making processes, ranging from loan applications and hiring to criminal justice and healthcare. While offering powerful predictive capabilities, these models can inadvertently perpetuate and even amplify existing societal biases present in the training data. One critical area where algorithmic bias is a significant concern is income prediction. Predicting an individual’s income based on demographic and socioeconomic factors can be used in various applications, including targeted marketing, resource allocation, and policy making. However, if these models exhibit bias with respect to sensitive attributes such as race or sex, they can lead to unfair or discriminatory outcomes. The problem of bias in machine learning stems from biased data collection, prejudiced feature selection, or algorithmic limitations. Addressing this issue requires a systematic approach that involves identifying potential biases, quantifying their extent, and applying appropriate mitigation strategies. The UCI Adult Income dataset, derived from the 1994 Census database, is a widely-used benchmark for this task. This study aims to provide a comprehensive analysis of bias in an income prediction model. We begin by training a baseline classification model and evaluating its overall performance. We then conduct a detailed fairness assessment, analyzing how the model performs across different groups defined by the ‘sex’ (or ‘gender’) sensitive attribute. Following the identification of biases, we apply the Exponentiated Gradient fairness mitigation technique. Finally, we evaluate the effectiveness of the mitigation strategy by comparing the performance and fairness metrics of the mitigated model against the baseline.

2. Methodology and System Implementation

The project methodology involves data acquisition, preprocessing, baseline model training, and the application of fairness mitigation techniques.

2.1. Data Loading and Preprocessing

The research utilizes the UCI Adult Income dataset, specifically the “Adult Census Income Dataset” from Kaggle. Preprocessing involved several steps:

1. Data Cleaning: Rows with missing values were dropped.
2. Target Target Variable Creation: A binary target variable, income_label, was created from the original income column, mapping values like ‘>50K’ to 1 and others to 0.
3. Sensitive Attribute Identification: The ‘sex’ (or ‘gender’) column was identified as the sensitive attribute for analysis.
4. Feature Selection: Features were selected by excluding the original target and potential identifiers.
5. Data Splitting: The dataset was split into training (80%) and testing (20%) sets, stratified on the target variable to maintain class distribution. A preprocessing pipeline was constructed using scikit-learn’s ColumnTransformer. Numerical features were imputed using the median strategy and then scaled using StandardScaler. Categorical features were imputed using the most frequent value and then encoded using OneHotEncoder. The scikit-learn pipeline implementation is as follows:

from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

num_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])
cat_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('ohe', OneHotEncoder(handle_unknown='ignore')) 
    # sparse=False was used in the project [cite: 396]
])
preprocessor = ColumnTransformer([
    ('num', num_pipeline, num_cols),
    ('cat', cat_pipeline, cat_cols)
])

2.2. Baseline Model

A baseline classification model was built using a Pipeline that combined the preprocessor and a RandomForestClassifier configured with 200 estimators. The pipeline was fitted on the training data (X_train, y_train) and saved as artifacts/baseline_rf.pkl.

from sklearn.ensemble import RandomForestClassifier

clf = Pipeline([
    ('preproc', preprocessor),
    ('rf', RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1))
])
clf.fit(X_train, y_train)
joblib.dump(clf, 'artifacts/baseline_rf.pkl')

2.3. Fairness Mitigation

To address biases, the Exponentiated Gradient (EG) reduction technique from the Fairlearn library was employed, aiming for Demographic Parity. EG is an in-processing algorithm that repeatedly calls a standard classifier with re-weighted training data. A fresh RandomForestClassifier pipeline (rf2) was defined to be used as the estimator within the ExponentiatedGradient algorithm. The mitigation model was then fit on the training data, specifying the sensitive features.

from fairlearn.reductions import ExponentiatedGradient, DemographicParity

# Define a new pipeline for the estimator
rf2 = Pipeline([
    ('preproc', preprocessor), 
    ('rf', RandomForestClassifier(n_estimators=200, random_state=0))
])

eg = ExponentiatedGradient(
    estimator=rf2,
    constraints=DemographicParity(),
    eps=0.02
)
eg.fit(X_train, y_train, sensitive_features=train_df[SENSITIVE])
joblib.dump(eg, 'artifacts/eg_dempar.pkl')

Predictions from this mitigated model (y_pred_eg) were then generated on the test set for comparison.

3. System Architecture and MLOps

Beyond the core model, the project defines a full MLOps system for monitoring, inference, and interaction, comprising a model API, an interactive dashboard, and a monitoring pipeline. The repository is structured to separate concerns: Code snippet graph TD A[fairlens/] —> B(notebook/01_train_and_fairness.ipynb); A —> C(app/); C —> D(streamlit_app.py); C —> E(api/main.py); C —> F(docker/Dockerfile_streamlit); A —> G(artifacts/); G —> H(baseline_rf.pkl); G —> I(eg_dempar.pkl); G —> J(metrics_summary.json); G —> K(evidently_report.html); The Streamlit dashboard (app/streamlit_app.py) provides an interactive UI for exploring model predictions and fairness metrics. It loads the metrics_summary.json file to display overall performance, group-specific metrics, and visualizations. The FastAPI endpoint (app/api/main.py) serves as the inference backend, loading the baseline model and providing a /predict route for real-time predictions. It is deployed on Render for public access. The monitoring pipeline is simulated in the notebook, generating Evidently reports to detect data drift and performance degradation.

3.1. Inference API (FastAPI)

A REST API is defined using FastAPI to serve model predictions programmatically. The API loads the saved baseline_rf.pkl model and exposes endpoints:

• /predict: Accepts a JSON payload with feature values (e.g., age, gender), converts it to a pandas.DataFrame, and returns the model’s prediction (0 or 1) and probability score.
• /health: A simple status check endpoint.
• /simulate_drift: An optional endpoint that generates synthetic data with shifted distributions (e.g., older population, different gender balance) to test monitoring.

This API is designed for deployment as a web service on platforms like Render.

3.2. Interactive Dashboard (Streamlit)

An interactive web dashboard is built using Streamlit to visualize model behavior and fairness metrics. The dashboard, defined in app/streamlit_app.py, is designed for deployment on Hugging Face Spaces.

Its key functions include:

1. Loading Artifacts: Loads the baseline model, mitigated model, and saved JSON metrics.
2. Displaying Metrics: Shows overall accuracy for baseline and mitigated models and displays the detailed group fairness metrics from the MetricFrame in a table.
3. Live Prediction: Provides a simple UI form for users to input features (e.g., age, gender) and receive a live prediction from the baseline model.
4. Embedding Reports: Embeds the full HTML report generated by Evidently AI for detailed data drift and performance analysis.

4. Results and Analysis 4.1. Evaluation Metrics

Model performance and fairness were evaluated using a comprehensive set of metrics:

• Overall Performance: Accuracy, ROC AUC, Precision, Recall (TPR), F1-score.
• Fairness Metrics (Group-Specific):
  • Selection Rate: Proportion of instances predicted as positive (>50K).
  • False Positive Rate (FPR): Proportion of negative instances incorrectly predicted as positive.
  • True Positive Rate (TPR): Proportion of positive instances correctly predicted as positive.
  • Demographic Parity Difference: Difference between the highest and lowest selection rates across groups.
  • Demographic Parity Ratio: Ratio of the lowest to highest selection rates.
  • Equalized Odds Difference: Maximum absolute difference in FPR and TPR across groups.
  • Disparate Impact: Ratio of a group’s selection rate to the reference group’s rate.

Fairlearn’s MetricFrame was used to calculate these metrics stratified by the ‘sex’ attribute.

4.2. Baseline Model Performance

The baseline RandomForest model achieved reasonable overall performance on the test set:

• Overall Accuracy: 0.8520
• Overall ROC AUC: 0.9005

The model performed better on the majority class ($<=50K$) (F1-score: 0.9048) than the minority class ($>50K$) (F1-score: 0.6680). The confusion matrix (Figure 1) visualizes the counts of true positive, true negative, false positive, and false negative predictions.

Classification Report

4.3. Baseline Fairness Analysis

A fairness analysis of the baseline model revealed significant disparities with respect to the ‘sex’ attribute.

Baseline Group Metrics (Sex)

Key disparities observed:

1. Selection Rate: The rate for females (8.04%) was significantly lower than for males (26.67%).
2. Accuracy: The model was more accurate for females (93.0%) than for males (81.4%).
3. Error Rates: The False Positive Rate was much lower for females (2.25%) than for males (10.66%). Conversely, the True Positive Rate was also lower for females (54.6%) than for males (63.2%).

The Disparate Impact ratio for females (comparing their selection rate to the reference group, males) was 0.3012. This is significantly below the common fairness threshold of 0.8, indicating substantial bias against the female group in being predicted as having an income >50K.

These disparities are visualized in Figure 3 (Accuracy by group), Figure 4 (Selection Rate by group), and Figure 5 (TPR / FPR by group).

The Disparate Impact ratio was calculated, comparing each group’s selection rate to the reference group (males, having the highest selection rate). The results are available in artifacts/disparate_impact.csv and artifacts/disparate_impact.json.

The disparate impact ratio for females is 0.3012, significantly below the common threshold of 0.8, indicating substantial bias against the female group in being predicted as having income >50K. This is visualized in Figure 6.

Confusion matrices for each group (Figures 7 and 8) provide a detailed view of prediction outcomes by group. These matrices visually confirm the differences in true/false positive/negative counts between the groups, aligning with the observed disparities in FPR and TPR.

4.4. Fairness Mitigation Results

The Exponentiated Gradient model aiming for Demographic Parity was evaluated. Comparison of Selection Rates

In this specific execution, the mitigation technique resulted in minimal changes to the selection rates80. The selection rate gap was not significantly reduced81. This highlights that mitigation effectiveness can vary and often involves a trade-off with overall accuracy82.

A comparison of baseline and mitigated metrics shows the impact of mitigation.

The accuracy by group comparison is visualized in Figure 9.

Observing the comparison, the mitigated model shows slight changes in both accuracy and selection rates for both groups. Ideally, mitigation would significantly reduce the gap in selection rates (bringing the disparate impact ratio closer to 1) while minimizing the impact on accuracy, especially for the unprivileged group. In this specific execution, the changes in selection rates were minimal. The effectiveness of mitigation techniques can vary based on the model, dataset, and specific technique applied, and often involves a trade-off with overall accuracy or performance for one or more groups.

4.5. Model Interpretation and Monitoring

Feature Importance
Feature importance analysis for the baseline model identified fnlwgt, age, capital.gain, hours.per.week, and marital.status_Married-civ-spouse as the most influential features. While predictive, features like occupation, education, and age can be highly correlated with sensitive attributes, potentially perpetuating historical biases even when ‘sex’ itself is not used as a direct feature.

Data Drift and Performance Monitoring
An MLOps monitoring component was implemented using Evidently AI. The notebook generates a comprehensive report comparing the training (reference) dataset and the test (current) dataset.

from evidently.report import Report
from evidently.metric_preset import DataDriftPreset, ClassificationPreset
from evidently import ColumnMapping

column_mapping = ColumnMapping()
column_mapping.target = 'income_label'

# Add predictions to the test dataframe
cur = test_df.copy()
cur['pred_baseline'] = clf.predict(cur[features])
cur['pred_mitigated'] = y_pred_eg

drift_report = Report(metrics=[DataDriftPreset(), ClassificationPreset()])
drift_report.run(reference_data=ref, current_data=cur, column_mapping=column_mapping)
drift_report.save_html('artifacts/evidently_report.html')

This report analyzes:

1. Data Drift: Checks for distribution shifts in all features between the training and test sets. In the initial analysis, numeric feature distributions appeared stable, suggesting low data drift.
2. Classification Performance: Provides a detailed breakdown of model performance on the test set, including confusion matrices, ROC curves, and class-specific precision/recall.

This report (evidently_report.html) is a crucial artifact for monitoring model health in production. The /simulate_drift API endpoint can be used to periodically generate new “current” data to test this monitoring pipeline.

The top 15 features are visualized in Figure 10, and the full list is in artifacts/feature_importance_full.csv. The most influential features include fnlwgt, age, capital.gain, hours.per.week, and marital.status_Married-civ-spouse. These features intuitively relate to factors influencing income, such as work experience, wealth accumulation, work intensity, and household structure. While predictive, some of these features (e.g., occupation, education, and even age due to historical wage gaps) can be highly correlated with sensitive attributes like sex or race. Heavy reliance on such features can inadvertently perpetuate biases present in historical data, even if the sensitive attribute itself is not used as a direct feature. Feature importance provides model explainability but must be considered alongside fairness metrics. Data drift analysis (cell pwP7M4ycNGYA) assessed the stability of feature distributions between the training and test datasets. Figure 11 shows the distribution of some numeric features. The distributions of numeric features appeared largely similar between the training and test sets, suggesting low data drift. This indicates that the test data is representative of the training data in terms of these variables, which is a positive sign for the model’s expected stability. Significant drift could signal changes in the real-world data distribution compared to training data, potentially degrading both performance and fairness over time.

5. Discussion and Conclusion

This study demonstrated a workflow for identifying, analyzing, and mitigating bias in an income prediction model. The baseline model, while reasonably accurate overall, exhibited significant fairness disparities, with a disparate impact ratio of 0.3012 for the ‘sex’ attribute, far below parity. The application of the Exponentiated Gradient mitigation technique showed the process of addressing such bias, although in this specific run, the impact on selection rates was modest. This underscores that mitigation requires careful tuning to balance fairness gains with overall model utility. Feature importance analysis highlighted the challenge of indirect bias, where features like age or marital.status may carry historical biases correlated with sensitive attributes. The integration of Evidently reports and a drift simulation API demonstrates a robust MLOps approach, essential for maintaining model reliability and fairness in production. Limitations include focusing on a single sensitive attribute, using one mitigation technique, and relying on a dataset from 1994. Future work should explore intersectional fairness, experiment with diverse mitigation techniques (pre-, in-, and post-processing), and analyze other fairness metrics like Equalized Odds. In conclusion, addressing algorithmic bias is paramount. This research provides a comprehensive blueprint—from initial training and fairness evaluation to a fully deployable MLOps system with monitoring—to support the development of more equitable AI systems.

6. Artifacts and Reproducibility

A comprehensive set of artifacts was generated to support reproducibility, deployment, and monitoring:

• Notebook: notebook/01_train_and_fairness.ipynb (The primary Colab notebook for training and analysis).
• Model Files: model/baseline_rf.pkl (Saved baseline model) and model/eg_dempar.pkl (Saved mitigated model).
• Application Code:
  • app/streamlit_app.py (Streamlit dashboard UI).
  • app/api/main.py (FastAPI inference endpoint).
  • app/requirements.txt and app/api/requirements.txt (Python dependencies).
• Metrics and Reports:
  • model/metrics_summary.json (Key performance/fairness metrics for the dashboard).
  • model/evidently_report.html (Interactive data drift and classification report).
  • Detailed metrics CSV/JSON files (e.g., group_metrics_baseline.csv, disparate_impact.json).
• Visualizations: PNG files for charts (e.g., confusion_matrix_baseline.png, disparate_impact.png, feature_importance_top15.png).
• Deployment: docker/Dockerfile_streamlit (Dockerfile for Hugging Face Spaces deployment).
• Sample Data: test_samples_with_predictions.csv (Test data subset with model predictions).

The test_samples_with_predictions.csv file is particularly useful for inspecting individual model predictions:

Displaying first few rows of sample predictions

import pandas as pd sample_predictions_df = pd.read_csv(‘artifacts/test_samples_with_predictions.csv’) print(sample_predictions_df.head().to_markdown(index=False))

7. References

This research utilized several key Python libraries and external resources:

• Dataset: Adult Census Income (Eman Fatima, 2025) via Kaggle.
• Core ML: Scikit-learn (model building, evaluation), Pandas (data manipulation), Numpy (numerical operations).
• Fairness: Fairlearn (fairness assessment and mitigation).
• MLOps/Monitoring: Evidently AI (data drift and model performance monitoring).
• API & UI: FastAPI (inference API), Uvicorn (ASGI server), Streamlit (interactive dashboard).
• Deployment: Render (API hosting), Hugging Face Spaces (Streamlit hosting).
• Utilities: Joblib (model serialization).
• Visualization: Matplotlib, Seaborn, Plotly.