Impact of PCA Dimensionality Reduction on Classification Algorithm Performance using Hepatitis Dataset

This project analyzes the impact of Principal Component Analysis (PCA) on the performance of several machine learning classification algorithms for predicting hepatitis patient outcomes. The study compares classification performance before and after dimensionality reduction using the Hepatitis dataset from the UCI Machine Learning Repository.

Overview

Hepatitis is an inflammatory liver disease caused by viral infections that can damage liver cells and organ function. Accurate prediction of patient outcomes is important to support medical decision-making and improve treatment effectiveness.

Medical datasets often contain many correlated features, leading to high-dimensional data that increases model complexity and computational cost. Principal Component Analysis (PCA) is applied to reduce dimensionality while preserving as much information as possible.

This project evaluates whether PCA improves the performance of machine learning classification models on the Hepatitis dataset.

Objectives

• Apply Principal Component Analysis (PCA) to reduce the dimensionality of the Hepatitis dataset.
• Compare the performance of classification algorithms before and after PCA.
• Analyze the effect of dimensionality reduction on prediction accuracy and evaluation metrics.

Dataset

• Dataset: Hepatitis Dataset
• Source: UCI Machine Learning Repository
• Number of Instances: 155
• Original Features: 20
• Features after PCA: 7 Principal Components
• Target Variable: Patient Outcome (Live / Die)

Methodology

The project follows these stages:

1. Data Cleaning
2. Missing Value Imputation
3. Data Normalization (Z-Score)
4. Exploratory Data Analysis (EDA)
5. Correlation Analysis
6. Principal Component Analysis (PCA)
7. Model Training
8. Model Evaluation
9. Performance Comparison

Machine Learning Models

The following classification algorithms were evaluated:

• Logistic Regression
• Decision Tree
• K-Nearest Neighbor (KNN)

Evaluation Metrics

Model performance was measured using:

• Accuracy
• Recall
• Specificity
• Precision
• Negative Predictive Value (NPV)
• F1-Score
• Training Time

Technologies Used

• Python
• Jupyter Notebook
• Pandas
• NumPy
• Scikit-learn
• Matplotlib
• Seaborn
• SPSS

Results

Before PCA

Model	Accuracy	F1-Score	Recall
Logistic Regression	87.1%	92.0%	95.8%
Decision Tree	93.5%	95.8%	95.8%
KNN	83.9%	89.8%	91.7%

After PCA

Model	Accuracy	F1-Score	Specificity
Logistic Regression	90.3%	93.9%	71.4%
Decision Tree	74.2%	82.6%	57.1%
KNN	83.9%	89.8%	57.1%

Key Findings

• PCA successfully reduced the dataset from 20 features to 7 principal components while maintaining the original 155 observations. :contentReference[oaicite:0]{index=0}
• Decision Tree achieved the best overall performance on the original dataset, reaching 93.5% accuracy and 95.8% F1-score. :contentReference[oaicite:1]{index=1}
• Logistic Regression showed improved performance after PCA, achieving 90.3% accuracy, 93.9% F1-score, and 71.4% specificity. :contentReference[oaicite:2]{index=2}
• PCA reduced the performance of Decision Tree but improved Logistic Regression, while KNN performance remained relatively stable. :contentReference[oaicite:3]{index=3}

Project Structure

├── dataset/
│   └── hepatitis.csv
├── notebooks/
│   ├── Data_Preprocessing.ipynb
│   ├── Exploratory_Data_Analysis.ipynb
│   ├── PCA.ipynb
│   └── Classification_Model.ipynb
├── images/
│   ├── correlation_matrix.png
│   ├── scree_plot.png
│   └── model_comparison.png
├── report/
│   └── Project_Report.pdf
├── requirements.txt
└── README.md

Workflow

Hepatitis Dataset
        │
        ▼
 Data Cleaning
        │
        ▼
 Missing Value Imputation
        │
        ▼
 Standardization (Z-Score)
        │
        ▼
 Exploratory Data Analysis
        │
        ▼
 Principal Component Analysis
        │
        ▼
 Train-Test Split (80:20)
        │
        ▼
 ┌────────────────────────────┐
 │ Logistic Regression        │
 │ Decision Tree              │
 │ K-Nearest Neighbor (KNN)   │
 └────────────────────────────┘
        │
        ▼
 Model Evaluation
        │
        ▼
 Performance Comparison