Machine Learning from Scratch using Python

Implementing classical ML models from scratch using only NumPy, and building small projects alongside for deeper understanding. No sklearn for model logic — just math and code.

Following the roadmap.sh/machine-learning roadmap — see ROADMAP.md for full progress tracker.

Progress

Category	Models	Status
Regression	Linear, Multiple, Lasso, Ridge, ElasticNet	Complete
Classification	KNN, Logistic Regression, SVM, Decision Trees, Random Forest, Gradient Boosting	Complete
Unsupervised	K-Means, Hierarchical, DBSCAN, PCA	Complete
Deep Learning	Perceptron, MLP, Adam optimizer, Fashion-MNIST classifier	In progress (CNNs next)

Models Implemented

Regression

Model	Code
Simple Linear Regression	SLR-byhand.py
Multiple Linear Regression	MLR-byhand.py
Ridge Regression	LR-ridge.py
Lasso Regression	LR-lasso.py
ElasticNet Regression	elasticnet_from_scratch.py

Classification

Model	Code
KNN (K-Nearest Neighbors)	KNN-byhand.py
Logistic Regression	log-regbyhand.py
SVM (Support Vector Machine)	SVM-byhand.py
Decision Trees	DecisionTrees-byhand.py
Random Forest	Randomforest-byhand.py
Gradient Boosting Machines	GradientBoostingMachines-byhand.py

Unsupervised

Model	Code
K-Means Clustering	Kmeans-byhand.py
Hierarchical Clustering	Hierarchical-byhand.py
DBSCAN	DBSCAN-byhand.py
PCA (Principal Component Analysis)	PCA-byhand.py

Deep Learning

Model	Code
Perceptron	Perceptron-byhand.py
MLP (XOR, SGD)	MLP-byhand.py
MLP with Momentum	MLP-momentum.py
MLP with Adam	MLP-adam.py
Softmax MLP (multi-class)	softmax-mlp.py
Moons MLP	projects/moons-mlp.py

Projects

Project	Dataset	Model	Code
Titanic Survival Prediction	Kaggle Titanic	Logistic Regression (from scratch)	log-reg-titanic.ipynb
Spam Detection	SMS Spam Collection	SVM (from scratch)	svm-spamdetection.ipynb
Healthcare Resource Predictor	WHO / World Bank	Decision Tree, Random Forest, Gradient Boosting	GitHub repo
Fashion-MNIST Classifier	Fashion-MNIST (HuggingFace)	MLP from scratch with Adam (NumPy only) — 87.62% test accuracy	GitHub repo

Visualisations

Regression

Model	Output
Simple Linear Regression
Multiple Linear Regression
Lasso Regression
Ridge & Lasso
ElasticNet

Classification

Model	Output
KNN