PRODUCT CLASSIFICATION CHALLENGE

Goal: Train a neural network to classify images of products belonging to 9 different classes, and explore different strategies to improve accuracy.

---

Requirements

torch>=1.7
torchvision
pyyaml
huggingface_hub>=0.17.0
safetensors>=0.2
numpy
scikit-learn
pandas
matplotlib
jupyterlab

We use HuggingFace's (and, more specifically Ross Wightman) pytorch-image-models for quickly train classification model models.

git clone https://github.com/huggingface/pytorch-image-models.git

Prepare and visualize datadata

From the data available from the challenge, we produce the list of images (in .txt) and later convert it to .csv file by running

$ cd ${ROOT}/data/train
$ ls -1 -R --color=none > listdir_train.txt
$ cd ${ROOT}/data/validation
$ ls -1 -R --color=none > listdir_validation.txt
$ cd ${ROOT}
$ python src/convert_database.py

For dataset stats and visualization, please run the notebooks data_stats.ipynb and batch_visualization.ipynb.

Train classification models

A simple template script for training a classification model with timm can be found at src/run_training_cmd.sh

sh src/run_training_cmd.sh convnext_atto "--gaussian-blur-prob 0.3" $OUTPUT_PATH

Trained models

Some models finetuned on the challende data can be found at /models, each with a different architecture.

Evaluating models

For a complete evaluation of the available models, please run

MODEL=convnext_atto
python src/eval_model.py --model_type=$MODEL --image_size=32 --model_path models/${MODEL}-32/best.pth.tar--num_classes 10 --root_dir data/validation

Other useful flags to this script include: --save_confusion_matrix (store_true): save the confusion matrix of the classified images and labels. --extract-features (store_true): extract and save image features (last layer before final FC layer), to be used for feature space visualization and label diffusion --extract_pseudo_labels (int): used together with the prior flag, to also save the indicator of which images are the top-k images more confidently classified per class and which are the most uncertain.

Best model: ConvNeXt-atto training on 32x32 resolution images (79.14%)

Overall Performance: Top-1 Accuracy: 79.14% Correct Predictions: 129/163

Performance Metrics: Total Evaluation Time: 0.20s Average Batch Time: 0.0337s Throughput: 806.95 images/second

Per-Class Accuracy: Class 0 (amaciador_confort): 100.00% Class 1 (bolachasRicanela_Quetara): 96.67% Class 3 (cereaisAllbran_kellog): 83.33% Class 4 (detergente_Woolite): 78.57% Class 5 (lixivia_domestos): 59.09% Class 6 (NestumMel): 100.00% Class 7 (papelHigienico_Scottex): 85.71% Class 8 (tostas_tostagrill): 95.45% Class 9 (pensoHigienico_Evax): 0.00%

======================================================================

====================================================================== DETAILED CLASSIFICATION REPORT

          precision    recall  f1-score   support

       0       0.56      1.00      0.71        15
       1       0.88      0.97      0.92        30
       3       0.77      0.83      0.80        12
       4       1.00      0.79      0.88        14
       5       0.93      0.59      0.72        22
       6       1.00      1.00      1.00        24
       7       0.35      0.86      0.50         7
       8       0.88      0.95      0.91        22
       9       0.00      0.00      0.00        17

accuracy                           0.79       163

macro avg 0.71 0.78 0.72 163 weighted avg 0.76 0.79 0.76 163

Plot t-SNE of extracted features

python simple_tsne.py

for a quick plot of the t-SNE visualization of the feature space of our best model. For visualizing other models, please change the config in the script.

Run and evaluate label diffusion

From extracted features and estimated top-k pseudo-labels from classifier, we can run label graph diffusion to address classifying images and a class not seen during training:

K=4; ALPHA=0.35; SIGMA=1; N_ITER=400; DIST=eucledean
  python src/eval_diffusion.py --alpha $ALPHA --sigma $SIGMA --n_iter $N_ITER --distance $DIST --features_path features/convnext_atto_res32_FT_pseudolabels${K}.npz

Best result: Eucledean pairwise distance (83,44%)

Per-Class Accuracy: Class 0 (amaciador_confort): 86.67% Class 1 (bolachasRicanela_Quetara): 96.67% Class 3 (cereaisAllbran_kellog): 83.33% Class 4 (detergente_Woolite): 92.86% Class 5 (lixivia_domestos): 72.73% Class 6 (NestumMel): 100.00% Class 7 (papelHigienico_Scottex): 85.71% Class 8 (tostas_tostagrill): 90.91% Class 9 (pensoHigienico_Evax): 29.41%

DETAILED CLASSIFICATION REPORT precision recall f1-score support

       0       0.87      0.87      0.87        15
       1       0.94      0.97      0.95        30
       3       0.71      0.83      0.77        12
       4       1.00      0.93      0.96        14
       5       0.76      0.73      0.74        22
       6       1.00      1.00      1.00        24
       7       0.35      0.86      0.50         7
       8       1.00      0.91      0.95        22
       9       0.62      0.29      0.40        17

accuracy                           0.83       163

macro avg 0.81 0.82 0.79 163 weighted avg 0.86 0.83 0.83 163