Skip to content

data_analysis/ml: refresh classifier results on Phase 1 (11 subjects)#6

Open
AlanYWu wants to merge 1 commit into
mainfrom
refresh-ml-results
Open

data_analysis/ml: refresh classifier results on Phase 1 (11 subjects)#6
AlanYWu wants to merge 1 commit into
mainfrom
refresh-ml-results

Conversation

@AlanYWu

@AlanYWu AlanYWu commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Re-trains v1 (band power), v2 (time-freq), v3 (EEGNet) MI pipelines and the blink detector across the current Phase 1 subject pool (sub02–sub12, 11 subjects) and regenerates data_analysis/ml/results.md.

The previous results.md described only sub02–sub05 (4 subjects). This PR brings results in line with the data we now have on Drive.

What changed

  • Sample size: 4 → 11 subjects, 77 → 171 MI epochs, 190 → 519 blink epochs.
  • MI still hovers near chance — best within-subject is now LDA / temporal_only at 56.7%; best LOSO is RF on v3 EEGNet / all_4ch at 59.4%. The wider pool brings the previous best (61.0% within-subject) down — likely the small-sample number was lucky.
  • Blink stays strong: 91.3% within-subject (LDA all_4ch) and 79.8% LOSO LDA — the threshold baseline now narrowly beats LDA on LOSO (80.7%).
  • Channel quality table now covers all 11 subjects; auto-channel selector still picks TP9, TP10.

Headline (best-of)

Task Best within-subject Best LOSO
MI v1 LDA / temporal_only — 56.7% SVM-rbf / temporal_only — 57.3%
MI v2 LDA / temporal_only — 55.7% RF / all_4ch — 56.3%
MI v3 LDA / temporal_only — 54.0% RF / all_4ch — 59.4%
Blink LDA / all_4ch — 91.3% LDA / all_4ch — 79.8% (Threshold 80.7%)

Files

  • `data_analysis/ml/refresh_results.py` — regeneration script (v1/v2/v3 + blink, 5-fold within-subject CV + LOSO, renders the markdown)
  • `data_analysis/ml/results.md` — refreshed tables + per-subject breakdowns
  • `data_analysis/ml/results_data.json` — raw means + per-subject scores

Test plan

  • `python data_analysis/ml/refresh_results.py` reproduces both files.
  • Reviewer sanity-checks the per-subject MI table — sub04 still tends to be the weakest, sub09–sub11 are noisy due to low MI epoch counts.

Re-runs v1 (band power), v2 (time-freq), v3 (EEGNet) MI pipelines and
the blink detector on the current Phase 1 pool (sub02–sub12) and
regenerates results.md. Previous file described only sub02–sub05.

Sample size: 4 → 11 subjects, 77 → 171 MI epochs, 190 → 519 blink epochs.

Headline (best-of, % accuracy):
  MI within-subject: LDA / temporal_only — 56.7 (was 61.0 on 4 subjects)
  MI LOSO:           RF / all_4ch (v3)   — 59.4 (was 62.2)
  Blink within-subj: LDA / all_4ch       — 91.3 (was 88.7)
  Blink LOSO:        LDA / all_4ch       — 79.8 (was 86.0)

Adds:
  data_analysis/ml/refresh_results.py — regeneration script
  data_analysis/ml/results_data.json  — raw means + per-subject scores
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant