NoteTaker-py

Turn any YouTube video or audio file into hierarchical study notes — the kind of indented, topic-grouped outline you'd actually take in a lecture, not a flat wall of bullet points.

YouTube / audio file
        │
        ▼
   yt-dlp (download best audio)
        │
        ▼
   faster-whisper (timestamped transcript)
        │
        ▼
   merge into ~60s timestamped chunks
        │
        ▼
   per-chunk hybrid summary
   (sentence-transformer salience filter
    → distilBART abstractive rewrite)
        │
        ▼
   cluster chunks → topic sections
   (KMeans + silhouette-chosen k)
        │
        ▼
   per-section title via headline-style generation
        │
        ▼
   Markdown notes  +  transcript.json  +  notes.json

Why this exists

Most auto-notetakers spit out a flat list of "important sentences." Real academic notes — especially the hierarchical style taught in high school — have topics, sub-topics, and supporting detail. NoteTaker-py reconstructs that structure from a spoken source:

• Topic sections come from clustering chunk embeddings along the timeline.
• Chunk summaries use embeddings to pick the most salient sentences, then an abstractive model (Google T5) rewrites them into clean prose instead of copying verbatim transcript fragments.
• Pass --no-abstractive to skip the transformer and fall back to a pure extractive pipeline (much faster, lower quality).

Tech stack

Stage	Library
Audio download	yt-dlp
Transcription	faster-whisper
Salience scoring	sentence-transformers
Abstractive rewrite & titles	transformers (distilbart-cnn-12-6 by default)
Clustering	scikit-learn (KMeans + silhouette)
CLI / UX	typer + rich
Data models	pydantic v2 + @dataclass

Install

pip install -r requirements.txt

faster-whisper and yt-dlp both need ffmpeg on your PATH. On Windows: winget install Gyan.FFmpeg.

Usage

# YouTube URL
python -m notetaker "https://www.youtube.com/watch?v=VIDEO_ID"

# Local audio/video file
python -m notetaker ./lecture.mp3

# Tune the pipeline
python -m notetaker URL \
    --model small.en \
    --chunk-seconds 45 \
    --bullets 4 \
    --output ./out

The legacy python transcribe.py ... entry point still works.

CLI options

Flag	Default	Purpose
--output / -o	out	Output directory
--model / -m	base.en	faster-whisper model size
--embedder / -e	all-MiniLM-L6-v2	sentence-transformers model
--chunk-seconds	60	Target chunk length before summarization
--bullets / -b	3	Bullets per chunk
--language / -l	auto	Force a language code (e.g. en)
--device	auto	auto / cpu / cuda
--compute-type	auto	faster-whisper compute precision
--summarizer-model	sshleifer/distilbart-cnn-12-6	HF model for rewrites + titles
--no-abstractive	off	Skip the transformer; pure extractive output
--prefer-youtube-transcript/--no-prefer-youtube-transcript	on	Use YouTube captions/subtitles

Output

For an input titled My Lecture, the pipeline writes:

out/
├── audio/<video_id>.mp3
├── my-lecture.md              # hierarchical notes
├── my-lecture.transcript.json # raw timestamped segments
└── my-lecture.notes.json      # full structured notes (pydantic dump)

Package layout

notetaker/
├── __init__.py
├── __main__.py     # python -m notetaker
├── cli.py          # typer app + pipeline orchestration
├── models.py       # TranscriptChunk (dataclass), pydantic models
├── download.py     # yt-dlp wrapper
├── transcribe.py   # faster-whisper wrapper
├── chunk.py        # merge raw segments → ~60s chunks
├── summarize.py   # salience filter + abstractive rewrite per chunk
├── hierarchy.py    # KMeans + silhouette → topic sections (with generated titles)
├── nlp.py          # lazy-loaded HF pipelines (summary + headline)
└── export.py       # Markdown + JSON writers

Each stage is independently importable, so you can swap in a different summarizer (e.g. a transformers-based abstractive model) without touching the rest of the pipeline.

Known Issues, Roadmap

Currently, the transformers package contains an issue in version 5 regarding the registration of summarization and text2text-generations tasks. Because of this, requirements.txt pins version 4, although the codebase was updated to be less dependent on that.

Future releases of this project will include transcript preprocessing to join segments and normalize them, further reduction in conversational language, and support for YouTube captioning to skip transcription stage.