Skip to content
View Atharvax16's full-sized avatar
🎯
Focusing
🎯
Focusing

Block or report Atharvax16

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Atharvax16/README.md

Hi, I'm Atharva Kocharekar πŸ‘‹

Typing SVG


πŸ”¬ Abstract

I'm an MSc Artificial Intelligence student at Dublin City University, supervised by Prof. Alessandra Mileo, working on robust and explainable machine learning for high-stakes vision.

The thread running through everything I do is one question:

Does a model actually work β€” or does it just look like it does?

My dissertation surfaces a diagnostic paradox: generative restorers can add pixel fidelity (Cold Diffusion, +12 dB PSNR) while failing to recover β€” and often harming β€” downstream diagnostic accuracy, because the "restored" images push the classifier off-distribution. Explanations also drift faster than accuracy as quality drops, meaning a model can be right for the wrong reasons. Currently under review at OMIA / MICCAI 2026.

That reliability lens shapes how I work, not just what I work on. I read a paper, then rebuild its mechanism small enough to watch it run β€” and I report the reproductions that failed as prominently as the ones that worked. Two of the results below are negative. They're the ones I'd want you to read.


🧡 Research threads

  • Generative restoration for DR screening β€” benchmarking CLAHE, A-ESRGAN, SwinIR+GAN, Cold Diffusion and a conditional DDPM as restorers against CNNs and a ViT-Base classifier under nine synthetic degradations (blur / exposure / noise Γ— 3 severities) on APTOS 2019. ViTs proved substantially more robust under severe noise (~62% vs ~20% accuracy) β€” a gap invisible on clean benchmarks.
  • Explainability under distribution shift β€” quantifying how Grad-CAM, SHAP and attention rollout drift as quality degrades, scored by insertion/deletion AUC for faithfulness and Spearman stability rather than eyeballed heatmaps β€” and finding where restoration silently breaks interpretability.
  • Generative-image forensics β€” separating authentic photographs from diffusion/GAN output via frequency artifacts and intermediate transformer features, extending toward reference-free, generator-agnostic hallucination detection for medical images.
  • Episodic memory for vision-language agents β€” the current track, below. πŸ”΄ active

🧭 Current track β€” VoxSight Recall

Episodic memory for vision-language agents. (active since July 2026 Β· reading β†’ reproducing β†’ building)

VoxSight can describe what the camera sees right now. It cannot answer "where did I leave my keys?" β€” because nothing it saw an hour ago survives the turn it was seen in. An agent that perceives continuously but remembers nothing has no episodic memory, only perception.

The bet: this is a memory-architecture problem before it is a model problem. Rather than fine-tune a bigger VLM and hope recall emerges from scale, I'm following the differentiable-memory literature from its origin and rebuilding each mechanism by hand.

One idea connects the whole reading list: you cannot hard-pick one memory slot, because a hard pick has no gradient β€” so you softly blend several. NTM blends notebook rows; RAG blends Wikipedia passages. Same trick, different scale. MemGPT then breaks the pattern in the useful direction β€” it gives up the gradient entirely and lets the model hard-pick at runtime by calling a function β€” making the real premise explicit: the bottleneck was never capacity, it was allocation.

Paper Status
βœ… Neural Turing Machines β€” Graves et al., 2014 read Β· notes written
βœ… RAG for Knowledge-Intensive NLP β€” Lewis et al., 2020 reproduced end to end ↓
βœ… MemGPT: LLMs as Operating Systems β€” Packer et al., 2023 read Β· notes written
πŸ”¨ Episodic-memory layer for VoxSight design β€” once the reading lands

The gap I'm aiming at: MemGPT never asks whether a memory can be trusted. Every self-edit lands with no source, no timestamp, no confidence, and no record of what it overwrote β€” and recursive summarisation compounds the error with no path back to the original observation. For a user who can't see, a confidently wrong memory is worse than an absent one. Provenance over episodic memory is the thread, and it's orthogonal to the paging mechanism.


πŸ§ͺ Reproductions β€” including the ones that didn't work

Rebuilt from the paper on Apple Silicon (MPS), not cloned from a reference repo. Every number here is measured in my own notebook.

RAG β€” Lewis et al. 2020, rebuilt end to end

15,077-passage index Β· 96 SQuAD-dev questions asked open-domain Β· exact MIPS Β· both equations written from scratch in log-space.

Claim My measurement
Eq (1) & (2) from scratch match transformers |Ξ”| = 0.0e+00 βœ…
Retrieval beats closed-book (same generator weights) 1.0 Β± 1.0 β†’ 24.0 Β± 4.4 EM βœ…
Retrieval beats random passages 2.1 Β± 1.5 β†’ 24.0 Β± 4.4 EM βœ…
Answer-only loss moves the retriever (no passage labels) gold-passage trust 0.097 β†’ 0.354 βœ…
Index hot-swap changes world knowledge, zero retraining 2016 β†’ 2020, weights untouched βœ…
answer-recall@k rises monotonically with k 26 β†’ 36 β†’ 50 β†’ 61 β†’ 71 βœ…
Retriever has not collapsed (Appendix H) 427 distinct docs / 480 slots βœ…
Learned DPR beats BM25 (paper: 43.5 vs 29.7) 24.0 ± 4.4 vs 44.8 ± 5.1 EM ❌

BM25 winning is the interesting result. A corpus-size artifact, not a refutation: DPR's query tower was fine-tuned on NaturalQuestions, my questions are SQuAD, and lexical overlap is unusually strong when the haystack is 15k passages instead of 21M. Reporting it beats quietly dropping the row.

SteerViT β€” Ruthardt et al. 2026, small-scale

Frozen DINOv2 + gated cross-attention Β· RefCOCOg / COCO Β· 4,500 images Β· 3,000 steps (the paper trains 20–50k).

Condition patch-grid IoU PR-AUC
baseline (frozen, no text) 0.1295 0.2208
steerability (correct prompt) 0.2940 0.5113
wrong prompt (mismatched) 0.2323 0.3926

Steering emerges β€” but the sanity check is the point. Feed a prompt describing a different image and localisation must collapse toward baseline. It didn't: collapse ratio 0.625 against a 0.30 pass bar, so 62.5% of the gain survived a mismatched prompt. At this scale the model is reading the image, not the text. FAIL β€” and it's the one number worth reporting. The gate (tanh(Ξ±Β·Ο‰), Ξ± initialised at 0) was verified to reproduce frozen DINOv2 exactly at Ο‰=0.


πŸ—οΈ The Architecture Lab

Every architecture I study gets rebuilt as a hand-drawn, steppable sketch you can walk through and turn the knobs on β€” patch size, kernel size, the prompt you steer with, how many documents you retrieve. Enter the Lab β†’

Built What you can step through
Vision Transformer image β†’ patchify β†’ flatten β†’ project β†’ +CLS/pos
CNN convolve Β· pool Β· receptive field Β· the transposed-conv step that becomes an AI-image detector
DINOv2 student–teacher β†’ EMA β†’ frozen embedding β†’ XGBoost / MLP head
SteerViT frozen ViT β†’ gated cross-attention β†’ prompt-steered features β†’ the sanity check that failed
RAG index β†’ MIPS β†’ concatenate β†’ marginalise β†’ the gradient that reaches the retriever
AI-image forensics six lenses on a fake: FFT Β· ELA Β· up-convolution Β· CLIP Β· DINOv2 Β· learned artifacts

On the bench: JEPA Β· Embedding space Β· VLM Β· NTM Β· MemGPT Β· Encoder–Decoder Β· Diffusion


πŸ“„ Publications & preprints

  • Towards a Robust and Explainable Pipeline for Diabetic Retinopathy Classification through Quality-Aware GenAI Image Restoration β€” under review, OMIA / MICCAI 2026
  • Autonomous Drone Navigation β€” pitched at the International Conference on Engineering Research and Innovations, 2025

🌟 Selected work

  • Robust & Explainable AI for Diabetic Retinopathy Β Β·Β  MSc thesis A five-phase pipeline: 26,370 synthetic degraded fundus images β†’ three DR graders (ResNet-50 / EfficientNet / ViT, ordinal focal loss, quadratic-weighted kappa) β†’ a quantitative XAI benchmark (IG, SHAP, attention-rollout, Grad-CAM) β†’ seven restorers (Cold Diffusion, conditional DDPM, SwinIR+GAN, …) β†’ a quality-aware trust router that decides per image whether to enhance, which grader to trust, and which explanation to return. The central contribution is a pathology-preserving DDPM β€” a restorer constrained not to hallucinate lesions it cannot justify.

  • VoxSight β€” Real-Time Accessibility Co-Pilot Β Β·Β  Cursor Hackathon β†’ product A conversational agent that "sees" for you: point a camera, hold a button, ask by voice, get a spoken answer. Audio + frame β†’ STT β†’ vision LLM β†’ TTS in a single WebSocket round-trip. Built end-to-end in one night in mock mode with zero API keys β€” every service behind a swappable mock/live flag β€” then wired to real models the next morning without touching orchestration. Since grown intent routing, persistent memory, and deterministic turn-by-turn walking navigation (a routing engine, so it costs no model calls per step). The memory gap in this system is what started VoxSight Recall above.

  • AI-Generated Image Detection β€” Etsy Research Challenge Β Β·Β  πŸ₯ˆ 2nd place Authentic photographs vs diffusion/GAN images on a stratified ~4,340-image marketplace dataset. Hand-crafted forensics (FFT F1 0.60, ELA F1 0.65) against learned representations β€” and intermediate CLIP layers beat the final embedding, 0.92 vs 0.85 F1. A 6,018-dim fusion with 5-fold CV (0.87 Β± 0.01), TTA and pseudo-labeling, then an Optuna-weighted 4-model ensemble reaching β‰ˆ0.94 F1. Placed 2nd on Etsy's research leaderboard and was invited to present at Etsy HQ, Dublin. (Repository private.)

  • HealEdge β€” Federated Learning for DR Detection with XAI Β Β·Β  πŸ₯‰ 3rd of 70 AdvanceHealth MedTech Hackathon, Trinity College Dublin. FedAvg-trained ResNet-50 across non-IID hospital data so raw images never leave the client, with Grad-CAM heatmaps so clinicians can verify each prediction β€” addressing both the privacy and the trust barrier at once.

  • Cyberattack Detection from CPU Usage Logs Β Β·Β  πŸ… AWS hackathon winner Isolation Forest vs Random Forest on CPU telemetry β€” 98% accuracy, 78% recall, recall deliberately optimised over precision, because a few false alarms beat missing the spike that matters. Deployed on SageMaker with Lambda inference and SNS real-time alerts.

  • 3D Spatial Reasoning β€” Nomad AI Neuro-symbolic system for collision-free, explainable object placement. The LLM parses intent into constraints; a deterministic geometry engine (AABB checks, clearance/walkway rules) computes valid candidates β€” avoiding LLM "coordinate hallucination."

More projects β€” agents, multimodal & analytics
  • CATAS β€” Compliance & Treasury Agents (Lyzr Hackathon) A dual-agent system automating treasury reconciliation and compliance, grounding LLM reasoning in deterministic ML β€” Isolation Forest, Prophet and logistic regression underneath, with an immutable audit log on every action.

  • Research Growth Companion A six-agent assistant (orchestrator + 5 specialists) keeping ML researchers current on papers, conferences, benchmarks and careers β€” sourcing from arXiv, Semantic Scholar and top venues.

  • VenueFlow β€” AI Wedding Coordinator (Build with Gemini XPRIZE) Four agents β€” intake Β· venue Β· supplier Β· quote β€” sourcing vendors, drafting timelines and tracking budgets, with an AgentDecisionLog on every action so the whole run is auditable.

  • Parkinson's β€” Gait, Voice & Tapping Multimodal screening fusing three signal types: gait, voice, and finger-tapping.

  • Credit-Card Fraud Analysis Recall-first detection over 1.29M+ highly imbalanced transactions β€” target encoding, infrequent-class weighting, and F1 over accuracy as the evaluation metric by design.


🧱 Foundations β€” built from scratch

Because you don't really know a mechanism until you've written its backward pass.


πŸ“– til β€” the paper journal

Papers read in my own words, with notebooks where the idea needs to be felt rather than read. Browse the journal β†’

Currently running threads on image forensics (Durall's up-convolution spectral tell, Frank's DCT counterpart, Gragnaniello's FFT baseline, and the argument that the detection arms race is unwinnable by design) and memory architectures (NTM β†’ RAG β†’ MemGPT), alongside Attention Is All You Need, Generative Adversarial Nets, and the MPNN β†’ SchNet β†’ DimeNet β†’ NequIP arc.


🧰 Methods & tools

Modeling & Deep Learning PyTorch Β· timm Β· TensorFlow / Keras Β· scikit-learn Β· HF Transformers
Generative & Diffusion DDPM Β· Cold Diffusion Β· GANs Β· SwinIR
Explainability (XAI) Integrated Gradients Β· SHAP Β· Grad-CAM Β· Attention Rollout
Representation & Forensics CLIP Β· DINOv2 Β· FFT / ELA Β· XGBoost
Memory & Retrieval DPR / dense retrieval Β· MIPS Β· BM25 Β· vector DBs Β· agent design
Foundations (from scratch) NumPy Β· BPTT / RNNs Β· Neural LMs Β· Optimization
Infra & MLOps AWS SageMaker Β· Lambda / SNS Β· Docker Β· FastAPI Β· WebSockets


πŸ“Š GitHub at a glance


πŸ† Highlights

  • πŸ₯ˆ 2nd place, Etsy Research Leaderboard β€” invited to present AI-image detection research at Etsy HQ, Dublin
  • πŸ₯‰ 3rd of 70 teams β€” Advanced HealthTech & MedTech Innovation Hackathon, Trinity College Dublin (2026)
  • πŸ… Winner β€” Motivalogic Hackathon, sponsored by AWS, Dublin (2025)
  • 🌍 Ambassador, Thinking About Thinking β€” responsible-AI community (AE Global Summit, London 2026)

πŸ“Œ What I'm focusing on right now
  • Resubmitting the OMIA/MICCAI paper with revised result tables and consistency fixes
  • VoxSight Recall β€” designing the episodic-memory layer, with provenance as a first-class property rather than an afterthought
  • Building a quality-aware triage pipeline (selective prediction + calibrated trust scores) as a practical fix for the diagnostic paradox
  • Extending AI-image detection toward reference-free, generator-agnostic hallucination detection in the medical domain
  • Working through the Architecture Lab backlog β€” NTM and MemGPT are next on the bench
πŸ—ΊοΈ How I got here
Year
2021 The Spark β€” TEC Mumbai B.E. in AI & Data Science. Fell for the question behind the model: why does this work?
2023 First Impact β€” Data Science Intern Production ML pipelines. Learned that clean data beats clever models.
2024 Going Global β€” Dublin MSc in Artificial Intelligence at DCU; began the thesis on robust, explainable DR screening.
2025 Research Mode Diffusion restoration + a quantitative XAI benchmark; a separate study on detecting AI-generated images. Reading and reproducing foundational papers in parallel.
2026 Etsy Recognition The detection research placed 2nd on Etsy's leaderboard β€” presented in person at Etsy HQ, Dublin.
2026 What's next ML / applied-research roles and collaborations β€” robustness, explainability, memory architectures.
🀝 For collaborators / researchers

I'm especially keen to connect with people working on medical imaging, generative models, memory architectures, model trustworthiness, or interpretability. Happy to talk about diagnostic reliability, synthetic data, episodic memory for agents, or where generative AI quietly breaks the things it claims to fix.

I care about correctness, clarity, and shipping β€” and about reporting the result that didn't go my way. Open to research and collaboration opportunities.


πŸ“« Let's connect

Pinned Loading

  1. 3D-Spatial-Reasoning-Nomad-AI 3D-Spatial-Reasoning-Nomad-AI Public

    Jupyter Notebook 1