Educhain TutorBench Evaluator

Educhain TutorBench is a benchmark for evaluating the performance of large language models (LLMs) in the education domain. The benchmark consists of a set of tasks that assess the ability of LLMs to understand and generate text in various educational contexts.

For Detailed Results Check the Results Directory

Models Evaluated

• 'openai/gpt-5'
• 'anthropic/claude-sonnet-4.5'
• 'google/gemini-2.5-pro'
• 'x-ai/grok-4-fast'
• 'deepseek/deepseek-v3.2-exp'
• 'z-ai/glm-4.5'
• 'openai/gpt-4.1'
• 'moonshotai/kimi-k2-0905'
• 'qwen/qwen3-235b-a22b'

Default Judge : x-ai/grok-4-fast

You can change the judge model in the llm_evaluator.py script . Just change the model name in the .env file (make sure to use official openrouter model id as model ) You can also edit the judge prompt in the judge_prompt.py script in the prompts directory.

Getting Started

git clone https://github.com/MissLostCodes/Ed_Metric.git

Installations

cd into the directory and run

pip install -r requirements.txt
# or if you have uv installed
uv add -r requirements.txt

Run the UI

streamlit run app.py

To evaluate models follow the steps below:

• Step 1 : Pick any prompt / question / educational task you want to evaluate the LLM on, and pass it int the ui , you can also add expected output .
• Step 2 : Run Evaluation and wait
• Step 3 : View results and use the view rankings button and see chart a overall ranking of models over multiple prompts which you passed till nbow . All these scores are stored in evaluations.csv file .

Metrics being tested :

• Confusion Recognition: Identifies the student's specific confusion point and current understanding level.
• Adaptive Response: Tailors the response to the student's level, emotional state, and specific need.
• Learning Facilitation: Ensures the student understands and can apply the concept.
• Strategic Decision-Making: Chooses the best approach (direct answer, guided discovery, etc.) for the student.
• Engagement & Emotional Intelligence: Addresses the student's emotional state and maintains motivation.
• Knowledge Pillar: Demonstrates accurate subject knowledge and correctness in solving the student's question.
• Error Analysis: Diagnoses the student's specific error or misconception and offers actionable feedback.
• Adaptive Capability: Adjusts the content complexity to match the student's needs.
• Curriculum Awareness: Aligns with age/developmental appropriateness and curriculum.
• Explanation Ability: Provides clear and pedagogically sound explanations.