Combining an LLM's language understanding with a symbolic solver's guaranteed-correct logic — so a scheduling assistant can be fluent and verifiably right.
Large language models are excellent at understanding messy, ambiguous human language. They are not reliable at guaranteed-correct logical reasoning — ask an LLM to schedule five people's meetings under real constraints, and it may produce a fluent, confident-sounding answer that quietly double-books someone or invents a person who was never mentioned.
Symbolic solvers are the opposite: give a constraint solver a precise logical problem, and it will find a provably correct answer, or prove that none exists. But it can't read a sentence.
This project explores what happens when you make each half do only what it's actually good at — an LLM to understand the request, a symbolic solver to guarantee the answer is correct — and empirically measures whether that combination actually outperforms just asking an LLM to do the whole job.
This is directly the kind of neural + symbolic integration that programs like the Erasmus Mundus AI (EMAI) curriculum are built around, and this repository documents both the system and the evaluation honestly, including where it didn't provide a clean advantage.
flowchart TD
A["Natural language request<br/><i>e.g. 'schedule 5 meetings...'</i>"] --> B["LLM Agent (LangGraph)<br/>Parses into structured constraints"]
B --> C["Symbolic Solver<br/>python-constraint CSP"]
C --> D{"Checks validity"}
D -- "invalid: retry with error" --> B
D -- valid --> E["Verified schedule<br/>returned in natural language"]
style A fill:#e8e8e8,stroke:#888,color:#000
style E fill:#e8e8e8,stroke:#888,color:#000
style B fill:#d8c7f0,stroke:#7a5aa8,color:#000
style C fill:#b9e3dc,stroke:#3f9b8a,color:#000
style D fill:#b9e3dc,stroke:#3f9b8a,color:#000
The retry loop is the core contribution. If the LLM's parse is malformed, references an unknown person, or the solver proves no valid schedule exists, the specific error is fed back to the LLM for another attempt — capped at 3 tries. This turns a one-shot "hope it's right" system into one that self-corrects and fails loudly and specifically rather than silently.
| Component | Role | Tech |
|---|---|---|
Parser (agent/parser.py) |
Extracts meetings (attendees, duration, preferences) from natural language. Refuses to invent people not in the known list. | Groq API, openai/gpt-oss-120b |
Solver (solver/solve.py) |
Assigns every meeting a time slot with zero conflicts, or proves no assignment exists. | python-constraint (CSP backtracking search) |
Agent graph (agent/graph.py) |
Wires parser and solver together with the retry-on-failure loop. | LangGraph |
Baseline (baseline/baseline.py) |
A pure-LLM version with no solver, used purely for comparison. | Same Groq model |
It's easy to claim a neuro-symbolic system is more reliable. This repo tries to actually measure it. eval/run_comparison.py runs both systems on the same 10 hand-designed requests spanning five categories — easy, shared-attendee, unknown-person, provably-impossible, and ambiguous — and an independent checker (eval/checker.py, deliberately written separately from the solver) grades every output for double-booking, invented people, and unresolved slots.
| Category | Result |
|---|---|
| Easy / shared-attendee (4 cases) | Both systems succeeded equally — no real gap here |
| Unknown-person / impossible (4 cases) | Both systems correctly declined, but the neuro-symbolic system gave a specific, verifiable reason each time; the baseline often just returned nothing |
| Ambiguous ("someone from the design team") | The baseline silently guessed a real person and a real time slot — passing every structural check despite the request never naming anyone. The neuro-symbolic system refused, because its parser is constrained to only use explicitly named people. |
That last row is the most important result in this repo. It shows that a schedule can look completely valid — real people, real slots, no conflicts — while still being wrong, because it was built on a fabricated premise no validity checker can catch after the fact. Preventing that requires stopping the ungrounded guess before it happens, not verifying it afterward.
This is reported as a genuine, if narrow, finding — not a clean win. On more clear-cut requests, both systems performed identically, and the honest limitation is discussed in Limitations below.
A Streamlit demo lets you type your own people, time slots, and request, and see both systems run side by side on the same input.
flowchart LR
U["You type:<br/>people, slots, request"] --> S1["Baseline: pure LLM"]
U --> S2["Neuro-symbolic system"]
S1 --> R["Side-by-side comparison"]
S2 --> R
style U fill:#e8e8e8,stroke:#888,color:#000
style R fill:#e8e8e8,stroke:#888,color:#000
style S1 fill:#f0c7c7,stroke:#a85a5a,color:#000
style S2 fill:#b9e3dc,stroke:#3f9b8a,color:#000
1. Clone and enter the repo
git clone https://github.com/JAI0705/neuro-symbolic-scheduler.git
cd neuro-symbolic-scheduler2. Create a virtual environment
python3 -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate3. Install dependencies
pip install langgraph groq python-dotenv python-constraint streamlit4. Add your Groq API key
Create a .env file in the project root:
GROQ_API_KEY=your_key_here
Get a free key at console.groq.com.
Run the interactive demo:
streamlit run app.pyRun the full baseline-vs-neuro-symbolic evaluation:
python3 eval/run_comparison.pyThis prints a live comparison for all 10 test cases and saves full results to eval/results.json.
Run just the symbolic solver in isolation (no LLM, no API calls):
python3 eval/test_manual.pyRun the agent end to end on a single request:
python3 agent/graph.pyneuro-symbolic-scheduler/
├── solver/
│ ├── schema.py # Shared data contract: Person, Meeting, TimeSlot, Schedule
│ └── solve.py # The symbolic CSP solver
├── agent/
│ ├── parser.py # LLM: natural language -> structured Meeting objects
│ └── graph.py # LangGraph retry loop wiring parser + solver together
├── baseline/
│ └── baseline.py # Pure-LLM baseline, no solver, for comparison
├── eval/
│ ├── test_set.py # 10 test cases across 5 categories
│ ├── checker.py # Independent grader for baseline output
│ ├── run_comparison.py # Runs both systems, prints + saves results
│ └── test_manual.py # Solver-only sanity tests
├── app.py # Streamlit side-by-side demo
├── .env.example
└── README.md
Being direct about what this project does not show:
- On clear-cut requests, there was no measurable advantage over a pure LLM. The gap only appears on genuinely ambiguous or adversarial input — which is itself a useful finding about where verification actually matters, but it means the headline isn't "solvers always win."
- The system's refusal on ambiguous input is a prompting choice, not deep reasoning. The parser refuses ungrounded references because its system prompt strictly requires explicitly named people — a more permissive prompt would make the same guess the baseline did. It demonstrates the value of enforcing grounding, not a fundamentally different reasoning capability.
- The test set is intentionally small (10 cases) and scheduling conflicts are fairly obvious (few people, few slots). A stronger version of this evaluation would scale up the number of people/meetings/slots so conflicts require real combinatorial reasoning rather than being visible at a glance — this is a natural next step.
- Availability constraints (who's free when) are handled entirely by the solver, not the LLM, by design — but this means the system's "intelligence" about scheduling logic is really the CSP solver's, not the LLM's. The LLM's contribution is narrower: turning language into structured requests.
This project isn't trying to prove neuro-symbolic AI is strictly superior — it's trying to characterize where combining neural and symbolic components actually changes outcomes, and where it doesn't. That distinction, and the evidence behind it, is the actual point.
MIT