Rework evidence agent into structured assessor#1371
Draft
SachaProbo wants to merge 1 commit into
Draft
Conversation
823405a to
4fef960
Compare
Replace the free-text evidence describer with an assessor that emits a structured assessment (status plus rationale) so downstream code can act on the verdict instead of parsing prose. Rename the evidencedescriber package and worker to evidenceassessor and introduce a dedicated EvidenceAssessment coredata model backed by an assessment status enum. Split the schema changes into focused migrations and scope the FAILED transition to the status columns so unrelated evidence fields are left untouched. Realign the surrounding configuration and wiring: rename the config to EvidenceAssessmentConfig, update probod aliases and builders, add a Thinking budget to the LLM agent config, and extract an OutputType.DecorateEnum helper. Bind jsonRawMessageOrNull through driver.Valuer so raw JSON assessments persist cleanly. Signed-off-by: Sacha Al Himdani <[email protected]>
4fef960 to
75a131e
Compare
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the free-text evidence describer with an assessor that emits a structured assessment (status plus rationale) so downstream code can act on the verdict instead of parsing prose.
Rename the evidencedescriber package and worker to evidenceassessor and introduce a dedicated EvidenceAssessment coredata model backed by an assessment status enum. Split the schema changes into focused migrations and scope the FAILED transition to the status columns so unrelated evidence fields are left untouched.
Realign the surrounding configuration and wiring: rename the config to EvidenceAssessmentConfig, update probod aliases and builders, add a Thinking budget to the LLM agent config, and extract an OutputType.DecorateEnum helper. Bind jsonRawMessageOrNull through driver.Valuer so raw JSON assessments persist cleanly.
Summary by cubic
Replaced the free-text evidence describer with a structured assessor that outputs a JSON assessment and explicit status. Added an
EvidenceAssessmentmodel and updated the worker, services, and configs so downstream code can act on verdicts without parsing prose.Coredata
+336-173assessment JSONBandEvidenceAssessmentStatus; renameddescription_*toassessment_*via 3 migrations.Evidence.SetAssessment,GetAssessment, andSetAssessmentFailed; addeddriver.Valuerfor raw JSON nulls.EvidenceFileID.Service
+303-198pkg/evidenceassessorandevidence-assessmentworker; removedpkg/evidencedescriber.DescriptionfromSummary; failure only touches status columns.OutputType.DecorateEnumand enforced enum forconfidence; addedThinkingbudget toLLMAgentConfig.EvidenceAssessmentConfig; adjusted services toEvidenceFileIDand new statuses.Tests
+145-8confidence.GraphQL API
+2-2EvidenceFileID.Other
+58-58Written for commit 75a131e. Summary will update on new commits.