Skip to content

additional info fields for details tab#222

Open
pranavrd wants to merge 1 commit into
mainfrom
pranavrd_issue_203
Open

additional info fields for details tab#222
pranavrd wants to merge 1 commit into
mainfrom
pranavrd_issue_203

Conversation

@pranavrd

Copy link
Copy Markdown

Validation note + one open design question

How this was validated

  • Unit tests for the field/value mapping against the spec's worked example and
    each branch (checkbox list, dict-with-values, textbox scalar, date&time split,
    empty result).
  • Handler tests covering all four error codes (DE 1000/1001/1002 and the empty
    additionalFields 200).
  • Real-data run: loaded req_add_info.csv (164 requests) into local Postgres and
    transformed every req_id, asserting each field resolves to a valid shape.

Data-quality finding (tracked separately)

Four list-type fields (1.1.A, 2.1.A, 5.1.1.A, 5.2.B) have malformed rows in the
data — item ids written into field_value, or free text on selection fields. The
Lambda maps them per spec; the rows themselves are wrong. Filed as a separate
data-quality issue; not addressed in this PR by design (the Lambda should not
special-case bad data).

Open design question: value-based date&time detection can misfire

Per the issue, date&time values are detected by value pattern
(^\d{4}-\d{2}-\d{2}T) and split into _date / _time, with an explicit
instruction not to hardcode date&time field_ids. That works, but it keys off the
value alone with no awareness of the field's actual type.

Real example — 4.3.3.C (PREFERRED_TUTORING_FEE, a list field whose items are
itemType: "currency"):

  • 7 requests store currency values → mapped correctly as
    {"4.3.3.C.1": "100", "4.3.3.C.2": "200"}.
  • 1 request (REQ-00-000-000-0204) stores 2026-05-10T10:00:00Z in that currency
    field → the regex fires and it's split into _date/_time, even though this is
    a fee field that should never hold a datetime.

So a value that merely looks like a datetime gets datetime treatment regardless
of field type. The offending row is arguably also bad data (a datetime in a fee
field), but it exposes the limitation.

Options:

  1. Keep as-is (current PR). Matches the spec's explicit "detect by value, don't
    hardcode field_ids" instruction. Treats REQ-...0204 as another bad-data row,
    consistent with the four fields above. No new dependencies.
  2. Make detection field-type aware. Only split _date/_time when the metadata
    marks the field as date&time. More correct, but pulls the metadata JSON into the
    Lambda — a dependency the spec didn't call for.

@pranavrd
pranavrd requested review from sana-desai and saquibb8 July 13, 2026 23:07
@pranavrd pranavrd linked an issue Jul 13, 2026 that may be closed by this pull request
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Get Additional info fields for Request Details- Details tab

1 participant