Skip to content

fix(parse): keep a leading whitespace-only field at the start of a row#1243

Open
spokodev wants to merge 1 commit into
C2FO:mainfrom
spokodev:fix-leading-whitespace-field
Open

fix(parse): keep a leading whitespace-only field at the start of a row#1243
spokodev wants to merge 1 commit into
C2FO:mainfrom
spokodev:fix-leading-whitespace-field

Conversation

@spokodev

Copy link
Copy Markdown

Problem

A whitespace-only field at the start of a row is dropped or truncated, even though the identical field in any other column is preserved (default options, trim: false):

const csv = require('fast-csv');

csv.parseString(' \n',   { headers: false }) // [[]]         expected [[' ']]      — row dropped
csv.parseString('  \n',  { headers: false }) // [[]]         expected [['  ']]
csv.parseString(' ,b\n', { headers: false }) // [['', 'b']]  expected [[' ', 'b']] — first field corrupted

csv.parseString('a,  ,c', { headers: false }) // [['a', '  ', 'c']]  // a whitespace-only non-first field is fine

058B...\t \t \t... (the existing tab-delimiter test) shows whitespace-only fields are expected to be preserved — it just only holds for non-first columns.

RFC 4180 section 2.4: "Spaces are considered part of a field and should not be ignored." With trim defaulting to false, dropping this whitespace is data loss / a wrong row count.

Cause

RowParser.getStartToken locates the row/field start with scanner.nextNonSpaceToken, which skips leading whitespace, so for " \n" it jumps to the row delimiter (empty row) and for " ,b" it jumps to the , (pushing an empty first column). The leading whitespace — the actual unquoted field content — never reaches the column parser.

Fix

Use scanner.nextCharacterToken for the row start, so leading whitespace flows into NonQuotedColumnParser as field content. The two getters differ only when the next character is whitespace, so any non-whitespace row start is byte-identical. Whitespace before a quoted field is unaffected because ColumnParser.parse still uses nextNonSpaceToken for quote detection ( "x",y still parses as ["x", "y"]).

Verification

  • Added RowParser.spec.ts cases for a leading whitespace-only field (' ,b\n'[' ', 'b'], ' \n'[' ']). They fail on main and pass with the fix.
  • Full @fast-csv/parse suite green (374 tests).
  • Round-trip fuzz (2,892 force-quoted records with whitespace-heavy fields) format → parse: 0 failures.

Note (out of scope, flagging for awareness): RowParser.isEmptyRow uses replace(/\s+/g, ''), so ignoreEmpty: true would also discard a legitimate whitespace-only row. Left untouched to keep this PR to one fix.

A whitespace-only field at the start of a row was dropped or truncated,
while the same field in any other column was preserved:

    parseString(' \n',   { headers: false })  // [[]]        expected [[' ']]
    parseString(' ,b\n', { headers: false })  // [['', 'b']] expected [[' ', 'b']]

`getStartToken` located the row/field start with `nextNonSpaceToken`, which
skips leading whitespace, so that whitespace never reached the column
parser. RFC 4180 section 2.4 treats spaces as part of a field, and `trim`
defaults to `false`, so the leading whitespace is field content.

Use `nextCharacterToken` for the row start so leading whitespace flows into
the column parser. It differs from `nextNonSpaceToken` only when the next
character is whitespace, and `ColumnParser.parse` still uses
`nextNonSpaceToken` for quote detection, so whitespace before a quoted
field is unaffected.
@c2fo-cibot c2fo-cibot Bot added the size/S Denotes a PR that changes 10-29 lines label Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Denotes a PR that changes 10-29 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant