CsvProvider: Incorrect truncation of sample file (by InferRows)

[nikoyak]

#r "nuget: FSharp.Data"
type TP = FSharp.Data.CsvProvider< @"C:\Temp\multiline.csv", InferRows = 2>

multiline.csv:

f1,f2
"multi-
line field",2

Error:

Error: input.fsx (1,11)-(1,76) typecheck error The type provider "ProviderImplementation.CsvProvider" reported an error: Cannot read sample CSV from 'C:\Temp\multiline.csv': Couldn't parse row 1 according to schema: Expected 2 columns, got 1

In the sample, one row of data occupies two lines of the file. InferRows pre-truncates file lines, not data rows:

FSharp.Data/src/CommonProviderImplementation/Helpers.fs

Lines 249 to 261 in d5287e4

	match maxNumberOfRows with
	| None -> reader.ReadToEnd()
	| Some max ->
	let sb = StringBuilder()
	let max = ref max
	while !max > 0 do
	let line = reader.ReadLine()
	if line = null then
	max := 0
	else
	line |> sb.AppendLine |> ignore
	decr max
	sb.ToString()

[github-actions]

🤖 This is an automated response from RepoAssist, the repository's AI assistant.

The root cause of this bug is in Helpers.fs around line 290–303. When InferRows (i.e. maxNumberOfRows) is set, the code reads the sample file using reader.ReadLine(), counting text lines rather than CSV rows:

| Some max ->
    let sb = StringBuilder()
    let mutable max = max
    while max > 0 do
        let line = reader.ReadLine()
        if isNull line then max <- 0
        else
            line |> sb.AppendLine |> ignore
            max <- max - 1
    sb.ToString()

A multi-line quoted CSV field (as in the repro) spans two text lines but counts as two against the InferRows limit, so the sample gets truncated mid-row. The CSV parser then sees an incomplete row and fails.

A correct fix would require CSV-aware line counting — i.e., parsing through quoting boundaries before deciding to stop — rather than plain text-line counting. That is non-trivial since the quote character itself is a type-provider parameter.

Workaround: set InferRows = 0 to disable truncation and infer from the entire file:

type TP = CsvProvider<"multiline.csv", InferRows=0>

Generated by Repo Assist

To install this workflow, run gh aw add githubnext/agentics/workflows/repo-assist.md@b0296681ad309e6455276244363810b1e7d98335. View source at https://github.com/githubnext/agentics/tree/b0296681ad309e6455276244363810b1e7d98335/workflows/repo-assist.md.

[dsyme]

A correct fix would require CSV-aware line counting — i.e., parsing through quoting boundaries before deciding to stop — rather than plain text-line counting. That is non-trivial since the quote character itself is a type-provider parameter.

This fix should be done

[github-actions]

🤖 This is an automated response from Repo Assist.

I've created a draft PR to implement this fix: PR #1625 — Fix #1439: InferRows counts CSV rows, not text lines

The approach was to pass None as maxNumberOfRows to the initial text-reading pass so the raw file is never truncated by line count. The actual row limit is then applied by the CSV parser itself, which correctly respects quoting boundaries. This avoids the issue where multi-line quoted fields consumed multiple "lines" against the limit. The fix is minimal and does not affect the non-InferRows code path.

Generated by Repo Assist

To install this workflow, run gh aw add githubnext/agentics/workflows/repo-assist.md@ee50a3b7d1d3eb4a8c409ac9409fd61c9a66b0f5. View source at https://github.com/githubnext/agentics/tree/ee50a3b7d1d3eb4a8c409ac9409fd61c9a66b0f5/workflows/repo-assist.md.