index out of range error about pgfutter

[fei0x]

Are you guys still looking for a fix? I can help!

Yes, still seeing this, thank you!

In linux, for some reason, for some files, you need to have windows style line breaks for it to work.

My file had no issues on csvLint except the line-break warning.

You can use the unix2dos command to convert the file first, then it worked for me.

[ryuvsken]

change to this code
https://github.com/lukasmartinelli/pgfutter/blob/master/csv.go#L100

		for i, col := range record {
			cols[i] = strings.Replace(col, "\x00", "", -1)
			// bytes.Trim(b, "\x00")
			// cols[i] = col
		}

to

		for idx, col := range record {
			cols[idx] = strings.Replace(col, "\x00", "", -1)
			// bytes.Trim(b, "\x00")
			// cols[i] = col
		}

[raginjason]

What version of pgfutter? I ask because I had a similar panic on v1.2, but when I built from source it ran fine. There have been a few fixes since v1.2 that may have addressed your issue.

[melcha]

I got the same error with a 4MB file, version v1.2.

The problem was that the CSV file was malformed, a few rows had extra columns. I discovered this by validating the CSV file here https://csvlint.io/, the output was very helpful.

[shmsr]

Are you guys still looking for a fix? I can help!

[wisefool769]

Are you guys still looking for a fix? I can help!

Yes, still seeing this, thank you!

[waynegraham]

Can confirm that running my CSV that was having the index out of range issue through unix2dos resolved this for macOS.

[raindog308]

Thank you@waynegraham

Can also confirm that running unix2dos on 18GB of tab-delimited files fixed this issue on Linux 👍

This issue isn't 100% consistent. Some files worked as-is (with Unix line breaks), some did not. After running unix2dos on all, they all worked.