Skip to content

Commit d25992a

Browse files
committed
Don't warn on final missing value when EOF has been reached
Fixes #948. As noted in the included code comment, if the last expected value to be parsed is `missing`, the delimited file may not include a final newline, so the EOF, while abrupt, can be treated as valid. In this specific case, we can avoid emitting the warning and just ensure that column is marked as having a `missing` value parsed.
1 parent 55e0fc9 commit d25992a

File tree

2 files changed

+16
-3
lines changed

2 files changed

+16
-3
lines changed

src/file.jl

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -720,9 +720,17 @@ Base.@propagate_inbounds function parserow(startpos, row, numwarnings, ctx::Cont
720720
else
721721
if i < ncols
722722
if Parsers.newline(code) || pos > len
723-
ctx.silencewarnings || numwarnings[] > ctx.maxwarnings || notenoughcolumns(i, ncols, rowoffset + row)
724-
!ctx.silencewarnings && numwarnings[] == ctx.maxwarnings && toomanywwarnings()
725-
numwarnings[] += 1
723+
# in https://github.com/JuliaData/CSV.jl/issues/948,
724+
# it was noticed that if we reached the EOF right before parsing
725+
# the last expected column, then the warning is a bit spurious.
726+
# The final value is `missing` and the csv writer chose to just
727+
# "close" the file w/o including a final newline
728+
# we can treat this special-case as "valid" and not emit a warning
729+
if !(pos > len && i == (ncols - 1))
730+
ctx.silencewarnings || numwarnings[] > ctx.maxwarnings || notenoughcolumns(i, ncols, rowoffset + row)
731+
!ctx.silencewarnings && numwarnings[] == ctx.maxwarnings && toomanywwarnings()
732+
numwarnings[] += 1
733+
end
726734
for j = (i + 1):ncols
727735
columns[j].anymissing = true
728736
end

test/basics.jl

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -743,4 +743,9 @@ f = CSV.File(IOBuffer(data); header=false, types=Dict(1 => String), typemap=Dict
743743
f = CSV.File(IOBuffer(data); header=false, types=Dict(1 => String), downcast=true);
744744
@test f.types == [i == 1 ? String : Int8 for i = 1:60_000]
745745

746+
# 948
747+
f = CSV.File(IOBuffer("a,b\n1,2\n3,"))
748+
@test f.a == [1, 3]
749+
@test isequal(f.b, [2, missing])
750+
746751
end

0 commit comments

Comments
 (0)