Empty/full whitespace value gets converted to null in CSV parser if it is in the first column

[lizdeika]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

from io import StringIO

import polars as pl

tsv_data = """
a	xxxxxx	12
b	vvvvvv	13
c	eee	14
 	 	0.0
d	ttt	66
e	ggg	444
f	 	44
 	 	0.1
"""

column_list = ["col1", "col2", "col3"]
schema = {"col1": pl.Utf8, "col2": pl.Utf8, "col3": pl.Utf8}
dtypes = {"col1": pl.Utf8, "col2": pl.Utf8, "col3": pl.Utf8}

df = pl.read_csv(
    StringIO(tsv_data),
    has_header=False,
    schema=schema,
    dtypes=dtypes,
    new_columns=column_list,
    separator="\t",
)

print(df)

Log output

shape: (8, 3)
┌──────┬────────┬──────┐
│ col1 ┆ col2   ┆ col3 │
│ ---  ┆ ---    ┆ ---  │
│ str  ┆ str    ┆ str  │
╞══════╪════════╪══════╡
│ a    ┆ xxxxxx ┆ 12   │
│ b    ┆ vvvvvv ┆ 13   │
│ c    ┆ eee    ┆ 14   │
│ null ┆        ┆ 0.0  │
│ d    ┆ ttt    ┆ 66   │
│ e    ┆ ggg    ┆ 444  │
│ f    ┆        ┆ 44   │
│ null ┆        ┆ 0.1  │
└──────┴────────┴──────┘

Issue description

Simple TSV file that has 4th and last rows' first column value as SPACE character
Those spaces get converted to nulls.
No problem for columns that are not first.

Expected behavior

Space is Space, not null

Installed versions

--------Version info---------
Polars:               0.19.18
Index type:           UInt32
Platform:             macOS-14.0-arm64-arm-64bit
Python:               3.11.6 (main, Oct  2 2023, 13:45:54) [Clang 15.0.0 (clang-1500.0.40.1)]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          <not installed>
connectorx:           <not installed>
deltalake:            <not installed>
fsspec:               2023.6.0
gevent:               <not installed>
matplotlib:           <not installed>
numpy:                1.25.1
openpyxl:             <not installed>
pandas:               2.0.3
pyarrow:              12.0.1
pydantic:             1.10.12
pyiceberg:            <not installed>
pyxlsb:               <not installed>
sqlalchemy:           2.0.16
xlsx2csv:             <not installed>
xlsxwriter:           <not installed>

[orlp]

This is not limited to tab-separated values, the same happens for CSV with commas as well.

[lizdeika]

Maybe this will help:
Setting missing_utf8_is_empty_string=True
Space(in first column) gets converted to empty string "" instead of null
Looks like space is not recognized as utf8 char if it is a value of the first column

[Wainberg]

Similar whitespace-related CSV bugs: #10587, #12763

[lizdeika]

Seems I should fallback to pandas

[orlp]

@lizdeika Pull requests are welcome!

[taki-mekhalfa]

Not able to reproduce using polars 0.20.6 anymore;
was fixed by: #13934