dataprep.eda TypeError: Please provide npartitions as an int, or possibly as None if you specify chunksize

[rfinatan]

I'm struggling to understand this TypeError coming out of the dataprep package. My setup is very simple and as follows:

import pandas as pd
import numpy as np
df = pd.DataFrame({
    "phone": [
        "555-234-5678", "(555) 234-5678", "555.234.5678", "555/234/5678",
        15551234567, "(1) 555-234-5678", "+1 (234) 567-8901 x. 1234",
        "2345678901 extension 1234", "2345678", "800-299-JUNK", "1-866-4ZIPCAR",
        "123 ABC COMPANY", "+66 91 889 8948", "hello", np.nan, "NULL"
    ]
})
df

from dataprep.clean import clean_phone
clean_phone(df, "phone")

The resulting error message gets thrown in the terminal (I've replaced sensitive values with x for security purposes) :

Traceback (most recent call last):
  File "c:\Users\x\x\Documents\Repositories\test.py", line 14, in <module>
    clean_phone(df, "phone")
  File "C:\Users\x\Anaconda3\envs\myenv\lib\site-packages\dataprep\clean\clean_phone.py", line 150, in clean_phone
    df = to_dask(df)
  File "C:\Users\x\Anaconda3\envs\myenv\lib\site-packages\dataprep\clean\utils.py", line 73, in to_dask
    return dd.from_pandas(df, npartitions=npartitions)
  File "C:\Users\x\Anaconda3\envs\myenv\lib\site-packages\dask\dataframe\io\io.py", line 236, in from_pandas
    raise TypeError(
TypeError: Please provide npartitions as an int, or possibly as None if you specify chunksize.

This is a direct attempt to replicate the tutorial shown by the dataprep package team found at: https://docs.dataprep.ai/user_guide/clean/clean_phone.html

@dovahcrow, any ideas what's wrong here? I've completed a successful installation of the package and its dependencies with no errors via Conda.

[rfinatan]

Hi all,

User SultanOrazbayev was able to provide a workaround for those replicating this issue.

Please see below for his response:

_

There is a small bug in dataprep package, you can track it in this PR.

In the meantime, one option to avoid the bug is to explicitly convert data to a dask dataframe and pass that into the function:

_

import numpy as np
import pandas as pd
from dask.dataframe import from_pandas
from dataprep.clean import clean_phone

df = pd.DataFrame(
    {
        "phone": [
            "555-234-5678",
            "(555) 234-5678",
            "555.234.5678",
            "555/234/5678",
            15551234567,
            "(1) 555-234-5678",
            "+1 (234) 567-8901 x. 1234",
            "2345678901 extension 1234",
            "2345678",
            "800-299-JUNK",
            "1-866-4ZIPCAR",
            "123 ABC COMPANY",
            "+66 91 889 8948",
            "hello",
            np.nan,
            "NULL",
        ]
    }
)

# to avoid the bug we are passing ddf, not df
ddf = from_pandas(df, npartitions=2)
clean_phone(ddf, "phone")

[jinglinpeng]

Thanks @rfinatan for proposing this issue and thanks @SultanOrazbayev for submitting the PR. Will merge it in the next release.