SNOW-969175: Decimals with precision greater than 18 are returned as object type when using the to_pandas method

[patrickhowerter]

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using?

3.8.5

2. What operating system and processor architecture are you using?

Windows-10-10.0.19041-SP0

3. What are the component versions in the environment (pip freeze)?

altair==4.2.2
asn1crypto==1.5.1
attrs==23.1.0
azure-core==1.29.3
azure-storage-blob==12.17.0
backports.zoneinfo==0.2.1
blinker==1.6.2
cachetools==5.3.1
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
click==8.1.7
cloudpickle==2.0.0
colorama==0.4.6
coverage==7.3.1
cryptography==41.0.3
dill==0.3.7
entrypoints==0.4
exceptiongroup==1.1.3
filelock==3.12.3
gitdb==4.0.10
GitPython==3.1.35
idna==3.4
importlib-metadata==6.8.0
importlib-resources==6.0.1
iniconfig==2.0.0
isodate==0.6.1
jaraco.classes==3.3.0
Jinja2==3.1.2
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
keyring==24.2.0
markdown-it-py==3.0.0
MarkupSafe==2.1.3
mdurl==0.1.2
more-itertools==10.1.0
numpy==1.24.4
oscrypto==1.3.0
packaging==23.1
pandas==2.0.3
Pillow==9.5.0
pkgutil_resolve_name==1.3.10
platformdirs==3.8.1
pluggy==1.3.0
protobuf==3.20.3
pyarrow==10.0.1
pycparser==2.21
pycryptodomex==3.18.0
pydeck==0.8.0
Pygments==2.16.1
PyJWT==2.8.0
Pympler==1.0.1
pyOpenSSL==23.2.0
pytest==7.4.2
pytest-azurepipelines==1.0.4
pytest-cov==4.1.0
pytest-nunit==1.0.3
python-dateutil==2.8.2
pytz==2023.3.post1
pytz-deprecation-shim==0.1.0.post0
pywin32-ctypes==0.2.2
snowflake-snowpark-python==1.8.0
sortedcontainers==2.4.0
streamlit==1.22.0
tenacity==8.2.3
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.1
toolz==0.12.0
tornado==6.3.3
typing_extensions==4.7.1
tzdata==2023.3
tzlocal==4.3.1
urllib3==1.26.16
validators==0.22.0
watchdog==3.0.0
zipp==3.16.2

4. What did you do?
   you can execute the following script to see that converting a decimal column from a snowpark dataframe to pandas dataframe can have different pandas data type, depending on the precision.

from snowflake.snowpark.types import DecimalType
from snowflake.snowpark.functions import col,cast


session = #create your snowpark session here

# returns data as float64 (expected)
snowpark_df = session.create_dataframe([1.5, 234.5], schema=["a"]).with_column("a", cast(col("a"), DecimalType(18, 6)))
print(snowpark_df.to_pandas().dtypes)

# returns as object (unexpected)
snowpark_df = session.create_dataframe([1.5, 234.5], schema=["a"]).with_column("a", cast(col("a"), DecimalType(19, 6)))
print(snowpark_df.to_pandas().dtypes)

5. What did you expect to see?
   I would expect for both to be returned as float64 in the pandas dataframe. If there is not possible because of a limitation with float64, then it would be helpful to have some type of option in the to_pandas() method, to return decimal types as float64 when the precision is greater than 18.