SNOW-977836: The withColumnRenamed fucntion fails to rename a column if the snowpark dataframe has multiple columns with same name but with different case style

[Ilyas-kipi]

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using?

Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]

2. What operating system and processor architecture are you using?

Windows-10-10.0.19045-SP0

3. What are the component versions in the environment (pip freeze)?

altair==4.2.2
asn1crypto==1.5.1
attrs==23.1.0
blinker==1.6.2
cachetools==5.3.1
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
click==8.1.7
cloudpickle==2.0.0
colorama==0.4.6
cron-descriptor==1.4.0
croniter==1.4.1
cryptography==41.0.3
entrypoints==0.4
filelock==3.12.3
gitdb==4.0.10
GitPython==3.1.32
greenlet==2.0.2
idna==3.4
importlib-metadata==6.8.0
Jinja2==3.1.2
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
markdown-it-py==3.0.0
MarkupSafe==2.1.3
mdurl==0.1.2
numpy==1.25.2
oscrypto==1.3.0
packaging==23.1
pandas==2.0.3
Pillow==10.0.0
platformdirs==3.8.1
plotly==5.16.1
protobuf==3.20.3
pyarrow==13.0.0
pycparser==2.21
pycryptodomex==3.18.0
pydeck==0.8.1b0
Pygments==2.16.1
PyJWT==2.8.0
Pympler==1.0.1
pyOpenSSL==23.2.0
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0.1
referencing==0.30.2
requests==2.31.0
rich==13.5.2
rpds-py==0.10.0
six==1.16.0
smmap==5.0.0
snowflake-connector-python==3.5.0
snowflake-snowpark-python==1.10.0
snowflake-sqlalchemy==1.5.0
sortedcontainers==2.4.0
SQLAlchemy==1.4.49
streamlit==1.22.0
tenacity==8.2.3
toml==0.10.2
tomlkit==0.12.1
toolz==0.12.0
tornado==6.3.3
typing_extensions==4.7.1
tzdata==2023.3
tzlocal==5.0.1
urllib3==1.26.16
validators==0.21.2
vega-datasets==0.9.0
watchdog==3.0.0
zipp==3.16.2

4. What did you do?

   from snowflake.snowpark.types import IntegerType, StringType, StructField
   schema = StructType([StructField("id", IntegerType()), StructField("Snow Flake", StringType()), StructField("SNOW FLAKE",StringType())])
   df = session.create_dataframe([[1, "snow", "flake"], [3, "snow", "flake"]], schema)
   df.with_column_renamed('"Snow Flake"','"Snow Flake Renamed"').show()
   

5. What did you expect to see?

   I expected the column 'Snow Flake' to be renamed to 'Snow Flake Renamed' but I ran into the following exception

   SnowparkColumnException: Unable to rename the column "Snow Flake" as "Snow Flake Renamed" because this DataFrame has 2 columns named "Snow Flake".

[suenalaba]

If by design, duplicate column names are allowable in Snowpark's API, then I think we should allow a way for the user to edit column names which are duplicated.

Example:

duplicated_col	duplicated_col
'some-value'	'some-value-2'

should be able to be changed to:

new_col_name_1	new_col_name_2
'some-value'	'some-value-2'

Instead of just throwing the error you showed.

[Ilyas-kipi]

Hello @suenalaba, Sry If I was not clear. By definition we cannot create a snowpark dataframe with ambigious column names but we can create a snowpark dataframe that has two columns with same name but different case (i.e)
Name of column 1 - "Snow Flake"
Name of column 2 - "SNOW FLAKE"

Example dataframe :-

When we try to rename Column - 1 (i.e) "Snow Flake" to "Snow Flake Renamed" using the below command
df.with_column_renamed('"Snow Flake"','"Snow Flake Renamed"').show()

we run into an exception -> SnowparkColumnException: Unable to rename the column "Snow Flake" as "Snow Flake Renamed" because this DataFrame has 2 columns named "Snow Flake".

This is because, in the current implementation, the with_column_renamed method, the column being renamed is converted to upper case and then it is checked if there are any more columns with same name, In our case, we have a column that matches upper("Snow Flake") in snowpark dataframe and hence we run into this exception. I've addressed this in my PR #1149