You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
SNOW-1879403: In functions.replace, snowflake-snowpark-python v1.26.0 leads to AssertionError: Do not use lit(Column(...)), type hint does not allow this syntax.
#2856
Open
ndevir opened this issue
Jan 11, 2025
· 2 comments
import logging
import snowflake.snowpark.functions as spf
from utils.snowpark.snowpark_client import SnowparkClient
for logger_name in ('snowflake.snowpark', 'snowflake.connector'):
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter('%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)
with SnowparkClient("PUBLIC") as my_session:
# create a new dataframe (my_session represents a correctly configured session)
df = my_session.create_dataframe(
data=[("this",)],
schema=["A"]
)
# this operation succeeds
df.with_column(
"B",
spf.replace(
spf.col("A"),
"this",
"that"
)
)
# this operation fails: AssertionError: Do not use lit(Column(...)), type hint does not allow this syntax.
df.with_column(
"B",
spf.replace(
spf.col("A"),
spf.lit("this"),
spf.lit("that")
)
)
What did you expect to see?
I expected the above code to run without issues. So far (until snowflake-snowpark-python v1.25.0) it did work. Now (with snowflake-snowpark-python v1.26.0), lit (and other functions, e.g., iff) inside replace fails with the AssertionError that I commented into the code in section 4 (which can also be seen in the below logs).
I saw that as of snowflake-snowpark-python v1.26.0 lit (and several other functions in snowflake.snowpark.functions such as replace) has the @publicapi decorator which might be the cause for this problem.
Can you set logging to DEBUG and collect the logs?
github-actionsbot
changed the title
In functions.lit, snowflake-snowpark-python v1.26.0 leads to AssertionError: Do not use lit(Column(...)), type hint does not allow this syntax.
SNOW-1879403: In functions.lit, snowflake-snowpark-python v1.26.0 leads to AssertionError: Do not use lit(Column(...)), type hint does not allow this syntax.
Jan 11, 2025
ndevir
changed the title
SNOW-1879403: In functions.lit, snowflake-snowpark-python v1.26.0 leads to AssertionError: Do not use lit(Column(...)), type hint does not allow this syntax.
SNOW-1879403: In functions.replace, snowflake-snowpark-python v1.26.0 leads to AssertionError: Do not use lit(Column(...)), type hint does not allow this syntax.
Jan 11, 2025
thanks for the suggestion! This works in case we want to work with literals directly. However, we also require it to work with an iff inside the replace which fails with the same AssertionError (potentially again due to the @publicapi decorator) such as in the following example:
Please answer these questions before submitting your issue. Thanks!
What version of Python are you using?
Python 3.11.4 (main, Feb 9 2024, 09:16:05) [GCC 9.4.0]
What operating system and processor architecture are you using?
Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35
What are the component versions in the environment (
pip freeze
)?I expected the above code to run without issues. So far (until snowflake-snowpark-python v1.25.0) it did work. Now (with snowflake-snowpark-python v1.26.0),
lit
(and other functions, e.g.,iff
) insidereplace
fails with the AssertionError that I commented into the code in section 4 (which can also be seen in the below logs).I saw that as of snowflake-snowpark-python v1.26.0
lit
(and several other functions insnowflake.snowpark.functions
such asreplace
) has the@publicapi
decorator which might be the cause for this problem.The text was updated successfully, but these errors were encountered: