Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1661619: Local testing - incorrect behavior when joining and filtering on "isin" #2283

Open
orrdermer1 opened this issue Sep 12, 2024 · 2 comments
Assignees
Labels
bug Something isn't working local testing Local Testing issues/PRs status-triage_done Initial triage done, will be further handled by the driver team

Comments

@orrdermer1
Copy link

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

    Python 3.10.13

  2. What operating system and processor architecture are you using?

    macOS-14.6.1-arm64-arm-64bit

  3. What are the component versions in the environment (pip freeze)?

    ...Snowpark 1.22.0

  4. What did you do?

from snowflake.snowpark import Session
session = Session.builder.config("local_testing", True).create()

users = session.create_dataframe(
        [
            {"user_id": 1, "username": "Alice"},
            {"user_id": 2, "username": "Bob"},
            {"user_id": 3, "username": "Charlie"},
        ]
    )

group_memberships = session.create_dataframe(
    [
        {"group_id": 1, "user_id": 1, "status": "Active"},
        {"group_id": 2, "user_id": 1, "status": "Active"},
        {"group_id": 1, "user_id": 2, "status": "Disabled"},
        {"group_id": 2, "user_id": 2, "status": "Active"},
        {"group_id": 1, "user_id": 3, "status": "Active"},
    ]
)

df = (
    users
    .join(group_memberships, users["user_id"] == group_memberships["user_id"])
    .select(users["username"], group_memberships["group_id"], group_memberships["status"])
)

df.show()
"""
--------------------------------------
|"USERNAME"  |"GROUP_ID"  |"STATUS"  |
--------------------------------------
|Alice       |1           |Active    |
|Alice       |2           |Active    |
|Bob         |1           |Disabled  |
|Bob         |2           |Active    |
|Charlie     |1           |Active    |
--------------------------------------
"""

df.where(df["status"].isin(["Active"])).show()
"""
--------------------------------------
|"USERNAME"  |"GROUP_ID"  |"STATUS"  |
--------------------------------------
|Alice       |1           |Active    |
|Alice       |2           |Active    |
--------------------------------------
"""
  1. What did you expect to see?

The "where" in the end filters 2 records we still expect to see.
"Regular" Snowpark session behave correctly - only the local testing is affected.

  1. Can you set logging to DEBUG and collect the logs?
    (Nothing shows during local testing)
@orrdermer1 orrdermer1 added bug Something isn't working needs triage Initial RCA is required labels Sep 12, 2024
@github-actions github-actions bot changed the title Local testing - incorrect behavior when joining and filtering on "isin" SNOW-1661619: Local testing - incorrect behavior when joining and filtering on "isin" Sep 12, 2024
@sfc-gh-sghosh sfc-gh-sghosh self-assigned this Sep 12, 2024
@sfc-gh-sghosh
Copy link

Hello @orrdermer1 ,

Thanks for raising the issue, we are investigating, will update.

Regards,
Sujan

@sfc-gh-sghosh sfc-gh-sghosh added status-triage Issue is under initial triage and removed needs triage Initial RCA is required labels Sep 17, 2024
@sfc-gh-sghosh
Copy link

Hello @orrdermer1 ,

We are able to reproduce the issue with local_testing and will work on eliminating it. will update.

Regards,
Sujan

@sfc-gh-sghosh sfc-gh-sghosh added status-triage_done Initial triage done, will be further handled by the driver team and removed status-triage Issue is under initial triage labels Sep 17, 2024
@sfc-gh-jrose sfc-gh-jrose added the local testing Local Testing issues/PRs label Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working local testing Local Testing issues/PRs status-triage_done Initial triage done, will be further handled by the driver team
Projects
None yet
Development

No branches or pull requests

4 participants