v0.20.13

v0.20.13 (2024-03-15)

• Use spark df.toPandas() function instead of private function converted spark df -> pandas -> polars.

v0.20.12

v0.20.12 (2024-03-12)

• Avoid instantiating spark if databricks-connect installed.

v0.20.9

v0.20.9 (2024-02-29)

• Add new function promote_model_to_production in ds_utils.
• Update ddtrace==2.6.5.
• Update httpx==0.27.0.
• Update orjson==3.9.15.
• Update pandas==2.2.1.
• Update pyodbc==5.1.0.
• Update polars==0.20.13.
• Update redis==5.0.2.
• Update python to 3.12.2.
• Add .show() to end of spark.sql(...) to display merge statistics.

v0.20.8

v0.20.8 (2024-02-19)

• Update aioboto3==12.3.0.
• Update boto3==1.34.34.
• Update botocore==1.34.34.
• Update dominodatalab==1.3.0.
• Update fakeredis==2.21.1.
• Update httpx==0.26.0.
• Update polars==0.20.10.
• Update pre-commit==3.6.2.
• Update pytest==8.0.1.
• Update twine==5.0.0.

v0.20.6

v0.20.6 (2023-12-14)

• Add stage_table optional argument to merge_spark_df_in_db ds.utils function.
• Update aioboto3==12.1.0.
• Update boto3==1.33.1.
• Update botocore==1.33.1.

v0.20.5

v0.20.5 (2023-12-08)

• Add function ese_db_catalog in ds_utils.
• Update pandas==2.1.4.
• Update polars==0.19.19.

v0.20.4

v0.20.4 (2023-12-01)

• Fix bug with sql statement in function ds_utils.read_constants_from_db.
• Update aiojobs==1.2.1.
• Update polars==0.19.18.

v0.20.3

v0.20.3 (2023-11-29)

• Add Databricks functions to read/write constants for Data Science to use across projects.

v0.20.2

v0.20.2 (2023-11-16)

• Hard-code 'us-east-1' and 'us-east-2' as regions in SQS AwsServiceManager.

v0.20.1

v0.20.1 (2023-11-16)

Add sqs.sync_add_regions for setting region besides us-east-1 during instantiation of LongRunningJobs.