Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

658 snippet support sqlcmd #721

Closed
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
a40bd59
snippet support for profile and explore
AnirudhVIyer Jul 11, 2023
120ccdb
snippet support for profile and explore 1
AnirudhVIyer Jul 11, 2023
f2deca6
Update table_explorer.ipynb
AnirudhVIyer Jul 11, 2023
e9251f7
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 12, 2023
fc11cc4
review changes 2
AnirudhVIyer Jul 12, 2023
ddb7cbf
Update table_explorer.ipynb
AnirudhVIyer Jul 12, 2023
44bdcda
fix lint
AnirudhVIyer Jul 12, 2023
62c5b83
Merge branch '658-snippet-support-sqlcmd' of https://github.com/Aniru…
AnirudhVIyer Jul 12, 2023
023d366
fix errors
AnirudhVIyer Jul 12, 2023
901a2c4
fix errors review
AnirudhVIyer Jul 12, 2023
e9aabba
modyfying snippet message
AnirudhVIyer Jul 13, 2023
aea58f5
refactored is_snippet and with-clause
AnirudhVIyer Jul 14, 2023
ed2ff7c
refactored is_snippet and with-clause removed
AnirudhVIyer Jul 14, 2023
466178d
Merge branch 'ploomber:master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 14, 2023
7276af3
is_snippet refactor, remove with_clause
AnirudhVIyer Jul 14, 2023
49ca7a7
Merge branch '658-snippet-support-sqlcmd' of https://github.com/Aniru…
AnirudhVIyer Jul 14, 2023
2f53599
Empty-Commit
AnirudhVIyer Jul 14, 2023
44652df
use dictionary to select messaged
AnirudhVIyer Jul 17, 2023
2651854
use dictionary to select messages
AnirudhVIyer Jul 17, 2023
9322172
use dictionary to select messages 1
AnirudhVIyer Jul 17, 2023
2d34975
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 17, 2023
a677d8b
use dictionary to select messages 2
AnirudhVIyer Jul 17, 2023
0b2069d
Update duckdb-native-sqlalchemy.md
AnirudhVIyer Jul 17, 2023
7a6ec04
use dictionary to select messages 3
AnirudhVIyer Jul 17, 2023
6920d50
replace store_query with prep_query
AnirudhVIyer Jul 18, 2023
0c235df
replace store_query with prep_query 1
AnirudhVIyer Jul 18, 2023
40a63e9
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 18, 2023
8abe684
Update duckdb-native-sqlalchemy.md
AnirudhVIyer Jul 18, 2023
f4e7e86
replace store_query with prep_query 2
AnirudhVIyer Jul 18, 2023
ae843a6
replace store_query with prep_query 3
AnirudhVIyer Jul 18, 2023
040f9eb
replace store_query with prep_query 4
AnirudhVIyer Jul 18, 2023
6d6b6d3
replace store_query with prep_query 5
AnirudhVIyer Jul 18, 2023
f391844
restore store_render
AnirudhVIyer Jul 19, 2023
5d3fbbb
restore prepare_query
AnirudhVIyer Jul 19, 2023
f612d15
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 20, 2023
839224e
integration test added
AnirudhVIyer Jul 20, 2023
8919b22
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 27, 2023
4f98ffd
refactor changes
AnirudhVIyer Jul 27, 2023
1e044ac
resolve merge conflicts
AnirudhVIyer Jul 27, 2023
46189c0
resolve merge conflicts 1
AnirudhVIyer Jul 27, 2023
b05c54e
resolve merge conflicts 2
AnirudhVIyer Jul 27, 2023
9fe8de6
resolve merge conflicts 3
AnirudhVIyer Jul 27, 2023
8507eca
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Jul 27, 2023
056c876
resolve merge conflicts 3
AnirudhVIyer Jul 27, 2023
04b77a3
merge conflict 4
AnirudhVIyer Jul 27, 2023
213757f
changes
AnirudhVIyer Jul 31, 2023
0194b32
Merge branch 'master' into 658-snippet-support-sqlcmd
AnirudhVIyer Aug 4, 2023
f0feedb
update tests
AnirudhVIyer Aug 7, 2023
e3d2889
update tests
AnirudhVIyer Aug 7, 2023
b28fed3
test commit
AnirudhVIyer Aug 8, 2023
e1049c1
change integration test
AnirudhVIyer Aug 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
* [Fix] Refactored `ResultSet` to lazy loading (#470)
* [Fix] Removed `WITH` when a snippet does not have a dependency (#657)
* [Fix] Used display module when generating CTE (#649)
* [Fix] Added support for `profile` and `explore` commands with saved snippets (#658)

## 0.7.9 (2023-06-19)

Expand Down
304 changes: 152 additions & 152 deletions doc/user-guide/table_explorer.ipynb
Original file line number Diff line number Diff line change
@@ -1,154 +1,154 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "b930f418",
"metadata": {},
"source": [
"# Table Explorer\n",
"\n",
"\n",
"```{versionadded} 0.7.6\n",
"~~~\n",
"pip install jupysql --upgrade\n",
"~~~\n",
"```\n",
"\n",
"In this guide, we demonstrate how to use JupySQL's table explorer to visualize SQL tables in HTML format and interact with them efficiently. By running SQL queries in the background instead of loading the data into memory, we minimize the resource consumption and processing time required for handling large datasets, making the interaction with the SQL tables faster and more streamlined.\n",
"\n",
"```{note}\n",
"If you are using JupyterLab or Binder, please ensure that you have installed the latest version of the JupySQL plugin by running the following command: `pip install jupysql-plugin --upgrade`.\n",
"```\n",
"\n",
"Let's start by preparing our dataset. We'll be using the [NYC taxi dataset](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page).\n",
"\n",
"## Download the data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67e9f89e",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"from pathlib import Path\n",
"from urllib.request import urlretrieve\n",
"\n",
"url = \"https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet\"\n",
"\n",
"if not Path(\"yellow_tripdata_2021-01.parquet\").is_file():\n",
" urlretrieve(url, \"yellow_tripdata_2021.parquet\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50e7c60f",
"metadata": {},
"source": [
"## Installation"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2708d4a7",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"%pip install jupysql jupysql-plugin --upgrade --quiet"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e41a3624",
"metadata": {},
"source": [
"## Set connection\n",
"\n",
"After our dataset is ready, we should set our connection.\n",
"\n",
"For this demonstration, we'll be using the `DuckDB` connection."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dbe40317",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"%load_ext sql\n",
"%sql duckdb://"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "08358b2d",
"metadata": {},
"source": [
"## Create the table\n",
"\n",
"To create the table, use the `explore` attribute and specify the name of the table that was just downloaded."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e6c6c7d",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"%sqlcmd explore --table \"yellow_tripdata_2021.parquet\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "23b547b7",
"metadata": {},
"source": [
"See interactive and live example on [Binder](https://binder.ploomber.io/v2/gh/ploomber/jupysql/master?urlpath=lab/tree/doc/user-guide/table_explorer.ipynb)."
]
}
],
"metadata": {
"jupytext": {
"notebook_metadata_filter": "myst"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"myst": {
"html_meta": {
"description lang=en": "Templatize SQL queries in Jupyter via JupySQL",
"keywords": "jupyter, sql, jupysql, jinja",
"property=og:locale": "en_US"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "b930f418",
"metadata": {},
"source": [
"# Table Explorer\n",
"\n",
"\n",
"```{versionadded} 0.7.6\n",
"~~~\n",
"pip install jupysql --upgrade\n",
"~~~\n",
"```\n",
"\n",
"In this guide, we demonstrate how to use JupySQL's table explorer to visualize SQL tables in HTML format and interact with them efficiently. By running SQL queries in the background instead of loading the data into memory, we minimize the resource consumption and processing time required for handling large datasets, making the interaction with the SQL tables faster and more streamlined.\n",
"\n",
"```{note}\n",
"If you are using JupyterLab or Binder, please ensure that you have installed the latest version of the JupySQL plugin by running the following command: `pip install jupysql-plugin --upgrade`.\n",
"```\n",
"\n",
"Let's start by preparing our dataset. We'll be using the [NYC taxi dataset](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page).\n",
"\n",
"## Download the data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67e9f89e",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"from pathlib import Path\n",
"from urllib.request import urlretrieve\n",
"\n",
"url = \"https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet\"\n",
"\n",
"if not Path(\"yellow_tripdata_2021-01.parquet\").is_file():\n",
" urlretrieve(url, \"yellow_tripdata_2021.parquet\")"
]
neelasha23 marked this conversation as resolved.
Show resolved Hide resolved
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50e7c60f",
"metadata": {},
"source": [
"## Installation"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2708d4a7",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"%pip install jupysql jupysql-plugin --upgrade --quiet"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e41a3624",
"metadata": {},
"source": [
"## Set connection\n",
"\n",
"After our dataset is ready, we should set our connection.\n",
"\n",
"For this demonstration, we'll be using the `DuckDB` connection."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dbe40317",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"%load_ext sql\n",
"%sql duckdb://"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "08358b2d",
"metadata": {},
"source": [
"## Create the table\n",
"\n",
"To create the table, use the `explore` attribute and specify the name of the table that was just downloaded."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e6c6c7d",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
"%sqlcmd explore --table \"yellow_tripdata_2021.parquet\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "23b547b7",
"metadata": {},
"source": [
"See interactive and live example on [Binder](https://binder.ploomber.io/v2/gh/ploomber/jupysql/master?urlpath=lab/tree/doc/user-guide/table_explorer.ipynb)."
]
}
],
"metadata": {
"jupytext": {
"notebook_metadata_filter": "myst"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"myst": {
"html_meta": {
"description lang=en": "Templatize SQL queries in Jupyter via JupySQL",
"keywords": "jupyter, sql, jupysql, jinja",
"property=og:locale": "en_US"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading
Loading