Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(datasets): Remove tracking datasets which are used in Kedro Viz Experiment Tracking #969

Merged
merged 5 commits into from
Jan 8, 2025
Merged

chore(datasets): Remove tracking datasets which are used in Kedro Viz Experiment Tracking #969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d1cdb58
remove et related kedro datasets
ravi-kumar-pilla Dec 17, 2024
1462f99
Merge branch 'main' into chore/remove-et-ds
ravi-kumar-pilla Dec 17, 2024
f7e1bce
Merge branch 'main' into chore/remove-et-ds
ravi-kumar-pilla Dec 18, 2024
20d636e
update release note and static json schema
ravi-kumar-pilla Jan 6, 2025
dec5322
temporary doc fix
ravi-kumar-pilla Jan 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions kedro-datasets/RELEASE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# Upcoming Release
## Major features and improvements
## Bug fixes and other changes

## Breaking Changes

- Removed `tracking.MetricsDataset` and `tracking.JSONDataset`

## Community contributions

# Release 6.0.0
Expand Down
2 changes: 0 additions & 2 deletions kedro-datasets/docs/source/api/kedro_datasets.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,4 @@ kedro_datasets
svmlight.SVMLightDataset
tensorflow.TensorFlowModelDataset
text.TextDataset
tracking.JSONDataset
tracking.MetricsDataset
yaml.YAMLDataset
5 changes: 0 additions & 5 deletions kedro-datasets/kedro_datasets/_typing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,3 @@
ImagePreview = NewType("ImagePreview", str)
PlotlyPreview = NewType("PlotlyPreview", dict)
JSONPreview = NewType("JSONPreview", str)


# experiment tracking datasets types
MetricsTrackingPreview = NewType("MetricsTrackingPreview", dict)
JSONTrackingPreview = NewType("JSONTrackingPreview", dict)
4 changes: 2 additions & 2 deletions kedro-datasets/kedro_datasets/dask/csv_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,9 @@ def __init__( # noqa: PLR0913
filepath: Filepath in POSIX format to a CSV file
CSV collection or the directory of a multipart CSV.
load_args: Additional loading options `dask.dataframe.read_csv`:
https://docs.dask.org/en/latest/generated/dask.dataframe.read_csv.html
https://docs.dask.org/en/stable/generated/dask.dataframe.read_csv.html
save_args: Additional saving options for `dask.dataframe.to_csv`:
https://docs.dask.org/en/latest/generated/dask.dataframe.to_csv.html
https://docs.dask.org/en/stable/generated/dask.dataframe.to_csv.html
credentials: Credentials required to get access to the underlying filesystem.
E.g. for ``GCSFileSystem`` it should look like `{"token": None}`.
fs_args: Optional parameters to the backend file system driver:
Expand Down
4 changes: 2 additions & 2 deletions kedro-datasets/kedro_datasets/dask/parquet_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,9 @@ def __init__( # noqa: PLR0913
filepath: Filepath in POSIX format to a parquet file
parquet collection or the directory of a multipart parquet.
load_args: Additional loading options `dask.dataframe.read_parquet`:
https://docs.dask.org/en/latest/generated/dask.dataframe.read_parquet.html
https://docs.dask.org/en/stable/generated/dask.dataframe.read_parquet.html
save_args: Additional saving options for `dask.dataframe.to_parquet`:
https://docs.dask.org/en/latest/generated/dask.dataframe.to_parquet.html
https://docs.dask.org/en/stable/generated/dask.dataframe.to_parquet.html
credentials: Credentials required to get access to the underlying filesystem.
E.g. for ``GCSFileSystem`` it should look like `{"token": None}`.
fs_args: Optional parameters to the backend file system driver:
Expand Down
26 changes: 0 additions & 26 deletions kedro-datasets/kedro_datasets/tracking/__init__.py
View file
Open in desktop

This file was deleted.

56 changes: 0 additions & 56 deletions kedro-datasets/kedro_datasets/tracking/json_dataset.py
View file
Open in desktop

This file was deleted.

76 changes: 0 additions & 76 deletions kedro-datasets/kedro_datasets/tracking/metrics_dataset.py
View file
Open in desktop

This file was deleted.

4 changes: 0 additions & 4 deletions kedro-datasets/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,10 +163,6 @@ tensorflow = ["kedro-datasets[tensorflow-tensorflowmodeldataset]"]
text-textdataset = []
text = ["kedro-datasets[text-textdataset]"]

tracking-jsondataset = []
tracking-metricsdataset = []
tracking = ["kedro-datasets[tracking-jsondataset, tracking-metricsdataset]"]

yaml-yamldataset = ["kedro-datasets[pandas-base]", "PyYAML>=4.2, <7.0"]
yaml = ["kedro-datasets[yaml-yamldataset]"]

Expand Down
72 changes: 0 additions & 72 deletions kedro-datasets/static/jsonschema/kedro-catalog-0.18.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,6 @@
"spark.SparkJDBCDataSet",
"tensorflow.TensorFlowModelDataset",
"text.TextDataSet",
"tracking.JSONDataSet",
"tracking.MetricsDataSet",
"yaml.YAMLDataSet"
]
}
Expand Down Expand Up @@ -1312,76 +1310,6 @@
}
}
},
{
"if": {
"properties": {
"type": {
"const": "tracking.JSONDataSet"
}
}
},
"then": {
"required": [
"filepath"
],
"properties": {
"filepath": {
"type": "string",
"description": "Filepath in POSIX format to a text file prefixed with a protocol like `s3://`.\nIf prefix is not provided, `file` protocol (local filesystem) will be used.\nThe prefix should be any protocol supported by ``fsspec``.\nNote: `http(s)` doesn't support versioning."
},
"save_args": {
"type": "object",
"description": "json options for saving JSON files (arguments passed\ninto ```json.dump``). Here you can find all available arguments:\nhttps://docs.python.org/3/library/json.html\nAll defaults are preserved, but \"default_flow_style\", which is set to False."
},
"credentials": {
"type": [
"object",
"string"
],
"description": "Credentials required to get access to the underlying filesystem.\nE.g. for ``GCSFileSystem`` it should look like `{\"token\": None}`."
},
"fs_args": {
"type": "object",
"description": "Extra arguments to pass into underlying filesystem class constructor\n(e.g. `{\"project\": \"my-project\"}` for ``GCSFileSystem``), as well as\nto pass to the filesystem's `open` method through nested keys\n`open_args_load` and `open_args_save`.\nHere you can find all available arguments for `open`:\nhttps://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.open\nAll defaults are preserved, except `mode`, which is set to `r` when loading\nand to `w` when saving."
}
}
}
},
{
"if": {
"properties": {
"type": {
"const": "tracking.MetricsDataSet"
}
}
},
"then": {
"required": [
"filepath"
],
"properties": {
"filepath": {
"type": "string",
"description": "Filepath in POSIX format to a text file prefixed with a protocol like `s3://`.\nIf prefix is not provided, `file` protocol (local filesystem) will be used.\nThe prefix should be any protocol supported by ``fsspec``.\nNote: `http(s)` doesn't support versioning."
},
"save_args": {
"type": "object",
"description": "json options for saving JSON files (arguments passed\ninto ```json.dump``). Here you can find all available arguments:\nhttps://docs.python.org/3/library/json.html\nAll defaults are preserved, but \"default_flow_style\", which is set to False."
},
"credentials": {
"type": [
"object",
"string"
],
"description": "Credentials required to get access to the underlying filesystem.\nE.g. for ``GCSFileSystem`` it should look like `{\"token\": None}`."
},
"fs_args": {
"type": "object",
"description": "Extra arguments to pass into underlying filesystem class constructor\n(e.g. `{\"project\": \"my-project\"}` for ``GCSFileSystem``), as well as\nto pass to the filesystem's `open` method through nested keys\n`open_args_load` and `open_args_save`.\nHere you can find all available arguments for `open`:\nhttps://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.open\nAll defaults are preserved, except `mode`, which is set to `r` when loading\nand to `w` when saving."
}
}
}
},
{
"if": {
"properties": {
Expand Down
72 changes: 0 additions & 72 deletions kedro-datasets/static/jsonschema/kedro-catalog-0.19.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@
"spark.SparkJDBCDataset",
"tensorflow.TensorFlowModelDataset",
"text.TextDataset",
"tracking.JSONDataset",
"tracking.MetricsDataset",
"yaml.YAMLDataset"
]
}
Expand Down Expand Up @@ -1277,76 +1275,6 @@
}
}
},
{
"if": {
"properties": {
"type": {
"const": "tracking.JSONDataset"
}
}
},
"then": {
"required": [
"filepath"
],
"properties": {
"filepath": {
"type": "string",
"description": "Filepath in POSIX format to a text file prefixed with a protocol like `s3://`.\nIf prefix is not provided, `file` protocol (local filesystem) will be used.\nThe prefix should be any protocol supported by ``fsspec``.\nNote: `http(s)` doesn't support versioning."
},
"save_args": {
"type": "object",
"description": "json options for saving JSON files (arguments passed\ninto ```json.dump``). Here you can find all available arguments:\nhttps://docs.python.org/3/library/json.html\nAll defaults are preserved, but \"default_flow_style\", which is set to False."
},
"credentials": {
"type": [
"object",
"string"
],
"description": "Credentials required to get access to the underlying filesystem.\nE.g. for ``GCSFileSystem`` it should look like `{\"token\": None}`."
},
"fs_args": {
"type": "object",
"description": "Extra arguments to pass into underlying filesystem class constructor\n(e.g. `{\"project\": \"my-project\"}` for ``GCSFileSystem``), as well as\nto pass to the filesystem's `open` method through nested keys\n`open_args_load` and `open_args_save`.\nHere you can find all available arguments for `open`:\nhttps://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.open\nAll defaults are preserved, except `mode`, which is set to `r` when loading\nand to `w` when saving."
}
}
}
},
{
"if": {
"properties": {
"type": {
"const": "tracking.MetricsDataset"
}
}
},
"then": {
"required": [
"filepath"
],
"properties": {
"filepath": {
"type": "string",
"description": "Filepath in POSIX format to a text file prefixed with a protocol like `s3://`.\nIf prefix is not provided, `file` protocol (local filesystem) will be used.\nThe prefix should be any protocol supported by ``fsspec``.\nNote: `http(s)` doesn't support versioning."
},
"save_args": {
"type": "object",
"description": "json options for saving JSON files (arguments passed\ninto ```json.dump``). Here you can find all available arguments:\nhttps://docs.python.org/3/library/json.html\nAll defaults are preserved, but \"default_flow_style\", which is set to False."
},
"credentials": {
"type": [
"object",
"string"
],
"description": "Credentials required to get access to the underlying filesystem.\nE.g. for ``GCSFileSystem`` it should look like `{\"token\": None}`."
},
"fs_args": {
"type": "object",
"description": "Extra arguments to pass into underlying filesystem class constructor\n(e.g. `{\"project\": \"my-project\"}` for ``GCSFileSystem``), as well as\nto pass to the filesystem's `open` method through nested keys\n`open_args_load` and `open_args_save`.\nHere you can find all available arguments for `open`:\nhttps://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.open\nAll defaults are preserved, except `mode`, which is set to `r` when loading\nand to `w` when saving."
}
}
}
},
{
"if": {
"properties": {
Expand Down
Empty file removed kedro-datasets/tests/tracking/__init__.py
View file
Open in desktop
Empty file.
Loading
Loading