Code

Paths as URIs#243

Code

Merged

TomNicholas merged 78 commits intomainzarr-developers/VirtualiZarr:mainfrom paths_as_uriszarr-developers/VirtualiZarr:paths_as_urisCopy head branch name to clipboardDec 3, 2024

+711-252

Commits

Commits on Oct 2, 2024

• validate that paths can be coerced to valid URIs

  TomNicholas

  committed
  c787da4

  Copy full SHA for c787da4

• add a test that paths are converted to URIs

  TomNicholas

  committed
  b2f3f26

  Copy full SHA for b2f3f26

• added test and better error if local path is not absolute

  TomNicholas

  committed
  6b15d9e

  Copy full SHA for 6b15d9e

• raise more informative error if path is not absolute

  TomNicholas

  committed
  045cc7a

  Copy full SHA for 045cc7a

• test that empty paths are allowed

  TomNicholas

  committed
  ddc6b7f

  Copy full SHA for ddc6b7f

• add failing test for raising on malformed paths

  TomNicholas

  committed
  61bf5d6

  Copy full SHA for 61bf5d6

• fix paths in tests

  TomNicholas

  committed
  2896bd8

  Copy full SHA for 2896bd8

• fix more tests

  TomNicholas

  committed
  584adf9

  Copy full SHA for 584adf9

• remove the file:/// prefix when writing to kerchunk format

  TomNicholas

  committed
  14b8ca3

  Copy full SHA for 14b8ca3

Commits on Oct 19, 2024

• Merge branch 'main' into paths_as_uris

  TomNicholas

  committed
  bad0b1b

  Copy full SHA for bad0b1b

• absolute paths in recently-added tests

  TomNicholas

  committed
  739a6bd

  Copy full SHA for 739a6bd

• absolute paths in recently-added tests

  TomNicholas

  committed
  06f3a4d

  Copy full SHA for 06f3a4d

Commits on Oct 20, 2024

• fix one more test

  TomNicholas

  committed
  3d50def

  Copy full SHA for 3d50def

Commits on Nov 21, 2024

• Merge branch 'main' into paths_as_uris

  TomNicholas

  committed
  b37f8d0

  Copy full SHA for b37f8d0

• stop wrapping specific error in less useful one

  TomNicholas

  committed
  a9bd0a3

  Copy full SHA for a9bd0a3

• moved remaining kerchunk parsing logic out into translator file

  TomNicholas

  committed
  f0bc445

  Copy full SHA for f0bc445

• add fs_root parameter to validation fn

  TomNicholas

  committed
  b224af8

  Copy full SHA for b224af8

• demote ChunkEntry to a TypedDict to separate validation fn

  TomNicholas

  committed
  c8d513c

  Copy full SHA for c8d513c

• actually instead add new constructor method to TypedDict

  TomNicholas

  committed
  4ad26cf

  Copy full SHA for 4ad26cf

• test kerchunk writer with absolute paths

  TomNicholas

  committed
  7aaaabf

  Copy full SHA for 7aaaabf

• make kerchunk reader tests pass

  TomNicholas

  committed
  f66ed79

  Copy full SHA for f66ed79

• try to implement the fs_root concatenation

  TomNicholas

  committed
  e30ab05

  Copy full SHA for e30ab05

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 0a4a11d
  pre-commit-ci[bot]

  committed
  0a4a11d

  Copy full SHA for 0a4a11d

Commits on Nov 22, 2024

• implementation using cloudpathlib working

  TomNicholas

  committed
  98644a7

  Copy full SHA for 98644a7

• check that fs_root doesn't have a filetype suffix

  TomNicholas

  committed
  4499cf9

  Copy full SHA for 4499cf9

• add cloudpathlib to dependencies

  TomNicholas

  committed
  d27bdf6

  Copy full SHA for d27bdf6

• allow http paths, and require file suffixes

  TomNicholas

  committed
  fefab90

  Copy full SHA for fefab90

• unit tests for path validation

  TomNicholas

  committed
  927121c

  Copy full SHA for 927121c

• test whether we can catch malformed paths

  TomNicholas

  committed
  c561a5d

  Copy full SHA for c561a5d

• test fs_root

  TomNicholas

  committed
  d42ff33

  Copy full SHA for d42ff33

Commits on Nov 23, 2024

• add (unimplemented) validate_entries kwarg to .from_arrays

  TomNicholas

  committed
  4b7cd2d

  Copy full SHA for 4b7cd2d

• add .keys(), .values(), .items()

  TomNicholas

  committed
  0f8b550

  Copy full SHA for 0f8b550

• test validate_paths during .from_arrays

  TomNicholas

  committed
  8055ba7

  Copy full SHA for 8055ba7

• ensure validation actually normalizes paths to uris

  TomNicholas

  committed
  f18bf59

  Copy full SHA for f18bf59

• test .rename_paths correctly validates paths

  TomNicholas

  committed
  96f6360

  Copy full SHA for 96f6360

• some release notes

  TomNicholas

  committed
  b2ed1cd

  Copy full SHA for b2ed1cd

• remove now-superfluous coercion to URI in icechunk writer

  TomNicholas

  committed
  87fad80

  Copy full SHA for 87fad80

• added link to icechunk writer performance benchmark

  TomNicholas

  committed
  1f41d69

  Copy full SHA for 1f41d69

• add reader_kwargs argument to open_virtual_dataset, and pass it down to every reader

  TomNicholas

  committed
  41ec08f

  Copy full SHA for 41ec08f

• Merge branch 'main' into reader_kwargs

  TomNicholas

  committed
  b4cc617

  Copy full SHA for b4cc617

• Merge branch 'main' into paths_as_uris

  TomNicholas

  committed
  ffff93e

  Copy full SHA for ffff93e

• Merge branch 'main' into paths_as_uris

  TomNicholas

  committed
  26e8950

  Copy full SHA for 26e8950

• ensure relative paths containing .. can be normalized

  TomNicholas

  committed
  19c13ae

  Copy full SHA for 19c13ae

• ensure HDF5 reader always returns absolute URIs

  TomNicholas

  committed
  6ce7ebd

  Copy full SHA for 6ce7ebd

• ensure HDF reader always returns absolute URIs

  TomNicholas

  committed
  24b6ea7

  Copy full SHA for 24b6ea7

• add relative path handling to other kerchunk-based readers

  TomNicholas

  committed
  5f0f362

  Copy full SHA for 5f0f362

• Merge branch 'reader_kwargs' into paths_as_uris

  TomNicholas

  committed
  001f95b

  Copy full SHA for 001f95b

Commits on Nov 24, 2024

• add dmrpp relative path integration test

  ayushnag

  committed
  15ae139

  Copy full SHA for 15ae139

Commits on Nov 26, 2024

• Merge branch 'main' into paths_as_uris

  TomNicholas

  committed
  b3432c6

  Copy full SHA for b3432c6

Commits on Nov 27, 2024

• Merge branch 'main' into paths_as_uris

  TomNicholas

  committed
  ae00ac3

  Copy full SHA for ae00ac3

• Merge branch 'main' into filepath_test_dmrpp

  TomNicholas

  authored
  ddc19c3

  Copy full SHA for ddc19c3

• fix kerchunk relative paths test by pluggin through fs_root kwarg

  TomNicholas

  committed
  8353a9b

  Copy full SHA for 8353a9b

• Merge branch 'filepath_test_dmrpp' of https://github.com/ayushnag/VirtualiZarr into paths_as_uris_dmrpp_fix

  TomNicholas

  committed
  251469f

  Copy full SHA for 251469f

• fix dmrpp tests by using absolute filepaths in DMR++ contents

  TomNicholas

  committed
  0915433

  Copy full SHA for 0915433

• clarify new dmrpp test

  TomNicholas

  committed
  3ca8f7f

  Copy full SHA for 3ca8f7f

• test handling of relative filepaths to dmrpp files

  TomNicholas

  committed
  8a90ec7

  Copy full SHA for 8a90ec7

• group related tests

  TomNicholas

  committed
  4afa34c

  Copy full SHA for 4afa34c

Commits on Dec 1, 2024

• removed cloudpathlib from validation code

  TomNicholas

  committed
  da9ee10

  Copy full SHA for da9ee10

Commits on Dec 2, 2024

• fix bug but restrict fs_root to only handle filesystem paths, not bucket url prefixes

  TomNicholas

  committed
  3557008

  Copy full SHA for 3557008

• global list of recognized URI prefixes

  TomNicholas

  committed
  ca09096

  Copy full SHA for ca09096

• cleanup

  TomNicholas

  committed
  7727d4e

  Copy full SHA for 7727d4e

• remove cloudpathlib from dependencies

  TomNicholas

  committed
  03aa6bc

  Copy full SHA for 03aa6bc

• fix/ignore some typing errors

  TomNicholas

  committed
  4d8ceea

  Copy full SHA for 4d8ceea

Commits on Dec 3, 2024

• rewrite tests to use a new dmrparser_factory

  TomNicholas

  committed
  cd5bec1

  Copy full SHA for cd5bec1

• rewrite using global dict of XML strings

  TomNicholas

  committed
  e521e20

  Copy full SHA for e521e20

• fix final test by explicitly passing in tmp_path instead of using a fixture which requests tmp_path

  TomNicholas

  committed
  4ff17d0

  Copy full SHA for 4ff17d0

• Merge branch 'dmrpp_tests_tmp_path' into paths_as_uris

  TomNicholas

  committed
  7e0be24

  Copy full SHA for 7e0be24

• fix bug with not converting Path objects to strings

  TomNicholas

  committed
  bf0bb87

  Copy full SHA for bf0bb87

• Merge branch 'dmrpp_tests_tmp_path' into paths_as_uris

  TomNicholas

  committed
  25acdf9

  Copy full SHA for 25acdf9

• dmrpp relative paths tests passing

  TomNicholas

  committed
  ac04cef

  Copy full SHA for ac04cef

• fix type hint for filetype kwarg

  TomNicholas

  committed
  39bbfaf

  Copy full SHA for 39bbfaf

• Merge branch 'main' into paths_as_uris

  TomNicholas

  authored
  9b6906b

  Copy full SHA for 9b6906b

• user documentation on fs_root

  TomNicholas

  committed
  0127f05

  Copy full SHA for 0127f05

• change example manifests to use URIs

  TomNicholas

  committed
  cd5da5d

  Copy full SHA for cd5da5d

• reminder that rename_paths exists

  TomNicholas

  committed
  df6663f

  Copy full SHA for df6663f

• Merge branch 'paths_as_uris' of https://github.com/TomNicholas/VirtualiZarr into paths_as_uris

  TomNicholas

  committed
  be20994

  Copy full SHA for be20994

• update release notes

  TomNicholas

  committed
  528f722

  Copy full SHA for 528f722

• remove note about .rename_paths

  TomNicholas

  committed
  a8d5a15

  Copy full SHA for a8d5a15