diff --git a/.omero b/.omero index 421c3ae..3824e85 160000 --- a/.omero +++ b/.omero @@ -1 +1 @@ -Subproject commit 421c3aeb9861fcdce1f29f815c425f877de50159 +Subproject commit 3824e8568d3eb1352eca3a413811b7d665fb3aef diff --git a/.omeroci/cli-build b/.omeroci/cli-build index ed09e48..8ef8e37 100755 --- a/.omeroci/cli-build +++ b/.omeroci/cli-build @@ -22,4 +22,5 @@ conda activate omero export OMERO_DIST=${OMERO_DIST:-/opt/omero/server/OMERO.server} omero $PLUGIN -h + python setup.py test -t test -i ${OMERO_DIST}/etc/ice.config -vs diff --git a/LICENSE b/LICENSE old mode 100644 new mode 100755 diff --git a/README.md b/README.md old mode 100644 new mode 100755 index 929b60d..f7714dd --- a/README.md +++ b/README.md @@ -63,6 +63,12 @@ about the files (name, mimetype). `--plugin` allows you to export omero data to a desired format by using an external plugin. See for example the [arc plugin](https://github.com/cmohl2013/omero-arc), which exports omero projects to ARC repositories. +`--binaries` allows to specify whether to archive binary data +(e.g images, ROIs, FileAnnotations) or only create the transfer.xml. +Default is `all` and will create the archive. With `none`, only the `transfer.xml` +file is created, in which case the last cli argument is the path where +the `transfer.xml` file will be written. + Examples: ``` @@ -71,6 +77,8 @@ omero transfer pack --zip Image:123 transfer_pack.zip omero transfer pack Dataset:1111 /home/user/new_folder/new_pack.tar omero transfer pack 999 tarfile.tar # equivalent to Project:999 omero transfer pack --plugin arc Project:999 path/to/my/arc/repo +omero transfer pack --binaries none Dataset:1111 /home/user/new_folder/ +omero transfer pack --binaries all Dataset:1111 /home/user/new_folder/new_pack.tar ``` ## `omero transfer unpack` diff --git a/requirements.txt b/requirements.txt old mode 100644 new mode 100755 diff --git a/setup.py b/setup.py old mode 100644 new mode 100755 diff --git a/src/omero_cli_transfer.py b/src/omero_cli_transfer.py index fa242d9..1d9fded 100644 --- a/src/omero_cli_transfer.py +++ b/src/omero_cli_transfer.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +# !/usr/bin/env python # -*- coding: utf-8 -*- # Copyright (C) 2022 The Jackson Laboratory # All rights reserved. @@ -85,12 +85,21 @@ orig_group`), other options are `none`, `img_id`, `timestamp`, `software`, `version`, `md5`, `hostname`, `db_id`, `orig_user`, `orig_group`. +--binaries allows to specify whether to archive binary data +(e.g images, ROIs, FileAnnotations) or only create the transfer.xml. +Default is `all` and will create the archive. +With `none`, only the `transfer.xml` file is created, in which case +the last cli argument is the path where the `transfer.xml` file +will be written. + Examples: omero transfer pack Image:123 transfer_pack.tar omero transfer pack --zip Image:123 transfer_pack.zip omero transfer pack Dataset:1111 /home/user/new_folder/new_pack.tar omero transfer pack 999 tarfile.tar # equivalent to Project:999 omero transfer pack 1 transfer_pack.tar --metadata img_id version db_id +omero transfer pack --binaries none Dataset:1111 /home/user/new_folder/ +omero transfer pack --binaries all Dataset:1111 /home/user/new_folder/pack.tar """) UNPACK_HELP = ("""Unpacks a transfer packet into an OMERO hierarchy. @@ -224,6 +233,14 @@ def _configure(self, parser): "--plugin", help="Use external plugin for packing.", type=str) pack.add_argument("filepath", type=str, help=file_help) + pack.add_argument( + "--binaries", + choices=["all", "none"], + default="all", + help="With `--binaries none`, only generate the metadata file " + "(transfer.xml or ro-crate-metadata.json). " + "With `--binaries all` (the default), both pixel data " + "and annotation are saved.") file_help = ("Path to where the zip file is saved") unpack.add_argument("filepath", type=str, help=file_help) @@ -396,6 +413,11 @@ def __pack(self, args): if args.simple: raise ValueError("Single plate or screen cannot be " "packaged in human-readable format") + + if (args.binaries == "none") and args.simple: + raise ValueError("The `--binaries none` and `--simple` options " + "are incompatible") + if isinstance(args.object, Image): src_datatype, src_dataid = "Image", args.object.id elif isinstance(args.object, Dataset): @@ -422,7 +444,12 @@ def __pack(self, args): " permissions for current user.") print("Populating xml...") tar_path = Path(args.filepath) - folder = str(tar_path) + "_folder" + if args.binaries == "all": + folder = str(tar_path) + "_folder" + else: + folder = os.path.splitext(tar_path)[0] + print(f"Output will be written to {folder}") + os.makedirs(folder, mode=DIR_PERM, exist_ok=True) if args.barchive: md_fp = str(Path(folder) / "submission.tsv") @@ -436,8 +463,11 @@ def __pack(self, args): args.barchive, args.simple, args.figure, self.metadata) - print("Starting file copy...") - self._copy_files(path_id_dict, folder, self.gateway) + + if args.binaries == "all": + print("Starting file copy...") + self._copy_files(path_id_dict, folder, self.gateway) + if args.simple: self._fix_pixels_image_simple(ome, folder, md_fp) if args.barchive: @@ -477,11 +507,11 @@ def __pack(self, args): tmp_path=Path(folder), image_filenames_mapping=path_id_dict, conn=self.gateway) - else: + elif args.binaries == "all": self._package_files(os.path.splitext(tar_path)[0], args.zip, folder) - print("Cleaning up...") - shutil.rmtree(folder) + print("Cleaning up...") + shutil.rmtree(folder) return def __unpack(self, args): diff --git a/test/integration/test_transfer.py b/test/integration/test_transfer.py index 613a0bc..e609055 100644 --- a/test/integration/test_transfer.py +++ b/test/integration/test_transfer.py @@ -188,6 +188,26 @@ def test_pack_special(self, target_name, tmpdir): assert len(f.getmembers()) == 6 self.delete_all() + @pytest.mark.parametrize('target_name', sorted(SUPPORTED)) + def test_pack_metadata_only(self, target_name, tmpdir): + if target_name == "datasetid" or target_name == "projectid" or\ + target_name == "idonly" or target_name == "imageid": + self.create_image(target_name=target_name) + elif target_name == "plateid" or target_name == "screenid": + self.create_plate(target_name=target_name) + target = getattr(self, target_name) + args = self.args + ["pack", target, "--binaries", "none", + str(tmpdir)] + self.cli.invoke(args, strict=True) + assert os.path.exists(str(tmpdir / 'transfer.xml')) + assert os.path.getsize(str(tmpdir / 'transfer.xml')) > 0 + + args = self.args + ["pack", target, "--binaries", "none", "--simple", + str(tmpdir)] + with pytest.raises(ValueError): + self.cli.invoke(args, strict=True) + self.delete_all() + @pytest.mark.parametrize('folder_name', TEST_FOLDERS) def test_unpack_folder(self, folder_name): self.args += ["unpack", "--folder", folder_name]