update documentation for cli

.github/workflows/mkdocs-publish.yml

@@ -40,7 +40,7 @@ jobs:
             echo "MKDOCS_VERSION=$(hatch version | grep -Po '.+dev')" >> $GITHUB_ENV
           fi
           
-          echo "$MKDOCS_VERSION $MIKE_ALIAS"
+          echo $MKDOCS_VERSION $MIKE_ALIAS
 
           echo "BADGE_MSG=fail" >> $GITHUB_ENV
           echo "BADGE_COLOUR=red" >> $GITHUB_ENV

docs/cli/about.md

@@ -7,7 +7,7 @@ functions to allow fast an easy interrogation of/interaction wtih SEG-Y files.
 For example, it is possible to scrape the text header from a SEG-Y file
 
 ```shell
-segysak ebcidc volve10r12-full-twt-sub3d.sgy
+segysak -f volve10r12-full-twt-sub3d.sgy ebcidc
 C 1 SEGY OUTPUT FROM Petrel 2017.2 Saturday, June 06 2020 10:15:00
 C 2 Name: ST10010ZDC12-PZ-PSDM-KIRCH-FULL-T.MIG_FIN.POST_STACK.3D.JS-017534
 ÝCroC 3
@@ -24,17 +24,8 @@ Standard linux redirects can be used to output the header to a file or to other
 command line tools.
 
 ```shell
-segysak ebcidc volve10r12-full-twt-sub3d.sgy > header.txt
-segysak ebcidc volve10r12-full-twt-sub3d.sgy | less
-```
-From `segysak>=0.5` file conversion is conducted with lazy loading, this should
-allow very large SEG-Y files to be converted to more performant file formats such
-as ZGY[^1] and NetCDF4.
-
-
-
-```shell
-segysak convert 
+segysak -f volve10r12-full-twt-sub3d.sgy ebcidc  > header.txt
+segysak -f volve10r12-full-twt-sub3d.sgy ebcidc | less
 ```
 
 A full list of sub-commands is available in the [CLI Reference](command-line-ref.md)

docs/cli/command-line-ref.md

@@ -1,35 +1,46 @@
 # CLI Reference
 
 ## Base command
-**segysak**
-
-```shell
+### segysak
+```text title="segysak"
 --8<-- "cli/segysak.txt"
 ```
 
 ## EBCIDC headers
-**segysak ebcidc**
-
-```shell
+### segysak ebcidc
+```text title="segysak ebcidc"
 --8<-- "cli/ebcidc.txt"
 ```
 
 ## Trace Headers
-**segysak scan**
-
-```shell
+### segysak scan
+```text title="segysak scan"
 --8<-- "cli/scan.txt"
 ```
 
-**segysak scrape**
-
-```shell
+### segysak scrape
+```text title="segysak scrape"
 --8<-- "cli/scrape.txt"
 ```
 
-## File format conversion
-**segysak convert**
+## File loading and conversion
+### segysak sgy
+```text title="segysak sgy"
+--8<-- "cli/sgy.txt"
+```
+
+### segysak netcdf
+```text title="segysak netcdf"
+--8<-- "cli/netcdf.txt"
+```
+
+### segysak pyzgy
+```text title="segysak pyzgy"
+--8<-- "cli/pyzgy.txt"
+```
 
-```shell
---8<-- "cli/convert.txt"
+## Data manipulation
+### segysak crop
+```text title="segysak crop"
+--8<-- "cli/crop.txt"
 ```

docs/cli/command-pipelines.md

@@ -0,0 +1,76 @@
+SEGY-SAK implements a methodology for chaining multiple cli commands through pipelines.
+Pipelines provide a simple interface to file conversion and volume manipulations (e.g. cropping) 
+in a single step process.
+
+### Pipeline format
+
+The pipeline format consists of a data loading command followed by any volume manipulations, and finally a data output command.
+Pipelines have the general form
+
+```shell
+segysak -f <input-file> <load> [options] <manipulate> [options] ... <output> [options]
+```
+
+This form is slightly different to Unix piping (`|`) operations because the program `segysak` does not terminate between options
+or use the `stdin` and `stdout` unix conventions.
+Instead the number of variables associated with each command and it's options are understood by the parser and the results
+of each command are passed to subsequent one. This approach means that Python objects can be kept in memory between operations
+enabling the transfer of data and state.
+
+For example, to load a 3D SEG-Y file and crop the inline dimension before output to new SEG-Y file, we could use (breaking 
+commands across lines)
+
+```shell
+segysak -f input_file.sgy \
+  sgy -d iline 189 -d xline 193 \
+  crop -c iline 10 20 \
+  sgy -o output_file.sgy
+```
+
+!!! note
+    Notice that the `-d` options for trace header locations are not required for output. This is because SEGY-SAK
+    conveniently remembers where the header locations were from the input. If a SEG-Y file was not the input, the 
+    trace header byte locations must be specified for output, as in the [file conversion](#file-conversion) example.
+
+
+### File conversion
+
+Chaining of commands allows for more flexible use of input and output operators. For example, a SEG-Y file
+can be converted to NetCDF format and back again.
+
+```shell
+segysak -f in.segy \
+  sgy -d iline 189 -d xline 193 -v cdpx 181 -v cdpy 185 \
+  netcdf -o out.nc
+
+segysak -f out.nc \
+  netcdf \
+  sgy -o out.segy -d iline 189 -d xline 193 -v cdpx 181 -v cdpy 185
+```
+
+### Troubleshooting
+
+Pipeline commands can be configured to provide additional output for troubleshooting/debugging
+using `--debug-level DEBUG` option.
+
+```shell
+segysak -f in.segy --debug-level DEBUG sgy -d iline 189 -d xline 193
+
+segysak:cli        segysak v0.5.2.dev4
+segysak:cli        in.sgy
+segysak:pipeline   Begin process pipeline
+segysak:sgy        PARAM: {'dimension': (('iline', 189), ('xline', 193)), 'variable': (), 'output': None}
+segysak:debug_ds   pipeline.ds:
+--------------
+<xarray.Dataset> Size: 372kB
+Dimensions:  (iline: 2, xline: 100, samples: 463)
+Coordinates:
+  * iline    (iline) int8 2B 105 106
+  * xline    (xline) int16 200B 300 301 302 303 304 305 ... 395 396 397 398 399
+  * samples  (samples) float32 2kB 0.0 4.0 8.0 ... 1.84e+03 1.844e+03 1.848e+03
+Data variables:
+    data     (iline, xline, samples) float32 370kB ...
+Attributes:
+    seisnc:   {"coord_scalar": -10.0, "coord_scaled": false}
+--------------
+```

mkdocs.yml

@@ -132,6 +132,7 @@ nav:
       - Lazy operations with Dask: examples/example_segysak_dask.ipynb
   - CLI:
       - About: cli/about.md
+      - Pipelines: cli/command-pipelines.md
       - Reference: cli/command-line-ref.md
   - Tutorial: tutorial.md
   - Meta:

pyproject.toml

@@ -117,7 +117,10 @@ cli-snippets = [
     "segysak ebcidc --help > $CLI_SNIPPETS/ebcidc.txt",
     "segysak scan --help > $CLI_SNIPPETS/scan.txt",
     "segysak scrape --help > $CLI_SNIPPETS/scrape.txt",
-    "segysak convert --help > $CLI_SNIPPETS/convert.txt",
+    "segysak sgy --help > $CLI_SNIPPETS/sgy.txt",
+    "segysak pyzgy --help > $CLI_SNIPPETS/pyzgy.txt",
+    "segysak netcdf --help > $CLI_SNIPPETS/netcdf.txt",
+    "segysak crop --help > $CLI_SNIPPETS/crop.txt",
 ]
 cff-snippets = [
     "echo $CFF_SNIPPETS",