Creating DSG datsets from non-DSG data or user-defined fake data

[AnsleyManke]

From Hankin notes in https://docs.google.com/document/d/19i-fbyA3XvPkwNp5lxXlnliIBiKwYDlZhz4B_oHKMGk/edit

Creating DSG datasets
The value of the ability to create “fake” datasets has been demonstrated countless times by Ferret users. The ability to create fake DSG datasets would be, if anything, even more valuable. For example, one could quickly and easily simulate an entire sampling program by creating a fake set of ship trajectories and sampling from a model.

The easiest way to add this capability to Ferret may be through extensions of the SAVE command. The extensions needed would be only

New qualifier /NFEATURES=max
SAVE/NFEATURE=max would be basically equivalent to SAVE/ELIMITS=1:max; it creates an output file whose dimensions are pre-set to something larger that the data being saved on the given command. NFEATURES sets the length of the instance dimension. The observations dimension must be the record dimension of the file (for netCDF3, that is, or its equivalent in netCDf 4)

/APPEND
Extensions to SAVE/APPEND would allow new features to be added to an existing file. For gridded datasets, SAVE/APPEND is dual-use -- adding additional variables or filling in additional time steps. To get the same level of flexibility for DSG files would require the use of SAVE/APPEND/E= to guide which feature is being added to.

Commonly users will have multi-feature datasets in the form of ASCII files (spreadsheets). Will a conversion tool be created based upon ERDDAP’s new ncCSV capabilities? If not, or if that tool isn’t flexible enough, then a Ferret script should be created that can perform the conversion, using the SAVE/APPEND capabilities discussed above.

Also explore additions to the ENSEMBLE capabilities in Ferret, to define virtual feature-collection datasets.