Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write functions to pretty datasets for publication after gap-filling #8

Open
caitlintwhite opened this issue Sep 26, 2023 · 0 comments
Labels
optional (would be helpful) If time, next step to build in (recommend in hindsight)

Comments

@caitlintwhite
Copy link
Contributor

I don't have suggestions for anything specific, but for all versions of C1, D1 and SDL datasets I've made that have been published on EDI, I always had a final script that "prettied" the datasets for publication. This involves things like:

  • making sure columns are in a consistent order,
  • that values are rounded to the precision the raw data are recorded to (predicted values will have more decimal places),
  • that colnames are correct (consistent with past version of dataset or consistent with comparable datasets if the dataset you are making is new),
  • that the source stations used in regressions have their standard for-publication labels,
  • that flag codes are appropriately assigned, and
  • any qualitative notes on flagging are clear and concise (I tend to standardize the notes/comments as much as I can).

The workflow could continue where a polishing script is made manually each time, but it seems like there are probably opportunities to automate some of the dataset "prettying/polishing" tasks. I didn't get that far in developing the workflow to start writing these sorts of functions and am now too far removed in time from the last time I made a NWT climate dataset to remember what tasks could be automated, but if you see any place for it, go for it!

@caitlintwhite caitlintwhite added the optional (would be helpful) If time, next step to build in (recommend in hindsight) label Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optional (would be helpful) If time, next step to build in (recommend in hindsight)
Projects
None yet
Development

No branches or pull requests

1 participant