Skip to content
This repository has been archived by the owner on May 4, 2021. It is now read-only.

CSV Parameters

Jump to bottom
jbraiuka edited this page Nov 25, 2015 · 32 revisions

#Outlines the various CSV input paramaters

Required Parameters

###USE Boolean Value. Says whether or not to run this line.
e.g: 1

###SOURCE The directory from which we will be scraping the images. Must be mounted on the system where the code is running
e.g: /network/phenocam/data_is_here

###DESTINATION The directory to where we will publish a timestream. Must be mounted on system where code is running e.g: /network/phenocam/timestream_goes_here

###ARCHIVE_DEST The directory where we are archiving this timestream. Must be mounted on system where code is running.
e.g. /network/phenocam/timestream_archive

###EXPT Experiment ID
e.g: fnqr-daintree

###CAM_NUM Camera Number
e.g: 1

###METHOD The method that the line will use.
Acceptable entries: copy, archive, move, resize, json

###MODE The mode that the system is running in. Usually batch e.g: batch

###INTERVAL The number of minutes between each camera shot e.g. 60

###EXPT_START The start date which you would like to include images from e.g. 2000_01_01

###EXPT_END The end date which you would like to include images until e.g. 2015_01_01, OR now

###IMAGE_TYPES The types of images stored here e.g. jpg

###LOCATION Location where data is from
e.g: overstory_nadir

Optional Paramaters

###SUNRISE The start time which you would like to include images from e.g 0700

###SUNSET The end time which you would like to include images until e.g 1700

###CAMERA_TIMEZONE The timezone in which the camera is situated e.g. 1300

###USER The user of the system e.g: Fred

###RESOLUTIONS The resolutions you would like to store it by. the optional resolution after original will store the size which it will be resized to. This value is the width of the resized image, unless it is rotated, then this becomes the height. Categorised by "~original" "original[width]" or "original[width]x[height]" e.g. original1920x1080

###PROJECT_OWNER The owner of the project e.g. Borevitz

###TS_STRUCTURE A custom structure used for storing the folders. This is a replaceable field. See Replaceable Paramaters. This field stores the folder structure for everything between the Destination folder, and the YYYY folder of the timestream. The default value is

EXPT/LOCATION-CCAM_NUM-DATASETID/{folder}/EXPT-CCAM_NUM-DATASETID

This would result in a folder structure BVZ-0000/GC-00-C01-F01/original/BVZ-0000-C01-F01~fullres-orig

The only difference here is the {folder} field which will be replaced with original, or outputs, accordingly.

This will ALWAYS be appended with ~{res}-{step} which will store the resolution, and setp of the timestream. This is done automatically for you.

###FILENAME_DATE_MASK The mask will will be used to extract date times from the filename (only if datetime not stored in exif data) e.g. %Y%m%d_%H%M%S

FN_PARSE

Will only grab images which have this exact string somewhere in their file name. Good for when images of different cameras are all mixed together. An empty entry will add all file names, regardless. **e.g. "Overstory" will add the image

../large_folder/Overstory_nadir/subfolder/asdf/1.jpg

But won't Add the image

../large_folder/Understory_nadir/subfolder/asdf/1.jpg

If the string is anywhere in the file path, then it will be matched up. So if you entered "asdf" both above images would be entered

ORIENTATION

Stores if the images need to be rotated anti-clockwise. Entry is rotation in degrees.

FN_Structure

A custom structure used for storing the filenames. This is a replaceable field. See Replaceable Paramaters. This sets the filename of all images. The default is:

EXPT-LOCATION-CCAM_NUM-DATASETID~{res}-{step}

This would result in a file named: BVZ-0000-GC-00-C01-F01~fullres-orig_2012_04_02_12_12_12.jpg

This will ALWAYS be appended with the _YEAR_MONTH_DAY_HOUR_MINUTE_SECOND fields.

DATASETID

Stores the field which identifies which dataset the images belong to. The idea being that if the same camera in the same location starts taking images of a new location, the DATASETID would change, signifying a new timestream ** e.g. 1 **

The -F will always be applied which adding a datasetID

TIMESHIFT

Shifts the timestream images times by the given value. Useful if images are in UTC, and need to be shifted for sunrise / sunset. Argument is number of hours. ** e.g. 10 **

USERFRIENDLYNAME

Stores the field which will be the name of the timestream. This is the "User Friendly" name. This is also a replaceable field. See Replacable Paramaters Below. ** e.g. Tumb Overstory Nadir **

Default value is EXPT-LOCATION-CCAM_NUM-DATASETID

JSON_UPDATES

Stores which fields will be entered in the new LARGE json. Done to avoid overwriting some fields when updating a specific timestream. If none, then all json fields will be populated. Else, needs to contain, in caps, the name of every json field the user wishes to have populated ** e.g. NAME,TS_END Choices are:

  • NAME
  • TS_ID
  • UTC
  • WIDTH_HIRES
  • TS_VERSION
  • TS_END
  • IMAGE_TYPE
  • HEIGHT_HIRES
  • EXPT
  • WIDTH
  • WEBROOT
  • PERIOD_IN_MINUTES
  • TIMEZONE
  • TS_START
  • HEIGHT
  • ACCESS
  • THUMBNAILS

LARGE_JSON

A boolean string, used for determining if you want a large json of all processed cameras outputted to the destination directory. Useful for uploading to the webpage. e.g. 1

Replaceable Parameters

This section highlights the changes that can be made by some of the Replaceable parameters as listed above. Generally, these fields will replace the field with their value, if its all capitals. For example

EXPT/LOCATION-CCAM_NUM will replace EXPT, LOCATION, and CAM_NUM with their values. Resulting in

BVZ-0000/GC00-C01

Replacement fields which can be used are:

  • use
  • location
  • expt
  • cam_num
  • source
  • destination
  • archive_dest
  • expt_end
  • expt_start
  • interval
  • image_types
  • method
  • resolutions
  • sunrise
  • sunset
  • timezone
  • user
  • mode
  • project_owner
Clone this wiki locally