-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 31ffb8b
Showing
51 changed files
with
19,009 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: ec8abc018b81dc419654f69a25c8c2f8 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
.. hpdocs documentation master file, created by | ||
sphinx-quickstart on Tue Jan 24 22:18:05 2023. | ||
You can adapt this file completely to your liking, but it should at least | ||
contain the root `toctree` directive. | ||
Welcome to hpdocs's documentation! | ||
================================== | ||
.. toctree:: | ||
:maxdepth: 1 | ||
|
||
pages/installation | ||
pages/hpdata | ||
|
||
|
||
.. Indices and tables | ||
.. ================== | ||
.. * :ref:`genindex` | ||
.. * :ref:`modindex` | ||
.. * :ref:`search` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
.. _hpdata: | ||
.. role:: raw-html(raw) | ||
:format: html | ||
|
||
============= | ||
Hydropop Data | ||
============= | ||
|
||
A full "run" of hydropop creation results in a set of files that provide hydropop delineations and a suite of attributes and parameters we have identified as relevant to the E3SM-Land Model. This documentation outlines the structure and contents of these data. Note that additional data may be provided (for example, data related to streamflow gages and watersheds). Information about those tables are also provided here, but are not necessarily part of the core hydropop functionality. | ||
|
||
Each "run" must be named, and that name is shown as {name} in the following documentation. Some filenames depend on the ids they represent, and these are shown in {} in the tree below. When the number of files in a directory can be variable, `...` is used. | ||
|
||
Directory and file structure | ||
---------------------------- | ||
|
||
The following tree shows how hydropop exports are structured on disk. Note that all core files are contained in the parent directory (called ``{name}``), while auxiliary/optional files will be in subdirectories that may or may not be present depending on the run parameters. That is, all files in the ``{name}`` directory will always exist for any hydropop export, but other directories (e.g. ``streamflow``, ``watersheds``, ``forcings``, etc.) might not. | ||
|
||
|
||
:: | ||
|
||
{name} <-- parent directory | ||
├── {name}_hpus.gpkg | ||
├── {name}_hpus.tif | ||
├── {name}_hpus.shp | ||
├── {name}_hpu_classes.gpkg | ||
├── {name}_hpu_classes.tif | ||
├── {name}_areagrid.tif | ||
├── {name}_adjacency.csv | ||
├── LICENCE.txt | ||
├── streamflow | ||
├── {id_gage_0}.csv | ||
├── {id_gage_...}.csv | ||
└── {id_gage_N}.csv | ||
├── watersheds | ||
├── {name}_basins.gpkg | ||
├── {name}_hpu_gages.csv | ||
└── {name}_gages.gpkg | ||
├── forcings | ||
├── daily | ||
├── {hpu_id_0}.csv | ||
├── {hpu_id_...}.cosv | ||
└── {hpu_id_N}.csv | ||
├── hourly | ||
├── {hpu_id_0}.csv | ||
├── {hpu_id_...}.cosv | ||
└── {hpu_id_N}.csv | ||
|
||
|
||
Core files | ||
---------- | ||
|
||
The following files will be present for any hydropop exports. Note that hydropop processing is all done in unprojected coordinates (EPSG:4326), but care is taken to correctly compute areas and distances when appropriate. All georeferenced outputs therefore also are in the EPSG:4326 coordinate reference system. | ||
|
||
.. csv-table:: | ||
:file: ../../doctables/hp_output_files.csv | ||
:widths: 10,90 | ||
:header-rows: 1 | ||
|
||
|
||
Auxiliary files | ||
--------------- | ||
|
||
Auxiliary files are not considered part of the core hydropop functionality, and therefore these files may not be present for a general user. | ||
|
||
The ``watersheds`` directory contains files about streamflow gages and their watersheds. These data were obtained from the Veins of the Earth (VotE) data platform, which is currently only available to LANL employees. Information about VotE can be found `here <https://www.essoar.org/doi/10.1002/essoar.10509913.2>`_, and LANL collaborators can access the private VotE repository upon request. | ||
|
||
The ``streamflow`` directory contains one `csv` per streamflow gage, and each filename corresponds to the `id_gage` provided by VotE (and found within the `watersheds` files). These data were obtained from the Veins of the Earth (VotE) data platform, which is currently only available to LANL employees. Information about VotE can be found `here <https://www.essoar.org/doi/10.1002/essoar.10509913.2>`_, and LANL collaborators can access the private VotE repository upon request. | ||
|
||
The ``forcings`` directory contains meterologic and other time-series data either required for or useful to running E3SM-Land models on hydropop units. The data were sampled from the `ERA5-Land Hourly <https://developers.google.com/earth-engine/datasets/catalog/ECMWF_ERA5_LAND_HOURLY>`_ dataset on Google Earth Engine (GEE). Some postprocessing to bring units to more standard formats (and other details) is performed, so the band descriptions provided by the GEE Data Catalog might not be exactly accuarte. Each `csv` file is named a corresponding hydropop id and contains time series. See the `hpus.gpkg`_ for more detailed descriptions of the contents of each file. | ||
|
||
.. csv-table:: | ||
:file: ../../doctables/auxiliary_files.csv | ||
:widths: 10, 20, 70 | ||
:header-rows: 1 | ||
|
||
|
||
Individual file contents | ||
------------------------ | ||
|
||
_`hpus.gpkg` | ||
^^^^^^^^^^^^ | ||
.. csv-table:: | ||
:file: ../../doctables/hpu_gpkg.csv | ||
:widths: 20, 10, 40, 30 | ||
:header-rows: 1 | ||
|
||
|
||
_`basins.gpkg` and _`gages.gpkg` | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
.. csv-table:: | ||
:file: ../../doctables/basins_gpkg.csv | ||
:widths: 20, 80 | ||
:header-rows: 1 | ||
|
||
|
||
Forcings csvs aka _`{hpu_id_n}.csv` | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
.. csv-table:: | ||
:file: ../../doctables/forcings.csv | ||
:widths: 20, 10, 70 | ||
:header-rows: 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
.. _hpdata: | ||
.. role:: raw-html(raw) | ||
:format: html | ||
|
||
============ | ||
Installation | ||
============ | ||
|
||
This repo is not provided as a conda/conda-forge/pip installable package. However, you may install the package by cloning | ||
the repo and performing a live install. Instructions for doing this within an Anaconda environment are provided here, and | ||
have only been tested on a Windows machine. | ||
|
||
I recommend using ``mamba`` as a drop-in replacement for most ``conda`` commands. | ||
It's a package-solver that's orders of magnitude faster. You can install it into your base environment with | ||
|
||
:: | ||
|
||
conda install mamba -c conda-forge | ||
|
||
or follow the actual `installation instructions <https://mamba.readthedocs.io/en/latest/installation.html>`_. | ||
|
||
Clone the ``hydropop`` repo to your PC. This will include the ``hp_enviornment.yml`` file containing the dependencies. I | ||
prefer `Github Desktop <https://desktop.github.com/>`_ for repo management and cloning. | ||
|
||
Open an Anaconda Terminal window and create an empty environment named ``hprepo``: | ||
|
||
:: | ||
|
||
conda create --name hprepo --no-default-packages | ||
|
||
Then update this enviornment using ``mamba`` and pointing to the ``hp_environment.yml`` file: | ||
|
||
:: | ||
|
||
mamba env update -n hprepo --file "path\to\hp_environment.yml" | ||
|
||
Finally, use ``pip`` to do a live-install of the repo into the environment. | ||
|
||
:: | ||
|
||
conda activate hprepo | ||
pip install -e "path\to\hydropop\repo\folder" | ||
|
||
And that's it! Note that this is a "live install" of the hydropop repo, which means that as you edit (or update) the | ||
repo's code on your PC, those changes will be immediately recognized. You should be able to use ``import hydropop.hp_utils`` | ||
etc. at this point. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
/* | ||
* _sphinx_javascript_frameworks_compat.js | ||
* ~~~~~~~~~~ | ||
* | ||
* Compatability shim for jQuery and underscores.js. | ||
* | ||
* WILL BE REMOVED IN Sphinx 6.0 | ||
* xref RemovedInSphinx60Warning | ||
* | ||
*/ | ||
|
||
/** | ||
* select a different prefix for underscore | ||
*/ | ||
$u = _.noConflict(); | ||
|
||
|
||
/** | ||
* small helper function to urldecode strings | ||
* | ||
* See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent#Decoding_query_parameters_from_a_URL | ||
*/ | ||
jQuery.urldecode = function(x) { | ||
if (!x) { | ||
return x | ||
} | ||
return decodeURIComponent(x.replace(/\+/g, ' ')); | ||
}; | ||
|
||
/** | ||
* small helper function to urlencode strings | ||
*/ | ||
jQuery.urlencode = encodeURIComponent; | ||
|
||
/** | ||
* This function returns the parsed url parameters of the | ||
* current request. Multiple values per key are supported, | ||
* it will always return arrays of strings for the value parts. | ||
*/ | ||
jQuery.getQueryParameters = function(s) { | ||
if (typeof s === 'undefined') | ||
s = document.location.search; | ||
var parts = s.substr(s.indexOf('?') + 1).split('&'); | ||
var result = {}; | ||
for (var i = 0; i < parts.length; i++) { | ||
var tmp = parts[i].split('=', 2); | ||
var key = jQuery.urldecode(tmp[0]); | ||
var value = jQuery.urldecode(tmp[1]); | ||
if (key in result) | ||
result[key].push(value); | ||
else | ||
result[key] = [value]; | ||
} | ||
return result; | ||
}; | ||
|
||
/** | ||
* highlight a given string on a jquery object by wrapping it in | ||
* span elements with the given class name. | ||
*/ | ||
jQuery.fn.highlightText = function(text, className) { | ||
function highlight(node, addItems) { | ||
if (node.nodeType === 3) { | ||
var val = node.nodeValue; | ||
var pos = val.toLowerCase().indexOf(text); | ||
if (pos >= 0 && | ||
!jQuery(node.parentNode).hasClass(className) && | ||
!jQuery(node.parentNode).hasClass("nohighlight")) { | ||
var span; | ||
var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg"); | ||
if (isInSVG) { | ||
span = document.createElementNS("http://www.w3.org/2000/svg", "tspan"); | ||
} else { | ||
span = document.createElement("span"); | ||
span.className = className; | ||
} | ||
span.appendChild(document.createTextNode(val.substr(pos, text.length))); | ||
node.parentNode.insertBefore(span, node.parentNode.insertBefore( | ||
document.createTextNode(val.substr(pos + text.length)), | ||
node.nextSibling)); | ||
node.nodeValue = val.substr(0, pos); | ||
if (isInSVG) { | ||
var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect"); | ||
var bbox = node.parentElement.getBBox(); | ||
rect.x.baseVal.value = bbox.x; | ||
rect.y.baseVal.value = bbox.y; | ||
rect.width.baseVal.value = bbox.width; | ||
rect.height.baseVal.value = bbox.height; | ||
rect.setAttribute('class', className); | ||
addItems.push({ | ||
"parent": node.parentNode, | ||
"target": rect}); | ||
} | ||
} | ||
} | ||
else if (!jQuery(node).is("button, select, textarea")) { | ||
jQuery.each(node.childNodes, function() { | ||
highlight(this, addItems); | ||
}); | ||
} | ||
} | ||
var addItems = []; | ||
var result = this.each(function() { | ||
highlight(this, addItems); | ||
}); | ||
for (var i = 0; i < addItems.length; ++i) { | ||
jQuery(addItems[i].parent).before(addItems[i].target); | ||
} | ||
return result; | ||
}; | ||
|
||
/* | ||
* backward compatibility for jQuery.browser | ||
* This will be supported until firefox bug is fixed. | ||
*/ | ||
if (!jQuery.browser) { | ||
jQuery.uaMatch = function(ua) { | ||
ua = ua.toLowerCase(); | ||
|
||
var match = /(chrome)[ \/]([\w.]+)/.exec(ua) || | ||
/(webkit)[ \/]([\w.]+)/.exec(ua) || | ||
/(opera)(?:.*version|)[ \/]([\w.]+)/.exec(ua) || | ||
/(msie) ([\w.]+)/.exec(ua) || | ||
ua.indexOf("compatible") < 0 && /(mozilla)(?:.*? rv:([\w.]+)|)/.exec(ua) || | ||
[]; | ||
|
||
return { | ||
browser: match[ 1 ] || "", | ||
version: match[ 2 ] || "0" | ||
}; | ||
}; | ||
jQuery.browser = {}; | ||
jQuery.browser[jQuery.uaMatch(navigator.userAgent).browser] = true; | ||
} |
Oops, something went wrong.