diff --git a/README.md b/README.md
index 74977d0..fa429a1 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,6 @@
-# Scikit-learn with Argo Observations Cookbook
+# Scikit-learn on Argo Observations
[](https://github.com/song-sangmin/sklearn-argo/actions/workflows/nightly-build.yaml)
[](https://binder.projectpythia.org/v2/gh/song-sangmin/sklearn-argo/main?labpath=notebooks)
@@ -18,7 +18,7 @@ This Project Pythia Cookbook covers two objectives:
## Authors
-[Song Sangmin (@song-sangmin), [Second Author](@second-author), etc. _Acknowledge primary content authors here_
+[Song Sangmin](@song-sangmin), [Michael Chen](@second-author).
### Contributors
@@ -30,17 +30,16 @@ This Project Pythia Cookbook covers two objectives:
(State one or more sections that will comprise the notebook. E.g., _This cookbook is broken up into two main sections - "Foundations" and "Example Workflows."_ Then, describe each section below.)
-This cookbook is broken up into __ sections:
+This cookbook is broken up into two main sections.
-1. Accessing BGC-Argo data with Argopy
-2. Applying scikit-learn to prepare data for machine learning
-3. Using sckikit-learn to develop regression models (e.g. RFR, XGBoost)
+1. Argo Foundations
+2. Scikit-learn Workflows
-### Section 1: Accessing BGC-Argo ( Replace with the title of this section, e.g. "Foundations" )
+### Section 1: Argo Foundations
(Add content for this section, e.g., "The foundational content includes ... ")
-### Section 2: Scikit-learn ( Replace with the title of this section, e.g. "Example workflows" )
+### Section 2: Scikit-learn Workflows
(Add content for this section, e.g., "Example workflows include ... ")
diff --git a/environment.yml b/environment.yml
index db2263d..e658246 100644
--- a/environment.yml
+++ b/environment.yml
@@ -8,6 +8,7 @@ dependencies:
- python=3.9
- numpy
- scipy
+ - pip
# Data packages
- pandas
@@ -34,3 +35,5 @@ dependencies:
- hyperopt
- dataframe_image
+ - pip:
+ - argovisHelpers==0.0.26
diff --git a/notebooks/argo-access.ipynb b/notebooks/argo-access.ipynb
index dad9f26..85e829a 100644
--- a/notebooks/argo-access.ipynb
+++ b/notebooks/argo-access.ipynb
@@ -17,9 +17,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Project Pythia Notebook Template\n",
- "\n",
- "Next, title your notebook appropriately with a top-level Markdown header, `#`. Do not use this level header anywhere else in the notebook. Our book build process will use this title in the navbar, table of contents, etc. Keep it short, keep it descriptive. Follow this with a `---` cell to visually distinguish the transition to the prerequisites section."
+ "# Accessing Argo Data"
]
},
{
@@ -34,13 +32,17 @@
"metadata": {},
"source": [
"## Overview\n",
- "If you have an introductory paragraph, lead with it here! Keep it short and tied to your material, then be sure to continue into the required list of topics below,\n",
"\n",
- "1. This is a numbered list of the specific topics\n",
- "1. These should map approximately to your main sections of content\n",
- "1. Or each second-level, `##`, header in your notebook\n",
- "1. Keep the size and scope of your notebook in check\n",
- "1. And be sure to let the reader know up front the important concepts they'll be leaving with"
+ "Building upon previous notebook, [Introduction to Argo](notebooks/argo-introduction.ipynb), we next explore how to access Argo data in a few different ways. \n",
+ "\n",
+ "1. Data formats for Argo profiles\n",
+ "2. Downloading [monthly snapshots](http://www.argodatamgt.org/Access-to-data/Argo-DOI-Digital-Object-Identifier) using Argo DOI's\n",
+ "3. Using [Argovis](https://argovis.colorado.edu/argo) for API-based queries \n",
+ "4. Using the [GO-BGC Toolbox](https://github.com/go-bgc/workshop-python)\n",
+ "5. Using [Argopy](https://argopy.readthedocs.io/en/latest/user-guide/fetching-argo-data/index.html), a dedicated Python package\n",
+ "\n",
+ "After going through this notebook, you will be able to retrieve Argo data of interest within a certain time frame, geographical location, or by platform identifier. There are many ways of working with Argo data which are not described here. \n",
+ "Further information on Argo access can be found on the [Argo website](https://argo.ucsd.edu/data/)."
]
},
{
@@ -48,23 +50,16 @@
"metadata": {},
"source": [
"## Prerequisites\n",
- "This section was inspired by [this template](https://github.com/alan-turing-institute/the-turing-way/blob/master/book/templates/chapter-template/chapter-landing-page.md) of the wonderful [The Turing Way](https://the-turing-way.netlify.app) Jupyter Book.\n",
- "\n",
- "Following your overview, tell your reader what concepts, packages, or other background information they'll **need** before learning your material. Tie this explicitly with links to other pages here in Foundations or to relevant external resources. Remove this body text, then populate the Markdown table, denoted in this cell with `|` vertical brackets, below, and fill out the information following. In this table, lay out prerequisite concepts by explicitly linking to other Foundations material or external resources, or describe generally helpful concepts.\n",
"\n",
"Label the importance of each concept explicitly as **helpful/necessary**.\n",
"\n",
"| Concepts | Importance | Notes |\n",
"| --- | --- | --- |\n",
- "| [Intro to Cartopy](https://foundations.projectpythia.org/core/cartopy/cartopy.html) | Necessary | |\n",
- "| [Understanding of NetCDF](https://foundations.projectpythia.org/core/data-formats/netcdf-cf.html) | Helpful | Familiarity with metadata structure |\n",
- "| Project management | Helpful | |\n",
+ "| [Intro to Numpy](https://numpy.org/learn/) | Necessary | |\n",
+ "| [Intro to NetCDF](https://foundations.projectpythia.org/core/data-formats/netcdf-cf.html) | Necessary | Familiarity with metadata structure |\n",
+ "| [Intro to Xarray](https://foundations.projectpythia.org/core/xarray.html) | Necessary | |\n",
"\n",
- "- **Time to learn**: estimate in minutes. For a rough idea, use 5 mins per subsection, 10 if longer; add these up for a total. Safer to round up and overestimate.\n",
- "- **System requirements**:\n",
- " - Populate with any system, version, or non-Python software requirements if necessary\n",
- " - Otherwise use the concepts table above and the Imports section below to describe required packages as necessary\n",
- " - If no extra requirements, remove the **System requirements** point altogether"
+ "- **Time to learn**: 20 min\n"
]
},
{
@@ -84,18 +79,624 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "ModuleNotFoundError",
+ "evalue": "No module named 'argovisHelpers'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
+ "Cell \u001b[0;32mIn[1], line 14\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mmatplotlib\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mcolors\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mmcolors\u001b[39;00m\n\u001b[1;32m 12\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mseaborn\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01msns\u001b[39;00m\n\u001b[0;32m---> 14\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01margovisHelpers\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m helpers \u001b[38;5;28;01mas\u001b[39;00m avh\n",
+ "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'argovisHelpers'"
+ ]
+ }
+ ],
+ "source": [
+ "# Import packages\n",
+ "import sys\n",
+ "import os\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "import scipy\n",
+ "import xarray as xr\n",
+ "from datetime import datetime, timedelta\n",
+ "\n",
+ "import matplotlib.pyplot as plt\n",
+ "import matplotlib.colors as mcolors\n",
+ "import seaborn as sns\n",
+ "\n",
+ "from argovisHelpers import helpers as avh"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Snapshots"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# # Base filepath. Need for Argo GDAC function.\n",
+ "root = '/Users/sangminsong/Library/CloudStorage/OneDrive-UW/Code/2024_Pythia/'\n",
+ "profile_dir = root + 'SOCCOM_GO-BGC_LoResQC_LIAR_28Aug2023_netcdf/'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
- "import sys"
+ "DSdict = {}\n",
+ "for filename in os.listdir(profile_dir):\n",
+ " if filename.endswith(\".nc\"):\n",
+ " fp = profile_dir + filename\n",
+ " single_dataset = xr.open_dataset(fp, decode_times=False)\n",
+ " DSdict[filename[0:7]] = single_dataset\n",
+ "# DSdict['5906030']"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[117 values with dtype=float64]
Pressure
(N_PROF, N_LEVELS)
float64
...
units :
dbar
[8073 values with dtype=float64]
Pressure_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Pressure_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Temperature
(N_PROF, N_LEVELS)
float64
...
units :
degree_C
[8073 values with dtype=float64]
Temperature_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Temperature_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Salinity
(N_PROF, N_LEVELS)
float64
...
units :
pss
[8073 values with dtype=float64]
Salinity_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Salinity_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Sigma_theta
(N_PROF, N_LEVELS)
float64
...
units :
kg/m^3
[8073 values with dtype=float64]
Sigma_theta_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Sigma_theta_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Depth
(N_PROF, N_LEVELS)
float64
...
units :
m
[8073 values with dtype=float64]
Depth_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Depth_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Oxygen
(N_PROF, N_LEVELS)
float64
...
units :
umol/kg
[8073 values with dtype=float64]
Oxygen_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Oxygen_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
OxygenSat
(N_PROF, N_LEVELS)
float64
...
units :
%
[8073 values with dtype=float64]
OxygenSat_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
OxygenSat_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Nitrate
(N_PROF, N_LEVELS)
float64
...
units :
umol/kg
[8073 values with dtype=float64]
Nitrate_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Nitrate_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Chl_a
(N_PROF, N_LEVELS)
float64
...
units :
mg/m^3
[8073 values with dtype=float64]
Chl_a_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
Chl_a_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
b_bp700
(N_PROF, N_LEVELS)
float64
...
units :
1/m
[8073 values with dtype=float64]
b_bp700_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
b_bp700_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
POC
(N_PROF, N_LEVELS)
float64
...
units :
mmol/m^3
[8073 values with dtype=float64]
POC_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
POC_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
pHinsitu
(N_PROF, N_LEVELS)
float64
...
units :
Total
[8073 values with dtype=float64]
pHinsitu_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
pHinsitu_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
pH25C
(N_PROF, N_LEVELS)
float64
...
units :
Total
[8073 values with dtype=float64]
pH25C_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
pH25C_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
TALK_LIAR
(N_PROF, N_LEVELS)
float64
...
units :
umol/kg
[8073 values with dtype=float64]
TALK_LIAR_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
TALK_LIAR_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
DIC_LIAR
(N_PROF, N_LEVELS)
float64
...
units :
umol/kg
[8073 values with dtype=float64]
DIC_LIAR_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
DIC_LIAR_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
pCO2_LIAR
(N_PROF, N_LEVELS)
float64
...
units :
uatm
[8073 values with dtype=float64]
pCO2_LIAR_QF
(N_PROF, N_LEVELS)
|S1
...
[8073 values with dtype=|S1]
pCO2_LIAR_QFA
(N_PROF, N_LEVELS)
float64
...
NOTES :
Variables ending in _QFA are an alternate format (double matrix instead of character array) for the variable ending in _QF
[8073 values with dtype=float64]
Type
()
|S1
...
[1 values with dtype=|S1]
mon_day_yr
(N_PROF)
|S10
...
[117 values with dtype=|S10]
hh_mm
(N_PROF)
|S5
...
[117 values with dtype=|S5]
Parameters
(NPARAMETER)
|S19
...
NOTES :
Parameters is not a FloatViz variable. Parameters lists parameters in the original FloatViz file, in the same column order. JULD is not a variable in the original FloatViz file.
[42 values with dtype=|S19]
JULD
(N_PROF)
float64
...
long_name :
Julian day (UTC) of the station relative to REFERENCE_DATE_TIME
standard_name :
time
units :
days since 1950-01-01 00:00:00 UTC
conventions :
Relative julian days with decimal part (as parts of day)
NOTES :
JULD not a FloatViz variable, but added for convenience and for similarity to ARGO-type files.
[117 values with dtype=float64]
REFERENCE_DATE_TIME
()
object
...
long_name :
Date of reference for Julian days
conventions :
YYYYMMDDHHMISS
NOTES :
REFERENCE_DATE_TIME not a FloatViz variable, but added for convenience and for similarity to ARGO-type files.
[1 values with dtype=object]
Comments :
\n",
+ "//0\n",
+ "//<Encoding>UTF-8</Encoding>\n",
+ "//File updated on 08/26/2023 15:30\n",
+ "//!! ADJUSTED DATA FILE !!\n",
+ "//WMO ID: 5906007\n",
+ "//Institution ID: 12880\n",
+ "//MBARI ID: ua12880\n",
+ "//Project Name: SOCCOM\n",
+ "//Region: SO\n",
+ "//No missing float profiles\n",
+ "//Missing Float position interpolated for station(s): 93 94 95 \n",
+ "//Latitude quality flag = 4 for interpolated float positions\n",
+ "//Missing Float data detected for adjusted station(s): 1 \n",
+ "//\n",
+ "//FLOAT VARIABLES:\n",
+ "//Variable Sensor Serial number Comment\n",
+ "//Pressure[dbar] Sbe41cp 10444 \n",
+ "//Temperature[°C] Sbe41cp 10444 \n",
+ "//Salinity[pss] Sbe41cp 10444 \n",
+ "//Sigma_theta[kg/m^3] Potential density at the sea surface\n",
+ "//Depth[m] Depth calculated from pressure and latitude\n",
+ "//Oxygen[µmol/kg] Aanderaa 4330 2857 \n",
+ "//OxygenSat[%] Calculation assumes atmospheric pressure= 1013.25 mbar\n",
+ "//Nitrate[µmol/kg] ISUS Lamp#L6.A01.0148 Spec#108002 Equitech Ti probe #1703.23 MBARI ARGO float#813 DF#225 CTD#10444 \n",
+ "//Chl_a[mg/m^3] FLBBAP2 4681 See: Boss, E.B. and N. Haëntjens, 2016. http://soccom.princeton.edu/sites/default/files/files/SOCCOM_2016-1_Bio-optics-primer.pdf\n",
+ "//b_bp700[1/m] FLBBAP2 4681 See: Boss, E.B. and N. Haëntjens, 2016. http://soccom.princeton.edu/sites/default/files/files/SOCCOM_2016-1_Bio-optics-primer.pdf\n",
+ "//POC[mmol/m^3] FLBBAP2 4681 See: Boss, E.B. and N. Haëntjens, 2016. http://soccom.princeton.edu/sites/default/files/files/SOCCOM_2016-1_Bio-optics-primer.pdf\n",
+ "//pHinsitu[Total] Durafet 225 \n",
+ "//pH25C[Total] Durafet 225 estimated with CO2SYS(TALK_LIAR,pHinsitu) for Matlab see note below\n",
+ "//TALK_LIAR[µmol/kg] TALK_LIAR = Alkalinity estimated with the LIAR V2.2 algorithim Eqn7, MeasIDVec = [1 6 7] (PSAL, DOXY_ADJ, TEMP), (Carter et al., 2016,doi:10.1002/lom3.10087; Carter et al., 2017, https://doi.org/10.1002/lom3.10232).LIAR code used was downloaded on 10/31/2018 from https://github.com/BRCScienceProducts/LIRs.\n",
+ "//DIC_LIAR[µmol/kg] estimated with CO2SYS(TALK_LIAR,pHinsitu) for Matlab (see note below)\n",
+ "//pCO2_LIAR[µatm] estimated with CO2SYS(TALK_LIAR,biased pHinsitu) for Matlab (see note below)\n",
+ "//\n",
+ "//NOTE ON CO2SYS CARBONATE SYSTEM CALCULATIONS:\n",
+ "//All carbonate system variables calculated with CO2SYS for Matlab\n",
+ "//(Sharp et al., 2020; see also Lewis and Wallace 1998)\n",
+ "//used the following conditions: pH was reported on the total scale.\n",
+ "//K1 and K2 dissociation constants were from Lueker et al., 2000, doi:\n",
+ "//10.1016/S0304-4203(00)00022-0. The KSO4 dissociation constant was\n",
+ "//from Dickson, 1990, doi: 10.1016/0021-9614(90)90074-Z. The KF dissociation\n",
+ "//constant was from Perez and Fraga 1987, doi: 10.1016/0304-4203(87)90036-3.\n",
+ "//The borate to salinity ratio was from Lee et al., 2010,\n",
+ "//doi:10.1016/j.gca.2009.12.027. Silicate and Phosphate were not\n",
+ "//measured by the float, but estimates based on Redfieldian ratios improved\n",
+ "//the carbonate system estimates. If a nitrate value was considered to\n",
+ "//be of good quality silicate = nitrate*2.5 and phosphate = nitrate/16,\n",
+ "//otherwise the best estimate for both was considered to be 0. When pCO2\n",
+ "//was estimated from TALK_LIAR and pHinsitu, a bias was first added to pHinsitu\n",
+ "//following Williams et al., 2017, doi: https://doi.org/10.1002/2016GB005541 , section 3.4, equation 3.\n",
+ "//This correction is not necessary for DIC and DIC is computed with the reported pH and the TALK_LIAR value.\n",
+ "//\n",
+ "//NOTE ON Chl_a [mg/m^3] CONCENTRATION:\n",
+ "//There is community-established calibration bias of 2 for the WET Labs 413 ECO-series fluorometers\n",
+ "//(Roesler et al, 2017, doi: 10.1002/lom3.10185).\n",
+ "//Chl_a has been recalculated using in situ measured dark counts. Chl_a is then divided by the Roesler factor of 2.\n",
+ "//Lastly, profiles with sun elevaltion > 0 are corrected for NPQ (Xing et al, 2012, doi: 10.4319/lom.2012.10.483)\n",
+ "//with uncorrected spikes (raw-filtered data) added back to the corrected profile.\n",
+ "//\n",
+ "//QUALITY CONTROLLED DATA CORRECTIONS:\n",
+ "//Measurement Station Gain Offset Drift\n",
+ "//Oxygen 1 1.0862 0.0000 0.0000\n",
+ "//Nitrate 1 1.0000 -0.4086 -42.7354\n",
+ "//Nitrate 3 1.0000 -1.6984 0.3811\n",
+ "//Nitrate 117 1.0000 -0.6807 0.0000\n",
+ "//pH 1 1.0000 -0.0166 0.9534\n",
+ "//pH 3 1.0000 0.0156 -0.0425\n",
+ "//pH 21 1.0000 -0.0256 1.0127\n",
+ "//pH 23 1.0000 -0.0310 0.0272\n",
+ "//pH 36 1.0000 0.0032 -0.2867\n",
+ "//pH 40 1.0000 -0.0014 0.0000\n",
+ "//pH 54 1.0000 -0.0151 0.0000\n",
+ "//pH 69 1.0000 -0.0044 -0.0139\n",
+ "//pH 107 1.0000 0.0003 0.0410\n",
+ "//pH 116 1.0000 -0.0122 0.0000\n",
+ "//pH 117 1.0000 -0.0055 0.0000\n",
+ "//\n",
+ "//Missing data value = -1e10\n",
+ "//Data quality flags: 0=Good, 4=Questionable, 8=Bad, 1=Missing or not inspected \n",
+ "//Note: all timestamps are in GMT.
"
+ ],
+ "text/plain": [
+ " Size: 2MB\n",
+ "Dimensions: (N_PROF: 117, N_LEVELS: 69, NPARAMETER: 42)\n",
+ "Dimensions without coordinates: N_PROF, N_LEVELS, NPARAMETER\n",
+ "Data variables: (12/60)\n",
+ " Cruise |S11 11B ...\n",
+ " Station (N_PROF) int32 468B ...\n",
+ " Lon (N_PROF) float64 936B ...\n",
+ " Lat (N_PROF) float64 936B ...\n",
+ " Lat_QF (N_PROF) |S1 117B ...\n",
+ " Lat_QFA (N_PROF) float64 936B ...\n",
+ " ... ...\n",
+ " Type |S1 1B ...\n",
+ " mon_day_yr (N_PROF) |S10 1kB ...\n",
+ " hh_mm (N_PROF) |S5 585B ...\n",
+ " Parameters (NPARAMETER) |S19 798B ...\n",
+ " JULD (N_PROF) float64 936B ...\n",
+ " REFERENCE_DATE_TIME object 8B ...\n",
+ "Attributes:\n",
+ " Comments: \\n//0\\n//UTF-8\\n//File updated on 08/26/2..."
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "DSdict['5906007']\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'/Users/sangminsong/Library/CloudStorage/OneDrive-UW/Code/2024_Pythia/SOCCOM_GO-BGC_LoResQC_LIAR_28Aug2023_netcdf/'"
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "profile_dir"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "ValueError",
+ "evalue": "unrecognized chunk manager dask - must be one of: []",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
+ "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mxr\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_mfdataset\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/Users/sangminsong/Library/CloudStorage/OneDrive-UW/Code/2024_Pythia/SOCCOM_GO-BGC_LoResQC_LIAR_28Aug2023_netcdf/*.nc\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n",
+ "File \u001b[0;32m/opt/homebrew/Caskroom/mambaforge/base/envs/sklearn-argo-dev/lib/python3.9/site-packages/xarray/backends/api.py:1054\u001b[0m, in \u001b[0;36mopen_mfdataset\u001b[0;34m(paths, chunks, concat_dim, compat, preprocess, engine, data_vars, coords, combine, parallel, join, attrs_file, combine_attrs, **kwargs)\u001b[0m\n\u001b[1;32m 1051\u001b[0m open_ \u001b[38;5;241m=\u001b[39m open_dataset\n\u001b[1;32m 1052\u001b[0m getattr_ \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mgetattr\u001b[39m\n\u001b[0;32m-> 1054\u001b[0m datasets \u001b[38;5;241m=\u001b[39m [open_(p, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mopen_kwargs) \u001b[38;5;28;01mfor\u001b[39;00m p \u001b[38;5;129;01min\u001b[39;00m paths]\n\u001b[1;32m 1055\u001b[0m closers \u001b[38;5;241m=\u001b[39m [getattr_(ds, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m_close\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;28;01mfor\u001b[39;00m ds \u001b[38;5;129;01min\u001b[39;00m datasets]\n\u001b[1;32m 1056\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m preprocess \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
+ "File \u001b[0;32m/opt/homebrew/Caskroom/mambaforge/base/envs/sklearn-argo-dev/lib/python3.9/site-packages/xarray/backends/api.py:1054\u001b[0m, in \u001b[0;36m\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 1051\u001b[0m open_ \u001b[38;5;241m=\u001b[39m open_dataset\n\u001b[1;32m 1052\u001b[0m getattr_ \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mgetattr\u001b[39m\n\u001b[0;32m-> 1054\u001b[0m datasets \u001b[38;5;241m=\u001b[39m [\u001b[43mopen_\u001b[49m\u001b[43m(\u001b[49m\u001b[43mp\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mopen_kwargs\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mfor\u001b[39;00m p \u001b[38;5;129;01min\u001b[39;00m paths]\n\u001b[1;32m 1055\u001b[0m closers \u001b[38;5;241m=\u001b[39m [getattr_(ds, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m_close\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;28;01mfor\u001b[39;00m ds \u001b[38;5;129;01min\u001b[39;00m datasets]\n\u001b[1;32m 1056\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m preprocess \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
+ "File \u001b[0;32m/opt/homebrew/Caskroom/mambaforge/base/envs/sklearn-argo-dev/lib/python3.9/site-packages/xarray/backends/api.py:577\u001b[0m, in \u001b[0;36mopen_dataset\u001b[0;34m(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)\u001b[0m\n\u001b[1;32m 570\u001b[0m overwrite_encoded_chunks \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124moverwrite_encoded_chunks\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m)\n\u001b[1;32m 571\u001b[0m backend_ds \u001b[38;5;241m=\u001b[39m backend\u001b[38;5;241m.\u001b[39mopen_dataset(\n\u001b[1;32m 572\u001b[0m filename_or_obj,\n\u001b[1;32m 573\u001b[0m drop_variables\u001b[38;5;241m=\u001b[39mdrop_variables,\n\u001b[1;32m 574\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mdecoders,\n\u001b[1;32m 575\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m 576\u001b[0m )\n\u001b[0;32m--> 577\u001b[0m ds \u001b[38;5;241m=\u001b[39m \u001b[43m_dataset_from_backend_dataset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 578\u001b[0m \u001b[43m \u001b[49m\u001b[43mbackend_ds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 579\u001b[0m \u001b[43m \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 580\u001b[0m \u001b[43m \u001b[49m\u001b[43mengine\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 581\u001b[0m \u001b[43m \u001b[49m\u001b[43mchunks\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 582\u001b[0m \u001b[43m \u001b[49m\u001b[43mcache\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 583\u001b[0m \u001b[43m \u001b[49m\u001b[43moverwrite_encoded_chunks\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 584\u001b[0m \u001b[43m \u001b[49m\u001b[43minline_array\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 585\u001b[0m \u001b[43m \u001b[49m\u001b[43mchunked_array_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 586\u001b[0m \u001b[43m \u001b[49m\u001b[43mfrom_array_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 587\u001b[0m \u001b[43m \u001b[49m\u001b[43mdrop_variables\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdrop_variables\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 588\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mdecoders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 589\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 590\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 591\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m ds\n",
+ "File \u001b[0;32m/opt/homebrew/Caskroom/mambaforge/base/envs/sklearn-argo-dev/lib/python3.9/site-packages/xarray/backends/api.py:370\u001b[0m, in \u001b[0;36m_dataset_from_backend_dataset\u001b[0;34m(backend_ds, filename_or_obj, engine, chunks, cache, overwrite_encoded_chunks, inline_array, chunked_array_type, from_array_kwargs, **extra_tokens)\u001b[0m\n\u001b[1;32m 368\u001b[0m ds \u001b[38;5;241m=\u001b[39m backend_ds\n\u001b[1;32m 369\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m--> 370\u001b[0m ds \u001b[38;5;241m=\u001b[39m \u001b[43m_chunk_ds\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 371\u001b[0m \u001b[43m \u001b[49m\u001b[43mbackend_ds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 372\u001b[0m \u001b[43m \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 373\u001b[0m \u001b[43m \u001b[49m\u001b[43mengine\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 374\u001b[0m \u001b[43m \u001b[49m\u001b[43mchunks\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 375\u001b[0m \u001b[43m \u001b[49m\u001b[43moverwrite_encoded_chunks\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 376\u001b[0m \u001b[43m \u001b[49m\u001b[43minline_array\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 377\u001b[0m \u001b[43m \u001b[49m\u001b[43mchunked_array_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 378\u001b[0m \u001b[43m \u001b[49m\u001b[43mfrom_array_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 379\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mextra_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 380\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 382\u001b[0m ds\u001b[38;5;241m.\u001b[39mset_close(backend_ds\u001b[38;5;241m.\u001b[39m_close)\n\u001b[1;32m 384\u001b[0m \u001b[38;5;66;03m# Ensure source filename always stored in dataset object\u001b[39;00m\n",
+ "File \u001b[0;32m/opt/homebrew/Caskroom/mambaforge/base/envs/sklearn-argo-dev/lib/python3.9/site-packages/xarray/backends/api.py:318\u001b[0m, in \u001b[0;36m_chunk_ds\u001b[0;34m(backend_ds, filename_or_obj, engine, chunks, overwrite_encoded_chunks, inline_array, chunked_array_type, from_array_kwargs, **extra_tokens)\u001b[0m\n\u001b[1;32m 307\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_chunk_ds\u001b[39m(\n\u001b[1;32m 308\u001b[0m backend_ds,\n\u001b[1;32m 309\u001b[0m filename_or_obj,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 316\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mextra_tokens,\n\u001b[1;32m 317\u001b[0m ):\n\u001b[0;32m--> 318\u001b[0m chunkmanager \u001b[38;5;241m=\u001b[39m \u001b[43mguess_chunkmanager\u001b[49m\u001b[43m(\u001b[49m\u001b[43mchunked_array_type\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 320\u001b[0m \u001b[38;5;66;03m# TODO refactor to move this dask-specific logic inside the DaskManager class\u001b[39;00m\n\u001b[1;32m 321\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(chunkmanager, DaskManager):\n",
+ "File \u001b[0;32m/opt/homebrew/Caskroom/mambaforge/base/envs/sklearn-argo-dev/lib/python3.9/site-packages/xarray/namedarray/parallelcompat.py:117\u001b[0m, in \u001b[0;36mguess_chunkmanager\u001b[0;34m(manager)\u001b[0m\n\u001b[1;32m 115\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(manager, \u001b[38;5;28mstr\u001b[39m):\n\u001b[1;32m 116\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m manager \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;129;01min\u001b[39;00m chunkmanagers:\n\u001b[0;32m--> 117\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m 118\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124munrecognized chunk manager \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmanager\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m - must be one of: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mlist\u001b[39m(chunkmanagers)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 119\u001b[0m )\n\u001b[1;32m 121\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m chunkmanagers[manager]\n\u001b[1;32m 122\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(manager, ChunkManagerEntrypoint):\n\u001b[1;32m 123\u001b[0m \u001b[38;5;66;03m# already a valid ChunkManager so just pass through\u001b[39;00m\n",
+ "\u001b[0;31mValueError\u001b[0m: unrecognized chunk manager dask - must be one of: []"
+ ]
+ }
+ ],
+ "source": [
+ "xr.open_mfdataset(\"/Users/sangminsong/Library/CloudStorage/OneDrive-UW/Code/2024_Pythia/SOCCOM_GO-BGC_LoResQC_LIAR_28Aug2023_netcdf/*.nc\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Your first content section"
+ "## Argovis"
]
},
{
@@ -295,7 +896,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.8"
+ "version": "3.9.19"
},
"nbdime-conflicts": {
"local_diff": [
diff --git a/notebooks/argo-introduction.ipynb b/notebooks/argo-introduction.ipynb
index dad9f26..e6148a9 100644
--- a/notebooks/argo-introduction.ipynb
+++ b/notebooks/argo-introduction.ipynb
@@ -4,22 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Let's start here! If you can directly link to an image relevant to your notebook, such as [canonical logos](https://github.com/numpy/numpy/blob/main/doc/source/_static/numpylogo.svg), do so here at the top of your notebook. You can do this with Markdown syntax,\n",
- "\n",
- "> `![](http://link.com/to/image.png \"image alt text\")`\n",
- "\n",
- "or edit this cell to see raw HTML `img` demonstration. This is preferred if you need to shrink your embedded image. **Either way be sure to include `alt` text for any embedded images to make your content more accessible.**\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Project Pythia Notebook Template\n",
- "\n",
- "Next, title your notebook appropriately with a top-level Markdown header, `#`. Do not use this level header anywhere else in the notebook. Our book build process will use this title in the navbar, table of contents, etc. Keep it short, keep it descriptive. Follow this with a `---` cell to visually distinguish the transition to the prerequisites section."
+ "# Introduction to Argo Observations"
]
},
{
@@ -36,11 +21,12 @@
"## Overview\n",
"If you have an introductory paragraph, lead with it here! Keep it short and tied to your material, then be sure to continue into the required list of topics below,\n",
"\n",
- "1. This is a numbered list of the specific topics\n",
- "1. These should map approximately to your main sections of content\n",
- "1. Or each second-level, `##`, header in your notebook\n",
- "1. Keep the size and scope of your notebook in check\n",
- "1. And be sure to let the reader know up front the important concepts they'll be leaving with"
+ "Here, we introduce Argo profiling floats, which are autonomous instruments that operate remotely and sample the ocean interior continuously. \n",
+ "\n",
+ "1. What are Argo floats?\n",
+ "2. What data are available, and in what formats?\n",
+ " \n",
+ "In the next notebook, [Accessing Argo Data](notebooks/argo-access.ipynb), we will explore different ways of downloading and retrieving float profiles. \n"
]
},
{
-# Scikit-learn with Argo Observations Cookbook
+# Scikit-learn on Argo Observations
[](https://github.com/song-sangmin/sklearn-argo/actions/workflows/nightly-build.yaml)
[](https://binder.projectpythia.org/v2/gh/song-sangmin/sklearn-argo/main?labpath=notebooks)
@@ -18,7 +18,7 @@ This Project Pythia Cookbook covers two objectives:
## Authors
-[Song Sangmin (@song-sangmin), [Second Author](@second-author), etc. _Acknowledge primary content authors here_
+[Song Sangmin](@song-sangmin), [Michael Chen](@second-author).
### Contributors
@@ -30,17 +30,16 @@ This Project Pythia Cookbook covers two objectives:
(State one or more sections that will comprise the notebook. E.g., _This cookbook is broken up into two main sections - "Foundations" and "Example Workflows."_ Then, describe each section below.)
-This cookbook is broken up into __ sections:
+This cookbook is broken up into two main sections.
-1. Accessing BGC-Argo data with Argopy
-2. Applying scikit-learn to prepare data for machine learning
-3. Using sckikit-learn to develop regression models (e.g. RFR, XGBoost)
+1. Argo Foundations
+2. Scikit-learn Workflows
-### Section 1: Accessing BGC-Argo ( Replace with the title of this section, e.g. "Foundations" )
+### Section 1: Argo Foundations
(Add content for this section, e.g., "The foundational content includes ... ")
-### Section 2: Scikit-learn ( Replace with the title of this section, e.g. "Example workflows" )
+### Section 2: Scikit-learn Workflows
(Add content for this section, e.g., "Example workflows include ... ")
diff --git a/environment.yml b/environment.yml
index db2263d..e658246 100644
--- a/environment.yml
+++ b/environment.yml
@@ -8,6 +8,7 @@ dependencies:
- python=3.9
- numpy
- scipy
+ - pip
# Data packages
- pandas
@@ -34,3 +35,5 @@ dependencies:
- hyperopt
- dataframe_image
+ - pip:
+ - argovisHelpers==0.0.26
diff --git a/notebooks/argo-access.ipynb b/notebooks/argo-access.ipynb
index dad9f26..85e829a 100644
--- a/notebooks/argo-access.ipynb
+++ b/notebooks/argo-access.ipynb
@@ -17,9 +17,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Project Pythia Notebook Template\n",
- "\n",
- "Next, title your notebook appropriately with a top-level Markdown header, `#`. Do not use this level header anywhere else in the notebook. Our book build process will use this title in the navbar, table of contents, etc. Keep it short, keep it descriptive. Follow this with a `---` cell to visually distinguish the transition to the prerequisites section."
+ "# Accessing Argo Data"
]
},
{
@@ -34,13 +32,17 @@
"metadata": {},
"source": [
"## Overview\n",
- "If you have an introductory paragraph, lead with it here! Keep it short and tied to your material, then be sure to continue into the required list of topics below,\n",
"\n",
- "1. This is a numbered list of the specific topics\n",
- "1. These should map approximately to your main sections of content\n",
- "1. Or each second-level, `##`, header in your notebook\n",
- "1. Keep the size and scope of your notebook in check\n",
- "1. And be sure to let the reader know up front the important concepts they'll be leaving with"
+ "Building upon previous notebook, [Introduction to Argo](notebooks/argo-introduction.ipynb), we next explore how to access Argo data in a few different ways. \n",
+ "\n",
+ "1. Data formats for Argo profiles\n",
+ "2. Downloading [monthly snapshots](http://www.argodatamgt.org/Access-to-data/Argo-DOI-Digital-Object-Identifier) using Argo DOI's\n",
+ "3. Using [Argovis](https://argovis.colorado.edu/argo) for API-based queries \n",
+ "4. Using the [GO-BGC Toolbox](https://github.com/go-bgc/workshop-python)\n",
+ "5. Using [Argopy](https://argopy.readthedocs.io/en/latest/user-guide/fetching-argo-data/index.html), a dedicated Python package\n",
+ "\n",
+ "After going through this notebook, you will be able to retrieve Argo data of interest within a certain time frame, geographical location, or by platform identifier. There are many ways of working with Argo data which are not described here. \n",
+ "Further information on Argo access can be found on the [Argo website](https://argo.ucsd.edu/data/)."
]
},
{
@@ -48,23 +50,16 @@
"metadata": {},
"source": [
"## Prerequisites\n",
- "This section was inspired by [this template](https://github.com/alan-turing-institute/the-turing-way/blob/master/book/templates/chapter-template/chapter-landing-page.md) of the wonderful [The Turing Way](https://the-turing-way.netlify.app) Jupyter Book.\n",
- "\n",
- "Following your overview, tell your reader what concepts, packages, or other background information they'll **need** before learning your material. Tie this explicitly with links to other pages here in Foundations or to relevant external resources. Remove this body text, then populate the Markdown table, denoted in this cell with `|` vertical brackets, below, and fill out the information following. In this table, lay out prerequisite concepts by explicitly linking to other Foundations material or external resources, or describe generally helpful concepts.\n",
"\n",
"Label the importance of each concept explicitly as **helpful/necessary**.\n",
"\n",
"| Concepts | Importance | Notes |\n",
"| --- | --- | --- |\n",
- "| [Intro to Cartopy](https://foundations.projectpythia.org/core/cartopy/cartopy.html) | Necessary | |\n",
- "| [Understanding of NetCDF](https://foundations.projectpythia.org/core/data-formats/netcdf-cf.html) | Helpful | Familiarity with metadata structure |\n",
- "| Project management | Helpful | |\n",
+ "| [Intro to Numpy](https://numpy.org/learn/) | Necessary | |\n",
+ "| [Intro to NetCDF](https://foundations.projectpythia.org/core/data-formats/netcdf-cf.html) | Necessary | Familiarity with metadata structure |\n",
+ "| [Intro to Xarray](https://foundations.projectpythia.org/core/xarray.html) | Necessary | |\n",
"\n",
- "- **Time to learn**: estimate in minutes. For a rough idea, use 5 mins per subsection, 10 if longer; add these up for a total. Safer to round up and overestimate.\n",
- "- **System requirements**:\n",
- " - Populate with any system, version, or non-Python software requirements if necessary\n",
- " - Otherwise use the concepts table above and the Imports section below to describe required packages as necessary\n",
- " - If no extra requirements, remove the **System requirements** point altogether"
+ "- **Time to learn**: 20 min\n"
]
},
{
@@ -84,18 +79,624 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "ModuleNotFoundError",
+ "evalue": "No module named 'argovisHelpers'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
+ "Cell \u001b[0;32mIn[1], line 14\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mmatplotlib\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mcolors\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mmcolors\u001b[39;00m\n\u001b[1;32m 12\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mseaborn\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01msns\u001b[39;00m\n\u001b[0;32m---> 14\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01margovisHelpers\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m helpers \u001b[38;5;28;01mas\u001b[39;00m avh\n",
+ "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'argovisHelpers'"
+ ]
+ }
+ ],
+ "source": [
+ "# Import packages\n",
+ "import sys\n",
+ "import os\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "import scipy\n",
+ "import xarray as xr\n",
+ "from datetime import datetime, timedelta\n",
+ "\n",
+ "import matplotlib.pyplot as plt\n",
+ "import matplotlib.colors as mcolors\n",
+ "import seaborn as sns\n",
+ "\n",
+ "from argovisHelpers import helpers as avh"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Snapshots"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# # Base filepath. Need for Argo GDAC function.\n",
+ "root = '/Users/sangminsong/Library/CloudStorage/OneDrive-UW/Code/2024_Pythia/'\n",
+ "profile_dir = root + 'SOCCOM_GO-BGC_LoResQC_LIAR_28Aug2023_netcdf/'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
- "import sys"
+ "DSdict = {}\n",
+ "for filename in os.listdir(profile_dir):\n",
+ " if filename.endswith(\".nc\"):\n",
+ " fp = profile_dir + filename\n",
+ " single_dataset = xr.open_dataset(fp, decode_times=False)\n",
+ " DSdict[filename[0:7]] = single_dataset\n",
+ "# DSdict['5906030']"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "![\"Project](\"images/ProjectPythia_Logo_Final-01-Blue.svg\"){kind=logo}
"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Project Pythia Notebook Template\n",
- "\n",
- "Next, title your notebook appropriately with a top-level Markdown header, `#`. Do not use this level header anywhere else in the notebook. Our book build process will use this title in the navbar, table of contents, etc. Keep it short, keep it descriptive. Follow this with a `---` cell to visually distinguish the transition to the prerequisites section."
+ "# Introduction to Argo Observations"
]
},
{
@@ -36,11 +21,12 @@
"## Overview\n",
"If you have an introductory paragraph, lead with it here! Keep it short and tied to your material, then be sure to continue into the required list of topics below,\n",
"\n",
- "1. This is a numbered list of the specific topics\n",
- "1. These should map approximately to your main sections of content\n",
- "1. Or each second-level, `##`, header in your notebook\n",
- "1. Keep the size and scope of your notebook in check\n",
- "1. And be sure to let the reader know up front the important concepts they'll be leaving with"
+ "Here, we introduce Argo profiling floats, which are autonomous instruments that operate remotely and sample the ocean interior continuously. \n",
+ "\n",
+ "1. What are Argo floats?\n",
+ "2. What data are available, and in what formats?\n",
+ " \n",
+ "In the next notebook, [Accessing Argo Data](notebooks/argo-access.ipynb), we will explore different ways of downloading and retrieving float profiles. \n"
]
},
{